scala - Spark print result of Arrays or saveAsTextFile -


i have been bogged down hours now... tried collect , mkstring(") , still not able print in console or save text file.

scala> val au1 = sc.parallelize(list(("a",array(1,2)),("b",array(1,2)))) scala> val au2 = sc.parallelize(list(("a",array(3)),("b",array(2)))) scala> val au3 = au1.union(au2) 

result of union

array[(string,array[int])] = array((a,array(1,2)),(b,array(1,2)),(a,array(3)),(b,array(2))) 

all print attempts resulting in following when x(0) , x(1)

 array[int]) not take parameters  

last attempt, performed following , resulting in index error

scala> val au4 = au3.map(x => (x._1, x._2._1._1, x._2._1._2)) <console>:33: error: value _1 not member of array[int]          val au4 = au3.map(x => (x._1, x._2._1._1, x._2._1._2)) 

._1 or ._2 can done in tuples , not in arrays

("a",array(1,2)) tuple ._1 a , ._2 array(1,2)

so if want elements of array need use () x._2(0)

but au2 arrays has 1 element x._2(1) work au1 , not au2. can use option or try below

val au4 = au3.map(x => (x._1, x._2(0), try(x._2(1)) getorelse(0))) 

Comments

Popular posts from this blog

android - InAppBilling registering BroadcastReceiver in AndroidManifest -

python Tkinter Capturing keyboard events save as one single string -

sql server - Why does Linq-to-SQL add unnecessary COUNT()? -