Hi x.split returns an array. So, after first map, you will get RDD of arrays. What is your expected outcome of 2nd map?
On Mon, Sep 5, 2016 at 11:30 PM, Ashok Kumar <ashok34...@yahoo.com.invalid> wrote: > Thank you sir. > > This is what I get > > scala> textFile.map(x=> x.split(",")) > res52: org.apache.spark.rdd.RDD[Array[String]] = MapPartitionsRDD[27] at > map at <console>:27 > > How can I work on individual columns. I understand they are strings > > scala> textFile.map(x=> x.split(",")).map(x => (x.getString(0)) > | ) > <console>:27: error: value getString is not a member of Array[String] > textFile.map(x=> x.split(",")).map(x => (x.getString(0)) > > regards > > > > > On Monday, 5 September 2016, 13:51, Somasundaram Sekar <somasundar.sekar@ > tigeranalytics.com> wrote: > > > Basic error, you get back an RDD on transformations like map. > sc.textFile("filename").map(x => x.split(",") > > On 5 Sep 2016 6:19 pm, "Ashok Kumar" <ashok34...@yahoo.com.invalid> wrote: > > Hi, > > I have a text file as below that I read in > > 74,20160905-133143,98. 11218069128827594148 > 75,20160905-133143,49. 52776998815916807742 > 76,20160905-133143,56. 08029957123980984556 > 77,20160905-133143,46. 63689526544407522777 > 78,20160905-133143,84. 88227141164402181551 > 79,20160905-133143,68. 72408602520662115000 > > val textFile = sc.textFile("/tmp/mytextfile. txt") > > Now I want to split the rows separated by "," > > scala> textFile.map(x=>x.toString). split(",") > <console>:27: error: value split is not a member of > org.apache.spark.rdd.RDD[ String] > textFile.map(x=>x.toString). split(",") > > However, the above throws error? > > Any ideas what is wrong or how I can do this if I can avoid converting it > to String? > > Thanking > > > > -- Best Regards, Ayan Guha