Re: SparkContext.wholeTextFiles throws not serializable exception

I have to use .lines.toArray.toSeq A little tricky. [image: --] Xi Shen [image: http://]about.me/davidshen On Fri, Mar 27, 2015 at 4:41 PM, Xi Shen wrote: > Hi, > > I want to load my data in this way: > > sc.wholeText

SparkContext.wholeTextFiles throws not serializable exception

Hi, I want to load my data in this way: sc.wholeTextFiles(opt.input) map { x => (x._1, x._2.lines.filter(!_.isEmpty).toSeq) } But I got java.io.NotSerializableException: scala.collection.Iterator$$anon$13 But if I use "x._2.split('\n')", I can get the expected result. I want to know what's wr