I believe that the implicit def is pulling in the enclosing class (in which the def is defined) in the closure which is not serializable.
On Wed, Apr 29, 2015 at 4:20 AM, guoqing0...@yahoo.com.hk < guoqing0...@yahoo.com.hk> wrote: > Hi guys, > I`m puzzled why i cant use the implicit function in spark streaming to > cause Task not serializable . > > code snippet: > > implicit final def str2KeyValue(s:String): (String,String) = { > val message = s.split("\\|") > if(message.length >= 2) > (message(0),message(1)) > else if(message.length == 1) { > (message(0), "") > } > else > ("","") > } > > def filter(stream:DStream[String]) :DStream[String] = { > stream.filter(s => { > (s._1=="Action" && s._2=="TRUE") > }) > > > Could you please give me some pointers ? Thank you . > > ------------------------------ > guoqing0...@yahoo.com.hk >