Could you put the implicit def in an object? That should work, as objects
are never serialized.
On Wed, Apr 29, 2015 at 6:28 PM, guoqing0...@yahoo.com.hk <
guoqing0...@yahoo.com.hk> wrote:
> Thank you for your pointers , it`s very helpful to me , in this scenario
> how can i use the implicit def in the enclosing class ?
>
>
>
> *From:* Tathagata Das <t...@databricks.com>
> *Date:* 2015-04-30 07:00
> *To:* guoqing0...@yahoo.com.hk
> *CC:* user <user@spark.apache.org>
> *Subject:* Re: implicit function in SparkStreaming
> I believe that the implicit def is pulling in the enclosing class (in
> which the def is defined) in the closure which is not serializable.
>
>
> On Wed, Apr 29, 2015 at 4:20 AM, guoqing0...@yahoo.com.hk <
> guoqing0...@yahoo.com.hk> wrote:
>
>> Hi guys,
>> I`m puzzled why i cant use the implicit function in spark streaming to
>> cause Task not serializable .
>>
>> code snippet:
>>
>> implicit final def str2KeyValue(s:String): (String,String) = {
>> val message = s.split("\\|")
>> if(message.length >= 2)
>> (message(0),message(1))
>> else if(message.length == 1) {
>> (message(0), "")
>> }
>> else
>> ("","")
>> }
>>
>> def filter(stream:DStream[String]) :DStream[String] = {
>> stream.filter(s => {
>> (s._1=="Action" && s._2=="TRUE")
>> })
>>
>>
>> Could you please give me some pointers ? Thank you .
>>
>> ------------------------------
>> guoqing0...@yahoo.com.hk
>>
>
>
