Thanks. I tried. The problem is I have to updateStatebyKey to maintain
other states related to keys.
Not sure where to pass this accumulator variable into updateStateBykey.
On Tue, Aug 18, 2015 at 2:17 AM, Hemant Bhanawat wrote:
> See if SparkContext.accumulator helps.
>
> On Tue, Aug 18, 2015 at 2:27 PM, Joanne Contact
> wrote:
>>
>> Hi Gurus,
>>
>> Please help.
>>
>> But please don't tell me to use updateStateByKey because I need a
>> global variable (something like the clock time) across the micro
>> batches but not depending on key. For my case, it is not acceptable to
>> maintain a state for each key since each key comes in different times.
>> Yes my global variable is related to time but cannot use machine
>> clock.
>>
>> Any hint? Or is this lack of global variable by design?
>>
>> Thanks!
>>
>> -
>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
>> For additional commands, e-mail: user-h...@spark.apache.org
>>
>
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org
