I have now a fair understanding of the situation after looking at javap
output. So as a reminder:
dstream.map(tuple = {
val w = StreamState.fetch[K,W](state.prefixKey, tuple._1)
(tuple._1, (tuple._2, w)) })
And StreamState being a very simple standalone object:
object StreamState {
def
Vaguely makes sense. :) Wow that's an interesting corner case.
On Wed, Apr 22, 2015 at 1:57 PM, Jean-Pascal Billaud j...@tellapart.com
wrote:
I have now a fair understanding of the situation after looking at javap
output. So as a reminder:
dstream.map(tuple = {
val w =
At this point I am assuming that nobody has an idea... I am still going to
give it a last shot just in case it was missed by some people :)
Thanks,
On Mon, Apr 20, 2015 at 2:20 PM, Jean-Pascal Billaud j...@tellapart.com
wrote:
Hey, so I start the context at the very end when all the piping is
Sure. But in general, I am assuming this Graph is unexpectedly null when
DStream is being serialized must mean something. Under which
circumstances, such an exception would trigger?
On Tue, Apr 21, 2015 at 4:12 PM, Tathagata Das t...@databricks.com wrote:
Yeah, I am not sure what is going on.
It is kind of unexpected, i can imagine a real scenario under which it
should trigger. But obviously I am missing something :)
TD
On Tue, Apr 21, 2015 at 4:23 PM, Jean-Pascal Billaud j...@tellapart.com
wrote:
Sure. But in general, I am assuming this Graph is unexpectedly null
when DStream is
Yeah, I am not sure what is going on. The only way to figure to take a look
at the disassembled bytecodes using javap.
TD
On Tue, Apr 21, 2015 at 1:53 PM, Jean-Pascal Billaud j...@tellapart.com
wrote:
At this point I am assuming that nobody has an idea... I am still going to
give it a last
Hey, so I start the context at the very end when all the piping is done.
BTW a foreachRDD will be called on the resulting dstream.map() right after
that.
The puzzling thing is why removing the context bounds solve the problem...
What does this exception mean in general?
On Mon, Apr 20, 2015 at
Hi,
I am getting this serialization exception and I am not too sure what Graph
is unexpectedly null when DStream is being serialized means?
15/04/20 06:12:38 INFO yarn.ApplicationMaster: Final app status: FAILED,
exitCode: 15, (reason: User class threw exception: Task not serializable)
Exception
