I'm trying to do a a job with Spark, using Java.
The thing is I need to have an index of words of about 3 GB in each
machine, so I'm trying to broadcast custom objects to represent the index
and the interface with it.
I'm using java standard serialization, so I tried to implement serializable
interface in each class involved, but some objects come from libraries so I
can't go any further.
Is there another way to make it works?
Should I try with Kryo?
Is there a way to work with non-serializable objects?
I use a fat-jar, so the code is available in all workers really. I thing it
should be a way to use it instead of being serializing and deserializing