Singletons aren't hacks; it can be an entirely appropriate pattern for this. What exception do you get? From Spark or your code? I think this pattern is orthogonal to using Spark. On Jan 21, 2015 8:11 AM, "octavian.ganea" <octavian.ga...@inf.ethz.ch> wrote:
> In case someone has the same problem: > > The singleton hack works for me sometimes, sometimes it doesn't in spark > 1.2.0, that is, sometimes I get nullpointerexception. Anyway, if you really > need to work with big indexes and you want to have the smallest amount of > communication between master and nodes, as well as if you have RAM > available > just for one instance of the indexes data per machine, than I suggest you > use spark with memcached . > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-share-a-NonSerializable-variable-among-tasks-in-the-same-worker-node-tp11048p21282.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
