Hello,
Once a broadcast variable is created using sparkContext.broadcast(), can it
ever be updated again? The use case is for something like the underlying
lookup data changing over time.
Thanks
NB
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com
of the intermediate
streams were not recovered properly and caused some of our computations to
produce incorrect values.
Any help/insights into how to go about tackling this will be appreciated.
Thanks
NB.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark
but
everything is recreated from the checkpointed data.
Hope this helps,
NB
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/kafka-Spark-Streaming-with-checkPointing-fails-to-restart-tp22864p22878.html
Sent from the Apache Spark User List mailing list archive
then
but was confused by documentation talking about never changing the
broadcasted variables.
I've tried it on a local mode process till now and does seem to work as
intended. When (and if !) we start running on a real cluster, I hope this
holds up.
Thanks
NB
On Tue, May 19, 2015 at 6:25 AM, Imran Rashid iras
Well, we haven't really enabled recovery after running into this issue in
Spark 1.2. I do intend to try this again soon with Spark 1.6.1 and see if it
works out this time.
NB
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Spark-recovery-takes-long