[
https://issues.apache.org/jira/browse/SPARK-6404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Patrick Wendell resolved SPARK-6404.
------------------------------------
Resolution: Invalid
I'm closing the issue because broadcast variables are immutable, so you can't
change their value. An approach like the one Saisai suggested is better... just
create a new broadcast variable for each batch. If there is some other issue
with that, we can create a new JIRA to fix it.
> Call broadcast() in each interval for spark streaming programs.
> ---------------------------------------------------------------
>
> Key: SPARK-6404
> URL: https://issues.apache.org/jira/browse/SPARK-6404
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Reporter: Yifan Wang
>
> If I understand it correctly, Spark’s broadcast() function will be called
> only once at the beginning of the batch. For streaming applications that need
> to run for 24/7, it is often needed to update variables that shared by
> broadcast() dynamically. It would be ideal if broadcast() could be called at
> the beginning of each interval.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
