Re: broadcast params to workers at the very beginning

[Yanbo Liang]

Hi,

The parameters should be broadcasted again after you update it at driver
side, then you can get updated version at worker side.

Thanks
Yanbo

2016-01-09 23:12 GMT+08:00 octavian.ganea <octavian.ga...@inf.ethz.ch>:

> Hi,
>
> In my app, I have a Params scala object that keeps all the specific
> (hyper)parameters of my program. This object is read in each worker. I
> would
> like to be able to pass specific values of the Params' fields in the
> command
> line. One way would be to simply update all the fields of the Params object
> using the values in the command line arguments. However, this will only
> update the Params local copy at the master node, while the worker nodes
> will
> still use the default Params version that is broadcasted by default at the
> very beginning of the program.
>
> Does anyone have an idea of how can I change at runtime the parameters of a
> specific object for each of its copies located at each worker ?
>
> Thanks,
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/broadcast-params-to-workers-at-the-very-beginning-tp25927.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>

broadcast params to workers at the very beginning

[octavian.ganea]

Hi,

In my app, I have a Params scala object that keeps all the specific
(hyper)parameters of my program. This object is read in each worker. I would
like to be able to pass specific values of the Params' fields in the command
line. One way would be to simply update all the fields of the Params object
using the values in the command line arguments. However, this will only
update the Params local copy at the master node, while the worker nodes will
still use the default Params version that is broadcasted by default at the
very beginning of the program.

Does anyone have an idea of how can I change at runtime the parameters of a
specific object for each of its copies located at each worker ?

Thanks, 



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/broadcast-params-to-workers-at-the-very-beginning-tp25927.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org