Is the broadcasted variable distributed to every executor or every worker? Now
i'm more confused
I thought it was supposed to save memory by distributing it to every worker and
the executors would share that copy
Date: Fri, 19 Feb 2016 16:48:59 -0800
Subject: Re: Access to broadcasted variable
From: shixi...@databricks.com
To: jeffsar...@hotmail.com
CC: user@spark.apache.org
The broadcasted object is serialized in driver and sent to the executors. And
in the executor, it will deserialize the bytes to get the broadcasted object.
On Fri, Feb 19, 2016 at 5:54 AM, jeff saremi <jeffsar...@hotmail.com> wrote:
could someone please comment on this? thanks
From: jeffsar...@hotmail.com
To: user@spark.apache.org
Subject: Access to broadcasted variable
Date: Thu, 18 Feb 2016 14:44:07 -0500
I'd like to know if the broadcasted object gets serialized when accessed by the
executor during the execution of a task?
I know that it gets serialized from the driver to the worker. This question is
inside worker when executor JVM's are accessing it
thanks
Jeff
