Hi,
Is there some way to customize the Akka configuration for Spark?
Specifically, I want to experiment with custom serialization for messages
that are send between the driver and executors in standalone mode.
Thanks,
Akshat
Hi,
Any input on this? I'm willing to instrument further and experiment
if there are any ideas.
On Mon, May 4, 2015 at 11:27 AM, Akshat Aranya wrote:
> Hi,
>
> I have been investigating scheduling delays in Spark and I found some
> unexplained anomalies. In my use case, I have
Hi,
I have been investigating scheduling delays in Spark and I found some
unexplained anomalies. In my use case, I have two stages after
collapsing the transformations: the first is a mapPartitions() and the
second is a sortByKey(). I found that the task serialization for the
first stage takes m
Hi,
I'm trying to figure out how the Spark standalone cluster mode works.
Specifically, I'm looking at the code to see how the user's application jar
makes it from the submission node, when it is on the local file system, to
the driver and the executors.
>From what I can see, spark-submit causes
