; t...@databricks.com
Subject: Re: [Spark Streaming] Streaming Dynamic Allocation is broken (at least
on YARN)
Any ideas @Tathagata? I'd be happy to contribute a patch if you can point me in
the right direction.
From: Karthik Palaniappan
Sent: Friday, Augu
017 2:34 AM
To: Karthik Palaniappan<mailto:karthik...@hotmail.com>
Cc: user@spark.apache.org<mailto:user@spark.apache.org>;
t...@databricks.com<mailto:t...@databricks.com>
Subject: Re: [Spark Streaming] Streaming Dynamic Allocation is broken (at least
on YARN)
Have you tried setting
mail.com>
Cc: user@spark.apache.org<mailto:user@spark.apache.org>;
t...@databricks.com<mailto:t...@databricks.com>
Subject: Re: [Spark Streaming] Streaming Dynamic Allocation is broken (at least
on YARN)
Have you tried setting spark.executor.instances=0 to a positive non-zero value?
Have you tried setting spark.executor.instances=0 to a positive non-zero
value? Also, since its a streaming application set executor cores > 1.
On Wed, Aug 23, 2017 at 3:38 AM, Karthik Palaniappan wrote:
> I ran the HdfsWordCount example using this command:
>
> spark-submit run-example \
> --c
I ran the HdfsWordCount example using this command:
spark-submit run-example \
--conf spark.streaming.dynamicAllocation.enabled=true \
--conf spark.executor.instances=0 \
--conf spark.dynamicAllocation.enabled=false \
--conf spark.master=yarn \
--conf spark.submit.deployMode=client \
o