[mailto:zoltan.zv...@gmail.com]
*Sent:* Thursday, May 07, 2015 10:05 PM
*To:* Taeyun Kim; user@spark.apache.org
*Subject:* Re: YARN mode startup takes too long (10+ secs)
Without considering everything, just a few hints:
You are running on YARN. From 09:18:34 to 09:18:37 your application
the
spark.yarn.scheduler.heartbeat.interval-ms.
I hope that the additional overhead it incurs would be negligible.
From: Zoltán Zvara [mailto:zoltan.zv...@gmail.com]
Sent: Thursday, May 07, 2015 10:05 PM
To: Taeyun Kim; user@spark.apache.org
Subject: Re: YARN mode startup takes too long (10+ secs)
Without
Hi,
I'm running a spark application with YARN-client or YARN-cluster mode.
But it seems to take too long to startup.
It takes 10+ seconds to initialize the spark context.
Is this normal? Or can it be optimized?
The environment is as follows:
- Hadoop: Hortonworks HDP 2.2 (Hadoop 2.6)
