Hello everyone,

For spark local mode, I noticed the executor fetching jars from local, is there 
a way to avoid these steps, as these jars already on the local filesystem, it 
takes seconds to fetching all the jars, we want to reduce the time on bootstrap 

we are using spark-submit with `--conf spark.jars` to define these libs.

[2019-09-26 06:19:59,651] [INFO ] o.a.s.e.Executor - Fetching 
spark:// with timestamp 1569478797007
[2019-09-26 06:19:59,729] [INFO ] o.a.s.n.c.TransportClientFactory - 
Successfully created connection to / after 55 ms (0 ms spent in 
[2019-09-26 06:19:59,754] [INFO ] o.a.s.u.Utils - Fetching 
spark:// to 
[2019-09-26 06:19:59,801] [INFO ] o.a.s.e.Executor - Adding 
 to class loader


Chen Bin

As a recipient of an email from Talend, your contact personal data will be on 
our systems. Please see our contacts privacy notice at Talend, Inc. 

Reply via email to