How to avoid spark executor fetching jars in Local mode

2019-10-11 Thread Bin Chen
Hello everyone, For spark local mode, I noticed the executor fetching jars from local, is there a way to avoid these steps, as these jars already on the local filesystem, it takes seconds to fetching all the jars, we want to reduce the time on bootstrap phase. we are using spark-submit with

Calculate Task Memory Usage

2019-10-11 Thread Jack Kolokasis
Hello to all, I am trying to calculate how much memory each task in Spark consumes. Is there any way to measure this ? Thanks, Iacovos - To unsubscribe e-mail: user-unsubscr...@spark.apache.org