lto:user@spark.apache.org>"
Subject: Re: spark performance - executor computing time
Is this repeatable? Do you always get one or two executors that are 6 times as
slow? It could be that some of your tasks have more work to do (maybe you are
filtering some records out? If it’s always one p
Is this repeatable? Do you always get one or two executors that are 6 times as
slow? It could be that some of your tasks have more work to do (maybe you are
filtering some records out? If it’s always one particular worker node is there
something about the machine configuration (e.g. CPU speed)
Hi,
I was running a job (on Spark 1.5 + Yarn + java 8). In a stage that
lookup
(org.apache.spark.rdd.PairRDDFunctions.lookup(PairRDDFunctions.scala:873))
there was an executor that took the executor computing time > 6 times of
median. This executor had almost the same shuffle read size and
