What Partitioner do you use ? Have you tried using RangePartitioner ?
Cheers On Wed, Feb 10, 2016 at 3:54 PM, daze5112 <david.zeel...@ato.gov.au> wrote: > Hi im trying to improve the performance of some code im running but have > noticed that my distribution of my RDD across executors isn't exactly even > (see pic below). Im using yarn and kicking it off with 11 executors. Not > sure how to get a more even spread or if this is normal. thanks > > < > http://apache-spark-user-list.1001560.n3.nabble.com/file/n26200/spark_partitions.png > > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/RDD-distribution-tp26200.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
