Hi,
I've sent a few emails to the user mailing list, but since I
believe this is bug, I think it's time to talk to the developpers.
So here is what happens: since we've migrated from 0.8.1 to 0.9,
whatever the value of spark.akka.frameSize I set, the Executors
lock when a collect tries to send more than 10MB of data to the
driver.
Since 10MB is the spark default, I suspect it could be related to
something in the configuration. We still use System.setProperty to
set the frameSize.
As a workaround, setting the frameSize back to 10 seems to work.
Guillaume
--
|
Guillaume
PITEL, Président
+33(0)6 25 48 86 80
eXenSa
S.A.S.
41, rue Périer -
92120 Montrouge - FRANCE
Tel +33(0)1 84 16 36 77 / Fax +33(0)9 72 28 37
05
|
|
