Could you clarify why you need a 10G akka frame size?
Best Regards,
Shixiong Zhu
2015-02-05 9:20 GMT+08:00 Shixiong Zhu :
> The unit of "spark.akka.frameSize" is MB. The max value is 2047.
>
> Best Regards,
> Shixiong Zhu
>
> 2015-02-05 1:16 GMT+08:00 sahanbull :
>
>> I am trying to run a spark
The unit of "spark.akka.frameSize" is MB. The max value is 2047.
Best Regards,
Shixiong Zhu
2015-02-05 1:16 GMT+08:00 sahanbull :
> I am trying to run a spark application with
>
> -Dspark.executor.memory=30g -Dspark.kryoserializer.buffer.max.mb=2000
> -Dspark.akka.frameSize=1
>
> and the job
I am trying to run a spark application with
-Dspark.executor.memory=30g -Dspark.kryoserializer.buffer.max.mb=2000
-Dspark.akka.frameSize=1
and the job fails because one or more of the akka frames are larger than
1mb (12000 ish).
When I change the Dspark.akka.frameSize=1 to 12000,1
