[
https://issues.apache.org/jira/browse/SPARK-1112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14011993#comment-14011993
]
Kevin (Sangwoo) Kim commented on SPARK-1112:
--------------------------------------------
[~matei]
I've found the default of spark.akka.frameSize is 10 from the config document,
http://spark.apache.org/docs/0.9.1/configuration.html
just tried to slightly larger and smaller (11 and 9) values.
I did collect() method on the userInfo and it might contains large data.
(edited the first comment.)
> When spark.akka.frameSize > 10, task results bigger than 10MiB block execution
> ------------------------------------------------------------------------------
>
> Key: SPARK-1112
> URL: https://issues.apache.org/jira/browse/SPARK-1112
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 0.9.0
> Reporter: Guillaume Pitel
> Priority: Critical
> Fix For: 0.9.2
>
>
> When I set the spark.akka.frameSize to something over 10, the messages sent
> from the executors to the driver completely block the execution if the
> message is bigger than 10MiB and smaller than the frameSize (if it's above
> the frameSize, it's ok)
> Workaround is to set the spark.akka.frameSize to 10. In this case, since
> 0.8.1, the blockManager deal with the data to be sent. It seems slower than
> akka direct message though.
> The configuration seems to be correctly read (see actorSystemConfig.txt), so
> I don't see where the 10MiB could come from
--
This message was sent by Atlassian JIRA
(v6.2#6252)
