Good question, I think inside of akka they will allocate a buffer of this size for every message. So if you set it super high you'll waste some memory temporarily allocating these buffers.
The main issue with this IIRC was for fetching results, which we fixed in 0.8.0 to use a different communication library. - Patrick On Mon, Nov 25, 2013 at 9:29 AM, Andrew Ash <[email protected]> wrote: > There have been a number of threads on this list about needing to set > spark.akka.frameSize to something higher than the default. The issue seems > to come up most when one key in a groupByKey has particularly large amounts > of data. > > What is the downside to setting this configuration parameter to the maximum > value by default? > > Andrew
