Which commit of master are you building off? It looks like there was a
bugfix for an issue related to KryoSerializer buffer configuration:
https://github.com/apache/spark/pull/5934
That patch was committed two weeks ago, but you mentioned that you're
building off a newer version of master. Could you confirm the commit that
you're running? If this used to work but now throws an error, then this is
a regression that should be fixed; we shouldn't require you to perform a mb
-> kb conversion to work around this.
On Sat, May 23, 2015 at 6:37 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> Pardon me.
>
> Please use '8192k'
>
> Cheers
>
> On Sat, May 23, 2015 at 6:24 PM, Debasish Das <debasish.da...@gmail.com>
> wrote:
>
>> Tried "8mb"...still I am failing on the same error...
>>
>> On Sat, May 23, 2015 at 6:10 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>>
>>> bq. it shuld be "8mb"
>>>
>>> Please use the above syntax.
>>>
>>> Cheers
>>>
>>> On Sat, May 23, 2015 at 6:04 PM, Debasish Das <debasish.da...@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am on last week's master but all the examples that set up the
>>>> following
>>>>
>>>> .set("spark.kryoserializer.buffer", "8m")
>>>>
>>>> are failing with the following error:
>>>>
>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>> spark.kryoserializer.buffer must be less than 2048 mb, got: + 8192 mb.
>>>> looks like buffer.mb is deprecated...Is "8m" is not the right syntax to
>>>> get 8mb kryo buffer or it shuld be "8mb"
>>>>
>>>> Thanks.
>>>> Deb
>>>>
>>>
>>>
>>
>
