Please update to the following: commit c2f0821aad3b82dcd327e914c9b297e92526649d Author: Zhang, Liye <liye.zh...@intel.com> Date: Fri May 8 09:10:58 2015 +0100
[SPARK-7392] [CORE] bugfix: Kryo buffer size cannot be larger than 2M On Sun, May 24, 2015 at 8:04 AM, Debasish Das <debasish.da...@gmail.com> wrote: > I am May 3rd commit: > > commit 49549d5a1a867c3ba25f5e4aec351d4102444bc0 > > Author: WangTaoTheTonic <wangtao...@huawei.com> > > Date: Sun May 3 00:47:47 2015 +0100 > > > [SPARK-7031] [THRIFTSERVER] let thrift server take SPARK_DAEMON_MEMORY > and SPARK_DAEMON_JAVA_OPTS > > On Sat, May 23, 2015 at 7:54 PM, Josh Rosen <rosenvi...@gmail.com> wrote: > >> Which commit of master are you building off? It looks like there was a >> bugfix for an issue related to KryoSerializer buffer configuration: >> https://github.com/apache/spark/pull/5934 >> >> That patch was committed two weeks ago, but you mentioned that you're >> building off a newer version of master. Could you confirm the commit that >> you're running? If this used to work but now throws an error, then this is >> a regression that should be fixed; we shouldn't require you to perform a mb >> -> kb conversion to work around this. >> >> On Sat, May 23, 2015 at 6:37 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> Pardon me. >>> >>> Please use '8192k' >>> >>> Cheers >>> >>> On Sat, May 23, 2015 at 6:24 PM, Debasish Das <debasish.da...@gmail.com> >>> wrote: >>> >>>> Tried "8mb"...still I am failing on the same error... >>>> >>>> On Sat, May 23, 2015 at 6:10 PM, Ted Yu <yuzhih...@gmail.com> wrote: >>>> >>>>> bq. it shuld be "8mb" >>>>> >>>>> Please use the above syntax. >>>>> >>>>> Cheers >>>>> >>>>> On Sat, May 23, 2015 at 6:04 PM, Debasish Das < >>>>> debasish.da...@gmail.com> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I am on last week's master but all the examples that set up the >>>>>> following >>>>>> >>>>>> .set("spark.kryoserializer.buffer", "8m") >>>>>> >>>>>> are failing with the following error: >>>>>> >>>>>> Exception in thread "main" java.lang.IllegalArgumentException: >>>>>> spark.kryoserializer.buffer must be less than 2048 mb, got: + 8192 mb. >>>>>> looks like buffer.mb is deprecated...Is "8m" is not the right syntax >>>>>> to get 8mb kryo buffer or it shuld be "8mb" >>>>>> >>>>>> Thanks. >>>>>> Deb >>>>>> >>>>> >>>>> >>>> >>> >> >