thanks. I was looking into this before a while back but got sidetracked and am just coming back to it. But I do remember thinking that the arguments may have been dropped by /bin/mahout spark-shell
i tried: $/bin/mahout spark-shell -Dspark.kryoserializer.buffer.mb=200 but its not showing up in propertires the environment tab of localhost:/8080 -> "Mahout Spark Shell" I'll look back into /bin/Mahout to see if there's a problem there. I'm getting the following error: com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 0, required: 1274 after doing some work on a Drm containing the output of seq2sparse from the 20newsgroups example. it seems to be failing at .collect > Date: Thu, 11 Sep 2014 09:39:32 -0700 > Subject: Re: setting spark config parameters for shell > From: [email protected] > To: [email protected] > > I remember i had a good answer for these type of things in context of the > shell, but have forgotten the answer... bummer: ) > > In spark, you can just pass them in with -Dname=value. May need tweaking > bin/mahout script though. that's what i dont remember. > > I thought we were setting a reasonable default though.. > > > On Thu, Sep 11, 2014 at 9:22 AM, Andrew Palumbo <[email protected]> wrote: > > > Does anybody know of an easy way to set the config parameters for the > > mahout spark-shell? > > > > I need to adjust: spark.kryoserializer.buffer.mb > > > > I've been diging through the spark docs but not having much luck. > > > >
