Hi for a 30 GB executor how much offheap should I give along with yarn memory over head is it ok?
On Thu, Jan 7, 2016 at 4:24 AM, Ted Yu <yuzhih...@gmail.com> wrote: > Turns out that I should have specified -i to my former grep command :-) > > Thanks Marcelo > > But does this mean that specifying custom value for parameter > spark.memory.offheap.size > would not take effect ? > > Cheers > > On Wed, Jan 6, 2016 at 2:47 PM, Marcelo Vanzin <van...@cloudera.com> > wrote: > >> Try "git grep -i spark.memory.offheap.size"... >> >> On Wed, Jan 6, 2016 at 2:45 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> > Maybe I looked in the wrong files - I searched *.scala and *.java files >> (in >> > latest Spark 1.6.0 RC) for '.offheap.' but didn't find the config. >> > >> > Can someone enlighten me ? >> > >> > Thanks >> > >> > On Wed, Jan 6, 2016 at 2:35 PM, Jakob Odersky <joder...@gmail.com> >> wrote: >> >> >> >> Check the configuration guide for a description on units >> >> ( >> http://spark.apache.org/docs/latest/configuration.html#spark-properties). >> >> In your case, 5GB would be specified as 5g. >> >> >> >> On 6 January 2016 at 10:29, unk1102 <umesh.ka...@gmail.com> wrote: >> >>> >> >>> Hi As part of Spark 1.6 release what should be ideal value or unit for >> >>> spark.memory.offheap.size I have set as 5000 I assume it will be 5GB >> is >> >>> it >> >>> correct? Please guide. >> >>> >> >>> >> >>> >> >>> -- >> >>> View this message in context: >> >>> >> http://apache-spark-user-list.1001560.n3.nabble.com/What-should-be-the-ideal-value-unit-for-spark-memory-offheap-size-tp25898.html >> >>> Sent from the Apache Spark User List mailing list archive at >> Nabble.com. >> >>> >> >>> --------------------------------------------------------------------- >> >>> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> >>> For additional commands, e-mail: user-h...@spark.apache.org >> >>> >> >> >> > >> >> >> >> -- >> Marcelo >> > >
