Re: Memory used in Spark-0.9.0-incubating

2014-09-25 Thread
istake 1024 and 1000? Best Regards, Yi Tian tianyi.asiai...@gmail.com <mailto:tianyi.asiai...@gmail.com> On Sep 25, 2014, at 18:41, 王晓雨 <mailto:wangxiao...@jd.com>> wrote: My yarn-site.xml config: yarn.nodemanager.resource.memory-mb 16384 ENV: Spark:0.9.0-incubating Hadoop:2.3.0

Re: Memory used in Spark-0.9.0-incubating

2014-09-25 Thread
My yarn-site.xml config: yarn.nodemanager.resource.memory-mb 16384 ENV: Spark:0.9.0-incubating Hadoop:2.3.0 I run spark task on Yarn. I see the log in Nodemanager: 2014-09-25 17:43:34,141 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory u

Memory used in Spark-0.9.0-incubating

2014-09-25 Thread
And where to config "5G" maxinum memory for container? -- 王晓雨 【云平台-开放云事业部】数据引擎组 - 手机:18600049984 电话:01062425760 邮箱:wangxiao...@jd.com 邮编:100195 地址:北京市海淀区杏石口路65号益园文创基地C区11号楼3层 -

Memory used in Spark-0.9.0-incubating

2014-09-25 Thread
ENV: Spark:0.9.0-incubating Hadoop:2.3.0 I run spark task on Yarn. I see the log in Nodemanager: 2014-09-25 17:43:34,141 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Memory usage of ProcessTree 549 for container-id container_1411635522254_0001_