MapReduce
http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/
"We’ll thus assign 4 GB for Map task Containers, and 8 GB for Reduce tasks
Containers."
You can try editing configuration file:
<name>mapreduce.map.memory.mb</name>
<value>?</value>
<name>mapreduce.reduce.memory.mb</name>
<value>?</value>
Or set these in a driver.
If you have custom AM, try this:
Driver:
capability.setMemory(amMemory);
appContext.setResource(capability);
AM:
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(containerMemory);
AMRMClient.ContainerRequest request = new
AMRMClient.ContainerRequest(capability, null, null, pri);
OR AM (use java command to set heap size):
vargs.add("-Xmx" + containerMemory + "m");
2013/11/15 Rob Blah <[email protected]>
> Question have you tried giving more memory to the containers?
>
>
> 2013/11/15 unmesha sreeveni <[email protected]>
>
>> ys found
>> tried -D mapred.child.java.opts=-Xmx4096M on the command line:
>>
>>
>> On Fri, Nov 15, 2013 at 3:01 PM, unmesha sreeveni
>> <[email protected]>wrote:
>>
>>> hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata /user/unmesha
>>> /out
>>>
>>>
>>>
>>> On Fri, Nov 15, 2013 at 2:53 PM, unmesha sreeveni <[email protected]
>>> > wrote:
>>>
>>>> When i tried to execute my program with 100 MB file i am getting
>>>> JavaHeapSpace Exception
>>>>
>>>>
>>>> >hadoop jar /home/my/hadoop2.jar /user/unmesha/inputdata
>>>> /user/inputdata/out
>>>>
>>>> How to increase the heap size through commandline
>>>>
>>>>
>>>> --
>>>> *Thanks & Regards*
>>>>
>>>> Unmesha Sreeveni U.B
>>>>
>>>> *Junior Developer*
>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> *Thanks & Regards*
>>>
>>> Unmesha Sreeveni U.B
>>>
>>> *Junior Developer*
>>>
>>>
>>>
>>
>>
>> --
>> *Thanks & Regards*
>>
>> Unmesha Sreeveni U.B
>>
>> *Junior Developer*
>>
>>
>>
>
