On Mon, Oct 8, 2012 at 4:29 PM, Arpit Gupta <ar...@hortonworks.com> wrote:
>
> i would recommended using the oracle jdk.
oracle version didn't help
> Also from your email below you mention that mapred.child.java.opts and
> mapred.child.ulimit were added to try to solve this problem. Are you setting
> memory settings for your map and reduce tasks? It might help if you share
> the full mapred-site.xml.
attached please find the mapred-site.xml, The configuration is simply
the default configuration generated by the RPM package provided by
Hadoop, with the extra mapred.child.java.opts and mapred.child.ulimit
setup discussed already
at this point we don't have any further clue on what to do except
switch back to 0.20.2
any help is appreciated,
Attila
>
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Oct 8, 2012, at 2:45 AM, Attila Csordas <attilacsor...@gmail.com> wrote:
>
> OpenJDK 64-Bit Server VM (build 19.0-b09, mixed mode)
>
> might the official Oracle Java be better?
>
> Thanks,
> Attila
>
> On Sun, Oct 7, 2012 at 8:37 PM, Arpit Gupta <ar...@hortonworks.com> wrote:
>
> are you using 32bit jdk for your task trackers?
>
> If so reduce the mem setting in mapred.child.java.opts
>
> --
> Arpit
>
> On Oct 7, 2012, at 12:29 PM, Attila Csordas <attilacsor...@gmail.com> wrote:
>
> <property>
> <name>mapred.tasktracker.map.tasks.maximum</name>
> <value>10</value>
> </property>
>
> <property>
> <name>mapred.tasktracker.reduce.tasks.maximum</name>
> <value>6</value>
> </property>
>
> Cheers,
> Attila
>
> On Sun, Oct 7, 2012 at 6:34 AM, Harsh J <ha...@cloudera.com> wrote:
>
> Hi,
>
> What is your # of slots per TaskTracker? Your ulimit seems pretty
> high. I'd set it to 1.5x times heap initially, i.e., 6291456 (6 GB)
> and try.
>
> On Sun, Oct 7, 2012 at 3:50 AM, Attila Csordas <attilacsor...@gmail.com>
> wrote:
>
> some details to this problem:
>
> 12/10/05 12:13:27 INFO mapred.JobClient: map 0% reduce 0%
> 12/10/05 12:13:40 INFO mapred.JobClient: Task Id :
> attempt_201210051158_0001_m_000002_0, Status : FAILED
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Task process exit with nonzero status of
> 134.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:258)
>
> attempt_201210051158_0001_m_000002_0: #
> attempt_201210051158_0001_m_000002_0: # There is insufficient memory
> for the Java Runtime Environment to continue.
> attempt_201210051158_0001_m_000002_0: # pthread_getattr_np
>
> in mapred-site.xml the following memory settings were set after a
> couple trials to get rid of the problem this way:
>
> <property>
> <name>mapred.child.java.opts</name>
> <value>-server -Xmx4096M -Djava.net.preferIPv4Stack=true</value>
> </property>
>
> <property>
> <name>mapred.child.ulimit</name>
> <value>16777216</value>
> </property>
>
> Cheers,
> Attila
>
>
>
> On Fri, Oct 5, 2012 at 10:50 AM, Steve Lewis <lordjoe2...@gmail.com> wrote:
>
> [We get 'There is insufficient memory for the Java Runtime Environment to
> continue.'
> any time we run any job including the most trivial word count process. It is
> true I am generating a jar for a larger job but only running a version of
> wordcount that worked well under 0.2
> Any bright ideas???
> This is a new 1.03 installation and nothing is known to work
>
> Steven M. Lewis PhD
> 4221 105th Ave NE
> Kirkland, WA 98033
> cell 206-384-1340
> skype lordjoe_com
>
>
>
>
> --
> Harsh J
>
>
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!-- Put site-specific property overrides in this file. -->
<configuration>
<property>
<name>mapred.tasktracker.tasks.sleeptime-before-sigkill</name>
<value>250</value>
<description>Normally, this is the amount of time before killing
processes, and the recommended-default is 5.000 seconds - a value of
5000 here. In this case, we are using it solely to blast tasks before
killing them, and killing them very quickly (1/4 second) to guarantee
that we do not leave VMs around for later jobs.
</description>
</property>
<property>
<name>mapred.system.dir</name>
<value>/mapred/mapredsystem</value>
<final>true</final>
</property>
<property>
<name>mapred.job.tracker</name>
<value>hadoop-master-02.ourdomain:9000</value>
<final>true</final>
</property>
<property>
<name>mapred.job.tracker.http.address</name>
<value>hadoop-master-02.ourdomain:50030</value>
<final>true</final>
</property>
<property>
<name>mapred.local.dir</name>
<value>/var/lib/hadoop/mapred</value>
<final>true</final>
</property>
<property>
<name>mapreduce.cluster.administrators</name>
<value>mapred</value>
</property>
<property>
<name>mapred.map.tasks.speculative.execution</name>
<value>false</value>
<description>If true, then multiple instances of some map tasks
may be executed in parallel.</description>
</property>
<property>
<name>mapred.reduce.tasks.speculative.execution</name>
<value>false</value>
<description>If true, then multiple instances of some reduce tasks
may be executed in parallel.</description>
</property>
<property>
<name>mapred.output.compression.type</name>
<value>BLOCK</value>
<description>If the job outputs are to compressed as SequenceFiles, how
should they be compressed? Should be one of NONE, RECORD or BLOCK.
</description>
</property>
<property>
<name>jetty.connector</name>
<value>org.mortbay.jetty.nio.SelectChannelConnector</value>
</property>
<property>
<name>mapred.task.tracker.task-controller</name>
<value>org.apache.hadoop.mapred.DefaultTaskController</value>
</property>
<property>
<name>mapred.child.root.logger</name>
<value>INFO,TLA</value>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-server -Xmx4096M -Djava.net.preferIPv4Stack=true</value>
</property>
<property>
<name>mapred.child.ulimit</name>
<value>6291456</value>
</property>
<property>
<name>mapred.job.tracker.persist.jobstatus.active</name>
<value>true</value>
<description>Indicates if persistency of job status information is
active or not.
</description>
</property>
<property>
<name>mapred.job.tracker.persist.jobstatus.dir</name>
<value>file:////var/log/hadoop/mapred/jobstatus</value>
<description>The directory where the job status information is persisted
in a file system to be available after it drops of the memory queue and
between jobtracker restarts.
</description>
</property>
<property>
<name>mapred.job.tracker.history.completed.location</name>
<value>/mapred/history/done</value>
</property>
<property>
<name>mapred.heartbeats.in.second</name>
<value>200</value>
<description>to enable HADOOP:5784</description>
</property>
<property>
<name>mapreduce.tasktracker.outofband.heartbeat</name>
<value>true</value>
<description>to enable MAPREDUCE:270</description>
</property>
<property>
<name>mapred.jobtracker.maxtasks.per.job</name>
<value>200000</value>
<final>true</final>
<description>The maximum number of tasks for a single job.
A value of -1 indicates that there is no maximum.
</description>
</property>
<property>
<name>mapreduce.jobtracker.kerberos.principal</name>
<value>jt/_HOST@${local.realm}</value>
<description>
JT principal
</description>
</property>
<property>
<name>mapreduce.tasktracker.kerberos.principal</name>
<value>tt/_HOST@${local.realm}</value>
<description>
TT principal.
</description>
</property>
<property>
<name>hadoop.job.history.user.location</name>
<value>none</value>
</property>
<property>
<name>mapreduce.jobtracker.keytab.file</name>
<value>/etc/security/keytabs/jt.service.keytab</value>
<description>
The keytab for the jobtracker principal.
</description>
</property>
<property>
<name>mapreduce.tasktracker.keytab.file</name>
<value>/etc/security/keytabs/tt.service.keytab</value>
<description>The filename of the keytab for the task tracker</description>
</property>
<property>
<name>mapreduce.jobtracker.staging.root.dir</name>
<value>/user</value>
<description>The Path prefix for where the staging directories should be
placed. The next level is always the user's
name. It is a path in the default file system.
</description>
</property>
<property>
<name>mapreduce.job.acl-modify-job</name>
<value></value>
</property>
<property>
<name>mapreduce.job.acl-view-job</name>
<value>Dr.Who</value>
</property>
<property>
<name>mapreduce.tasktracker.group</name>
<value>hadoop</value>
<description>The group that the task controller uses for accessing the
task controller. The mapred user must be a member and users should *not*
be members.
</description>
</property>
<property>
<name>mapred.acls.enabled</name>
<value>true</value>
</property>
<property>
<name>mapred.jobtracker.taskScheduler</name>
<value>org.apache.hadoop.mapred.CapacityTaskScheduler</value>
</property>
<property>
<name>mapred.queue.names</name>
<value>default</value>
</property>
<!-- settings for the history server -->
<property>
<name>mapreduce.history.server.embedded</name>
<value>false</value>
</property>
<property>
<name>mapreduce.history.server.http.address</name>
<value>hadoop-master-02.ourdomain:51111</value>
</property>
<property>
<name>mapreduce.jobhistory.kerberos.principal</name>
<value>jt/_HOST@${local.realm}</value>
<description>history server principal</description>
</property>
<property>
<name>mapreduce.jobhistory.keytab.file</name>
<value>/etc/security/keytabs/jt.service.keytab</value>
<description>
The keytab for the jobtracker principal.
</description>
</property>
<property>
<name>mapred.hosts</name>
<value>/etc/hadoop/mapred.include</value>
<description>Names a file that contains the list of nodes that may
connect to the jobtracker. If the value is empty, all hosts are
permitted.</description>
</property>
<property>
<name>mapred.hosts.exclude</name>
<value>/etc/hadoop/mapred.exclude</value>
<description>Names a file that contains the list of hosts that
should be excluded by the jobtracker. If the value is empty, no
hosts are excluded.</description>
</property>
<property>
<name>mapred.jobtracker.retirejob.check</name>
<value>10000</value>
</property>
<property>
<name>mapred.jobtracker.retirejob.interval</name>
<value>0</value>
</property>
<property>
<name>mapred.cluster.map.memory.mb</name>
<value>-1</value>
<description>The size, in terms of virtual memory, of a single map slot
in the Map-Reduce framework, used by the scheduler.
A job can ask for multiple slots for a single map task via
mapred.job.map.memory.mb, upto the limit specified by
mapred.cluster.max.map.memory.mb, if the scheduler supports the feature.
The value of -1 indicates that this feature is turned off.
</description>
</property>
<property>
<name>mapred.cluster.reduce.memory.mb</name>
<value>-1</value>
<description>The size, in terms of virtual memory, of a single reduce slot
in the Map-Reduce framework, used by the scheduler.
A job can ask for multiple slots for a single reduce task via
mapred.job.reduce.memory.mb, upto the limit specified by
mapred.cluster.max.reduce.memory.mb, if the scheduler supports the feature.
The value of -1 indicates that this feature is turned off.
</description>
</property>
<property>
<name>mapred.cluster.max.map.memory.mb</name>
<value>-1</value>
<description>The maximum size, in terms of virtual memory, of a single map
task launched by the Map-Reduce framework, used by the scheduler.
A job can ask for multiple slots for a single map task via
mapred.job.map.memory.mb, upto the limit specified by
mapred.cluster.max.map.memory.mb, if the scheduler supports the feature.
The value of -1 indicates that this feature is turned off.
</description>
</property>
<property>
<name>mapred.cluster.max.reduce.memory.mb</name>
<value>-1</value>
<description>The maximum size, in terms of virtual memory, of a single reduce
task launched by the Map-Reduce framework, used by the scheduler.
A job can ask for multiple slots for a single reduce task via
mapred.job.reduce.memory.mb, upto the limit specified by
mapred.cluster.max.reduce.memory.mb, if the scheduler supports the feature.
The value of -1 indicates that this feature is turned off.
</description>
</property>
<property>
<name>mapred.job.map.memory.mb</name>
<value>-1</value>
<description>The size, in terms of virtual memory, of a single map task
for the job.
A job can ask for multiple slots for a single map task, rounded up to the
next multiple of mapred.cluster.map.memory.mb and upto the limit
specified by mapred.cluster.max.map.memory.mb, if the scheduler supports
the feature.
The value of -1 indicates that this feature is turned off iff
mapred.cluster.map.memory.mb is also turned off (-1).
</description>
</property>
<property>
<name>mapred.job.reduce.memory.mb</name>
<value>-1</value>
<description>The size, in terms of virtual memory, of a single reduce task
for the job.
A job can ask for multiple slots for a single map task, rounded up to the
next multiple of mapred.cluster.reduce.memory.mb and upto the limit
specified by mapred.cluster.max.reduce.memory.mb, if the scheduler supports
the feature.
The value of -1 indicates that this feature is turned off iff
mapred.cluster.reduce.memory.mb is also turned off (-1).
</description>
</property>
<property>
<name>mapred.tasktracker.map.tasks.maximum</name>
<value>10</value>
</property>
<property>
<name>mapred.tasktracker.reduce.tasks.maximum</name>
<value>6</value>
</property>
</configuration>
