Do you use java 1.4 or 1.5 ? In general have a look to the hadoop code base: TaskRunner.java line: 145.
Stefan Am 05.06.2006 um 10:51 schrieb Murat Ali Bayir: > Hi eveybody, I have problem in running Jprofiler in remote side. I > am using DFS and submitting crawl job. > I configure LD library path and hadoop-default.xml in machine which > I run hadoop tasktracker, > in hadoop-default.xml, my java child compile option is given as: > > ---------------------------------------------------------------------- > ---------------------------------------------------------------------- > --------------------------------------------------------------- > > <property> > <name>mapred.child.java.opts</name> > <value>-Xmx200m - > agentlib:jprofilerti=port=8849,nowait,id=121,config=/home/agmlab/ > jprofiler4/config/config.xml -Xbootclasspath/a:/home/agmlab/ > jprofiler4/bin/agent.jar</value> > <description>Java opts for the task tracker child processes. > Subsumes > 'mapred.child.heap.size' (If a mapred.child.heap.size value is found > in a configuration, its maximum heap size will be used and a warning > emitted that heap.size has been deprecated). Also, the following > symbols, > if present, will be interpolated: @taskid@ is replaced by current > TaskID; > and @port@ will be replaced by mapred.task.tracker.report.port + 1 > (A second > child will fail with a port-in-use if > mapred.tasktracker.tasks.maximum is > greater than one). Any other occurrences of '@' will go unchanged. > For > example, to enable verbose gc logging to a file named for the > taskid in > /tmp and to set the heap maximum to be a gigabyte, pass a 'value' of: > > -Xmx1024m -verbose:gc -Xloggc:/tmp/@[EMAIL PROTECTED] > </description> > </property> > > ---------------------------------------------------------------------- > ---------------------------------------------------------------------- > --------------------------------------------------------------- > > When I submit a job, I run ps aux | grep java from the shell in > tasktracker, my child process related to crawl operation is given as: > > ********************************************************************** > ******************************************************* > > agmlab 8775 0.0 1.0 495460 22356 pts/1 Rl+ 11:39 0:00 /opt/ > jdk1.5.0_07/jre/bin/java -Xmx200m -classpath /home/agmlab/source/ > Hadoop/conf:/opt/jdk1.5.0_07/lib/tools.jar:/home/agmlab/source/ > Hadoop/build/classes > :/home/agmlab/source/Hadoop/build:/home/agmlab/source/Hadoop/build/ > test/classes:/home/agmlab/source/Hadoop: > /home/agmlab/source/Hadoop/hadoop-0.2.0-examples.jar:/home/agmlab/ > source/Hadoop/hadoop-0.2.0-test.jar > :/home/agmlab/source/Hadoop/hadoop-0.2.1.jar:/home/agmlab/source/ > Hadoop/lib/commons-logging-api-1.0.4.jar: > /home/agmlab/source/Hadoop/lib/jetty-5.1.4.jar:/home/agmlab/source/ > Hadoop/lib/junit-3.8.1.jar: > /home/agmlab/source/Hadoop/lib/lucene-core-1.9.1.jar:/home/agmlab/ > source/Hadoop/lib/servlet-api.jar: > /home/agmlab/source/Hadoop/lib/jetty-ext/ant.jar:/home/agmlab/ > source/Hadoop/lib/jetty-ext/commons-el.jar: > /home/agmlab/source/Hadoop/lib/jetty-ext/jasper-compiler.jar:/home/ > agmlab/source/Hadoop/lib/jetty-ext/jasper-runtime.jar > :/home/agmlab/source/Hadoop/lib/jetty-ext/jsp-api.jar: > :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/ > jetty-ext:/home/agmlab/temp/local/taskTracker/ > task_0010_m_000001_0/work/lib/pmd-ext:/home/agmlab/temp/local/ > taskTracker/task_0010_m_000001_0/ > work/lib/lucene-core-1.9.1.jar > :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/ > xerces-2_6_2-apis.jar > :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/ > jetty-5.1.4.jar:/home/agmlab/temp/local/taskTracker > /task_0010_m_000001_0/work/lib/commons-codec-1.3.jar: > /home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/ > xerces-2_6_2.jar > :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/ > taglibs-i18n.jar:/home/agmlab/temp/local/ > taskTracker/task_0010_m_000001_0/work/lib/commons-logging- > api-1.0.4.jar:/home/agmlab/temp/local/taskTracker/ > task_0010_m_000001_0/work/lib/je-2.0.83.jar > :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/ > lucene-misc-1.9.1.jar:/home/agmlab/temp/local/ > taskTracker/task_0010_m_000001_0/work/lib/xmlrpc-2.0.jar:/home/ > agmlab/temp/local/taskTracker/ > task_0010_m_000001_0/work/lib/servlet.jar:/home/agmlab/temp/local/ > taskTracker/task_0010_m_000001_0/work/lib/ > concurrent-1.3.4.jar: > /home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/ > jakarta-oro-2.0.7.jar > :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/ > commons-lang-2.1.jar > :/home/agmlab/temp/local/taskTracker/task_0010_m_000001_0/work/lib/ > servlet-api.jar:/home/agmlab/temp/local/ > taskTracker/task_0010_m_000001_0/work/lib/junit-3.8.1.jar:/home/ > agmlab/ > temp/local/taskTracker/task_0010_m_000001_0/work/classes:/home/ > agmlab/temp/local/taskTracker/ > task_0010_m_000001_0/work org.apache.hadoop.mapred.TaskTracker > $Child 18850 task_0010_m_000001_0 > agmlab 8789 0.0 0.0 51088 664 pts/0 R+ 11:39 0:00 grep > java > > ********************************************************************** > ************************************************************* > The options given below in XML document after -Xmx200m does not > work, what am I doing wrong in this configurations? > -agentlib:jprofilerti=port=8849,nowait,id=121,config=/home/agmlab/ > jprofiler4/config/config.xml -Xbootclasspath/a:/home/agmlab/ > jprofiler4/bin/agent.jar > > > > _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
