Yes it does. We could get around this by changing it to look for
nutch-*.jar instead of *.jar but then there are nutch references in the
hadoop scripts. Is there a better way to do this?
Dennis Kubes
Nigel Daley wrote:
-1
It sounds like this undoes
http://issues.apache.org/jira/browse/HADOOP-700
On Jan 31, 2007, at 3:51 PM, Dennis Kubes (JIRA) wrote:
[
https://issues.apache.org/jira/browse/HADOOP-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dennis Kubes updated HADOOP-964:
--------------------------------
Status: Patch Available (was: Open)
This patch adds any jars in the HADOOP_HOME directory to the hadoop
classpath that starts the Tasktracker.
Hadoop Shell Script causes ClassNotFoundException for Nutch processes
---------------------------------------------------------------------
Key: HADOOP-964
URL: https://issues.apache.org/jira/browse/HADOOP-964
Project: Hadoop
Issue Type: Bug
Components: scripts
Environment: windows xp and fedora core 6 linux, java
1.5.10...should affect all systems
Reporter: Dennis Kubes
Priority: Critical
Fix For: 0.11.0
Attachments: classpath.patch
In the ReduceTaskRunner constructor lin 339 a sorter is created that
attempts to get the map output key and value classes from the
configuration object. This is before the TaskTracker$Child process
is spawned off into into own separate JVM so here the classpath for
the configuration is the classpath that started the TaskTracker. The
current hadoop script includes the hadoop jars, meaning that any
hadoop writable type will be found, but it doesn't include nutch
jars so any nutch writable type or any other writable type will not
be found and will throw a ClassNotFoundException.
I don't think it is a good idea to have a dependecy on specific Nutch
jars in the Hadoop script but it is a good idea to allow jars to be
included if they are in specific locations, such as the HADOOP_HOME
where the nutch jar resides. I have attached a patch that adds any
jars in the HADOOP_HOME directory to the hadoop classpath. This
fixes the issues with getting ClassNotFoundExceptions inside of Nutch
processes.
--This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
