[
https://issues.apache.org/jira/browse/NUTCH-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120714#comment-13120714
]
Ferdy commented on NUTCH-1148:
------------------------------
For completeness: a fetcher job fails when run as 'hadoop jar ...' with the
following exception. This is using the current nutchgora branch. (Btw is anyone
able to affirm this? I could not find much about this on the mailing list and
issuetracker. I want to make sure that is not caused by something else.)
11/10/05 08:27:05 WARN plugin.PluginRepository: Plugins: directory not found:
plugins
11/10/05 08:27:05 INFO plugin.PluginRepository: Plugin Auto-activation mode:
[true]
11/10/05 08:27:05 INFO plugin.PluginRepository: Registered Plugins:
11/10/05 08:27:05 INFO plugin.PluginRepository: NONE
11/10/05 08:27:05 INFO plugin.PluginRepository: Registered Extension-Points:
11/10/05 08:27:05 INFO plugin.PluginRepository: NONE
Exception in thread "main" java.lang.RuntimeException: x-point
org.apache.nutch.protocol.Protocol not found.
at
org.apache.nutch.protocol.ProtocolFactory.<init>(ProtocolFactory.java:55)
at org.apache.nutch.fetcher.FetcherJob.getFields(FetcherJob.java:144)
at org.apache.nutch.fetcher.FetcherJob.run(FetcherJob.java:183)
at org.apache.nutch.fetcher.FetcherJob.fetch(FetcherJob.java:224)
at org.apache.nutch.fetcher.FetcherJob.run(FetcherJob.java:309)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.nutch.fetcher.FetcherJob.main(FetcherJob.java:315)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:186)
> PluginManifestParser cannot load plugins from classpath that is dynamically
> set using a contextClassLoader.
> -----------------------------------------------------------------------------------------------------------
>
> Key: NUTCH-1148
> URL: https://issues.apache.org/jira/browse/NUTCH-1148
> Project: Nutch
> Issue Type: Bug
> Reporter: Ferdy
> Attachments: NUTCH-1148-v1.patch
>
>
> This affects running nutchgora using Hadoop it's RunJar mechanism (hadoop jar
> ...). The mr tasks are perfectly able to load the plugins (please note
> NUTCH-937). But, when the plugins are loaded from the *job submitter* process
> itself, loading plugins might fail due to classloading issues. This is caused
> by the fact that PluginManifestParser does not use the contextClassLoader
> that is set by RunJar. This classloader contains the plugins folder. At least
> the FetcherJob is affected by this, because the job submitter uses getFields
> of Protocol implementations, therefore loading the plugins.
> The current 1.x is not affected because it does not load plugins at any point
> during the job submission. This might of course change so I propose to 'fix'
> the issue in the 1.x branch as well.
> The solution is fairly simple, PluginManifestParser should use the
> contextClassLoader of the current thread instead of using the system
> classloader. I will attach patch right away. It currently works but it still
> needs some further testing.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
