Re: Plugin dependancies do not get added to classpath while running Nutch in local mode

Hi Sujen, strange: other Kafka classes are found (ConfigDef, ProducerConfig, ConfigException) but DefaultPartitioner which is contained in the same jar not. Need to check the Kafka code how the partitioner classes are loaded. Could be that Kafka loads classes dynamically in a way incompatible w

Re: Plugin dependancies do not get added to classpath while running Nutch in local mode

Hi Sebastian, Here is the complete log trace from the haddop.log file 2016-09-25 19:14:08,455 INFO fetcher.FetchItemQueues - Using queue mode : byHost 2016-09-25 19:14:08,455 INFO fetcher.Fetcher - Fetcher: threads: 50 2016-09-25 19:14:08,455 INFO fetcher.Fetcher - Fetcher: time-out divisor: 2

Re: Plugin dependancies do not get added to classpath while running Nutch in local mode

Hi Sujen, could you send the complete stack trace? Just to be sure from where the error stems. > I looked at the code here > https://github.com/apache/nutch/blob/master/src/bin/nutch#L155-L164 > and > cannot understand the u

Re: Plugin dependancies do not get added to classpath while running Nutch in local mode

Thank you Sebastian for your response. I followed the steps as per your suggestion and added the required jars under runtime in plugin.xml. My code is at - https://github.com/ sujen1412/nutch/blob/kafka/src/plugin/publish-kafka/plugin.xml. Now after compiling and running ./bin/crawl in local mode

Re: Plugin dependancies do not get added to classpath while running Nutch in local mode

Hi Sujen, are the jars also listed in the plugin.xml? That's required. The plugin-specific ivy.xml is only used at compile time to fetch the library and its dependencies and get the plugin compiled. At runtime all required libs have to be listed in the plugin.xml, e.g., https://github.com/apache