Hi Sebastian and lewis,
Did build on other machine and diffed the runtime log. Got the issues
pretty clear
yes, the build was not proper. Got it resolved.
Happy crawling.
Regards,
GoViNd
On Mon, Jan 15, 2018 at 2:04 AM, Sebastian Nagel <wastl.na...@googlemail.com
> wrote:
> Hi Govind,
>
> thanks. At least, although it's caught later it seems a little bit clearer
> what is happening:
>
> > Exception in thread "main" java.lang.NoSuchMethodError:
> >
> org.apache.nutch.util.NutchJob.getInstance(Lorg/apache/hadoop/conf/
> Configuration;Ljava/lang/String;)Lorg/apache/nutch/util/NutchJob;
>
> However, there is a method in NutchJob.java:
>
> public static NutchJob getInstance(Configuration conf, String jobName)
> throws IOException {
>
> As said, I'm able to run Injector of the current 2.x branch with the
> changes described (use
> MongoDB), so this is really weired. Looks more like a build or class path
> issue...
>
> Best,
> Sebastian
>
>
> On 01/13/2018 08:19 AM, govind nitk wrote:
> >
> > Hi Sebastian,
> >
> > Thanks for clarification.
> >
> > $cat /tmp/urls/seeds.txt
> > http://nutch.apache.org/
> >
> > $export 'NUTCH_OPTS=-Xverify:none'
> > $./bin/nutch inject /tmp/urls/
> >
> > SLF4J: Class path contains multiple SLF4J bindings.
> > SLF4J: Found binding in
> > [jar:file:/home/govind/apache/nutch/runtime/local/lib/slf4j-
> log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: Found binding in
> > [jar:file:/home/govind/apache/nutch/runtime/local/lib/slf4j-
> log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> > InjectorJob: starting at 2018-01-13 12:09:33
> > InjectorJob: Injecting urlDir: /tmp/urls
> > Exception in thread "main" java.lang.NoSuchMethodError:
> > org.apache.nutch.util.NutchJob.getInstance(Lorg/apache/hadoop/conf/
> Configuration;Ljava/lang/String;)Lorg/apache/nutch/util/NutchJob;
> > at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:229)
> > at org.apache.nutch.crawl.InjectorJob.inject(InjectorJob.java:270)
> > at org.apache.nutch.crawl.InjectorJob.run(InjectorJob.java:293)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > at org.apache.nutch.crawl.InjectorJob.main(InjectorJob.java:302)
> >
> >
> > Attached is the java setting & properties for crosscheck.
> >
> > Regards,
> > Govind
> >
> > On Fri, Jan 12, 2018 at 5:12 PM, Sebastian Nagel <
> wastl.na...@googlemail.com
> > <mailto:wastl.na...@googlemail.com>> wrote:
> >
> > No. Please use Java 8. Nutch requires Java 8, see default.properties.
> >
> > The Dockerfile is outdated. If possible please open a Jira issue to
> update it.
> >
> > The error is really weired:
> > NutchJob extends org.apache.hadoop.mapreduce.Job, so there should
> be no
> > verification [1] error. I'm not able to reproduce it.
> >
> > Could you explain more in which environment Nutch is executed and how
> > you launch it? Ev. try with "java -Xverify:none" (for bin/nutch set
> > the environment variable NUTCH_OPTS=-Xverify:none) to see what
> happens.
> >
> > Thanks,
> > Sebastian
> >
> >
> > [1] https://static.rainfocus.com/oracle/oow16/sess/
> 1461563392709001ttyE/ppt/bcv_J1SF_2016.pdf
> > <https://static.rainfocus.com/oracle/oow16/sess/
> 1461563392709001ttyE/ppt/bcv_J1SF_2016.pdf>
> >
> >
> > On 01/12/2018 09:09 AM, govind nitk wrote:
> > > Hi Lewis,
> > >
> > > Tried with oracle java8, but issue persists and the error is same.
> > >
> > > Nutch might be compatiable with java8,
> > > but in docker file for hbase(nutch/docker/hbase/Dockerfile),
> java7 is used.
> > > So do we need to use java7 only ?
> > >
> > >
> > > Regards,
> > > GoViNd
> > >
> > >
> > >
> > > On Thu, Jan 11, 2018 at 8:22 PM, lewis john mcgibbney <
> lewi...@apache.org
> > <mailto:lewi...@apache.org>>
> > > wrote:
> > >
> > >> I unfortunately do not use the OpenJDK so i don't know if this is
> where
> > >> your issue stems from.
> > >> All of your config looks absolutely fine.
> > >> Lewis
> > >>
> > >> On Thu, Jan 11, 2018 at 8:26 AM, <user-digest-help@nutch.
> apache.org
> > <mailto:user-digest-h...@nutch.apache.org>>
> > >> wrote:
> > >>
> > >>>
> > >>> From: govind nitk <govind.n...@gmail.com <mailto:
> govind.n...@gmail.com>>
> > >>> To: user@nutch.apache.org <mailto:user@nutch.apache.org>
> > >>> Cc:
> > >>> Bcc:
> > >>> Date: Wed, 10 Jan 2018 14:06:53 +0530
> > >>> Subject: Re: Getting Error
> > >>> $java -version
> > >>> openjdk version "1.8.0_141"
> > >>> OpenJDK Runtime Environment (build 1.8.0_141-8u141-b15-3~14.04-
> b15)
> > >>> OpenJDK 64-Bit Server VM (build 25.141-b15, mixed mode)
> > >>>
> > >>>
> > >>> config edits:
> > >>>
> > >>> Gora properties:
> > >>> gora.datastore.default=org.apache.gora.mongodb.store.MongoStore
> > >>> gora.mongodb.override_hadoop_configuration=false
> > >>> gora.mongodb.mapping.file=/gora-mongodb-mapping.xml
> > >>> gora.mongodb.servers=localhost:27017
> > >>> gora.mongodb.db=crawler
> > >>> #gora.mongodb.login=login
> > >>> #gora.mongodb.secret=secret
> > >>>
> > >>>
> > >>> nutch-site.xml:
> > >>> <property>
> > >>> <name>storage.data.store.class</name>
> > >>> <value>org.apache.gora.mongodb.store.MongoStore</value>
> > >>> <description>Default class for storing data</description>
> > >>> </property>
> > >>>
> > >>>
> > >>> mongod running on default port: 27017.
> > >>>
> > >>>
> > >>> And before generating snapshot , uncommented the goa backend to
> use
> > >> mongo.
> > >>> as:
> > >>> <dependency org="org.apache.gora" name="gora-mongodb" rev="0.8"
> > >>> conf="*->default" />
> > >>>
> > >>>
> > >>> Am I missing anything else?
> > >>>
> > >>>
> > >>> regards,
> > >>> govind
> > >>>
> > >>>
> > >>>
> > >>> On Wed, Jan 10, 2018 at 12:31 PM, govind nitk <
> govind.n...@gmail.com
> > <mailto:govind.n...@gmail.com>>
> > >>> wrote:
> > >>>
> > >>>>
> > >>>> hi Lewis,
> > >>>>
> > >>>> uname -a: Linux data 4.4.0-108-generic #131~14.04.1-Ubuntu SMP
> Sun Jan
> > >> 7
> > >>>> 15:54:10 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
> > >>>>
> > >>>> On Tue, Jan 9, 2018 at 7:56 PM, lewis john mcgibbney <
> > >> lewi...@apache.org <mailto:lewi...@apache.org>
> > >>>>
> > >>>> wrote:
> > >>>>
> > >>>>> Hi govind,
> > >>>>> Very strange. Which operating system are you using?
> > >>>>> Lewis
> > >>>>>
> > >>>>> On Tue, Jan 9, 2018 at 5:15 AM, <user-digest-help@nutch.
> apache.org
> > <mailto:user-digest-h...@nutch.apache.org>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> From: govind nitk <govind.n...@gmail.com <mailto:
> govind.n...@gmail.com>>
> > >>>>>> To: user@nutch.apache.org <mailto:user@nutch.apache.org>
> > >>>>>> Cc:
> > >>>>>> Bcc:
> > >>>>>> Date: Tue, 9 Jan 2018 15:45:08 +0530
> > >>>>>> Subject: Getting Error
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> 1. running nutch compiled from branch 2.x. Build succeed.
> > >>>>>> 2. using mongo as db storage. changed the
> storage.data.store.class
> > >> to
> > >>>>> point
> > >>>>>> to mongo class.
> > >>>>>>
> > >>>>>>
> > >>>>>> Getting this error while running nutch inject
> /tmp/urls/seeds.txt ?
> > >>>>>>
> > >>>>>>
> > >>>>>> Error: A has occurred, please check your
> installation and
> > >>> try
> > >>>>>> again
> > >>>>>> Exception in thread "main" java.lang.VerifyError: Bad type on
> > >> operand
> > >>>>> stack
> > >>>>>> Exception Details:
> > >>>>>> Location:
> > >>>>>> org/apache/nutch/crawl/InjectorJob.run(Ljava/util/Map;)
> > >>>>> Ljava/util/Map;
> > >>>>>> @85: putfield
> > >>>>>> Reason:
> > >>>>>> Type 'org/apache/nutch/util/NutchJob' (current frame,
> stack[1])
> > >>> is
> > >>>>> not
> > >>>>>> assignable to 'org/apache/hadoop/mapreduce/Job'
> > >>>>>> Current Frame:
> > >>>>>> bci: @85
> > >>>>>> flags: { }
> > >>>>>> locals: { 'org/apache/nutch/crawl/InjectorJob',
> > >> 'java/util/Map',
> > >>>>>> 'org/apache/hadoop/fs/Path', 'java/lang/Object' }
> > >>>>>> stack: { 'org/apache/nutch/crawl/InjectorJob',
> > >>>>>> 'org/apache/nutch/util/NutchJob' }
> > >>>>>> Bytecode:
> > >>>>>> 0x0000000: 2ab6 0004 1205 b800 06b6 0007 2b12 09b9
> > >>>>>> 0x0000010: 000a 0200 4e2d c100 0b99 000b 2dc0 000b
> > >>>>>> 0x0000020: 4da7 000f bb00 0b59 2db6 000c b700 0d4d
> > >>>>>> 0x0000030: 2a04 b500 0e2a 03b5 000f 2a2a b600 04bb
> > >>>>>> 0x0000040: 0010 59b7 0011 1212 b600 132c b600 14b6
> > >>>>>> 0x0000050: 0015 b800 16b5 0017 2ab4 0017 2cb8 0018
> > >>>>>> 0x0000060: 2ab4 0017 1219 b600 1a2a b400 1712 1bb6
> > >>>>>> 0x0000070: 001c 2ab4 0017 121d b600 1e2a b400 1712
> > >>>>>> 0x0000080: 1fb6 0020 2ab4 0017 b600 2112 1b12 1db8
> > >>>>>> 0x0000090: 0022 3a04 2ab4 0017 1904 04b8 0023 2ab4
> > >>>>>> 0x00000a0: 0017 b600 21b8 0024 3a05 b200 25bb 0010
> > >>>>>> 0x00000b0: 59b7 0011 1226 b600 1319 05b6 0014 1227
> > >>>>>> 0x00000c0: b600 13b6 0015 b900 2802 002a b400 1712
> > >>>>>> 0x00000d0: 29b6 002a 2ab4 0017 03b6 002b 2ab4 0017
> > >>>>>> 0x00000e0: 04b6 002c 5701 2ab4 0017 2ab4 002d b800
> > >>>>>> 0x00000f0: 2e2a b400 17b6 002f 1230 1231 b600 32b9
> > >>>>>> 0x0000100: 0033 0100 3706 2ab4 0017 b600 2f12 3012
> > >>>>>> 0x0000110: 34b6 0032 b900 3301 0037 08b2 0025 bb00
> > >>>>>> 0x0000120: 1059 b700 1112 35b6 0013 1608 b600 36b6
> > >>>>>> 0x0000130: 0015 b900 2802 00b2 0025 bb00 1059 b700
> > >>>>>> 0x0000140: 1112 37b6 0013 1606 b600 36b6 0015 b900
> > >>>>>> 0x0000150: 2802 002a b400 2db0
> > >>>>>> Stackmap Table:
> > >>>>>> append_frame(@36,Top,Object[#148])
> > >>>>>> full_frame(@48,{Object[#149],Object[#150],Object[#151],
> > >>>>>> Object[#148]},{})
> > >>>>>>
> > >>>>>> at java.lang.Class.getDeclaredMethods0(Native Method)
> > >>>>>> at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
> > >>>>>> at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
> > >>>>>> at java.lang.Class.getMethod0(Class.java:3018)
> > >>>>>> at java.lang.Class.getMethod(Class.java:1784)
> > >>>>>> at sun.launcher.LauncherHelper.validateMainClass(
> LauncherHelper
> > >>>>> .java:544)
> > >>>>>> at sun.launcher.LauncherHelper.checkAndLoadMain(
> LauncherHelper.
> > >>>>> java:526)
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> Regards,
> > >>>>>> govind
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> http://home.apache.org/~lewismc/ <http://home.apache.org/~
> lewismc/>
> > >>>>> @hectorMcSpector
> > >>>>> http://www.linkedin.com/in/lmcgibbney <
> http://www.linkedin.com/in/lmcgibbney>
> > >>>>>
> > >>>>
> > >>>>
> > >>>
> > >>>
> > >>
> > >>
> > >> --
> > >> http://home.apache.org/~lewismc/ <http://home.apache.org/~
> lewismc/>
> > >> http://people.apache.org/keys/committer/lewismc <
> http://people.apache.org/keys/committer/lewismc>
> > >>
> > >
> >
> >
>
>
