Re: build with ant
I've not built this particular package myself, but generally, one doesn't need to build it -- you can most likely do what you need to using the configuration xml. If not, I suggest you respond with what you need to do so others may answer. -- H On 7 January 2015 at 19:09, Shadi Saleh propat...@gmail.com wrote: Thank you so much for quick reply, I still have the problem, here is the full output: uildfile: /root/svn_nutch/nutch/build.xml Trying to override old definition of task javac [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found. ivy-probe-antlib: ivy-download: [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found. ivy-download-unchecked: ivy-init-antlib: ivy-init: init: [mkdir] Created dir: /root/svn_nutch/nutch/build/test/lib clean-lib: [delete] Deleting directory /root/svn_nutch/nutch/build/test/lib resolve-default: [ivy:resolve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file = /root/svn_nutch/nutch/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] WARNINGS [ivy:resolve] module not found: org.restlet.jse#org.restlet;2.0.12 [ivy:resolve] local: tried [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet/2.0.12/ivys/ivy.xml [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet/2.0.12/jars/org.restlet.jar [ivy:resolve] maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.jar [ivy:resolve] apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.jar [ivy:resolve] sonatype: tried [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.jar [ivy:resolve] module not found: org.restlet.jse#org.restlet.ext.jackson;2.0.13 [ivy:resolve] local: tried [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet.ext.jackson/2.0.13/ivys/ivy.xml [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet.ext.jackson/2.0.13/jars/org.restlet.ext.jackson.jar [ivy:resolve] maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.jar [ivy:resolve] apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.jar [ivy:resolve] sonatype: tried [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.jar [ivy:resolve] :: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :: [ivy:resolve] :: org.restlet.jse#org.restlet;2.0.12: not found [ivy:resolve] :: org.restlet.jse#org.restlet.ext.jackson;2.0.13: not found [ivy:resolve] :: [ivy:resolve] [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS BUILD FAILED
Re: Problems with DomainStatistics
Hi Markus, On Wed, Jan 7, 2015 at 7:42 PM, user-digest-h...@nutch.apache.org wrote: Hi - it is a strange piece indeed. You cannot just tell it where the crawldb is, you need to tell it where the directory is, so specifying current is ok, but not part-* Thanks very much. I'll cook a patch up and get it to Jira. Lewis
build with ant
Dear all, I am trying to build nutch ver 2.2.1 with ant, but I got this error: [ivy:resolve] :: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :: [ivy:resolve] :: org.restlet.jse#org.restlet;2.0.5: not found [ivy:resolve] :: org.restlet.jse#org.restlet.ext.jackson;2.0.5: not found Build failed. Any idea please? Best -- *Shadi Saleh*
Re: build with ant
On 7 January 2015 at 18:56, Shadi Saleh propat...@gmail.com wrote: [ivy:resolve] :: org.restlet.jse#org.restlet;2.0.5: not found dependency org=org.restlet.jee name=org.restlet rev=2.0.12 / [ivy:resolve] :: org.restlet.jse#org.restlet.ext.jackson;2.0.5: not found dependency org=org.restlet.jee name=org.restlet.ext.jackson rev=2.0.13 / Just update the versions of the dependencies with those and it will compile. If you have further problems, do let the list know... -- H -- OpenPGP: https://hasan.d8u.us/gpg.key Sent from my mobile device Envoyé de mon portable
Re: build with ant
Thank you so much for quick reply, I still have the problem, here is the full output: uildfile: /root/svn_nutch/nutch/build.xml Trying to override old definition of task javac [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found. ivy-probe-antlib: ivy-download: [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found. ivy-download-unchecked: ivy-init-antlib: ivy-init: init: [mkdir] Created dir: /root/svn_nutch/nutch/build/test/lib clean-lib: [delete] Deleting directory /root/svn_nutch/nutch/build/test/lib resolve-default: [ivy:resolve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file = /root/svn_nutch/nutch/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] WARNINGS [ivy:resolve] module not found: org.restlet.jse#org.restlet;2.0.12 [ivy:resolve] local: tried [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet/2.0.12/ivys/ivy.xml [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet/2.0.12/jars/org.restlet.jar [ivy:resolve] maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.jar [ivy:resolve] apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.jar [ivy:resolve] sonatype: tried [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.jar [ivy:resolve] module not found: org.restlet.jse#org.restlet.ext.jackson;2.0.13 [ivy:resolve] local: tried [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet.ext.jackson/2.0.13/ivys/ivy.xml [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet.ext.jackson/2.0.13/jars/org.restlet.ext.jackson.jar [ivy:resolve] maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.jar [ivy:resolve] apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.jar [ivy:resolve] sonatype: tried [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.jar [ivy:resolve] :: [ivy:resolve] :: UNRESOLVED DEPENDENCIES :: [ivy:resolve] :: [ivy:resolve] :: org.restlet.jse#org.restlet;2.0.12: not found [ivy:resolve] :: org.restlet.jse#org.restlet.ext.jackson;2.0.13: not found [ivy:resolve] :: [ivy:resolve] [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS BUILD FAILED /root/svn_nutch/nutch/build.xml:471: impossible to resolve dependencies: resolve failed - see output for details Total time: 5 seconds Best On Thu, Jan 8, 2015 at 4:03 AM, Hasan Diwan hasan.di...@gmail.com wrote: On 7 January 2015 at 18:56, Shadi Saleh propat...@gmail.com wrote: [ivy:resolve] :: org.restlet.jse#org.restlet;2.0.5: not found dependency org=org.restlet.jee name=org.restlet rev=2.0.12 / [ivy:resolve] ::
Re: build with ant
I just removed the previous two lines and it works, I do not see them now on nutch git repository that's why I removed them, hope it will not affect on the future. best On Thu, Jan 8, 2015 at 4:28 AM, Hasan Diwan hasan.di...@gmail.com wrote: I've not built this particular package myself, but generally, one doesn't need to build it -- you can most likely do what you need to using the configuration xml. If not, I suggest you respond with what you need to do so others may answer. -- H On 7 January 2015 at 19:09, Shadi Saleh propat...@gmail.com wrote: Thank you so much for quick reply, I still have the problem, here is the full output: uildfile: /root/svn_nutch/nutch/build.xml Trying to override old definition of task javac [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found. ivy-probe-antlib: ivy-download: [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found. ivy-download-unchecked: ivy-init-antlib: ivy-init: init: [mkdir] Created dir: /root/svn_nutch/nutch/build/test/lib clean-lib: [delete] Deleting directory /root/svn_nutch/nutch/build/test/lib resolve-default: [ivy:resolve] :: Ivy 2.2.0 - 20100923230623 :: http://ant.apache.org/ivy/ :: [ivy:resolve] :: loading settings :: file = /root/svn_nutch/nutch/ivy/ivysettings.xml [ivy:resolve] [ivy:resolve] :: problems summary :: [ivy:resolve] WARNINGS [ivy:resolve] module not found: org.restlet.jse#org.restlet;2.0.12 [ivy:resolve] local: tried [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet/2.0.12/ivys/ivy.xml [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet/2.0.12/jars/org.restlet.jar [ivy:resolve] maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.jar [ivy:resolve] apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.jar [ivy:resolve] sonatype: tried [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet;2.0.12!org.restlet.jar: [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet/2.0.12/org.restlet-2.0.12.jar [ivy:resolve] module not found: org.restlet.jse#org.restlet.ext.jackson;2.0.13 [ivy:resolve] local: tried [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet.ext.jackson/2.0.13/ivys/ivy.xml [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] /root/.ivy2/local/org.restlet.jse/org.restlet.ext.jackson/2.0.13/jars/org.restlet.ext.jackson.jar [ivy:resolve] maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] http://repo1.maven.org/maven2/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.jar [ivy:resolve] apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.jar [ivy:resolve] sonatype: tried [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.pom [ivy:resolve] -- artifact org.restlet.jse#org.restlet.ext.jackson;2.0.13!org.restlet.ext.jackson.jar: [ivy:resolve] http://oss.sonatype.org/content/repositories/releases/org/restlet/jse/org.restlet.ext.jackson/2.0.13/org.restlet.ext.jackson-2.0.13.jar [ivy:resolve] :: [ivy:resolve] ::
Re: build with ant
On 7 January 2015 at 19:40, Shadi Saleh propat...@gmail.com wrote: I just removed the previous two lines and it works, I do not see them now on nutch git repository that's why I removed them, hope it will not affect on the future. Good to hear it was a simple fix. -- H -- OpenPGP: https://hasan.d8u.us/gpg.key Sent from my mobile device Envoyé de mon portable
RE: Problems with DomainStatistics
Hi - it is a strange piece indeed. You cannot just tell it where the crawldb is, you need to tell it where the directory is, so specifying current is ok, but not part-* M -Original message- From:Lewis John Mcgibbney lewis.mcgibb...@gmail.com Sent: Wednesday 7th January 2015 19:48 To: user@nutch.apache.org Subject: Problems with DomainStatistics Hi Folks, Does anyone else have problems with the DomainStaticstics [0] tool? I use it as follows ./bin/nutch domainstats /usr/local/.../crawldb/old/part-0/ output tld Although it is generated, nothing is written to the output directory ./bin/nutch domainstats /usr/local/.../crawldb/current/part-0/ output tld Although it is generated, nothing is written to the output directory The command line is less than helpful however I;ve even looked into the code and can quite see where I'm missing things. Any help is appreciated. Thanks Lewis [0] https://github.com/apache/nutch/blob/trunk/src/java/org/apache/nutch/util/domain/DomainStatistics.java -- *Lewis*
Re: Potential Bug in 2.X HostDbUpdateReducer
Hi Renato, On Thu, Dec 11, 2014 at 5:52 AM, user-digest-h...@nutch.apache.org wrote: From quickly checking out the code (Host.java + HostDB + HostDBUpdateReducer) it would seems like there is a bug exactly where you pointed. LOGGED!!! https://issues.apache.org/jira/browse/NUTCH-1907 WOW am I behind in email due to the holidays :)
Re: Help regarding headings plugin
Hi Krishna, On Thu, Dec 11, 2014 at 5:52 AM, user-digest-h...@nutch.apache.org wrote: When I dump data from segments, I am getting entire html data. Shouldnot it be just headings read from crawling. Why am I getting entire data? Please help me. Thanks in advance. No this is not the case at all! The headings plugin merely enables you to make the headings explicit within your manifestation of the webpage itself. Please look into the Parse Metadata... or possibly Content Metadata I cannot remeber. Your headings will all be in there. hth Lewis
Problems with DomainStatistics
Hi Folks, Does anyone else have problems with the DomainStaticstics [0] tool? I use it as follows ./bin/nutch domainstats /usr/local/.../crawldb/old/part-0/ output tld Although it is generated, nothing is written to the output directory ./bin/nutch domainstats /usr/local/.../crawldb/current/part-0/ output tld Although it is generated, nothing is written to the output directory The command line is less than helpful however I;ve even looked into the code and can quite see where I'm missing things. Any help is appreciated. Thanks Lewis [0] https://github.com/apache/nutch/blob/trunk/src/java/org/apache/nutch/util/domain/DomainStatistics.java -- *Lewis*
custom plugin with nutch job jar
Hi, I have a custom plugin for nutch 1.9, which runs well when I run the crawl script in local mode. But, when I run the crawl script in deploy mode, the custom plugin dint work. I believe this is due to the fact that my custom plugin is not incorporated in nutch job jar. Can anyone please guide me how can i incorporate my plugin into nutch job jar. I mean what changes I have to make in my nutch build.xml file to achieve that. Thanks
