Hello, we are running Nutch 1.10 on Hadoop 2.7 for quite some time now. But since Hadoop kept its binary compatibility, even older Nutch' should work just as fine.
Markus -----Original message----- > From:Imtiaz Shakil Siddique <[email protected]> > Sent: Monday 14th September 2015 18:08 > To: [email protected] > Subject: Re: Compatible Hadoop version with Nutch 1.10 > > Hi, > > Nutch 1.10 is already released. Are you mentioning about any newer nutch > release(nutch 1.11). > > Thanks for the help Sir. > Imtiaz Shakil Siddique > On Sep 14, 2015 8:57 PM, "Sebastian Nagel" <[email protected]> > wrote: > > > Hi, > > > > Nutch 1.10 is supposed to run with Hadoop 1.2.0. > > 1.10 (to be released soon) will run with 2.4.0, > > and probably also with newer Hadoop versions. > > > > If you need Nutch with a recent Hadoop version > > right now, you could build it by yourself from trunk. > > > > Cheers, > > Sebastian > > > > 2015-09-11 16:14 GMT+02:00 Imtiaz Shakil Siddique <[email protected] > > >: > > > > > Hi, > > > > > > I was trying to test nutch 1.10 with Hadoop-2.7.1 but during the inject > > > phase I came across with some errors. > > > > > > > I was executing $NUTCH_HOME/runtime/deploy/bin/crawl -i > > /home/nutch/urls > > > > /home/nutch/crawl/ 1 > > > > > > 15/09/10 19:41:17 ERROR crawl.Injector: Injector: > > > > java.lang.IllegalArgumentException: Wrong FS: > > > > hdfs://localhost:9000/user/root/inject-temp-875522145, expected: > > file:/// > > > > at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:646) > > > > at > > > > > > > > > org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:82) > > > > at > > > > > > > > > org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:601) > > > > at > > > > > > > > > org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:819) > > > > at > > > > > > > > > org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:596) > > > > at org.apache.hadoop.fs.FileSystem.isDirectory(FileSystem.java:1437) > > > > at > > > > > > > > > org.apache.hadoop.fs.ChecksumFileSystem.rename(ChecksumFileSystem.java:506) > > > > at org.apache.nutch.crawl.CrawlDb.install(CrawlDb.java:168) > > > > at org.apache.nutch.crawl.Injector.inject(Injector.java:356) > > > > at org.apache.nutch.crawl.Injector.run(Injector.java:379) > > > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > > > > at org.apache.nutch.crawl.Injector.main(Injector.java:369) > > > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > > > at > > > > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > > > at > > > > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > at java.lang.reflect.Method.invoke(Method.java:497) > > > > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > > > > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) > > > > > > > > > my conf file (hadoop-2.7.1) is given below > > > -------- core-site.xml -------- > > > > > > <property> > > > <name>fs.default.name</name> > > > <value>hdfs://localhost:9000</value> > > > </property> > > > <property> > > > <name>hadoop.tmp.dir</name> > > > <value>/home/nutch/hadoopData/hadoopTmpDir</value> > > > </property> > > > > > > -------- hdfs-site.xml -------- > > > <property> > > > <name>dfs.namenode.name.dir</name> > > > <value>/home/nutch/hadoopData/nameNodeData</value> > > > </property> > > > > > > <property> > > > <name>dfs.datanode.data.dir</name> > > > <value>/home/nutch/hadoopData/dataNodeData</value> > > > </property> > > > > > > <property> > > > <name>dfs.replication</name> > > > <value>1</value> > > > </property> > > > -------- mapred-site.xml -------- > > > <property> > > > <name>mapred.job.tracker</name> > > > <value>localhost:9001</value> > > > </property> > > > <property> > > > > > > <name>mapred.system.dir</name> > > > > > > <value>/home/nutch/hadoopData/mapredJobTrackerData</value> > > > </property> > > > <property> > > > > > > <name>mapred.local.dir</name> > > > > > > <value>/home/nutch/hadoopData/mapredTaskTrackerData</value> > > > > > > </property> > > > > > > But the same command works successfully when I use Hadoop-1.2.1. > > > What is the preferred version of Hadoop that we should use with Apache > > > Nutch 1.10 > > > > > > > > > Thank you so much. > > > Imtiaz Shakil Siddique > > > > > >
