Re: Does Nutch work with Hadoop Versions greater than 3.1.3?

2022-06-13 Thread Markus Jelsma
To add to Sebastian, it runs on Hadoop 3.3.x very good as well. Actually, i never had any Hadoop version that could not run Nutch out of the box and without issues. Op ma 13 jun. 2022 om 11:54 schreef Sebastian Nagel : > Hi Michael, > > Nutch (1.18, and trunk/master) should work together with mor

Re: Does Nutch work with Hadoop Versions greater than 3.1.3?

2022-06-13 Thread Sebastian Nagel
Hi Michael, Nutch (1.18, and trunk/master) should work together with more recent Hadoop versions. At Common Crawl we use a modified Nutch version based on the recent trunk running on Hadoop 3.2.2 (soon 3.2.3) and Java 11, even on a mixed Hadoop cluster with x64 and arm64 AWS EC2 instances. But I

Does Nutch work with Hadoop Versions greater than 3.1.3?

2022-06-12 Thread Michael Coffey
Do current 1.x versions of Nutch (1.18, and trunk/master) work with versions of Hadoop greater than 3.1.3? I ask because Hadoop 3.1.3 is from October 2019, and there are many newer versions available. For example, 3.1.4 came out in 2020, and there are 3.2.x and 3.3.x versions that came out this