> If I remember correct;ly one could not run multiple Nutch instances from the > same user > because all those instances would write to the same TMP file or something > like this...
Just make sure that every instance has it's own temp folder configured by setting -Dhadoop.tmp.dir=... That's required for local mode (which was thought for testing). In single-node pseudo-distributed mode the temp folders are automatically configured per job. > try and use Hadoop 3, say like till the end of next week and report back. Then you might try pseudo-distributed mode: http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/SingleCluster.html Best, Sebastian On 03/14/2018 12:47 PM, BlackIce wrote: > I'm redoing everything on my end.... pretty much from scratch.... > > The question I had when I woke up this morning... Do I really need to > configure VM's in order to run > multiple nodes? Or does running multiple, Nutch-Solr nodes, sufice if they > are under their own user > space? If I remember correct;ly one could not run multiple Nutch instances > from the same user > because all those instances would write to the same TMP file or something > like this... But this > shouldn't be the case when each instance is run from its own user account. > > With that sayd... If its really as easy as running the instances from their > own account... I could > try and use Hadoop 3, say like till the end of next week and report back. > > Greetz > > On Wed, Mar 14, 2018 at 12:01 AM, Lewis John McGibbney <lewi...@apache.org > <mailto:lewi...@apache.org>> wrote: > > Hi Seb, > > On 2018/03/12 11:00:52, Sebastian Nagel <wastl.na...@googlemail.com > <mailto:wastl.na...@googlemail.com>> wrote: > > Hi, > > > > > seeing as we have just merged in the 'new' MR patch > > > > yep, but there's still something to do (NUTCH-2517, > > ACK, this needs more testing. > > > NUTCH-2518). > > I honestly didn't see this come through but yes you are right. > > > Better to address this before any upgrade of the Hadoop version. > > ACK > > > But since there seem to be no breaking MapReduce API changes > > http://hadoop.apache.org/docs/r3.0.0/index.html > <http://hadoop.apache.org/docs/r3.0.0/index.html> > > I would even expect that the Nutch job jar (built for 2.7) > > will run on Hadoop 3.0, or does it not? > > > > I have absolutely no idea. I've certainly not had an opportunity to run > on H v3 cluster. > >
