Hi Hannes, You are welcome. Forking is optional, based on a config parameter. The interface can be used within a single JVM as well. I use this for debugging and small crawls.
Regards, Arkadi > -----Original Message----- > From: Hannes Carl Meyer [mailto:[email protected]] > Sent: Tuesday, May 25, 2010 5:01 PM > To: Kosmynin, Arkadi (CASS, Marsfield); [email protected] > Subject: Re: Running Nutch in a single VM > > Hi Arkadi, > > thanks for the great reference. I guess I can't fork child JVMs in my > use > case though. Fortunately I don't have to crawl more than 100k sites and > also > don't need a lot of multiple threads. > I'm going to take a look at your referenced class, thank you! > > Regards, > > Hannes > > On Mon, May 24, 2010 at 1:07 AM, <[email protected]> wrote: > > > Hi Hannes, > > > > This is done in Arch. See the au.csiro.cass.arch.utils.Starter class > and > > its use from other classes. It was not straightforward because the > used RAM > > tends to grow with iterations. To get around this, I had to fork > child JVMs. > > > > You can find Arch here: > > > > http://www.atnf.csiro.au/computing/software/arch/ > > > > Regards, > > > > Arkadi > > > > > -----Original Message----- > > > From: Hannes Carl Meyer [mailto:[email protected]] > > > Sent: Friday, May 21, 2010 7:33 PM > > > To: [email protected] > > > Subject: Running Nutch in a single VM > > > > > > Hi, > > > > > > is it possible to run nutch in a single virtual machine for > intranet > > > crawling? Even inside a Java Application Server? > > > > > > Normally I'm using custom Nutch crawl scripts and start from the OS > > > command > > > line by cron. In a new project it is required to use a running > Virtual > > > Machine for deloyment and invocation of crawler tasks. > > > > > > Does anybody has experiences in deploying Nutch in such a scenario? > > > > > > Kind Regards > > > > > > Hannes > > > > > > -- > > > > > > -- > > https://www.xing.com/profile/HannesCarl_Meyer > http://de.linkedin.com/in/hannescarlmeyer > http://twitter.com/hannescarlmeyer
