Hi Arkadi, thanks for the great reference. I guess I can't fork child JVMs in my use case though. Fortunately I don't have to crawl more than 100k sites and also don't need a lot of multiple threads. I'm going to take a look at your referenced class, thank you!
Regards, Hannes On Mon, May 24, 2010 at 1:07 AM, <[email protected]> wrote: > Hi Hannes, > > This is done in Arch. See the au.csiro.cass.arch.utils.Starter class and > its use from other classes. It was not straightforward because the used RAM > tends to grow with iterations. To get around this, I had to fork child JVMs. > > You can find Arch here: > > http://www.atnf.csiro.au/computing/software/arch/ > > Regards, > > Arkadi > > > -----Original Message----- > > From: Hannes Carl Meyer [mailto:[email protected]] > > Sent: Friday, May 21, 2010 7:33 PM > > To: [email protected] > > Subject: Running Nutch in a single VM > > > > Hi, > > > > is it possible to run nutch in a single virtual machine for intranet > > crawling? Even inside a Java Application Server? > > > > Normally I'm using custom Nutch crawl scripts and start from the OS > > command > > line by cron. In a new project it is required to use a running Virtual > > Machine for deloyment and invocation of crawler tasks. > > > > Does anybody has experiences in deploying Nutch in such a scenario? > > > > Kind Regards > > > > Hannes > > > > -- > -- https://www.xing.com/profile/HannesCarl_Meyer http://de.linkedin.com/in/hannescarlmeyer http://twitter.com/hannescarlmeyer
