Oh yes, please up your memory to like at least 2Gb.. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Chief Architect Instrument Software and Science Data Systems Section (398) NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 168-519, Mailstop: 168-527 Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Associate Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-----Original Message----- From: Shuo Li <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Friday, February 13, 2015 at 10:38 AM To: "[email protected]" <[email protected]> Cc: Mo Omer <[email protected]> Subject: Re: Vagrant Crushed When using Nutch-Selenium >Hey Mo and Prof Mattmann, > > >I will try to crawl the 3 websites in the homework tonight (NASA AMD, NSF >ACADIS and NSIDC Arctic Data Explorer). I will let you know what's going >on. > > >Is memory an issue? My vagrant only has 512MB of memory. > > >Regards, >Shuo Li > > >On Fri, Feb 13, 2015 at 10:25 AM, Mattmann, Chris A (3980) ><[email protected]> wrote: > >Hi Shuo, > >Thanks for your email. I wonder if using selenium grid would >help? > >Please see this plugin: > >https://github.com/momer/nutch-selenium-grid-plugin > > >I’m CC’ing Mo the author of the plugin to see if he experienced >this while running the original selenium plugin - Mo did using >selenium grid help the issue that Shuo is experiencing below? > >Mo: are you cool with portion the grid plugin, or if Lewis or >I do it to trunk (with full credit to you of course?) > >Cheers, >Chris > > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Chief Architect >Instrument Software and Science Data Systems Section (398) >NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA >Office: 168-519, Mailstop: 168-527 >Email: [email protected] >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Adjunct Associate Professor, Computer Science Department >University of Southern California, Los Angeles, CA 90089 USA >++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > >-----Original Message----- >From: Shuo Li <[email protected]> >Reply-To: "[email protected]" <[email protected]> >Date: Friday, February 13, 2015 at 10:12 AM >To: "[email protected]" <[email protected]> >Subject: Vagrant Crushed When using Nutch-Selenium > >>Hey guys, >> >> >>I'm trying to use Nutch-Selenium to crawl >>nutch.apache.org <http://nutch.apache.org> <http://nutch.apache.org>. >>However, my vagrant seems >>crushed after a few minutes. I forced it to shut down and it turns out it >>only crawled 59 websites. My nutch version is 1.10 and my OS is Ubuntu >>Trusty, 14.04. >> >> >>Is there anything I can provide to you guys? Or is there anybody have the >>same issue? Or 59 websites is the complete crawling? >> >> >>Any suggestion would be appreciated. >> >> >>Regards, >>Shuo Li >> > > > > > > > >
