Hi Andrzej,
when I ps nutch-java process (while fetching) I get:
25276 pts/0 Sl+ 0:07 /usr/lib/jvm/java-6-sun/bin/java -Xmx1000m
-Dhadoop.log.dir=/home/nutch/test/trunk/logs
-Dhadoop.log.file=hadoop.log -Djava.library.path=/home/nutch...
so this should mean that I have dedicated 1 Gb to this java process
(this are default settings I didn't changed it), right?
I am running nutch-1.0-dev (from trunk 2007-08-08) with sun jdk6 on
Ubuntu Feisty (dual-core AMD64, 2Gb RAM). Should I try with nutch-0.9?
Thanks,
Tomislav
On Tue, 2007-09-11 at 11:32 +0200, Andrzej Bialecki wrote:
> Tomislav Poljak wrote:
> > Hi Andrtzej,
> > I am running fetcher in non-parsing mode, I have this in nutch-site.xml:
> >
> > <property>
> > <name>fetcher.parse</name>
> > <value>false</value>
> > <description>If true, fetcher will parse content.</description>
> > </property>
> >
> > Maybe I didn't post a question correctly. I get a couple of fetcher
> > threads failing with java.lang.OutOfMemoryError like this (from
> > hadoop.log):
> >
> > 2007-09-09 01:07:24,150 INFO fetcher.Fetcher - fetching
> > http://scholar.google.com/intl/en/scholar/libraries.html
> > 2007-09-09 01:07:27,084 INFO fetcher.Fetcher - fetch of
>
> Sure looks strange to me. And you are sure that your Java heap size is
> set to the right amount? You didn't lose any 'm or 'g suffix in -Xmx ?
>
> ;) just checking ...
>
>
