I'm not sure if not the key problem is the real source of the
problem. In general I suggest using nutch 0.8 that fix a set of issues.
E.g. writes syncs to the files and creates checksums since people not
problems with hdd's.
At least a nutch map file require to have ordered keys and in your
case you have one key two times. Do you may create your segment file
by your self?
I suggest testing nutch 0.8.
Stefan
Am 16.02.2006 um 15:31 schrieb keren nutch:
Hi,
We are running Nutch 0.7.1 on Mandrake (Linux). We have 2G memory
and set NUTCH_HEAPSIZE=2000. When fetching, we got a severe error:
out of memory, see below.
060215 224055 fetching http://www.edu.pe.ca/vrcs/grassroots/1998/
chijnew/shipbu~1.
htm
060215 224057 SEVERE error writing output:java.lang.OutOfMemoryError
java.lang.OutOfMemoryError
060215 224057 - protocol redirect to http://www.amazon.ca/exec/
obidos/tg/detail/-
/music/B00006NSQ9/customer-reviews
060215 224057 fetching http://www.amazon.ca/exec/obidos/tg/
detail/-/music/B00006NS
Q9/customer-reviews
060215 224057 SEVERE error writing output:java.io.IOException: key
out of order: 3
91009 after 391009
java.io.IOException: key out of order: 391009 after 391009
at org.apache.nutch.io.MapFile$Writer.checkKey
(MapFile.java:134)
at org.apache.nutch.io.MapFile$Writer.append(MapFile.java:
120)
at org.apache.nutch.io.ArrayFile$Writer.append
(ArrayFile.java:39)
at org.apache.nutch.fetcher.Fetcher
$FetcherThread.outputPage(Fetcher.java:
278)
at org.apache.nutch.fetcher.Fetcher
$FetcherThread.handleFetch(Fetcher.java
:261)
at org.apache.nutch.fetcher.Fetcher$FetcherThread.run
(Fetcher.java:148)
Best regards,
Keren
---------------------------------
Find your next car at Yahoo! Canada Autos
