Hi Stefan, Thank you for reply. I didn't create segment file by myself. It was created via "nutch generate". Please let me know what you mean "yuo have one key two times". Best regards, Keren
Stefan Groschupf <[EMAIL PROTECTED]> wrote: I'm not sure if not the key problem is the real source of the problem. In general I suggest using nutch 0.8 that fix a set of issues. E.g. writes syncs to the files and creates checksums since people not problems with hdd's. At least a nutch map file require to have ordered keys and in your case you have one key two times. Do you may create your segment file by your self? I suggest testing nutch 0.8. Stefan Am 16.02.2006 um 15:31 schrieb keren nutch: > Hi, > > We are running Nutch 0.7.1 on Mandrake (Linux). We have 2G memory > and set NUTCH_HEAPSIZE=2000. When fetching, we got a severe error: > out of memory, see below. > > 060215 224055 fetching http://www.edu.pe.ca/vrcs/grassroots/1998/ > chijnew/shipbu~1. > htm > 060215 224057 SEVERE error writing output:java.lang.OutOfMemoryError > java.lang.OutOfMemoryError > 060215 224057 - protocol redirect to http://www.amazon.ca/exec/ > obidos/tg/detail/- > /music/B00006NSQ9/customer-reviews > 060215 224057 fetching http://www.amazon.ca/exec/obidos/tg/ > detail/-/music/B00006NS > Q9/customer-reviews > 060215 224057 SEVERE error writing output:java.io.IOException: key > out of order: 3 > 91009 after 391009 > java.io.IOException: key out of order: 391009 after 391009 > at org.apache.nutch.io.MapFile$Writer.checkKey > (MapFile.java:134) > at org.apache.nutch.io.MapFile$Writer.append(MapFile.java: > 120) > at org.apache.nutch.io.ArrayFile$Writer.append > (ArrayFile.java:39) > at org.apache.nutch.fetcher.Fetcher > $FetcherThread.outputPage(Fetcher.java: > 278) > at org.apache.nutch.fetcher.Fetcher > $FetcherThread.handleFetch(Fetcher.java > :261) > at org.apache.nutch.fetcher.Fetcher$FetcherThread.run > (Fetcher.java:148) > > Best regards, > Keren > > > > --------------------------------- > Find your next car at Yahoo! Canada Autos __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com
