PS. Ulimit -a
core file size (blocks, -c) 0 data seg size (kbytes, -d) unlimited max nice (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 138239 max locked memory (kbytes, -l) 32 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 max rt priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 138239 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited -----Original Message----- From: Sean Dean [mailto:[EMAIL PROTECTED] Sent: Friday, January 19, 2007 8:25 PM To: [email protected] Subject: Re: java.lang.OutOfMemoryError - trunk What OS are you using with Nutch, and what version of JVM? If its Linux, paste the output of "ulimit -a", if its BSD use "limits". You can also try inserting "-Xms2000m" before you set the max heap, so it would look like "-Xms2000m -Xmx2000m". I'm also assuming you have at least 2g free of RAM, or even more? ----- Original Message ---- From: Gal Nitzan <[EMAIL PROTECTED]> To: [email protected] Sent: Friday, January 19, 2007 10:57:01 AM Subject: java.lang.OutOfMemoryError - trunk Thanks Sean, I get out of memory errors. I have set max heap for both nutch and hadoop 2000mb each but it doesn't seem to affect anything. The out of memory happenes immediately after start of a task. Any idea? java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2786) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94) at java.io.DataOutputStream.write(DataOutputStream.java:90) at org.apache.hadoop.io.Text.writeString(Text.java:399) at org.apache.nutch.parse.Outlink.write(Outlink.java:52) at org.apache.nutch.parse.ParseData.write(ParseData.java:163) at org.apache.nutch.parse.ParseImpl.write(ParseImpl.java:55) at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:323) at org.apache.nutch.parse.ParseSegment.map(ParseSegment.java:96) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:183) at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1367) ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
