Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Eric Osgood
During a crawl of about 3.8M tlds to a depth of 2, when I try to index the segments, I get the following error: java.lang.StackOverflowError at java.util.regex.Pattern$Loop.match(Pattern.java:4295) Any help with this error would be much appreciated, I have encountered this before.

Re: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Godmar Back
Very intriguing, considering that we teach our students to avoid recursion where possible for this very reason. Googling reveals http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4675952 and http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5050507 so you could try increasing the Java stack

Re: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Eric Osgood
bin/nutch -Xss1024k index crawl1/indexes crawl1/crawldb crawl1/linkdb crawl/segments/* Exception in thread main java.lang.NoClassDefFoundError: index Caused by: java.lang.ClassNotFoundException: index at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at

Re: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Godmar Back
On Mon, Jan 11, 2010 at 11:50 AM, Eric Osgood e...@lakemeadonline.com wrote: Do you have to set the -Xss flag somewhere else? Yes, in bin/nutch - looking for where it sets -Xmx - Godmar

RE: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Fuad Efendi
Also, put it in Hadoop settings for tasks... http://www.tokenizer.ca/ -Original Message- From: Godmar Back [mailto:god...@gmail.com] Sent: January-11-10 11:53 AM To: nutch-user@lucene.apache.org Subject: Re: Help Needed with Error: java.lang.StackOverflowError On Mon, Jan 11

Re: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Eric Osgood
-10 11:53 AM To: nutch-user@lucene.apache.org Subject: Re: Help Needed with Error: java.lang.StackOverflowError On Mon, Jan 11, 2010 at 11:50 AM, Eric Osgood e...@lakemeadonline.com wrote: Do you have to set the -Xss flag somewhere else? Yes, in bin/nutch - looking for where it sets -Xmx

RE: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Fuad Efendi
Googling reveals http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4675952 and http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5050507 so you could try increasing the Java stack size in bin/nutch (-Xss), or use an alternate regexp if you can. Just out of curiosity, why does a

Re: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Mischa Tuffield
, at 9:22 AM, Fuad Efendi wrote: Also, put it in Hadoop settings for tasks... http://www.tokenizer.ca/ -Original Message- From: Godmar Back [mailto:god...@gmail.com] Sent: January-11-10 11:53 AM To: nutch-user@lucene.apache.org Subject: Re: Help Needed with Error

Re: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Eric Osgood
[mailto:god...@gmail.com] Sent: January-11-10 11:53 AM To: nutch-user@lucene.apache.org Subject: Re: Help Needed with Error: java.lang.StackOverflowError On Mon, Jan 11, 2010 at 11:50 AM, Eric Osgood e...@lakemeadonline.com wrote: Do you have to set the -Xss flag somewhere else? Yes, in bin

Re: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Godmar Back
On Mon, Jan 11, 2010 at 12:30 PM, Fuad Efendi f...@efendi.ca wrote: Googling reveals http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4675952 and http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5050507 so you could try increasing the Java stack size in bin/nutch (-Xss), or use an

Re: Help Needed with Error: java.lang.StackOverflowError

2010-01-11 Thread Andrzej Bialecki
On 2010-01-11 18:40, Godmar Back wrote: On Mon, Jan 11, 2010 at 12:30 PM, Fuad Efendif...@efendi.ca wrote: Googling reveals http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4675952 and http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5050507 so you could try increasing the Java stack size