Folks, I had around 3200 URLs, It is a linux virtual dedicated server (Godaddy) JAVA_TOOL_OPTIONS=-Xmx40000000 --> Required to be set to some value without which JVM would not start
JAVA_HEAP_MAX=-Xmx40000000 NUTCH_OPTS=-Xmx40000000 For 3200 URLs, I still got out of memory error. I am using 0.8dev. I have my plugins which captures more data from the given URL other than title and metatags. For this I use regular expression. Is this the cuase of the problem. With reduced seed, like 100-200, I do not have any problems. BTW, I am just running local file system. Can any one please shed some insights on this ? Thanks Sudhi
