Nodemanager crashing repeatedly

2018-09-04 Thread Gajanan Watkar
I am running Nutch-2.3.1 over Hadoop-2.5.2 and Hbase-1.2.3 with integration to Solr-6.5.1. I have crawled over 10 million pages. But while doing all this I am continuously facing two problems: 1. My Nodemanager is crashing repeatedly during different phases of crawl. It crashes my linux session

redirect bin/crwal log output to some other file

2018-09-04 Thread Amarnatha Reddy
Hi All, We are using bin/crawl command to crawl and index data into solr, currently the output is writing into default logs/hadoop.log file, so my requirement is how can i log data writing into different file bin/crawl -i -D solr.server.url=http://localhost:8983/solr/jeepkr -s urls/ crawl/ 1

IndexWriter interface in 1.15

2018-09-04 Thread Yossi Tamari
Hi, I missed it at the time, but I just realized (the hard way) that the IndexWriter interface was changed in 1.15 in ways that are not backward compatible. That means that any custom IndexWriter implementation will no longer compile, and probably will not run either. I think this was a