Re: Please share your experience of using Nutch in production

2014-06-24 Thread Gora Mohanty
On 23 June 2014 01:44, Meraj A. Khan wrote: > Gora, > > Thanks for sharing your admin perspective , rest assured I am not trying > to circumvent any politeness requirements in any way , as I mentioned > earlier , I am with in the crawl-delay limits that are being set by the web > masters if any ,

Re: Please share your experience of using Nutch in production

2014-06-23 Thread Jorge Luis Betancourt Gonzalez
Why are you assuming that the web masters are effectively going to block you? In my experience this is the least probable escenario. On Jun 22, 2014, at 4:14 PM, Meraj A. Khan wrote: > Gora, > > Thanks for sharing your admin perspective , rest assured I am not trying > to circumvent any polit

Re: Please share your experience of using Nutch in production

2014-06-22 Thread Meraj A. Khan
Gora, Thanks for sharing your admin perspective , rest assured I am not trying to circumvent any politeness requirements in any way , as I mentioned earlier , I am with in the crawl-delay limits that are being set by the web masters if any , however , you have confirmed my hunch that I might have

Re: Please share your experience of using Nutch in production

2014-06-22 Thread Gora Mohanty
On 22 June 2014 22:07, Meraj A. Khan wrote: > > Hello Folks, > > I have noticed that Nutch resources and mailing lists are mostly geared > towards the usage of Nutch in research oriented projects , I would like to > know from those of you who are using Nutch in production for large scale > crawli