Hi Manish, If you are pointing at the links retrieved from a page, I would recommend you to have a look at the Nutch configuration properties "db.max.outlinks.per.page" and "db.max.inlinks". Hope it helps.
Thanks & Regards, Karanjeet Singh CS Graduate Student University of Southern California [email protected] On Sun, Dec 20, 2015 at 8:33 PM, Manish Verma <[email protected]> wrote: > Hi, > > I am using notch 1.10 and using crawl script and I see from logs it uses > -topn 50000, I want to consider all pages equally and want to crawl > everything. > > Thanks MV > > >
