[ANNOUNCE] CFP open for ApacheCon North America 2016

2015-11-25 Thread Rich Bowen
Community growth starts by talking with those interested in your project. ApacheCon North America is coming, are you? We are delighted to announce that the Call For Presentations (CFP) is now open for ApacheCon North America. You can submit your proposed sessions at

RE: Manipulate queues

2015-11-25 Thread Markus Jelsma
Hello - please see the freegen tool. It takes a list of URL's and generates a fetch list for them that you can crawl immediately. The drawback is that regular metadata changes differently when running updatedb vs regular generated URL's. We use it too to prioritize URL's while also running

Manipulate queues

2015-11-25 Thread Gaspar Pizarro
Hi, I am doing whole-web crawling, but from time to time I need to fetch a list of urls immediately, with top priority. I was watching what happens with the injected urls when there are already non-fetched urls, and, when fetching the new injected urls are not on top on the list. Is there a way