Hi Everyone, Thanks for stoping by. I am working on a feature to crawl website content every 1 min. I am curious to know if there any good open source project for this specific scenario.
Specifically, I have many urls, and I want to maintain a thread pool so that each thread will repeatedly crawl content from the given url. It could be a hundreds thread at the same time. Your help is greatly appreciated. ;) -- https://mail.python.org/mailman/listinfo/python-list
