I've multiple spiders running in multiple instances (4) parallelly. All of them are using almost 100% cpu usage.
I've deployed them using scrapyd. Tried changing scrapyd settings like max_concurrent_requests,CONCURRENT_REQUESTS,CONCURRENT_REQUESTS_PER_DOMAIN to minimum but no luck. I'm using python 2.7.5 and scrapy 0.24 -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
