I am trying to build a CrawlSpider using Scrapy to crawl sites in Tor.
To do this I am using the Polipo proxy as described by this guide: http://blog.privatenode.in/torifying-scrapy-project-on-ubuntu/ So right now my spider works and I am able to access .onion sites in Scrapy. However what I would like to do is, for each request, send a NEWNYM signal in order to change my Tor identity. How can this be done in Scrapy? -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
