I need to scrape a website, where landing page A redirects to page B using Javascript, and then redirects back to page A--all while using cookies. Is scrapy a bad option? I've already done most of what I needed in Selenium, except that:
1) downloading images in Selenium is difficult (many StackOverflow pages confirm this) 2) Selenium and scrapy don't really interact <http://stackoverflow.com/questions/28420078/scrapy-with-selenium-for-a-webpage-requiring-authentication> 3) Mechanize doesn't handle javascript well either <http://stackoverflow.com/questions/802225/how-do-i-use-mechanize-to-process-javascript> Does anyone have sample snippets of how to do use scrapy in such a scenario? Should I use mechanize instead? -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
