Hi, I think you can do it in one spider a callback that generate ids and then call the second callback to construct the huge URL. If you want to do it using two spiders you need to run them programmatically http://kirankoduru.github.io/python/running-scrapy-programmatically.html
Best Regards. Lhassan. 2017-03-26 7:24 GMT+02:00 Billy John <archf...@gmail.com>: > Hi. How can I make a callback from a pipeline? How can I make a callback > to different spider when spider 1 has finished extraction? Say in first > cycle I generate ids. Then I split them into chunks of 200 and create one > huge URL for each chunk. In my understanding this can be accomplished from > pipeline or by using 2 all spiders. > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to scrapy-users+unsubscr...@googlegroups.com. > To post to this group, send email to scrapy-users@googlegroups.com. > Visit this group at https://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
