I have set the domain and I have hidden the original URL in my previous post.
On Friday, October 30, 2015 at 7:51:56 PM UTC+5:30, Travis Leleu wrote: > > You need to set the domain you want to crawl. Example.com isn't a real > domain... > > On Fri, Oct 30, 2015 at 9:45 AM, Prabhakar <drprabh...@gmail.com > <javascript:>> wrote: > >> Yes, the scrapy spider prints traceback. >> >> 2015-10-30 17:41:35+0530 [myspider] DEBUG: Crawled (200) <GET >> https://example.com/searchid?ss=1&id=1000> (referer: >> https://example.com/search?ss=1) ['partial'] >> 2015-10-30 17:41:35+0530 [myspider] DEBUG: Crawled (200) <GET >> https://example.com/searchid?ss=2&id=2000> (referer: >> https://example.com/search?ss=2) ['partial'] >> 2015-10-30 17:41:35+0530 [myspider] DEBUG: Crawled (200) <GET >> https://example.com/searchid?ss=3&id=3000> (referer: >> https://example.com/search?ss=3) ['partial'] >> >> >> And How can I fix this. >> >> >> On Friday, October 30, 2015 at 2:15:02 PM UTC+5:30, Nikolaos-Digenis >> Karagiannis wrote: >>> >>> You may be experiencing data loss in your proxy. >>> How do you know that your crawl fails? Does it print a traceback? >>> >>> >>> On Saturday, 4 January 2014 15:16:45 UTC+2, Shivkumar Agrawal wrote: >>>> >>>> Hi >>>> >>>> I have created a crawler in scrapy. It uses a list of proxies to crawl. >>>> I have 3-4 sites to crawl daily. During crawling, scrapy logs show >>>> ['partial'] message and my crawl is failed on those request. I have spent >>>> lot of time on googling with no luck. >>>> Can anybody help in this matter >>>> >>>> >>>> >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "scrapy-users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to scrapy-users...@googlegroups.com <javascript:>. >> To post to this group, send email to scrapy...@googlegroups.com >> <javascript:>. >> Visit this group at http://groups.google.com/group/scrapy-users. >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
