Thank you very much. понедельник, 12 декабря 2016 г., 12:15:01 UTC+3 пользователь Paul Tremberth написал: > > Hello Ramzay Ak > > At load time, HttpProxyMiddleware checks whether it can find proxies using > six.moves.urllib.request.getproxies(), > (in Python 2, this is > https://docs.python.org/2/library/urllib.html#urllib.getproxies) > > If it cannot find proxies this way, it will not be configured for the > crawl. > See > https://github.com/scrapy/scrapy/blob/129421c7e31b89b9b0f9c5f7d8ae59e47df36091/scrapy/downloadermiddlewares/httpproxy.py#L23 > > HttpProxyMiddleware does not currently understand HTTP_PROXY setting. > > You can call your spider with this to set an env var for the proxy > > $ http_proxy="http://127.0.0.1:8080"; scrapy crawl myspider > > Hope this helps. > > /Paul > > On Monday, December 5, 2016 at 12:23:16 PM UTC+1, Ramzay Ak wrote: >> >> Hello. I want that each request goes throw local proxy (127.0.0.1:8080). >> But when i start spider, i did not see HttpProxyMiddleware in list of >> enabled middleware: >> >> 2016-12-04 10:27:12 [scrapy] INFO: Enabled downloader middlewares: >>> ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', >>> >>> 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', >>> 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', >>> 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', >>> 'scrapy.downloadermiddlewares.retry.RetryMiddleware', >>> 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', >>> >>> 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', >>> 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', >>> 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', >>> 'scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware', >>> 'scrapy.downloadermiddlewares.stats.DownloaderStats'] >> >> >> in settings.py i add strings: >> >> >>> HTTP_PROXY = "http://127.0.0.1:8080"; >>> >>> DOWNLOADER_MIDDLEWARES = { >>> 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware': 100, >>> } >>> >>> >> Please tell me what should I do to make it work >> >
-- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at https://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
