New submission from Julia Dolgova:

I've found that urllib works sometimes slowly on windows with proxy.

To reproduce the issue:
on Windows:
1. Turn on the option "use proxy" in "browser settings" in "control panel".
No real proxy needed. The problem will come out before addressing to proxy. 
Just don't pay attention to exception.
2. Make sure that the list of addresses for proxy bypass is not empty
3. Execute with (attached here) in the same directory

The result output could be:
A (not a problem):
Before call to _socket.gethostbyaddr("")
After call to _socket.gethostbyaddr("")

B (little problem):
Before call to _socket.gethostbyaddr("")
Exception in call to _socket.gethostbyaddr("")

C (worse problem):
Before call to _socket.gethostbyaddr("")
Exception in call to _socket.gethostbyaddr("")

The result A,B or C depends on what DNS server you use, what url you pass into 
urllib2.urlopen(), and could differ at different time because dns is not a 
stable thing. 
However, no matter what result you have, this test shows that a hostname is 
passed into gethostbyaddr instead of IP as expected and described in MSDN. It 
should be changed to gethostbyname_ex here. compare performance of gethostbyaddr and gethostbyname_ex. 
It sets different dns servers on the system and calls these functions with 
different hostnames passed into. Run on my computer shows that gethostbyname_ex 
is 3 times more productive and doesn't raise exceptions.

Attached files: - just make a call to urllib2.urlopen("";) - not a patched lib. Has debug lines near 141 line. Use it with - compare performance of gethostbyaddr with gethostbyname_ex
log.txt - result of on my computer (Windows 8, 64 bit)
socket.patch - patch

components: Library (Lib), Windows
messages: 287597
nosy: juliadolgova, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
status: open
title: urllib2 works slowly with proxy on windows
type: performance
versions: Python 2.7
Added file:

Python tracker <>
Python-bugs-list mailing list

Reply via email to