Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Chris Angelico
On Thu, Jun 19, 2014 at 7:22 PM, Robin Becker ro...@reportlab.com wrote:
 I want to run torture tests against an https server on domain A; I have
 configured apache on the server to respond to a specific hostname ipaddress.

 I don't want to torture the live server so I have set up an alternate
 instance on a different ip address.

Since you mention urllib2, I'm assuming this is Python 2.x, not 3.x.
The exact version may be significant.

Can you simply query the server by IP address rather than host name?
According to the docs, urllib2.urlopen() doesn't check the
certificate, so it should be accepted. Or does the server insist on
the hostname being correct?

Failing that, you could monkey-patch socket.create_connection, which
seems to be the thing that ultimately does the work. Something like
this:

import socket.
orig_create_connection = socket.create_connection
def create_connection(address, *args, **kwargs):
if address == domainA: address = 1.2.3.4
return orig_create_connection(address, *args, **kwargs)
socket.create_connection = create_connection
# Proceed to use urllib2.urlopen()

Untested, but may do what you want.

Normally, though, I'd look at just changing the hosts file, if at all
possible. You're right that it does change state for your whole
computer, but it's generally the easiest solution.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Robin Becker

..


Since you mention urllib2, I'm assuming this is Python 2.x, not 3.x.
The exact version may be significant.


I can use python = 3.3 if required.



Can you simply query the server by IP address rather than host name?
According to the docs, urllib2.urlopen() doesn't check the
certificate, so it should be accepted. Or does the server insist on
the hostname being correct?

Failing that, you could monkey-patch socket.create_connection, which
seems to be the thing that ultimately does the work. Something like
this:

import socket.
orig_create_connection = socket.create_connection
def create_connection(address, *args, **kwargs):
 if address == domainA: address = 1.2.3.4
 return orig_create_connection(address, *args, **kwargs)
socket.create_connection = create_connection
# Proceed to use urllib2.urlopen()

Untested, but may do what you want.



this seems like a way forward



Normally, though, I'd look at just changing the hosts file, if at all
possible. You're right that it does change state for your whole
computer, but it's generally the easiest solution.

ChrisA

me too, but I want to start torturing from about 10 different servers so plumbum 
+ a python script seems like a good choice and I would not really want to hack 
the hosts files back and forth on a regular basis.

--
Robin Becker

--
https://mail.python.org/mailman/listinfo/python-list


Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Chris Angelico
On Thu, Jun 19, 2014 at 9:51 PM, Robin Becker ro...@reportlab.com wrote:
 Since you mention urllib2, I'm assuming this is Python 2.x, not 3.x.
 The exact version may be significant.

 I can use python = 3.3 if required.

The main reason I ask is in case something's changed. Basically, what
I did was go to my Python 2 installation (which happens to be 2.7.3,
because that's what Debian Wheezy ships with - not sure why it hasn't
been updated beyond that), pull up urllib2.py, and step through
manually, seeing where the hostname gets turned into an IP address.
Hence, this code:

 import socket.
 orig_create_connection = socket.create_connection
 def create_connection(address, *args, **kwargs):
  if address == domainA: address = 1.2.3.4
  return orig_create_connection(address, *args, **kwargs)
 socket.create_connection = create_connection
 # Proceed to use urllib2.urlopen()

 Untested, but may do what you want.


 this seems like a way forward

So if it works, that's great! If it doesn't, and you're on a different
version of Python (2.6? 2.4 even?), you might want to look at
repeating the exercise I did, with your actual Python.

But as a general rule, I'd recommend going with Python 3.x unless you
have a good reason for using 2.x. If a feature's been added to let you
mock in a different IP address, it'll be in 3.something but probably
not in 2.7.

 Normally, though, I'd look at just changing the hosts file, if at all
 possible. You're right that it does change state for your whole
 computer, but it's generally the easiest solution.

 ChrisA

 me too, but I want to start torturing from about 10 different servers so
 plumbum + a python script seems like a good choice and I would not really
 want to hack the hosts files back and forth on a regular basis.

Fair enough. In that case, the best thing to do would probably be
monkey-patching, with code along the lines of what I posted above.
Make the change as small and as specific as you can.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Robin Becker

On 19/06/2014 13:03, Chris Angelico wrote:
.

I can use python = 3.3 if required.


The main reason I ask is in case something's changed. Basically, what
I did was go to my Python 2 installation (which happens to be 2.7.3,
because that's what Debian Wheezy ships with - not sure why it hasn't
been updated beyond that), pull up urllib2.py, and step through
manually, seeing where the hostname gets turned into an IP address.
Hence, this code:

.
in practice this approach worked well with urllib in python27.
--
Robin Becker
--
https://mail.python.org/mailman/listinfo/python-list


Re: urllib/urllib2 support for specifying ip address

2014-06-19 Thread Chris Angelico
On Fri, Jun 20, 2014 at 12:19 AM, Robin Becker ro...@reportlab.com wrote:
 in practice [monkeypatching socket] worked well with urllib in python27.

Excellent! That's empirical evidence of success, then.

Like with all monkey-patching, you need to keep it as visible as
possible, but if your driver script is only a page or two of code, it
should be pretty clear what's going on.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list