On Tue, Apr 8, 2008 at 5:00 AM, Trent Nelson <[EMAIL PROTECTED]> wrote:
> > I've forwarding my most recent update to issue 2550 here such that the > proposed patch (and in general, the approach to network-oriented test cases) > can be vetted by a wider audience: > > http://bugs.python.org/file9980/trunk.2550-2.patch > > This patch works towards fixing a large proportion of the tests that were > written in such a way that often leads to buildbot errors when port > conflicts arise. With this patch applied, I can run many instances of the > test suite in parallel and none of the network-oriented tests fail (which > they do currently if you try and do this). > > Plenty of discussion (mostly me replying to my own comments) on the > subject at: http://bugs.python.org/issue2550. > > Anyone have any issues with this new approach? I'm particularily > interested in whether or not people disagree with the assumption I've drawn > regarding never using SO_REUSEADDR in tests for TCP/IP sockets (see below). > > Trent. I think your assumptions are fair. Note that not using SO_REUSEADDR can reserve the port for a few minutes beyond the actual unbinding destruction of the server socket on some OSes but with tens of thousands of ports available and likely not more than 1-200 being needed by a full test suite run and how long the test suite takes to run I think that is acceptable and is not a problem we're likely to ever run into (fix it only iffwe ever do). -gps > > > > ________________________________________ > From: [EMAIL PROTECTED] > [EMAIL PROTECTED] On Behalf Of Trent Nelson [ > [EMAIL PROTECTED] > Sent: 08 April 2008 12:49 > To: [EMAIL PROTECTED] > Subject: [issue2550] SO_REUSEADDR doesn't have the same semantics on > Windows as on Unix > > Trent Nelson <[EMAIL PROTECTED]> added the comment: > > Invested quite a few cycles on this issue last night. The more time I > spent on it, the more I became convinced that every single test working > with sockets should be changed in one fell swoop in order to facilitate > (virtually unlimited) parallel test execution without fear of port > conflicts. > > I've attached a second patch, trunk.2550-2.patch, which is my progress > so far on doing just this. The main changes can be expressed by the > following two points: > > a) do whatever it takes in network-oriented tests to ensure > unique ports are obtained (relying on the bind_port() and > find_unused_port() methods exposed by test_support) > > b) never, ever, ever call SO_REUSEADDR on a socket from a test; > because we're putting so much effort into obtaining a unique > port, this should never be necessary -- in the rare cases that > our attempts to obtain a unique port fail, then we absolutely > should fail with EADDRINUSE, as the ability to obtain a unique > port for the duration of a client/server test is an invariant > that we *must* be able to depend upon. If the invariant is > broken, fail immediately (don't mask the problem with > SO_REUSEADDR). > > With this patch applied, I can spawn a handful of Python processes and > run the entire test suite (without -r, ensuring all tests are run in > the same order, which should encourage port conflicts (if there were > any)) without any errors*. Doing that now is completely and utterly > impossible. > > [*] Well, almost without error. All the I/O related tests that try and > open @test fail. > > I believe there's still outstanding work to do with this patch with > regards to how the intracacies of SO_REUSEADDR and SO_EXCLUSIVEADDRUSE > should be handled in the rest of the stdlib. I'm still thinking about > the best approach for this. However, the patch as it currently stands > is still quite substantial so I wanted to get it out sooner rather than > later for review. > > (I'll forward this to python-dev@ to try and encourage more eyes from > people with far more network-fu than I.) > > Added file: http://bugs.python.org/file9980/trunk.2550-2.patch > > __________________________________ > Tracker <[EMAIL PROTECTED]> > <http://bugs.python.org/issue2550> > __________________________________ > _______________________________________________ > Python-bugs-list mailing list > Unsubscribe: > http://mail.python.org/mailman/options/python-bugs-list/tnelson%40onresolve.com > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
