Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-08 Thread glyph


On 02:28 am, s...@pobox.com wrote:
   Glyph ... which is exactly why I have volunteered to explain to 
someone

   Glyph how to separate the core event-loop bits



   Neil This sounds great.



Anybody interested in working on this at a PyCon Sprint?  I won't be
attending the conference proper, but plan to spend a couple days 
sprinting,
one on Mailman/SpamBayes integration and one on Python core stuff. 
This
might fit well into my Python core slot.  I will probably have little 
time
before the sprint to do much, but any brain dumps or Twisted pointers 
people

could give me in the interim would be appreciated.


I'll try to make sure we get those notes to you in advance of the 
sprints :).  I should be at the Twisted sprint the whole time - will you 
be at the physical sprint, or following along at home?

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-08 Thread glyph


On 01:50 am, n...@arctrix.com wrote:

gl...@divmod.com gl...@divmod.com wrote:



we're not even talking about actually putting any Twisted code
into the standard library, just standardizing the protocol API, 
which

basically boils down to:

[...]

This sounds great.  I'm interested on working on this since it
scratches an itch of mine but I don't know if I will have time.  Do
you think if this part of Twisted became part of the standard
library that it would be used by Twisted or would it continue to
have its own version?


As I said above, right now I'm just talking about the interface.  There 
are several implementations of the main loop that call those interfaces, 
as well as test implementations.


Of course I hope we can eliminate some redundancy, but that's step 2: 
before we refactor, we have to make the code that we're refactoring 
actually repetitive rather than just similar.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-08 Thread skip

Glyph I'll try to make sure we get those notes to you in advance of the
Glyph sprints :).  I should be at the Twisted sprint the whole time -
Glyph will you be at the physical sprint, or following along at home?

Thanks.  I will be at the physical sprints.  (I live in the Chicago area.)

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-08 Thread Daniel Stutzbach
On Sat, Mar 7, 2009 at 8:28 PM, s...@pobox.com wrote:

 Anybody interested in working on this at a PyCon Sprint?  I won't be
 attending the conference proper, but plan to spend a couple days sprinting,


I'll be there and interested. :)

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-08 Thread glyph

On 02:46 pm, dan...@stutzbachenterprises.com wrote:

On Sat, Mar 7, 2009 at 8:28 PM, s...@pobox.com wrote:

Anybody interested in working on this at a PyCon Sprint?  I won't be
attending the conference proper, but plan to spend a couple days 
sprinting,



I'll be there and interested. :)


Great!  I plan to hold both of you to that :).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-07 Thread Neil Schemenauer
gl...@divmod.com gl...@divmod.com wrote:
 ... which is exactly why I have volunteered to explain to someone how to 
 separate the core event-loop bits (suitable for inclusion in the 
 standard library) from the huge pile of protocol implementations which 
 are not necessarily useful.

 Despite the FUD to the contrary, Twisted's internal factoring is quite 
 good; it's not a single, undifferentiated pile of code.  Plus, at this 
 point, we're not even talking about actually putting any Twisted code 
 into the standard library, just standardizing the protocol API, which 
 basically boils down to:
[...]

This sounds great.  I'm interested on working on this since it
scratches an itch of mine but I don't know if I will have time.  Do
you think if this part of Twisted became part of the standard
library that it would be used by Twisted or would it continue to
have its own version?

  Neil

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-07 Thread skip
Glyph ... which is exactly why I have volunteered to explain to someone
Glyph how to separate the core event-loop bits (suitable for inclusion
Glyph in the standard library) from the huge pile of protocol
Glyph implementations which are not necessarily useful.

Neil This sounds great.  I'm interested on working on this since it
Neil scratches an itch of mine but I don't know if I will have time.
Neil Do you think if this part of Twisted became part of the standard
Neil library that it would be used by Twisted or would it continue to
Neil have its own version?

Anybody interested in working on this at a PyCon Sprint?  I won't be
attending the conference proper, but plan to spend a couple days sprinting,
one on Mailman/SpamBayes integration and one on Python core stuff.  This
might fit well into my Python core slot.  I will probably have little time
before the sprint to do much, but any brain dumps or Twisted pointers people
could give me in the interim would be appreciated.

Skip
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-06 Thread Hrvoje Niksic

Greg Ewing wrote:

Antoine Pitrou wrote:


For starters, since py3k is supposed to support non-blocking IO, why not write a
portable API to make a raw file or socket IO object non-blocking?


I think we need to be clearer what we mean when we talk
about non-blocking in this context. Normally when you're
using select/poll you *don't* make the underlying file
descriptor non-blocking in the OS sense. The non-blockingness
comes from the fact that you're using select/poll to make
sure the fd is ready before you access it.

So I don't think it makes sense to talk about having a
non-blocking API as a separate thing from a select/poll
wrapper. The select/poll wrapper *is* the non-blocking
API.


This is not necessarily the case.  In fact, modern sources often 
recommend using poll along with the non-blocking sockets for (among 
other things) performance reasons.  For example, when a non-blocking 
socket becomes readable, you don't read from it only once and go back to 
the event loop, you read from it in a loop until you get EAGAIN.  This 
allows for processing of fast-incoming data with fewer system calls.


Linux's select(2) man page includes a similar advice with different 
motivation:


   Under Linux, select() may report a socket file descriptor
   as ready for reading,  while  nevertheless
   a subsequent read blocks.  This could for example
   happen when data has arrived but upon
   examination has wrong checksum and is discarded.  There may
   be other circumstances  in  which  a
   file  descriptor  is  spuriously  reported  as ready.
   Thus it may be safer to use O_NONBLOCK on
   sockets that should not block.

Even if you don't agree that using O_NONBLOCK with select/poll is the 
best approach to non-blocking, I think there is enough existing practice 
of doing this to warrant separate consideration of non-blocking sockets 
(in the OS sense) and select/poll.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-06 Thread Greg Ewing

Hrvoje Niksic wrote:


   Under Linux, select() may report a socket file descriptor
   as ready for reading,  while  nevertheless
   a subsequent read blocks.


Blarg. Linux is broken, then. This should not happen.


   This could for example
   happen when data has arrived but upon
   examination has wrong checksum and is discarded.


That's no excuse -- the kernel should check all its
checksums *before* waking up selecting processes!

Even if you don't agree that using O_NONBLOCK with select/poll is the 
best approach to non-blocking, I think there is enough existing practice 
of doing this to warrant separate consideration of non-blocking sockets 
(in the OS sense) and select/poll.


I'm not saying there isn't merit in having support for
non-blocking file descriptors, only that it's not in
any sense a prerequisite or first step towards a
select/poll wrapper. They're orthogonal issues, even
if you might sometimes want to use them together.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-06 Thread Hrvoje Niksic

Greg Ewing wrote:
Even if you don't agree that using O_NONBLOCK with select/poll is the 
best approach to non-blocking, I think there is enough existing practice 
of doing this to warrant separate consideration of non-blocking sockets 
(in the OS sense) and select/poll.


I'm not saying there isn't merit in having support for
non-blocking file descriptors, only that it's not in
any sense a prerequisite or first step towards a
select/poll wrapper. They're orthogonal issues, even
if you might sometimes want to use them together.


In that case we are in agreement.  Looking back, I was somewhat confused 
by this paragraph:


So I don't think it makes sense to talk about having a
non-blocking API as a separate thing from a select/poll
wrapper. The select/poll wrapper *is* the non-blocking
API.

If they're orthogonal, then it does make sense to talk about having a 
separate non-blocking socket API and poll API, even if the latter can be 
used to implement non-blocking *functionality* (hypothetical Linux 
issues aside).

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-06 Thread glyph


On 10:01 am, greg.ew...@canterbury.ac.nz wrote:

Hrvoje Niksic wrote:

   Under Linux, select() may report a socket file descriptor
   as ready for reading,  while  nevertheless
   a subsequent read blocks.


Blarg. Linux is broken, then. This should not happen.


You know what else is broken?  MacOS, FreeBSD, Solaris, and every 
version of Windows.  I haven't tried running Twisted on AmigaOS but I 
bet it has some problems too.


On occasion Linux has been so badly broken that Twisted has motivated a 
fix.  For example, 
http://lkml.indiana.edu/hypermail/linux/kernel/0502.3/1160.html


But even if we ignore difficulties at the OS level (which should, after 
all, be worked around rather than catered to in API design) there are 
other good reasons why the general async API should be fairly distant 
from both the select/poll wrapper and the questions of blocking vs. non- 
blocking sockets.  For another example, consider the issue of non- 
blocking SSL sockets.  Sometimes, in order to do a read, you actually 
need to do a write and then another read.  Which means that application 
code, if it wants to be compatible with SSL, needs to deal with any 
failure that may come from a write as coming from a read, unless you 
abstract all this away somehow.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread glyph

On 4 Mar, 09:14 pm, krs...@solarsail.hcs.harvard.edu wrote:
I spent about a half hour sometime in the last month talking this 
through with Itamar, though not in great detail. I'd be interested in 
sitting down with both of you and trying to establish more precisely 
how much work is necessary to get something to actually happen here. I 
won't outright promise a PEP, but I'll promise at the very least to 
write down elaborate notes that someone could turn into a PEP 
relatively straightforwardly. Deal?


Absolutely.  I really appreciate the offer.  As the other gentlemen 
suggested, PyCon would be an ideal venue for doing this.  Are you going 
to be there?


I'll hopefully be seeing your talk this evening, but I suspect that 
would be a pretty bad venue to try to get this done ;-).

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread glyph


On 4 Mar, 08:28 pm, ba...@python.org wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mar 4, 2009, at 2:44 PM, gl...@divmod.com wrote:
Maintaining compatibility with the 2.6.x version of asyncore 
presupposes that *someone* has written some software against that 
version of asyncore and it might break if they installed an upgrade, 
though.


FWIW, I use smtpd.py and a few of the asyncore APIs  (.loop(), 
.socket_map.clear(), and .close_all()) in the Mailman 3 test  suite. 
That only works on Python 2.6 and I don't recall even a hiccup  when 
moving from 2.5 to 2.6.


Well, in that case, I withdraw even my disinterested suggestion :).
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread glyph


On 4 Mar, 09:36 pm, dan...@stutzbachenterprises.com wrote:
Will any or all of you be at PyCon?  I'd be willing to put in the extra 
work

to turn your notes into a PEP.


I definitely will be.  I'll see you there!
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Neil Schemenauer
Chris McDonough chr...@plope.com wrote:
 As far as I can tell, asyncore/asynchat is all undocumented
 internals.  Any use of asyncore in anger will use internals;
 there never was any well-understood API to these modules.

What I would like to see is a module that provides a low-level API
for doing cross-platform asynchronous IO.  The two necessary parts
are:

* a wrapper that allows non-blocking reads and writes on
  channels (sockets, file descriptors, serial ports, etc) 

* a select() or epoll like interface that allows waiting on
  multiple channels

The implementation requires some intricate and platform specific
code which is why it would be nice to be a standard library feature.

I'm sure that Twisted has the necessary parts but the problem, IMHO,
is that it does so much more else.

  Neil

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Daniel Stutzbach
On Thu, Mar 5, 2009 at 1:30 PM, Neil Schemenauer n...@arctrix.com wrote:

 What I would like to see is a module that provides a low-level API
 for doing cross-platform asynchronous IO.  The two necessary parts
 are:

* a wrapper that allows non-blocking reads and writes on
  channels (sockets, file descriptors, serial ports, etc)

* a select() or epoll like interface that allows waiting on
  multiple channels


Two thoughts:

If you have a working select(), it will tell you the sockets on which read()
and write() won't block, so non-blocking reads and writes are not necessary.

On Windows, sockets, pipes, and files are each completely distinct types
with their own functions and unifying them under one select()-like interface
requires faking it using threads behind-the-scenes AFAIK.  Personally, I'd
be happy with continuing to only support socket objects on Windows.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Greg Ewing

Neil Schemenauer wrote:


What I would like to see is a module that provides a low-level API
for doing cross-platform asynchronous IO.  The two necessary parts
are:

* a wrapper that allows non-blocking reads and writes on
  channels (sockets, file descriptors, serial ports, etc) 


* a select() or epoll like interface that allows waiting on
  multiple channels


+1

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Greg Ewing

Daniel Stutzbach wrote:

If you have a working select(), it will tell you the sockets on which 
read() and write() won't block, so non-blocking reads and writes are not 
necessary.


No, but there should be an interface that lets you say
when something comes in on this fd, call this function
for me.

In other words it should be a light wrapper around
select/poll/whatever that provides a callback interface.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Josiah Carlson
On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing greg.ew...@canterbury.ac.nz wrote:
 Daniel Stutzbach wrote:

 If you have a working select(), it will tell you the sockets on which
 read() and write() won't block, so non-blocking reads and writes are not
 necessary.

 No, but there should be an interface that lets you say
 when something comes in on this fd, call this function
 for me.

 In other words it should be a light wrapper around
 select/poll/whatever that provides a callback interface.

A read callback, a write callback.  What about close, error, connect,
and accept callbacks?

I hate to say it (not really), but that's pretty much the handle_*()
methods of asyncore :/ .

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Bill Janssen
Josiah Carlson josiah.carl...@gmail.com wrote:

 On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing greg.ew...@canterbury.ac.nz 
 wrote:
  Daniel Stutzbach wrote:
 
  If you have a working select(), it will tell you the sockets on which
  read() and write() won't block, so non-blocking reads and writes are not
  necessary.
 
  No, but there should be an interface that lets you say
  when something comes in on this fd, call this function
  for me.
 
  In other words it should be a light wrapper around
  select/poll/whatever that provides a callback interface.
 
 A read callback, a write callback.  What about close, error, connect,
 and accept callbacks?
 
 I hate to say it (not really), but that's pretty much the handle_*()
 methods of asyncore :/ .

What asyncore was missing was a timer API (a way to register functions
to be called periodically).  Then it would be pretty much like any other
event loop system.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Greg Ewing

Josiah Carlson wrote:


A read callback, a write callback.  What about close, error, connect,
and accept callbacks?


Yep, all those as well.


I hate to say it (not really), but that's pretty much the handle_*()
methods of asyncore :/ .


Well, then, what's the problem?

Is there anything else people want that asyncore doesn't
provide, or is it just a matter of nailing down the existing
API a little?

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Josiah Carlson
On Thu, Mar 5, 2009 at 1:09 PM, Bill Janssen jans...@parc.com wrote:
 Josiah Carlson josiah.carl...@gmail.com wrote:

 On Thu, Mar 5, 2009 at 12:46 PM, Greg Ewing greg.ew...@canterbury.ac.nz 
 wrote:
  Daniel Stutzbach wrote:
 
  If you have a working select(), it will tell you the sockets on which
  read() and write() won't block, so non-blocking reads and writes are not
  necessary.
 
  No, but there should be an interface that lets you say
  when something comes in on this fd, call this function
  for me.
 
  In other words it should be a light wrapper around
  select/poll/whatever that provides a callback interface.

 A read callback, a write callback.  What about close, error, connect,
 and accept callbacks?

 I hate to say it (not really), but that's pretty much the handle_*()
 methods of asyncore :/ .

 What asyncore was missing was a timer API (a way to register functions
 to be called periodically).  Then it would be pretty much like any other
 event loop system.

There are two variants of patches to offer timer API functionality in
the bug tracker right now.  One from Giampaolo that uses a variant of
Twisted's scheduler, one from me that uses an updated sched.py .
Giampaolo's is more complete (it has documentation and tests), but
mine is more efficient with nontrivial task lists.

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Antoine Pitrou
Greg Ewing greg.ewing at canterbury.ac.nz writes:
 
 Neil Schemenauer wrote:
 
  What I would like to see is a module that provides a low-level API
  for doing cross-platform asynchronous IO.  The two necessary parts
  are:
  
  * a wrapper that allows non-blocking reads and writes on
channels (sockets, file descriptors, serial ports, etc) 

For starters, since py3k is supposed to support non-blocking IO, why not write a
portable API to make a raw file or socket IO object non-blocking?
(I'm only suggesting it here, I don't intend to do this myself)


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread glyph


On 07:30 pm, n...@arctrix.com wrote:

Chris McDonough chr...@plope.com wrote:

As far as I can tell, asyncore/asynchat is all undocumented
internals.  Any use of asyncore in anger will use internals;
there never was any well-understood API to these modules.



The implementation requires some intricate and platform specific
code which is why it would be nice to be a standard library feature.

I'm sure that Twisted has the necessary parts but the problem, IMHO,
is that it does so much more else.


... which is exactly why I have volunteered to explain to someone how to 
separate the core event-loop bits (suitable for inclusion in the 
standard library) from the huge pile of protocol implementations which 
are not necessarily useful.


Despite the FUD to the contrary, Twisted's internal factoring is quite 
good; it's not a single, undifferentiated pile of code.  Plus, at this 
point, we're not even talking about actually putting any Twisted code 
into the standard library, just standardizing the protocol API, which 
basically boils down to:


 connectionMade() - your connection has begun
 dataReceived(data) - you got some bytes, handle them
 connectionLost(reason) - your connection has gone away (with an object 
explaining why)


and the inverse, transport, which is:

 write(data) - deliver some data to the dataReceived on the other end 
of this connection (non-blocking, with buffering)

 loseConnection() - goodbye

There are a few other minor details related to how you set these up to 
talk to each other and tell when the out-buffer is empty, but it's all 
pretty straightforward.  The main point is that you don't ever call 
recv() or send() and deal with buffering or handling weird errno values. 
For example, if your connection goes away, the notification you get is 
your connection went away, not oops you tried to read some bytes, but 
your connection was gone by the time you tried, even though I just told 
you it was ready for reading or other similarly obtuse failure modes.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Josiah Carlson
On Thu, Mar 5, 2009 at 3:11 PM,  gl...@divmod.com wrote:

 On 07:30 pm, n...@arctrix.com wrote:

 Chris McDonough chr...@plope.com wrote:

 As far as I can tell, asyncore/asynchat is all undocumented
 internals.  Any use of asyncore in anger will use internals;
 there never was any well-understood API to these modules.

 The implementation requires some intricate and platform specific
 code which is why it would be nice to be a standard library feature.

 I'm sure that Twisted has the necessary parts but the problem, IMHO,
 is that it does so much more else.

 ... which is exactly why I have volunteered to explain to someone how to
 separate the core event-loop bits (suitable for inclusion in the standard
 library) from the huge pile of protocol implementations which are not
 necessarily useful.

 Despite the FUD to the contrary, Twisted's internal factoring is quite good;
 it's not a single, undifferentiated pile of code.  Plus, at this point,
 we're not even talking about actually putting any Twisted code into the
 standard library, just standardizing the protocol API, which basically
 boils down to:

  connectionMade() - your connection has begun
  dataReceived(data) - you got some bytes, handle them
  connectionLost(reason) - your connection has gone away (with an object
 explaining why)

 and the inverse, transport, which is:

  write(data) - deliver some data to the dataReceived on the other end of
 this connection (non-blocking, with buffering)
  loseConnection() - goodbye

 There are a few other minor details related to how you set these up to talk
 to each other and tell when the out-buffer is empty, but it's all pretty
 straightforward.  The main point is that you don't ever call recv() or
 send() and deal with buffering or handling weird errno values. For example,
 if your connection goes away, the notification you get is your connection
 went away, not oops you tried to read some bytes, but your connection was
 gone by the time you tried, even though I just told you it was ready for
 reading or other similarly obtuse failure modes.

Those obtuse failure modes are handled fairly well by asynchat.  It
works pretty well, and includes sample implementations to get you 99%
of the way towards the 5-method API you describe (which is why a lot
of people use asynchat instead of asyncore).  If it weren't for the
fact that asynchat had a previously existing API (including the
.push(), collect_incoming_data(), and found_terminator()), I think it
could stand for methods to make it fit better with asyncore's
handle_*() methods.

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-05 Thread Greg Ewing

Antoine Pitrou wrote:


For starters, since py3k is supposed to support non-blocking IO, why not write a
portable API to make a raw file or socket IO object non-blocking?


I think we need to be clearer what we mean when we talk
about non-blocking in this context. Normally when you're
using select/poll you *don't* make the underlying file
descriptor non-blocking in the OS sense. The non-blockingness
comes from the fact that you're using select/poll to make
sure the fd is ready before you access it.

So I don't think it makes sense to talk about having a
non-blocking API as a separate thing from a select/poll
wrapper. The select/poll wrapper *is* the non-blocking
API.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Guido van Rossum
On Tue, Mar 3, 2009 at 9:31 PM,  gl...@divmod.com wrote:

 On 3 Mar, 10:20 pm, gu...@python.org wrote:

 On Tue, Mar 3, 2009 at 1:17 PM,  gl...@divmod.com wrote:

 At the very least, this might serve as a basis for an abstract API for
 asyncore:


 http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html

 I hope we have learned from asyncore that a future-proof and maximally
 flexible design of such an API is not easy. This is one of those cases
 where I believe thinking hard up front will really pay off. In this
 case I would recommend going through a full-blown PEP process before
 committing to an API, to avoid making the same mistake twice. (I don't
 mean this as a dig at Twisted, just as an observation about this
 particular design problem. If Twisted solves all the problems already,
 writing the PEP should be easy.)

 I didn't see it as a dig at Twisted; you're right, this is a hard problem.
  Twisted doesn't solve all the problems perfectly, either; much has been
 made of some rather unfortunate nuances of the IProducer and IConsumer
 interfaces.

 I also regret that no Twisted people (myself included) seem to have the time
 to go through the PEP process and get something accepted.

Same here.

 Even if Twisted's interfaces aren't perfect, there's a lot of working
 application code which demonstrates that they are sufficient.  Perhaps more
 importantly, they are future-proof.  Every proposal we've come up with for
 addressing the more subtle shortcomings of these APIs has been completely
 backwards-compatible; there are no APIs with vague or broken semantics, just
 sub-optimal spellings.  The fact that there are already multiple reactor
 implementations which share very little code helps.

Sounds like it's not so much the code that's future proof but the
process used for evolving it. That seems to be missing for asyncore.
:-(

 If someone who has PEP-writing skills and some free time is updating
 asyncore to be more modern, I'd be very happy to walk that person through
 Twisted's API design and explain how either portions of our implementation
 could be lifted for the stdlib, or how a high level layer could be written
 into asyncore so that the author of a particular chunk of low-level
 networking code could ignore whether they're using the stdlib mainloop or
 one of the Twisted ones.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Sidnei da Silva
On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum gu...@python.org wrote:
 Sounds like it's not so much the code that's future proof but the
 process used for evolving it. That seems to be missing for asyncore.
 :-(

Turning the issue around a bit, has anyone considered polishing up the
current fix to restore it's backwards compatibility, instead of
starting a discussion about a full-blown replacement?

I think that would be enough for most asyncore users (or even the
couple few affected) for the moment, and then we can think about a
possible future replacement.

-- 
Sidnei da Silva
Canonical Ltd.
 Landscape · Changing the way you manage your systems
http://landscape.canonical.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Guido van Rossum
On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva
sidnei.da.si...@gmail.com wrote:
 On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum gu...@python.org wrote:
 Sounds like it's not so much the code that's future proof but the
 process used for evolving it. That seems to be missing for asyncore.
 :-(

 Turning the issue around a bit, has anyone considered polishing up the
 current fix to restore it's backwards compatibility, instead of
 starting a discussion about a full-blown replacement?

 I think that would be enough for most asyncore users (or even the
 couple few affected) for the moment, and then we can think about a
 possible future replacement.

If it can be done while maintaining backwards compatibility with both
the 2.6 version and the pre-2.6 version, that would be great of
course! But can it?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Jean-Paul Calderone

On Wed, 4 Mar 2009 10:21:26 -0800, Guido van Rossum gu...@python.org wrote:

On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva
sidnei.da.si...@gmail.com wrote:

On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum gu...@python.org wrote:

Sounds like it's not so much the code that's future proof but the
process used for evolving it. That seems to be missing for asyncore.
:-(


Turning the issue around a bit, has anyone considered polishing up the
current fix to restore it's backwards compatibility, instead of
starting a discussion about a full-blown replacement?

I think that would be enough for most asyncore users (or even the
couple few affected) for the moment, and then we can think about a
possible future replacement.


If it can be done while maintaining backwards compatibility with both
the 2.6 version and the pre-2.6 version, that would be great of
course! But can it?



Is it really necessary to retain compatibility with the Python 2.6
version?  Python 2.6.0 and Python 2.6.1 contain a regression (as
compared to basically all previous versions of Python) which prevents
asyncore-based programs which are years old from working on them.
Restoring the pre-2.6 behavior will fix these old, presumably stable,
widely used programs for users who install 2.6.2 and newer.

The downside (which you were imagining, I'm sure) is that any new
software developed against the Python 2.6.0 or 2.6.1 behavior will
then break in 2.6.2 and later.  While this is unfortunate, it is
clearly the far lesser of two evils.  The choice must be made, though.
Either leave old software broken or break new software.  Just because
the leave old software broken choice is made through inaction doesn't
make it the better choice (though obviously since it requires action,
someone will have to do it, and I'm not volunteering - if inaction is
the choice because no one wants to do the work, fine, but that's a
different motivation than avoiding breaking newly written software).

So, as a disinterested party in this specific case, I'd say revert to
the pre-2.6 behavior.  It does less harm than leaving the current
behavior.

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Guido van Rossum
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone exar...@divmod.com wrote:
 - Show quoted text -
 On Wed, 4 Mar 2009 10:21:26 -0800, Guido van Rossum gu...@python.org
 wrote:

 On Wed, Mar 4, 2009 at 10:14 AM, Sidnei da Silva
 sidnei.da.si...@gmail.com wrote:

 On Wed, Mar 4, 2009 at 3:04 PM, Guido van Rossum gu...@python.org
 wrote:

 Sounds like it's not so much the code that's future proof but the
 process used for evolving it. That seems to be missing for asyncore.
 :-(

 Turning the issue around a bit, has anyone considered polishing up the
 current fix to restore it's backwards compatibility, instead of
 starting a discussion about a full-blown replacement?

 I think that would be enough for most asyncore users (or even the
 couple few affected) for the moment, and then we can think about a
 possible future replacement.

 If it can be done while maintaining backwards compatibility with both
 the 2.6 version and the pre-2.6 version, that would be great of
 course! But can it?


 Is it really necessary to retain compatibility with the Python 2.6
 version?  Python 2.6.0 and Python 2.6.1 contain a regression (as
 compared to basically all previous versions of Python) which prevents
 asyncore-based programs which are years old from working on them.
 Restoring the pre-2.6 behavior will fix these old, presumably stable,
 widely used programs for users who install 2.6.2 and newer.

 The downside (which you were imagining, I'm sure) is that any new
 software developed against the Python 2.6.0 or 2.6.1 behavior will
 then break in 2.6.2 and later.  While this is unfortunate, it is
 clearly the far lesser of two evils.  The choice must be made, though.
 Either leave old software broken or break new software.  Just because
 the leave old software broken choice is made through inaction doesn't
 make it the better choice (though obviously since it requires action,
 someone will have to do it, and I'm not volunteering - if inaction is
 the choice because no one wants to do the work, fine, but that's a
 different motivation than avoiding breaking newly written software).

 So, as a disinterested party in this specific case, I'd say revert to
 the pre-2.6 behavior.  It does less harm than leaving the current
 behavior.

Sorry, but I really do think that we should maintain backward
compatibility *within* the 2.6 series as well. If that makes it
impossible to also maintain the 2.5 behavior, perhaps some flag could
be added to restore 2.5 compatibility, e.g.

import asyncore
asyncore.python_25_compat = True

Note that this API is designed to work in 2.5 as well. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Jean-Paul Calderone

On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum gu...@python.org wrote:

On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone exar...@divmod.com wrote:
[snip]


So, as a disinterested party in this specific case, I'd say revert to
the pre-2.6 behavior.  It does less harm than leaving the current
behavior.


Sorry, but I really do think that we should maintain backward
compatibility *within* the 2.6 series as well. If that makes it
impossible to also maintain the 2.5 behavior, perhaps some flag could
be added to restore 2.5 compatibility, e.g.

import asyncore
asyncore.python_25_compat = True

Note that this API is designed to work in 2.5 as well. :-)



But why?  The argument I made had the objective of minimizing developer
effort.  What's the objective of maintaining backward compatibility within
the 2.6 series in this case (sorry if it appeared earlier in this thread
and I missed it)?

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Guido van Rossum
On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone exar...@divmod.com wrote:
 On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum gu...@python.org
 wrote:

 On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone exar...@divmod.com
 wrote:
 [snip]

 So, as a disinterested party in this specific case, I'd say revert to
 the pre-2.6 behavior.  It does less harm than leaving the current
 behavior.

 Sorry, but I really do think that we should maintain backward
 compatibility *within* the 2.6 series as well. If that makes it
 impossible to also maintain the 2.5 behavior, perhaps some flag could
 be added to restore 2.5 compatibility, e.g.

 import asyncore
 asyncore.python_25_compat = True

 Note that this API is designed to work in 2.5 as well. :-)

 But why?  The argument I made had the objective of minimizing developer
 effort.  What's the objective of maintaining backward compatibility within
 the 2.6 series in this case (sorry if it appeared earlier in this thread
 and I missed it)?

The same as always. We don't change APIs in bugfix releases.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Jean-Paul Calderone

On Wed, 4 Mar 2009 10:54:19 -0800, Guido van Rossum gu...@python.org wrote:

On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone exar...@divmod.com wrote:

On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum gu...@python.org
wrote:


On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone exar...@divmod.com
wrote:
[snip]


So, as a disinterested party in this specific case, I'd say revert to
the pre-2.6 behavior.  It does less harm than leaving the current
behavior.


Sorry, but I really do think that we should maintain backward
compatibility *within* the 2.6 series as well. If that makes it
impossible to also maintain the 2.5 behavior, perhaps some flag could
be added to restore 2.5 compatibility, e.g.

import asyncore
asyncore.python_25_compat = True

Note that this API is designed to work in 2.5 as well. :-)



But why?  The argument I made had the objective of minimizing developer
effort.  What's the objective of maintaining backward compatibility within
the 2.6 series in this case (sorry if it appeared earlier in this thread
and I missed it)?


The same as always. We don't change APIs in bugfix releases.



Okay.  Thanks for explaining.

Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Steve Holden
Guido van Rossum wrote:
 On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone exar...@divmod.com 
 wrote:
 On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum gu...@python.org
 wrote:
 On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone exar...@divmod.com
 wrote:
 [snip]
 So, as a disinterested party in this specific case, I'd say revert to
 the pre-2.6 behavior.  It does less harm than leaving the current
 behavior.
 Sorry, but I really do think that we should maintain backward
 compatibility *within* the 2.6 series as well. If that makes it
 impossible to also maintain the 2.5 behavior, perhaps some flag could
 be added to restore 2.5 compatibility, e.g.

 import asyncore
 asyncore.python_25_compat = True

 Note that this API is designed to work in 2.5 as well. :-)
 
 But why?  The argument I made had the objective of minimizing developer
 effort.  What's the objective of maintaining backward compatibility within
 the 2.6 series in this case (sorry if it appeared earlier in this thread
 and I missed it)?
 
 The same as always. We don't change APIs in bugfix releases.
 
Which unfortunately stops us from categorizing API breakages as bugs.

regards
 Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread glyph


On 06:51 pm, exar...@divmod.com wrote:
On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum gu...@python.org 
wrote:
On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone 
exar...@divmod.com wrote:



So, as a disinterested party in this specific case, I'd say revert to
the pre-2.6 behavior.



Sorry, but I really do think that we should maintain backward
compatibility *within* the 2.6 series as well.



But why?  The argument I made had the objective of minimizing developer
effort.  What's the objective of maintaining backward compatibility 
within
the 2.6 series in this case (sorry if it appeared earlier in this 
thread

and I missed it)?


Python's compatibility policy dictates that compatibility between micro- 
version releases be maintained *extremely* carefully.


As you all well know, normally I'd be strongly in favor of adhering to 
such a policy.  However, the ostensible purpose of the policy here is 
that users can *always* install micro-version upgrades to get bugfixes 
and security fixes, without worrying about compatibility upgrades.


Maintaining compatibility with the 2.6.x version of asyncore presupposes 
that *someone* has written some software against that version of 
asyncore and it might break if they installed an upgrade, though.  If 
that's the case - if there's even one person who has written or run any 
asyncore software against the version in 2.6 - then I think maintaining 
bug-for-bug compatibility is important.


However, my guess - and I'm assuming that JP was thinking the same thing 
- is that literally nobody has done that, or even *would* ever do that, 
so there's no software anywhere in the world that would break.  asyncore 
is relatively unpopular (thanks in part to the excellent alternatives 
:-)); its major users are the same people who have already complained.


Maybe it's too late to do something like this for 2.6, or even 3.0, but 
I've thought on a few occasions that projects (both Python and Twisted, 
at least) could use a special very-low-traffic policy-deviations mailing 
list for We *REALLY* don't think this is going to ever break anything 
anyone has actually done, but just to be sure... notifications for 
situations like this.  The implications being that (A) _everyone_ who 
uses the software should subscribe, and (B) if anyone ever responded to 
a message, the deviation wouldn't take place.


Anyway, I'm also a disinterested observer, so please don't take this as 
a strong endorsement of doing the fix in 2.6; the set-an-attribute-to- 
fix-it idea is fine by me too.  I thought I'd lay out the reasoning for 
violating an otherwise extremely reasonable policy, though.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Barry Warsaw

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On Mar 4, 2009, at 2:44 PM, gl...@divmod.com wrote:

Maintaining compatibility with the 2.6.x version of asyncore  
presupposes that *someone* has written some software against that  
version of asyncore and it might break if they installed an upgrade,  
though.  If that's the case - if there's even one person who has  
written or run any asyncore software against the version in 2.6 -  
then I think maintaining bug-for-bug compatibility is important.


However, my guess - and I'm assuming that JP was thinking the same  
thing - is that literally nobody has done that, or even *would* ever  
do that, so there's no software anywhere in the world that would  
break.  asyncore is relatively unpopular (thanks in part to the  
excellent alternatives :-)); its major users are the same people who  
have already complained.


FWIW, I use smtpd.py and a few of the asyncore APIs  
(.loop(), .socket_map.clear(), and .close_all()) in the Mailman 3 test  
suite.  That only works on Python 2.6 and I don't recall even a hiccup  
when moving from 2.5 to 2.6.


Barry

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSa7kiXEjvBPtnXfVAQI2BQP/XD71ipSv5OhtdOJt+OF2jbfOyFU/2vPR
iJ33g9JJ2A0CvDWXmc81t9hMH3U8waagkXaDUF4r2MNM3JbONSjQGGZqX6X/1UOB
XCT+jNOrbRw2VQR0qjrweQ5A8u0Y1zsDo/rD4CXe0K1pFFWLubaU3lVgqEBMBL8r
xvf77EMoUuA=
=vmM5
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Bill Janssen
Guido van Rossum gu...@python.org wrote:

 On Wed, Mar 4, 2009 at 10:51 AM, Jean-Paul Calderone exar...@divmod.com 
 wrote:
  On Wed, 4 Mar 2009 10:46:28 -0800, Guido van Rossum gu...@python.org
  wrote:
 
  On Wed, Mar 4, 2009 at 10:27 AM, Jean-Paul Calderone exar...@divmod.com
  wrote:
  [snip]
 
  So, as a disinterested party in this specific case, I'd say revert to
  the pre-2.6 behavior.  It does less harm than leaving the current
  behavior.
 
  Sorry, but I really do think that we should maintain backward
  compatibility *within* the 2.6 series as well. If that makes it
  impossible to also maintain the 2.5 behavior, perhaps some flag could
  be added to restore 2.5 compatibility, e.g.
 
  import asyncore
  asyncore.python_25_compat = True
 
  Note that this API is designed to work in 2.5 as well. :-)
 
  But why?  The argument I made had the objective of minimizing developer
  effort.  What's the objective of maintaining backward compatibility within
  the 2.6 series in this case (sorry if it appeared earlier in this thread
  and I missed it)?
 
 The same as always. We don't change APIs in bugfix releases.

OK, seems reasonable.

But in this case, isn't the broken API the bug that's being fixed?  Do
we need a different way to fix broken APIs in bugfix releases?

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Daniel Stutzbach
On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum gu...@python.org wrote:

 The same as always. We don't change APIs in bugfix releases.


This question is actually for the Zope folks and others who have had
problems with the 2.6 asyncore/asynchat:

Are any of the problems due to a change in the documented API... or are they
all due to changes in undocumented internals that your code relied on?

Myself, the only change that I noticed going from 2.5 to 2.6 was that the
undocumented signature of asynchat.__init__() changed from taking a
parameter called conn to one with the same meaning called sock.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Ivan Krstić

Glyph,

On Mar 4, 2009, at 12:31 AM, gl...@divmod.com wrote:
If someone who has PEP-writing skills and some free time is updating  
asyncore to be more modern, I'd be very happy to walk that person  
through Twisted's API design and explain how either portions of our  
implementation could be lifted for the stdlib, or how a high level  
layer could be written into asyncore so that the author of a  
particular chunk of low-level networking code could ignore whether  
they're using the stdlib mainloop or one of the Twisted ones.


I spent about a half hour sometime in the last month talking this  
through with Itamar, though not in great detail. I'd be interested in  
sitting down with both of you and trying to establish more precisely  
how much work is necessary to get something to actually happen here. I  
won't outright promise a PEP, but I'll promise at the very least to  
write down elaborate notes that someone could turn into a PEP  
relatively straightforwardly. Deal?


Cheers,

--
Ivan Krstić krs...@solarsail.hcs.harvard.edu | http://radian.org

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Chris McDonough
Daniel Stutzbach wrote:
 On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum gu...@python.org wrote:
 
 The same as always. We don't change APIs in bugfix releases.

 
 This question is actually for the Zope folks and others who have had
 problems with the 2.6 asyncore/asynchat:
 
 Are any of the problems due to a change in the documented API... or are they
 all due to changes in undocumented internals that your code relied on?

As far as I can tell, asyncore/asynchat is all undocumented internals.  Any
use of asyncore in anger will use internals; there never was any well-understood
API to these modules.  Medusa itself (from which asyncore and asynchat were
derived) appears to itself break with the changes to asyncore/asynchat in 2.6
(at least it appears to use attributes like ac_out_buffer which were removed
in 2.6; this is not Zope's version; there is no such animal; this is plain old
Medusa 0.5.4).

Count me in as one who believes that it would be the lesser of two evils to
revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2
rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost
certainly has fewer, if any, consumers); somebody messed up in 2.6 by making its
asyncore/asynchat more forward compatible with 3.0's than backwards compatible
with 2.5's and prior; this was just a mistake and keeping old code running
should trump any theoretical or puritanical benefit in a dot release.

- C

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Guido van Rossum
On Wed, Mar 4, 2009 at 1:27 PM, Chris McDonough chr...@plope.com wrote:
 Daniel Stutzbach wrote:
 On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum gu...@python.org wrote:

 The same as always. We don't change APIs in bugfix releases.


 This question is actually for the Zope folks and others who have had
 problems with the 2.6 asyncore/asynchat:

 Are any of the problems due to a change in the documented API... or are they
 all due to changes in undocumented internals that your code relied on?

 As far as I can tell, asyncore/asynchat is all undocumented internals.  Any
 use of asyncore in anger will use internals; there never was any 
 well-understood
 API to these modules.  Medusa itself (from which asyncore and asynchat were
 derived) appears to itself break with the changes to asyncore/asynchat in 2.6
 (at least it appears to use attributes like ac_out_buffer which were removed
 in 2.6; this is not Zope's version; there is no such animal; this is plain 
 old
 Medusa 0.5.4).

 Count me in as one who believes that it would be the lesser of two evils to
 revert to the older (2.5 and prior) asyncore/asynchat implementations in 2.6.2
 rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost
 certainly has fewer, if any, consumers); somebody messed up in 2.6 by making 
 its
 asyncore/asynchat more forward compatible with 3.0's than backwards compatible
 with 2.5's and prior; this was just a mistake and keeping old code running
 should trump any theoretical or puritanical benefit in a dot release.

Then I'd like to hear from the folks who made and reviewed those
changes to asyncore in 2.6. I can't imagine the changes were done
without good intentions.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Daniel Stutzbach
On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krstić 
krs...@solarsail.hcs.harvard.edu wrote:

 I spent about a half hour sometime in the last month talking this through
 with Itamar, though not in great detail. I'd be interested in sitting down
 with both of you and trying to establish more precisely how much work is
 necessary to get something to actually happen here. I won't outright promise
 a PEP, but I'll promise at the very least to write down elaborate notes that
 someone could turn into a PEP relatively straightforwardly. Deal?


Will any or all of you be at PyCon?  I'd be willing to put in the extra work
to turn your notes into a PEP.

--
Daniel Stutzbach, Ph.D.
President, Stutzbach Enterprises, LLC http://stutzbachenterprises.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Steve Holden
Daniel Stutzbach wrote:
 On Wed, Mar 4, 2009 at 3:14 PM, Ivan Krstić
 krs...@solarsail.hcs.harvard.edu
 mailto:krs...@solarsail.hcs.harvard.edu wrote:
 
 I spent about a half hour sometime in the last month talking this
 through with Itamar, though not in great detail. I'd be interested
 in sitting down with both of you and trying to establish more
 precisely how much work is necessary to get something to actually
 happen here. I won't outright promise a PEP, but I'll promise at the
 very least to write down elaborate notes that someone could turn
 into a PEP relatively straightforwardly. Deal?
 
 
 Will any or all of you be at PyCon?  I'd be willing to put in the extra
 work to turn your notes into a PEP.
 
OPEN SPACE!
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC  http://www.holdenweb.com/

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-04 Thread Josiah Carlson
On Wed, Mar 4, 2009 at 1:31 PM, Guido van Rossum gu...@python.org wrote:
 On Wed, Mar 4, 2009 at 1:27 PM, Chris McDonough chr...@plope.com wrote:
 Daniel Stutzbach wrote:
 On Wed, Mar 4, 2009 at 12:54 PM, Guido van Rossum gu...@python.org wrote:

 The same as always. We don't change APIs in bugfix releases.


 This question is actually for the Zope folks and others who have had
 problems with the 2.6 asyncore/asynchat:

 Are any of the problems due to a change in the documented API... or are they
 all due to changes in undocumented internals that your code relied on?

 As far as I can tell, asyncore/asynchat is all undocumented internals.  Any
 use of asyncore in anger will use internals; there never was any 
 well-understood
 API to these modules.  Medusa itself (from which asyncore and asynchat were
 derived) appears to itself break with the changes to asyncore/asynchat in 2.6
 (at least it appears to use attributes like ac_out_buffer which were 
 removed
 in 2.6; this is not Zope's version; there is no such animal; this is plain 
 old
 Medusa 0.5.4).

 Count me in as one who believes that it would be the lesser of two evils to
 revert to the older (2.5 and prior) asyncore/asynchat implementations in 
 2.6.2
 rather than soldiering on with the 2.6 and 2.6.1 implementation (which almost
 certainly has fewer, if any, consumers); somebody messed up in 2.6 by making 
 its
 asyncore/asynchat more forward compatible with 3.0's than backwards 
 compatible
 with 2.5's and prior; this was just a mistake and keeping old code running
 should trump any theoretical or puritanical benefit in a dot release.

 Then I'd like to hear from the folks who made and reviewed those
 changes to asyncore in 2.6. I can't imagine the changes were done
 without good intentions.

Interestingly enough, I just accidentally found this thread with an
unrelated web search, but it makes sense to reply.

Why the changes to asyncore:
1) the internals of asyncore were inefficient
2) the API and general behavior of the handle_*(), readable() and
writable() methods are unchanged, which are *exactly* what a user
should be overriding with subclassing
3) anyone modifying the *internals* of the implementation for the sake
of speed, shouldn't need to anymore

Maybe I have misread the posts in this thread, but does anyone have
any examples of their code breaking when only overriding handle_*(),
readable(), and writable() methods?

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-03 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Tres Seaver wrote:
 Guido van Rossum wrote:
 On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen jans...@parc.com wrote:
 Josiah Carlson josiah.carl...@gmail.com wrote:

 But yes, zope needs to be changed to reflect the updated
 asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
 easier to use now.
 Just for completeness, I built a fresh 2.6, installed Medusa (from
 http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
 as well as it does on 2.5, anyway).  I think this is a Zope issue.
 Way back in the day, Zope monkeypatched whole parts of asyncore,
 replacing them with some of its own code. If that's still the case,
 this breakage should be no surprise.
 
 Zope has been hooking the 'asyncore.loop' function (wrapping it in order
 to add a clean shutdown' flog) since 2001 at least (the 2.5 branch is
 the earliest checkout I have, and it was branched in early January 2002).
 
 Zope also began patched asyncore's logging functions in 2.7, and later
 updated that patch to make the logger use the stdlib 'logging' module.
 
 I think the change we need to make (in ZServer/medusa/http_server.py) is
 to emulate the new 'writeable' implementation in asynchat, at least when
 running under 2.6.
 
 Zope's 'ZServer.medusa.http_server.http_request.writable()'
 implementation is::
 
 def writable (self):
 # this is just the normal async_chat 'writable',
 # here for comparison
 return self.ac_out_buffer or len(self.producer_fifo)
 
 
 The Python 2.5 asynchat.asynchat.writable does::
 
 def writable (self):
 predicate for inclusion in the writable for select()
 # return len(self.ac_out_buffer) or len(self.producer_fifo) or
 # (not self.connected)
 # this is about twice as fast, though not as clear.
 return not (
 (self.ac_out_buffer == '') and
 self.producer_fifo.is_empty() and
 self.connected
 )
 
 The Python 2.6 asynchat.asynchat.writable now does::
 
 def writable (self):
 predicate for inclusion in the writable for select()
 return self.producer_fifo or (not self.connected)
 
 
 medusa 0.5.4's medusa.http_server.http_request doesn't override
 'writable', but it does have a broken 'writeable_for_proxy':
 
 
 def writable_for_proxy (self):
 # this version of writable supports the idea of a 'stalled'
 # producer
 # [i.e., it's not ready to produce any output yet] This is
 # needed by
 # the proxy, which will be waiting for the magic combination of
 # 1) hostname resolved
 # 2) connection made
 # 3) data available.
 if self.ac_out_buffer:
 return 1
 elif len(self.producer_fifo):
 p = self.producer_fifo.first()
 if hasattr (p, 'stalled'):
 return not p.stalled()
 else:
 return 1

*crickets*.

The 2.6 version of asyncore *also* breaks supervisord, which does *not*
use Zope's medusa:  it uses medusa 0.5.4.  How is anybody supposed to
write a package which sits atop a library like asyncore in a fashion
portable across Python versions?  The changes to the implementation in
2.6 (there is no real API) can't be reconciled, AFAICT.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJrZG0+gerLs4ltQ4RAnsmAJ9v/vPkHgE3AdP5ngVuYaKlxDGhJACgsCi2
3awbUffi2BU41qQgd6eJV18=
=WBt6
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-03 Thread Guido van Rossum
On Tue, Mar 3, 2009 at 12:23 PM, Tres Seaver tsea...@palladion.com wrote:
 How is anybody supposed to
 write a package which sits atop a library like asyncore in a fashion
 portable across Python versions?  The changes to the implementation in
 2.6 (there is no real API) can't be reconciled, AFAICT.

This seems to be the crux of the problem with asyncore, ever since it
was added to the stdlib -- there's no real API, so every change
potentially breaks something. I wish we could start over with a proper
design under a new name. Maybe packages requiring asyncore
functionality should just copy the version of asyncore they like into
their own three and stick with that.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-03 Thread glyph


On 08:46 pm, gu...@python.org wrote:

This seems to be the crux of the problem with asyncore, ever since it
was added to the stdlib -- there's no real API, so every change
potentially breaks something. I wish we could start over with a proper
design under a new name.


Might I suggest reactor... or possibly twisted, as that new name? 
;-)


(Sorry, I was trying to avoid this thread, but that was an opening I 
could drive a truck through).


In all seriousness, I seem to recall that Thomas Wouters was interested 
in doing integrating some portion of Twisted core into the standard 
library as of last PyCon.  I mention him specifically by name in the 
hopes that it will jog his memory.


At the very least, this might serve as a basis for an abstract API for 
asyncore:


http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-03 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Guido van Rossum wrote:
 On Tue, Mar 3, 2009 at 12:23 PM, Tres Seaver tsea...@palladion.com wrote:
 How is anybody supposed to
 write a package which sits atop a library like asyncore in a fashion
 portable across Python versions?  The changes to the implementation in
 2.6 (there is no real API) can't be reconciled, AFAICT.
 
 This seems to be the crux of the problem with asyncore, ever since it
 was added to the stdlib -- there's no real API, so every change
 potentially breaks something. I wish we could start over with a proper
 design under a new name. Maybe packages requiring asyncore
 functionality should just copy the version of asyncore they like into
 their own three and stick with that.

That was the very solution Chris came up with earlier today:  stick a
fork in it, its done!


Tres.
- --
===
Tres Seaver  +1 540-429-0999  tsea...@palladion.com
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJraty+gerLs4ltQ4RAjBSAJ4niecZJusKY4XiioJ18mdhdMixxQCfWvcQ
Dwkh1ZBuxtGRbhUI4qy96Sc=
=ms0I
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-03 Thread Guido van Rossum
On Tue, Mar 3, 2009 at 1:17 PM,  gl...@divmod.com wrote:

 On 08:46 pm, gu...@python.org wrote:

 This seems to be the crux of the problem with asyncore, ever since it
 was added to the stdlib -- there's no real API, so every change
 potentially breaks something. I wish we could start over with a proper
 design under a new name.

 Might I suggest reactor... or possibly twisted, as that new name? ;-)

 (Sorry, I was trying to avoid this thread, but that was an opening I could
 drive a truck through).

 In all seriousness, I seem to recall that Thomas Wouters was interested in
 doing integrating some portion of Twisted core into the standard library as
 of last PyCon.  I mention him specifically by name in the hopes that it will
 jog his memory.

 At the very least, this might serve as a basis for an abstract API for
 asyncore:

 http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html

I hope we have learned from asyncore that a future-proof and maximally
flexible design of such an API is not easy. This is one of those cases
where I believe thinking hard up front will really pay off. In this
case I would recommend going through a full-blown PEP process before
committing to an API, to avoid making the same mistake twice. (I don't
mean this as a dig at Twisted, just as an observation about this
particular design problem. If Twisted solves all the problems already,
writing the PEP should be easy.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2009-03-03 Thread glyph


On 3 Mar, 10:20 pm, gu...@python.org wrote:

On Tue, Mar 3, 2009 at 1:17 PM,  gl...@divmod.com wrote:



At the very least, this might serve as a basis for an abstract API for
asyncore:

http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.interfaces.IProtocol.html



I hope we have learned from asyncore that a future-proof and maximally
flexible design of such an API is not easy. This is one of those cases
where I believe thinking hard up front will really pay off. In this
case I would recommend going through a full-blown PEP process before
committing to an API, to avoid making the same mistake twice. (I don't
mean this as a dig at Twisted, just as an observation about this
particular design problem. If Twisted solves all the problems already,
writing the PEP should be easy.)


I didn't see it as a dig at Twisted; you're right, this is a hard 
problem.  Twisted doesn't solve all the problems perfectly, either; much 
has been made of some rather unfortunate nuances of the IProducer and 
IConsumer interfaces.


I also regret that no Twisted people (myself included) seem to have the 
time to go through the PEP process and get something accepted.


Even if Twisted's interfaces aren't perfect, there's a lot of working 
application code which demonstrates that they are sufficient.  Perhaps 
more importantly, they are future-proof.  Every proposal we've come up 
with for addressing the more subtle shortcomings of these APIs has been 
completely backwards-compatible; there are no APIs with vague or broken 
semantics, just sub-optimal spellings.  The fact that there are already 
multiple reactor implementations which share very little code helps.


If someone who has PEP-writing skills and some free time is updating 
asyncore to be more modern, I'd be very happy to walk that person 
through Twisted's API design and explain how either portions of our 
implementation could be lifted for the stdlib, or how a high level 
layer could be written into asyncore so that the author of a particular 
chunk of low-level networking code could ignore whether they're using 
the stdlib mainloop or one of the Twisted ones.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2008-10-09 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Guido van Rossum wrote:
 On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen [EMAIL PROTECTED] wrote:
 Josiah Carlson [EMAIL PROTECTED] wrote:

 But yes, zope needs to be changed to reflect the updated
 asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
 easier to use now.
 Just for completeness, I built a fresh 2.6, installed Medusa (from
 http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
 as well as it does on 2.5, anyway).  I think this is a Zope issue.
 
 Way back in the day, Zope monkeypatched whole parts of asyncore,
 replacing them with some of its own code. If that's still the case,
 this breakage should be no surprise.

Zope has been hooking the 'asyncore.loop' function (wrapping it in order
to add a clean shutdown' flog) since 2001 at least (the 2.5 branch is
the earliest checkout I have, and it was branched in early January 2002).

Zope also began patched asyncore's logging functions in 2.7, and later
updated that patch to make the logger use the stdlib 'logging' module.

I think the change we need to make (in ZServer/medusa/http_server.py) is
to emulate the new 'writeable' implementation in asynchat, at least when
running under 2.6.

Zope's 'ZServer.medusa.http_server.http_request.writable()'
implementation is::

def writable (self):
# this is just the normal async_chat 'writable',
# here for comparison
return self.ac_out_buffer or len(self.producer_fifo)


The Python 2.5 asynchat.asynchat.writable does::

def writable (self):
predicate for inclusion in the writable for select()
# return len(self.ac_out_buffer) or len(self.producer_fifo) or
# (not self.connected)
# this is about twice as fast, though not as clear.
return not (
(self.ac_out_buffer == '') and
self.producer_fifo.is_empty() and
self.connected
)

The Python 2.6 asynchat.asynchat.writable now does::

def writable (self):
predicate for inclusion in the writable for select()
return self.producer_fifo or (not self.connected)


medusa 0.5.4's medusa.http_server.http_request doesn't override
'writable', but it does have a broken 'writeable_for_proxy':


def writable_for_proxy (self):
# this version of writable supports the idea of a 'stalled'
# producer
# [i.e., it's not ready to produce any output yet] This is
# needed by
# the proxy, which will be waiting for the magic combination of
# 1) hostname resolved
# 2) connection made
# 3) data available.
if self.ac_out_buffer:
return 1
elif len(self.producer_fifo):
p = self.producer_fifo.first()
if hasattr (p, 'stalled'):
return not p.stalled()
else:
return 1


Tres.
- --
===
Tres Seaver  +1 540-429-0999  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFI7ix3+gerLs4ltQ4RAldnAKC/QLJHmdE9dxInkWuIGja0gtSXYwCcCJcH
6XooEwW/AkJ1ntmGyxi8urM=
=1kps
-END PGP SIGNATURE-

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2008-10-08 Thread Bill Janssen
Sidnei da Silva [EMAIL PROTECTED] wrote:

 https://bugs.edge.launchpad.net/zope2/+bug/280020

I think there are real issues here with both asynchat and Medusa.
Asynchat has been heavily re-written, and the ac_out_buffer has
apparently disappeared.  But ac_out_buffer_size is still there.  That
strikes me as odd, and probably means that asynchat.py needs more
changes.  However, Medusa (basically just an application layer on top
of asyncore/asynchat) also needs to be re-written to take account of
the changes in asynchat.

Bill

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2008-10-08 Thread Josiah Carlson
On Wed, Oct 8, 2008 at 10:30 AM, Bill Janssen [EMAIL PROTECTED] wrote:
 Sidnei da Silva [EMAIL PROTECTED] wrote:

 https://bugs.edge.launchpad.net/zope2/+bug/280020

 I think there are real issues here with both asynchat and Medusa.
 Asynchat has been heavily re-written, and the ac_out_buffer has
 apparently disappeared.  But ac_out_buffer_size is still there.  That
 strikes me as odd, and probably means that asynchat.py needs more
 changes.  However, Medusa (basically just an application layer on top
 of asyncore/asynchat) also needs to be re-written to take account of
 the changes in asynchat.

ac_out_buffer was removed because it is unneeded (we have a deque; why
rely on an extra attribute?).
ac_out_buffer_size still remains as a blocksize in which to pre-split
outgoing data (if you have a 100k string you want to send, repeatedly
slicing it as you are able to send pieces is slow, but pre-slicing
chunks is fast, and generally results in being able to pass full chunk
to the underlying TCP/IP stack).

But yes, zope needs to be changed to reflect the updated
asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
easier to use now.

 - Josiah
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2008-10-08 Thread Sidnei da Silva
On Wed, Oct 8, 2008 at 6:26 PM, Josiah Carlson [EMAIL PROTECTED] wrote:
 ac_out_buffer was removed because it is unneeded (we have a deque; why
 rely on an extra attribute?).
 ac_out_buffer_size still remains as a blocksize in which to pre-split
 outgoing data (if you have a 100k string you want to send, repeatedly
 slicing it as you are able to send pieces is slow, but pre-slicing
 chunks is fast, and generally results in being able to pass full chunk
 to the underlying TCP/IP stack).

 But yes, zope needs to be changed to reflect the updated
 asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
 easier to use now.

Great to know. So medusa should be changed similarly to the changes
made to asynchat? Your suggestion on the bugtracker is even better: to
subclass asynchat as much as possible.

-- 
Sidnei da Silva
Enfold Systemshttp://enfoldsystems.com
Fax +1 832 201 8856 Office +1 713 942 2377 Ext 214
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2008-10-08 Thread Bill Janssen
Sidnei da Silva [EMAIL PROTECTED] wrote:

 Great to know. So medusa should be changed similarly to the changes
 made to asynchat?

Hmmm.  Been a long time since a Medusa release, but I need a working
Medusa, too.

 Your suggestion on the bugtracker is even better: to
 subclass asynchat as much as possible.

That's basically what Medusa is, a set of subclasses.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2008-10-08 Thread Bill Janssen
Josiah Carlson [EMAIL PROTECTED] wrote:

 But yes, zope needs to be changed to reflect the updated
 asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
 easier to use now.

Just for completeness, I built a fresh 2.6, installed Medusa (from
http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
as well as it does on 2.5, anyway).  I think this is a Zope issue.

Bill
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] asyncore fixes in Python 2.6 broke Zope's version of medusa

2008-10-08 Thread Guido van Rossum
On Wed, Oct 8, 2008 at 6:39 PM, Bill Janssen [EMAIL PROTECTED] wrote:
 Josiah Carlson [EMAIL PROTECTED] wrote:

 But yes, zope needs to be changed to reflect the updated
 asyncore/asynchat semantics.  Trust me; it's faster, cleaner, and
 easier to use now.

 Just for completeness, I built a fresh 2.6, installed Medusa (from
 http://www.amk.ca/python/code/medusa.html), and it runs just fine (well,
 as well as it does on 2.5, anyway).  I think this is a Zope issue.

Way back in the day, Zope monkeypatched whole parts of asyncore,
replacing them with some of its own code. If that's still the case,
this breakage should be no surprise.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com