Re: [Python-Dev] Support for async read/write

2011-02-04 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 20/10/10 00:58, Antoine Pitrou wrote:
 It would be nice to know about the use case Jesus has in mind.

I am thinking about the cases where a request come, you do some disk
read in behalf of it and you reply.

If the read is slow (if not cached, you have to deal with a physical
harddisk), you stop the main-loop for a while, unless you use threads.

If, instead, I can schedule a read and keep processing other requests
(possibly queueing new reads), be notified when the read is done and
complete the reply, I think this is more simple and performing (and far
less memory hungry) that using threads.

I just opened a issue, assigned to me. I plan to do the implementation
myself, at least for Solaris and possibly (recent) Linux:
http://bugs.python.o rg/issue7.

Some people say AIO OS implementations are flaky. Well, you must deal
with it, like you deal with other flaky OS corners, like exhausting
memory or whatever.

I would suppose that if Python exposes AIO and in some OS support is
underpar, the exposing would incentivate a better support for OS. Like
happened to threads and linux years ago.

- -- 
Jesus Cea Avion _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
Things are not so easy  _/_/  _/_/_/_/  _/_/_/_/  _/_/
My name is Dump, Core Dump   _/_/_/_/_/_/  _/_/  _/_/
El amor es poner tu felicidad en la felicidad de otro - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBTUwbeZlgi5GaxT1NAQK4uwP+J3PwGY1dvBi+OyBo9M9UWsma0khgzdUS
6ewHhCrCK+U5HK0e/g9cLbesBSsYvVfNjPe+fb9cQuwMBK0lpF3kOzfsEf82RIxR
NlsqOba31CE1tW9uS4wja0TddFDob3IImqgwSB7NptBOZTNVjDvB6k0V7KHqvPWX
9g01GqaQ9uQ=
=hx6M
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-20 Thread Glyph Lefkowitz

On Oct 19, 2010, at 9:55 PM, exar...@twistedmatrix.com wrote:

 Not only is the performance usually worse than expected, the behavior of 
 aio_* functions require all kinds of subtle and mysterious coordination with 
 signal handling, which I'm not entirely sure Python would even be able to 
 pull off without some modifications to the signal module.  (And, as 
 Jean-Paul mentioned, if your OS kernel runs out of space in a queue 
 somewhere, completion notifications might just never be delivered at all.)
 
 Just to be clear, James corrected me there.  I thought Jesus was talking 
 about the mostly useless Linux AIO APIs, which have the problems I described. 
  He was actually talking about the POSIX AIO APIs, which have a different set 
 of problems making them a waste of time.

I know, I'm referring to the behavior of POSIX AIO.

Perhaps I'm overstating the case with 'subtle and mysterious', then, but the 
POISX 'aiocb' structure still includes an 'aio_sigevent' member which is the 
way to find out about I/O event completion.  If you're writing an application 
that uses AIO, basically all of your logic ends up living in the context of a 
signal handler, and as 
http://www.opengroup.org/onlinepubs/95399/functions/xsh_chap02_04.html#tag_02_04_01
 puts it,

When signal-catching functions are invoked asynchronously with process 
execution, the behavior of some of the functions defined by this volume of IEEE 
Std 1003.1-2001 is unspecified if they are called from a signal-catching 
function.

Of course, you could try using signalfd(), but that's not in POSIX.

(Or, you could use SIGEV_THREAD, but that would be functionally equivalent to 
running read() in a thread, except much more difficult.)

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-20 Thread Glyph Lefkowitz

On Oct 20, 2010, at 12:31 AM, Jeffrey Yasskin wrote:

 No comment on the rest of your claim, but this is a silly argument.
 The standard says the same thing about at least fcntl.h, signal.h,
 pthread.h, and ucontext.h, which clearly are useful.

It was meant to be tongue-in-cheek :).  Perhaps I should not have assumed that 
everyone else was as familiar with the POSIX documentation; I figured that most 
readers would know that most pages say that.

But, that was the result of a string of many different searches attempting to 
find someone explaining why this was a good idea or why anyone would want to 
use it.  I think in this case, it's accurate.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-20 Thread Jeffrey Yasskin
On Wed, Oct 20, 2010 at 2:55 PM, Glyph Lefkowitz
gl...@twistedmatrix.com wrote:

 On Oct 20, 2010, at 12:31 AM, Jeffrey Yasskin wrote:

 No comment on the rest of your claim, but this is a silly argument.
 The standard says the same thing about at least fcntl.h, signal.h,
 pthread.h, and ucontext.h, which clearly are useful.

 It was meant to be tongue-in-cheek :).  Perhaps I should not have assumed
 that everyone else was as familiar with the POSIX documentation; I figured
 that most readers would know that most pages say that.

Oops, sorry. My joke-detector failed.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


[Python-Dev] Support for async read/write

2010-10-19 Thread Jesus Cea
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Current Python lacks support for aio_* syscalls to do async IO. I
think this could be a nice addition for python 3.3.

If you agree, I will create an issue in the tracker. If you think the
idea is of no value, please say so for me to move on. Maybe an 3th party
module, but I think this functionality sould be available in core python.

Thanks!.

PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read,
aio_return, aio_write.

- -- 
Jesus Cea Avion _/_/  _/_/_/_/_/_/
j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
.  _/_/  _/_/_/_/  _/_/  _/_/
Things are not so easy  _/_/  _/_/_/_/  _/_/_/_/  _/_/
My name is Dump, Core Dump   _/_/_/_/_/_/  _/_/  _/_/
El amor es poner tu felicidad en la felicidad de otro - Leibniz
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQCVAwUBTL3MUplgi5GaxT1NAQKv5QQAnDan88kXJ67fucz2rT/mZze+065lm9E4
+XJ2JfqGMVE1/qMsXwg81l19RHSYReBgBjd7zyXWE9Fk/1Rfq4gjOZEtoG0IpGZG
E3CH+Ab5/o/PjJZNKQaPpe0cwGSXFPKkPpgepKS1d8ZXyf6VXMc8UTSWjMI5jIVe
4M+yvw5m4I0=
=nsdO
-END PGP SIGNATURE-
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread Peter Portante
Yes, that would be a nice addition. -peter


On 10/19/10 12:50 PM, Jesus Cea j...@jcea.es wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 Current Python lacks support for aio_* syscalls to do async IO. I
 think this could be a nice addition for python 3.3.
 
 If you agree, I will create an issue in the tracker. If you think the
 idea is of no value, please say so for me to move on. Maybe an 3th party
 module, but I think this functionality sould be available in core python.
 
 Thanks!.
 
 PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read,
 aio_return, aio_write.
 
 - -- 
 Jesus Cea Avion _/_/  _/_/_/_/_/_/
 j...@jcea.es - http://www.jcea.es/ _/_/_/_/  _/_/_/_/  _/_/
 jabber / xmpp:j...@jabber.org _/_/_/_/  _/_/_/_/_/
 .  _/_/  _/_/_/_/  _/_/  _/_/
 Things are not so easy  _/_/  _/_/_/_/  _/_/_/_/  _/_/
 My name is Dump, Core Dump   _/_/_/_/_/_/  _/_/  _/_/
 El amor es poner tu felicidad en la felicidad de otro - Leibniz
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.10 (GNU/Linux)
 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
 
 iQCVAwUBTL3MUplgi5GaxT1NAQKv5QQAnDan88kXJ67fucz2rT/mZze+065lm9E4
 +XJ2JfqGMVE1/qMsXwg81l19RHSYReBgBjd7zyXWE9Fk/1Rfq4gjOZEtoG0IpGZG
 E3CH+Ab5/o/PjJZNKQaPpe0cwGSXFPKkPpgepKS1d8ZXyf6VXMc8UTSWjMI5jIVe
 4M+yvw5m4I0=
 =nsdO
 -END PGP SIGNATURE-
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 http://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe: 
 http://mail.python.org/mailman/options/python-dev/peter.a.portante%40gmail.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread exarkun

On 04:50 pm, j...@jcea.es wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Current Python lacks support for aio_* syscalls to do async IO. I
think this could be a nice addition for python 3.3.


Adding more platform wrappers is always nice.  Keep in mind that the 
quality of most (all?) aio_* implementations is spotty at best, though. 
On Linux, they will sometimes block (for example, if you fail to align 
buffers properly, or open a file without O_DIRECT, or if there are too 
many other aio operations active on the system at the time, etc).  Thus, 
these APIs are finicky at best, and the Python bindings will be 
similarly fragile.


Jean-Paul

If you agree, I will create an issue in the tracker. If you think the
idea is of no value, please say so for me to move on.


Do you have an application in mind?

Maybe an 3th party
module, but I think this functionality sould be available in core 
python.


Starting off as a 3rd party library to try to develop some interest and 
users (and thus experience) before adding it to the standard library 
might make sense (as it frequently does).

Thanks!.

PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read,
aio_return, aio_write.


Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread Giampaolo Rodolà
You should  file a new issue on the bug tracker but unless you have a
patch to propose it's unlikely that someone else is gonna implement
it.


Regards

--- Giampaolo
http://code.google.com/p/pyftpdlib/
http://code.google.com/p/psutil/


2010/10/19 Jesus Cea j...@jcea.es:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Current Python lacks support for aio_* syscalls to do async IO. I
 think this could be a nice addition for python 3.3.

 If you agree, I will create an issue in the tracker. If you think the
 idea is of no value, please say so for me to move on. Maybe an 3th party
 module, but I think this functionality sould be available in core python.

 Thanks!.

 PS: The function calls are: aio_cancel, aio_error, aio_fsync, aio_read,
 aio_return, aio_write.

 - --
 Jesus Cea Avion                         _/_/      _/_/_/        _/_/_/
 j...@jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
 jabber / xmpp:j...@jabber.org         _/_/    _/_/          _/_/_/_/_/
 .                              _/_/  _/_/    _/_/          _/_/  _/_/
 Things are not so easy      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
 My name is Dump, Core Dump   _/_/_/        _/_/_/      _/_/  _/_/
 El amor es poner tu felicidad en la felicidad de otro - Leibniz
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.10 (GNU/Linux)
 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

 iQCVAwUBTL3MUplgi5GaxT1NAQKv5QQAnDan88kXJ67fucz2rT/mZze+065lm9E4
 +XJ2JfqGMVE1/qMsXwg81l19RHSYReBgBjd7zyXWE9Fk/1Rfq4gjOZEtoG0IpGZG
 E3CH+Ab5/o/PjJZNKQaPpe0cwGSXFPKkPpgepKS1d8ZXyf6VXMc8UTSWjMI5jIVe
 4M+yvw5m4I0=
 =nsdO
 -END PGP SIGNATURE-
 ___
 Python-Dev mailing list
 Python-Dev@python.org
 http://mail.python.org/mailman/listinfo/python-dev
 Unsubscribe: 
 http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread James Y Knight

On Oct 19, 2010, at 1:47 PM, exar...@twistedmatrix.com wrote:

 Adding more platform wrappers is always nice.  Keep in mind that the quality 
 of most (all?) aio_* implementations is spotty at best, though. On Linux, 
 they will sometimes block (for example, if you fail to align buffers 
 properly, or open a file without O_DIRECT, or if there are too many other aio 
 operations active on the system at the time, etc).  

You're thinking of the linux-specific AIO calls. Those have the properties 
you're describing (which makes them pretty useless for most code too), but 
they're completely different from the aio_* functions.

The POSIX aio_* calls don't do any of that. They aren't syscalls implemented in 
the kernel, they're implemented in glibc. They simply create a threadpool in 
your process to call the standard synchronous operations, and make it difficult 
to reliably get completion notification (completion notification takes place 
via Real-Time signals (SIGEV_SIGNAL), which can be dropped if linux runs out of 
space in its RT-signal-queue, and when that happens you get no indication that 
that has occurred. You can also do completion notification via calling a 
function on a thread (SIGEV_THREAD), but, for that, glibc will always spawns a 
brand new thread for each notification, which is quite slow.)

Basically: you shouldn't ever use those APIs. Especially on linux, but probably 
everywhere else. 

So, in conclusion, I disagree that adding wrappers for these would be nice. It 
wouldn't. It would cause some people to think they would be useful things to 
call, and they would always be wrong.

James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread Antoine Pitrou

Hello Jesus,

 Current Python lacks support for aio_* syscalls to do async IO. I
 think this could be a nice addition for python 3.3.
 
 If you agree, I will create an issue in the tracker. If you think the
 idea is of no value, please say so for me to move on. Maybe an 3th party
 module, but I think this functionality sould be available in core python.

Well, if you think this is interesting, it would be nice if you could
do a bit more research instead of just filing issues for us to solve.
After all, you probably already have an idea about why this is useful,
how to interface it into the stdlib, etc. Just saying we should expose
these functions doesn't bring a lot of added value IMO (there are many
system functions which could be exposed and currently aren't).

Also, the canonical way to do file I/O in Python 3 is the `io` lib,
therefore it would be a bit of a shame to have separate, non-integrated
`aio_*` functions.

Also, since the `io` lib is already supposed to support non-blocking
IO, perhaps it would be valuable to stress this support and propose any
interesting patches for fixing and/or improving it.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread Martin v. Löwis
Am 19.10.2010 19:47, schrieb exar...@twistedmatrix.com:
 On 04:50 pm, j...@jcea.es wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Current Python lacks support for aio_* syscalls to do async IO. I
 think this could be a nice addition for python 3.3.
 
 Adding more platform wrappers is always nice. 

Exactly so. Exposing platform APIs as-is really doesn't need much
discussion, except when the platform wrapper is going to deviate
significantly from the platform API. There is a long tradition of
exposing even obscure unixish APIs in Python, making it the #1 language
to write exotic scripts in. We definitely want to continue this
tradition.

Please refrain from trying to come up with abstractions, though;
this should be outside of Python for the moment. Covering both Unix
AIO and Windows completion ports (say) may be quite a challenge
(but then, JP may have some view on this).

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread Martin v. Löwis
 So, in conclusion, I disagree that adding wrappers for these would be
 nice. It wouldn't. It would cause some people to think they would be
 useful things to call, and they would always be wrong.

We are all consenting adults. If people want to shoot themselves in
their feet, we let them. For example, we have os.open, even though
there is no garbage collection for file handles, and we have
os._exit, even though it doesn't call finalizers.

People should trust that the APIs we expose go *literally* into the
library, so we can blame any consequences that this has onto the
library (and possibly the kernel, in turn).
People readily accept that explanation.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread Martin v. Löwis
 Also, the canonical way to do file I/O in Python 3 is the `io` lib,
 therefore it would be a bit of a shame to have separate, non-integrated
 `aio_*` functions.

I disagree. We also have posix.open, posix.dup, etc. We always expose
POSIX functions in the posix module (except for the socket functions,
unfortunately, and a few other exceptions), and I see no reason to break
with this tradition. The io module should be thought of as sitting
on top of the posix module (and it initially was).

 Also, since the `io` lib is already supposed to support non-blocking
 IO, perhaps it would be valuable to stress this support and propose any
 interesting patches for fixing and/or improving it.

I think that's entirely independent.

Regards,
Martin
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread Antoine Pitrou
Le mercredi 20 octobre 2010 à 00:48 +0200, Martin v. Löwis a écrit :
  Also, the canonical way to do file I/O in Python 3 is the `io` lib,
  therefore it would be a bit of a shame to have separate, non-integrated
  `aio_*` functions.
 
 I disagree. We also have posix.open, posix.dup, etc. We always expose
 POSIX functions in the posix module (except for the socket functions,
 unfortunately, and a few other exceptions), and I see no reason to break
 with this tradition. The io module should be thought of as sitting
 on top of the posix module (and it initially was).

I'm not suggesting to break with any tradition, but that building
duplicate APIs to do file I/O has little value and only makes things
more cumbersome.

  Also, since the `io` lib is already supposed to support non-blocking
  IO, perhaps it would be valuable to stress this support and propose any
  interesting patches for fixing and/or improving it.
 
 I think that's entirely independent.

I don't think it is. If the goal is to give ways to improve file I/O
performance for specialized use cases, then it certainly deserves
integration with the current I/O stack. If the goal is not that, then I
don't really know what it is.

It would be nice to know about the use case Jesus has in mind.

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread James Y Knight
On Oct 19, 2010, at 6:44 PM, Martin v. Löwis wrote:

 So, in conclusion, I disagree that adding wrappers for these would be
 nice. It wouldn't. It would cause some people to think they would be
 useful things to call, and they would always be wrong.
 
 We are all consenting adults. If people want to shoot themselves in
 their feet, we let them. For example, we have os.open, even though
 there is no garbage collection for file handles, and we have
 os._exit, even though it doesn't call finalizers.


There's a difference.

os._exit is useful. os.open is useful. aio_* are *not* useful. For anything. If 
there's anything you think you want to use them for, you're wrong. It either 
won't work properly or it will worse performing than the simpler alternatives.

It would absolutely be a waste of time (of both the implementor of the wrapper 
and the poor users who stumble across them in documentation and try to use 
them) to bother adding wrappers to these functions for python. 

James
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread Glyph Lefkowitz

On Oct 19, 2010, at 8:09 PM, James Y Knight wrote:

 There's a difference.
 
 os._exit is useful. os.open is useful. aio_* are *not* useful. For anything. 
 If there's anything you think you want to use them for, you're wrong. It 
 either won't work properly or it will worse performing than the simpler 
 alternatives.


I'd like to echo this sentiment.  This is not about providing a 'safe' wrapper 
to hide some powerful feature of these APIs: the POSIX aio_* functions are 
really completely useless.

To quote the relevant standard 
http://www.opengroup.org/onlinepubs/95399/basedefs/aio.h.html:

APPLICATION USAGE

None.

RATIONALE

None.

FUTURE DIRECTIONS

None.

Not only is the performance usually worse than expected, the behavior of aio_* 
functions require all kinds of subtle and mysterious coordination with signal 
handling, which I'm not entirely sure Python would even be able to pull off 
without some modifications to the signal module.  (And, as Jean-Paul mentioned, 
if your OS kernel runs out of space in a queue somewhere, completion 
notifications might just never be delivered at all.)

I would love for someone to prove me wrong.  In particular, I would really love 
for there to be a solution to asynchronous filesystem I/O better than start a 
thread, read until you block.  But, as far as I know, there isn't, and 
wrapping these functions will just confuse and upset anyone who attempts to use 
them in any way.

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread exarkun

On 01:37 am, gl...@twistedmatrix.com wrote:


On Oct 19, 2010, at 8:09 PM, James Y Knight wrote:

There's a difference.

os._exit is useful. os.open is useful. aio_* are *not* useful. For 
anything. If there's anything you think you want to use them for, 
you're wrong. It either won't work properly or it will worse 
performing than the simpler alternatives.



I'd like to echo this sentiment.  This is not about providing a 'safe' 
wrapper to hide some powerful feature of these APIs: the POSIX aio_* 
functions are really completely useless.


To quote the relevant standard 
http://www.opengroup.org/onlinepubs/95399/basedefs/aio.h.html:



APPLICATION USAGE

None.

RATIONALE

None.

FUTURE DIRECTIONS

None.

Not only is the performance usually worse than expected, the behavior 
of aio_* functions require all kinds of subtle and mysterious 
coordination with signal handling, which I'm not entirely sure Python 
would even be able to pull off without some modifications to the signal 
module.  (And, as Jean-Paul mentioned, if your OS kernel runs out of 
space in a queue somewhere, completion notifications might just never 
be delivered at all.)


Just to be clear, James corrected me there.  I thought Jesus was talking 
about the mostly useless Linux AIO APIs, which have the problems I 
described.  He was actually talking about the POSIX AIO APIs, which have 
a different set of problems making them a waste of time.


Jean-Paul
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com


Re: [Python-Dev] Support for async read/write

2010-10-19 Thread Jeffrey Yasskin
On Tue, Oct 19, 2010 at 8:37 PM, Glyph Lefkowitz
gl...@twistedmatrix.com wrote:
 I'd like to echo this sentiment.  This is not about providing a 'safe'
 wrapper to hide some powerful feature of these APIs: the POSIX aio_*
 functions are really completely useless.
 To quote the relevant standard
 http://www.opengroup.org/onlinepubs/95399/basedefs/aio.h.html:

 APPLICATION USAGE
 None.
 RATIONALE
 None.
 FUTURE DIRECTIONS
 None.


No comment on the rest of your claim, but this is a silly argument.
The standard says the same thing about at least fcntl.h, signal.h,
pthread.h, and ucontext.h, which clearly are useful.

Jeffrey
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com