Am 02.01.2006 um 08:36 schrieb Stephen Deasey:
On 12/31/05, Zoran Vasiljevic <[EMAIL PROTECTED]> wrote:
Am 31.12.2005 um 20:12 schrieb Vlad Seryakov:
aio_read/aio_write system calls look like supported ubnder Linux
yes. most of modern os's support some kind of kaio.
I have checked solaris, linux and darwin and all do.
Hmm, not sure about that...
Linux aio_read() is implemented within glibc with via threads. The
2.6 Linux epoll() is great for socket file descriptors, but doesn't
work with files. Same for realtime signals in 2.4. The kernel stuff,
io_submit() etc. only works on files opened with O_DIRECT, i.e. no
file system buffering. io_*() via libaio, and epoll() are not
portable.
The BSDs (FreeBSD first, others later) have kqueue, and so does Mac/OS
from 10.3 on, but again this is not portable. This didn't work with
threads in the past. Don't know if/when that got fixed...
Solaris 10 has some brand new port_*() calls. Again, not portable.
Not sure how aio_read() etc. are implemented on older versions of
Solaris.
I think Windows has pretty good support for AIO, including to files.
Obviously, not portable.
Look:
On Darwin (10.4+)
aio_cancel(2) - cancel an outstanding asynchronous I/O
operation (REALTIME)
aio_error(2) - retrieve error status of asynchronous I/O
operation (REALTIME)
aio_read(2) - asynchronous read from a file (REALTIME)
aio_return(2) - retrieve return status of asynchronous I/O
operation (REALTIME)
aio_suspend(2) - suspend until asynchronous I/O operations
or timeout complete (REALTIME)
aio_write(2) - asynchronous write to a file (REALTIME)
On Solaris:
On Linux:
aio_return (3) - get return status of asynchronous I/O operation
aio.h (0p) - asynchronous input and output (REALTIME)
aio_cancel (3) - cancel an outstanding asynchronous I/O request
aio_error (3p) - retrieve errors status for an asynchronous I/O
operation (REALTIME)
aio_error (3) - get error status of asynchronous I/O operation
aio_suspend (3) - wait for asynchronous I/O operation or timeout
aio_read (3p) - asynchronous read from a file (REALTIME)
aio_return (3p) - retrieve return status of an asynchronous I/O
operation (REALTIME)
aio_write (3p) - asynchronous write to a file (REALTIME)
aio_cancel (3p) - cancel an asynchronous I/O request (REALTIME)
aio_fsync (3) - asynchronous file synchronization
aio_fsync (3p) - asynchronous file synchronization (REALTIME)
aio_write (3) - asynchronous write
aio_suspend (3p) - wait for an asynchronous I/O request (REALTIME)
aio_read (3) - asynchronous read
On Solaris (2.8+)
# apropos aio
aio aio (3head) - asynchronous input and output
aio_cancel aio_cancel (3rt) - cancel asynchronous I/O request
aio_error aio_error (3rt) - retrieve errors status for an
asynchronous I/O operation
aio_fsync aio_fsync (3rt) - asynchronous file synchronization
aio_read aio_read (3rt) - asynchronous read from a file
aio_req aio_req (9s) - asynchronous I/O request structure
aio_return aio_return (3rt) - retrieve return status of an
asynchronous I/O operation
aio_suspend aio_suspend (3rt) - wait for asynchronous I/O request
aio_write aio_write (3rt) - asynchronous write to a file
aiocancel aiocancel (3aio) - cancel an asynchronous operation
aioread aioread (3aio) - read or write asynchronous I/O
operations
aiowait aiowait (3aio) - wait for completion of asynchronous
I/O operation
aiowrite aioread (3aio) - read or write asynchronous I/O
operations
libaio libaio (3lib) - the asynchronous I/O library
What we'd need is available on all above platforms:
aio_write
aio_cancel
aio_suspend
So, what is the problem?
If we only consider the problem of how to spool large uploads to disk,
then AIO is the most direct solution. It would allow the driver
thread to handle everything. But this doesn't help us with upload
statistics or access control etc. so we still have to tackle those
problems with some other solution.
This is right and I said that already: the upload statistics and/or
access ctrl etc, needs to be solved additionally by some other means.
The biggest problem I see with AIO is that it will be a real pain to
implement. At the minimum you're going to have to come up with some
kind of higher level abstraction, then add a different implementation
for each system we want to support, then add a fallback mechanism for
the systems which don't have AIO to offer.
As I see, all platforms we support now have needed aio_* calls. It might
be that we'd have to write some glue-code if API's are not 100%
compatible
(which I believe they are):
Darwin: int aio_write(struct aiocb *iocb);
Solaris: int aio_write(struct aiocb *aiocbp);
Linux: int aio_write(struct aiocb *aiocbp);
Oh, and it would be nice to handle network IO with that too, which may
mean using a different implementation depending on descriptor type.
Why network traffic? The only thing we really need there now is to
make spooling of the large-file-upload non-blocking, or?
Looks tough... :-(
Depends. I do not believe that it is that difficult. Admittently
I havent done anything in that direction yet, but judging from
the man-page reading it is all really simple. Might be that the
problems will expose when we dig more deeply inthere, but just
from the surface it looks ok.
