On Thu, Sep 08, 2011 at 01:17:22AM -0700, Scott Lamb wrote: > On Tue, Sep 6, 2011 at 3:25 AM, Nicholas Marriott > <nicholas.marri...@gmail.com> wrote: > > OpenBSD is libevent 1.4.13. > > > > OS X polling mechanisms are pretty bad: their poll() and kqueue() only > > support sockets, so you will need to use select (set EVENT_NOPOLL=1 > > EVENT_NOKQUEUE= in the environment). > > This isn't really limited to OS X. My understanding is that there's no > Unix or Unix-like system that supports non-blocking IO on regular > files except maybe through the special aio interface described in the > following link. That interface is often unimplemented, buggy, and/or > emulated in userspace with blocking IO done from hidden threads. And > it's not supported by libevent, perhaps for these reasons.
Nonblocking I/O has not much to do with it, you don't necessarily need nonblocking I/O if you have working poll(2) or select(2). It is a limitation of OS X. Every other platform with kqueue(2) and all I am aware of with poll(2) support it on all file descriptors. OS X doesn't support it on anything other than sockets - so not on ttys, not on files, not anything except sockets. This is annoying because it means you can't manage tty file descriptors or file descriptors linked to devices such as /dev/null (both of which are commonly used for eg stdout) with libevent unless you force it to use select. Linux epoll doesn't support /dev/null either, but it does support ttys and IIRC files too. > > http://pubs.opengroup.org/onlinepubs/009695399/basedefs/aio.h.html > > In particular, read() and write() on "regular files" will never return > EAGAIN, usually use "uninterruptible IO" (meaning even a SIGKILL will > not terminate the thread/process doing the IO if the device is hung; > you'll just see the dreaded state "D" in ps), and don't really work > with any of the notification interfaces used by libevent. They may be > reported as always available, simply cause EINVAL or some such error, > or do something more strange. > > fwiw, if you browse around the aio docs I linked above, you'll see the > following rationale: > > """ > Rationale for New Interface > > Non-blocking I/O does not satisfy the needs of either realtime or > high-performance computing models; these models require that a process > overlap program execution and I/O processing. Realtime applications > will often make use of direct I/O to or from the address space of the > process, or require synchronized (unbuffered) I/O; they also require > the ability to overlap this I/O with other computation. In addition, > asynchronous I/O allows an application to keep a device busy at all > times, possibly achieving greater throughput. Supercomputing and > database architectures will often have specialized hardware that can > provide true asynchrony underlying the logical asynchrony provided by > this interface. In addition, asynchronous I/O should be supported by > all types of files and devices in the same manner. > """ > > I don't think that's entirely accurate or an adequate justification > for not supporting the existing (and much simpler) non-blocking > interfaces, but I wasn't in the room when the decision was made... > > libevent works well with sockets (including TCP, UDP, and Unix-domain) > and pipes (named and unnamed) but not so well with regular files. > > > > > > > On Thu, Sep 01, 2011 at 11:19:52PM +0100, Bernd Schoeller wrote: > >> Dear List, > >> > >> I am currently taking my first steps in experimenting with libevent. > >> To do that, I have written the small example below that should copy > >> a file. I am currently trying to restrict myself to the core > >> functions, and I know that the buffer handling is too primitive. > >> > >> I am trying the code on 3 different platforms: > >> > >> a) On OpenBSD 4.9, the code seems to work and the file is copied. I > >> am not sure what version OpenBSD is using and what they changed. > >> > >> b) On MacOS Lion, using libevent 2.0.10, the program copies the > >> file, but seems to never receive the 0 bytes EOF read, thus blocking > >> after the last write. Output ends with: > >> > >> ... > >> read callback > >> Read: 6, 2 - 81953 > >> write callback > >> Written: 7, 4 - 81953 (81953) of 81953 > >> Sub read. > >> > >> c) On Linux (Ubuntu LTS), using libevent 1.4.13, the read operation > >> is never triggered and the main loop exits immediately. The output > >> printed is: > >> > >> Starting. > >> Opened read file handle 6. > >> Opened write file handle 7. > >> Exited with 1 > >> > >> With these 3 different behaviors, I am pretty sure that I have not > >> understood some fundamental detail of libevent. I do not think the > >> different versions are to blame, and it should be possible to create > >> some code that runs on all versions, as I am using very basic > >> functions that seem to be the same in all versions. > >> > >> Thanks for your help, > >> Bernd > >> > >> > >> ---> c_libevent_test.c <--- > >> > >> #include <event.h> > >> #include <stdio.h> > >> #include <fcntl.h> > >> #include <sys/types.h> > >> #include <sys/stat.h> > >> > >> #define READ_FILE_NAME "in.dat" > >> #define WRITE_FILE_NAME "out.dat" > >> > >> #define BUFFER_SIZE 100000 > >> > >> struct event fd_read; > >> struct event fd_write; > >> > >> int buffer[BUFFER_SIZE]; > >> int full; > >> int written; > >> > >> void onRead(int fd, short mode, void* data) > >> { > >> ? printf("read callback\n"); > >> ? full = read(fd, buffer, BUFFER_SIZE); > >> ? printf("Read: %d, %d - %d\n", fd, mode, full); > >> ? if (full > 0) { > >> ? ? written = 0; > >> ? ? event_add(&fd_write, NULL); > >> ? } > >> } > >> > >> void onWrite(int fd, short mode, void *data) > >> { > >> ? printf("write callback\n"); > >> ? int wr = write(fd, buffer+written, full-written); > >> ? written += wr; > >> ? printf("Written: %d, %d - %d (%d) of %d\n", fd, mode, > >> ? ? ? ? ?wr, written, full); > >> ? if (wr >= full) { > >> ? ? written = 0; > >> ? ? full = 0; > >> ? ? printf("Sub read.\n"); > >> ? ? event_add(&fd_read, NULL); > >> ? } else { > >> ? ? event_add(&fd_write, NULL); > >> ? } > >> } > >> > >> int main(int argc, char** argv) > >> { > >> ? full = 0; > >> > >> ? event_init(); > >> > >> ? printf("Starting.\n"); > >> > >> ? int fd; > >> ? fd = open(READ_FILE_NAME, O_RDONLY); > >> ? printf("Opened read file handle %d.\n", fd); > >> ? event_set(&fd_read, fd, EV_READ, onRead, 0); > >> > >> ? fd = open(WRITE_FILE_NAME, O_WRONLY | O_CREAT, 0600); > >> ? printf("Opened write file handle %d.\n", fd); > >> ? event_set(&fd_write, fd, EV_WRITE, onWrite, 0); > >> > >> ? event_add(&fd_read, NULL); > >> ? printf("Exited with %d\n", event_dispatch()); > >> ? return 0; > >> } > >> > >> *********************************************************************** > >> To unsubscribe, send an e-mail to majord...@freehaven.net with > >> unsubscribe libevent-users ? ?in the body. > > *********************************************************************** > > To unsubscribe, send an e-mail to majord...@freehaven.net with > > unsubscribe libevent-users ? ?in the body. > > > > > > -- > Scott Lamb <http://www.slamb.org/> > *********************************************************************** > To unsubscribe, send an e-mail to majord...@freehaven.net with > unsubscribe libevent-users in the body. *********************************************************************** To unsubscribe, send an e-mail to majord...@freehaven.net with unsubscribe libevent-users in the body.
