On Sun, 2007-04-08 at 11:27 -0700, Erick Tryzelaar wrote: > skaller wrote: > > OK, I see two options here: > > > > A. Modify the stream typeclass so all stream I/O requires > > a job_queue argument. > > > > B. Make a new type > > > > afd_t = fd_t * job_queue > > > > Now there are two file types, one for sync I/O and one > > for async I/O .. but only for posix lol (since Windows > > has no problem with async I/O). Also there's a problem > > with memory management of the job_queue. > > > > Solution B locks the file handling into async mode > > with a particular queue at time afd_t object is > > constructed. So actually you could do this: > > > > read ( (q,fd), .... > > > > Comments? > > I think you mean "read ( (fd, q), ...".
Yep. > The async difference between > posix and windows can't be hidden behind in the RTL, can it? No. That's the problem. A more detailed explanation: The whole Async system uses queued requests, however some of the requests are serviced by a single kind specific thread, where the scheduling is handled by the OS. For example, timers and sockets are both handled by a single management thread plus OS level notifications. On Windows and Solaris, async fileio is handled the same way .. using IO completion ports. On Linux, libaio handles it, either using Linux new magical async notification service, or just by spawning a pthread. But in general Posix doesn't support it, and RF has designed the async file I/O to use the general job scheduler, which is a queue plus a thread pool. Arbitrary synchronous (blocking) operations can be made asynchronous by just running them in a thread, forcing the underlying OS to schedule and notify using thread services instead of a dedicated notification service. In the old Felix, the timer thread was actually hidden from the end user. In the new system the end user must create the server object: OLD way: Faio::sleep 10.0; NEW way: val clock = Faio::mk_alarm_clock(); Faio:: sleep$ clock, 10.0; Moving the service object out of the system into user space allows the user to control service sharing and creation. You could actually slow down a game by just making a special slow_clock, and change one instantiation. Or, if you're into telescopes you might have atomic_clock() earth_clock() and system_clock() depending on what kind of time base you want. Sockets don't seem to need this .. I'm not sure how they start up the service thread. So for files -- if the implementation uses OS supported aio, or some 3pl library (like libaio), then no job_queue is needed. but if the 'in felix' emulation using a job queue is used .. well you have to supply the job queue. This could be fixed by conditionally creating a static job queue .. but that's not very nice: there are good reasons to use more than one such queue, such as the native vs. NFS file system example. The problem here is that we want an abstraction layer in which I/O is guaranteed to be asynchronous, so that for example you can confidently write a file stream processing program which pre-loads buffers at full speed .. crapping on all the other stupid C programs that have to pause in the middle of say lexing a file to load the next part of it in: in Felix you can just spawn an fthread (fibre) to load 3 pages of the file and synchronise with it via an fchannel, allowing the OS to load the pages asynchronously. This allows, say, a web-server to be written using fthreads that does all the I/O at maxed out OS thruput .. and indeed there exist a demo webserver (which is where i found the problem). It also allows a specialised configuration to discover we're running Linux, and gather up 20 web page requests into a single linux kernel call .. that is, try to use the new Linux kernel async I/O features directly. > Secondly, > will there ever be a use case for an end user to modify the job_queue? Yes, the user can change the number of threads in the thread pool and the bound on the queue. > If so, I vote for B. For simple stuff, we can have a function like > "fopen_async" that returns a afd_t for ease of use and has the default > job_queue built into the type. For more complicated things, they could > just pass in their own job_queue. There's no default job_queue: there used to be, but it's gone now. Just like there's no default clock. You have to actually create a clock, and you have to actually create a job queue. The clock is no problem .. it's needed on all OS: the job queue is the problem because it's only needed on generic posix. One solution is to create a single 'OS support' object which can be passed, it has a default clock and file aio object: for Windows the aio object is just a NULL pointer. The problem is this is hard to use because it aggregates unrelated services together: that makes incremental extension of the supported services hard, if not impossible. Asynchronous graphics makes sense too.. indeed asynchronous calculations (matrix math etc) also make sense. -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Felix-language mailing list Felix-language@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/felix-language
