On Sat, 2007-04-07 at 11:13 -0400, Chris King wrote: > On 4/7/07, skaller <[EMAIL PROTECTED]> wrote: > > Just to make sure we understand: the worker queue is a neutral > > synchronisation technique: there's a pool of threads which > > pull jobs of the queue, run them, signal they're finished, > > and then run the next one. The pool size is user controlled. > > This is probably a dumb idea, but maybe instead of a fixed pool and a > queue, forget the queue and just have a pool that dynamically resizes > itself when all the worker threads are busy. Reclaim worker threads > that have been inactive for so many seconds.
The number of threads in the pool isn't fixed, you can change it dynamically: it doesn't resize itself, but you could always write a monitor thread that measures the load on it and does that according to some algorithm. The purpose of a pool, however, is partly to bound the number of threads (as well as reduce thread creation time). The queue itself is also bounded. So basically at some point, dispatching a job will lock up the client thread, which is necessary to force the OS to context switch and actually run some of the dang threads in the pool! > Since fthreads block on > I/O the pool size will be bounded by the number of fthreads. Of > course, if the user spawns a thousand fthreads and has each of them > perform some I/O operation, we'd end up spawning (temporarily) a > thousand worker threads, the effects of which would vary system to > system. (My warning to a user would be "don't do that!" :)) The whole idea of Felix is that you use millions of fthreads: it isn't clear from the docs and tutorial material, but this provides a modular programming model impossible with other systems. You can basically write the code for 'black boxes' or 'chips' with inputs and outputs, and plug them together to make a program: the code for the 'chips' is self contained. A chip is basically an object with a callback, but you program it control inverted: you read and write the I/O pins (fchannels) the same way a C program reads and writes files -- after all, an Operating System does nothing more than control invert blocking operations by context switching, .. and indeed, Felix 'driver' is actually a user space operating system -- we even have a generic 'kernel call' which is a compiler primitive (BEXE_svc). In a game environment, you'd make a thread for every object: every sprite/monster in the game is an fthread. in a telco environment -- every phone call, and they run at around 600 calls/sec average 3 minute calls on a city exchange, or around 500,000 concurrent calls. But, you only EVER need: one worker thread for each CPU PLUS roughly one thread for each I/O device, the number of which is strictly bounded on any computer. So actually Pthreads are heavily abused by those ignorant of superior technology like Felix .. :) > Returning to the (admittedly more sane) idea of using a queue... It's a first approximation: as I said, we almost certainly have to support some kind of priority system down the track. > > To make it seem more 'intuitive' that a queue really is > > required, just consider fileio requests on your local > > disk as opposed to an NFS mounted volume. > I would argue that with multiple worker threads this wouldn't be much > of a problem... Well .. it won't be a problem until it actually IS a problem.. :) One want some real-time guarantees, and the simple thread pool with a queue doesn't provide that. It may work for a while, then suddenly all the threads hang on NFS accesses against a machine that is down, and wait while it reboots .. meanwhile 20,000 customers had their phone calls disconnected due to a lag in the accounting system .. :) -- John Skaller <skaller at users dot sf dot net> Felix, successor to C++: http://felix.sf.net ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Felix-language mailing list Felix-language@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/felix-language
