Tom <[EMAIL PROTECTED]> writes: > A quick perusal of multi-proc.lisp suggests that it uses > some kind of user-level threading based on signals. What > seems to be happening is that the I/O operations performed > by CLX hang the process until some signal interrupts it > and a context switch happens.
Not exactly. By default thread switching occurs only when the current thread blocks on an IO operation. The blocking thread registers an event-handler (with add-fd-handler) and invokes the scheduler (process-yield). The scheduler then picks the next runnable thread. If there is no runnable thread, e.g. all threads are waiting for a file-descriptor to become ready, the initial thread invokes sub-serve-event to perform a select on all interesting file descriptors. If a file-descriptor becomes ready, select returns and the previously registered event-handler will invoke the scheduler. The scheduler is also invoked if the timeout to select (*max-event-to-sec*) expires. This gives threads, that don't wait on a file-descriptor, a chance to run. Obviously this strategy leads to problems when the current thread never blocks on IO operations. Less obvious is a problem in the implementation of mp:process-wait-until-fd-usable. The problem is that the *initial-process* calls sys:wait-until-fd-usable direclty. This means, processes that don't wait for a file-descriptor have to wait until *max-event-to-sec* expires, even if they would be runnable immediately. This often leads to the slow context switches. I had this problem with the following code: (loop (let ((client (accept-tcp-connection listener))) (make-process (lambda () (select-method client))))) I called this snipped in the initial thread, and it took 1 second (the default value of *max-event-to-sec*) until select-method was called. This is because the newly created thread doesn't wait for a file-descriptor. > Is that the only way to get any kind of multiprocessing in > CMU CL, or is there some other implementation based on > system threads anywhere? A commercial variant of CMUCL, SCL <http://www.scieneer.com/scl/> supports native threads. > How do other people write > multithreaded applications in CMU CL? This would seem > to be important for, say, a CL web server. I call process-yield explicitly in appropriate places. In the example above, I called process-yield after make-process. Some other approaches to increase responsiveness: - Start the idle loop with mp::startup-idle-and-top-level-loops - Don't use multi-threading at all. Use serve-event and friends directly. This should work reasonably well with CLX code. Hemlock is written this way. - Start the SIGALRM based preemptive scheduler (mp:start-sigalrm-yield). People say this is "dangerous" because some parts of the system have not been updated for handle multi-threading correctly. (But no one seems to know what parts are dangerous ;-) - Set *max-event-to-sec* to a smaller value. 200 millisecs may be fast enough for interactive use. - Some people say it helps to make some file-descriptors non-blocking. I think this is even more dangerous than the preemptive scheduler. --helmut
