Re: [ql-developers] Job re-scheduling
Peter Graf writes: <> > Hm. I know this is perfectly allright, and I think I even understand the > priciples of background device drivers a bit. Oops. Should have known ;) However, sometimes the obvious is missed even by experts, so its good to get this out of the way first. > The issue I'm dealing with is the latency time until an external interrupt can > (indirectly) cause re-scheduling of the user-mode jobs, so a high-priority job > blocking for I/O resumes. I am not concerned about sequential throughput > but response. <> > For a simple example, imagine you have a usermode job that implements TCP echo. > No matter how fast and elegant the implementation of the ISR and the buffering, > if QDOS/SMS can only re-schedule jobs after event (A) or (B), the TCP echo > response will need between 0 and 20 milisec., average 10 milisec - and this > is slow on usual ethernet LAN's. There doesnt seem to be any system facility to cater for this sort of thing. For anything that requires a guaranteed response time less than the resolution provided by the 50/60 Hz frame, you may need to put something in your ISR, ie it should be part of the driver, not the application. > If my fear is correct, then QDOS/SMS has a principle limitation here that > stems from the 50/60 Hz frame interrupt heritage. > > >Having said that, it will be interesting to see how such a device will > >respond to the copying of a file from the network to a floppy disk ;( > >Perhaps you can fix the floppy driver while youre at it ;) > > The SMSQ/E IDE driver (principle: interrupts off) could even make copying > to harddisk interesting. 8-O I have often wondered why. There cant be any physical reason for it such as processing speed or controllers, as other systems - even Msdos - seem to manage much better in this field. Per
Re: [ql-developers] Job re-scheduling
Per wrote: > The issue I'm dealing with is the latency time until an external interrupt can > (indirectly) cause re-scheduling of the user-mode jobs, so a high-priority job > blocking for I/O resumes. I am not concerned about sequential throughput > but response. <> > For a simple example, imagine you have a usermode job that implements TCP echo. > No matter how fast and elegant the implementation of the ISR and the buffering, > if QDOS/SMS can only re-schedule jobs after event (A) or (B), the TCP echo > response will need between 0 and 20 milisec., average 10 milisec - and this > is slow on usual ethernet LAN's. There doesnt seem to be any system facility to cater for this sort of thing. For anything that requires a guaranteed response time less than the resolution provided by the 50/60 Hz frame, you may need to put something in your ISR, ie it should be part of the driver, not the application. Thanks for confirming my suspicion. So at least I know that this is a principal limitation of QDOS/SMS, and I'm not the only one who is aware. Or maybe we are both wrong?? I don't ask for a *guaranteed* response time less than 10 msec. That would obviously need to be handled within an ISR, not a usermode job anyway. I think about an *average* response time of about 1 msec or less, depending on system load. Just as other operating systems like Linux and eCos (sometimes even Windows) provide. If QDOS/SMS forces me to integrate the application into the ISR to achieve that goal, it has a principal (and quite severe) shortcoming for use on LAN's. I see only two possible solutions for QDOS/SMS: (1) A higher rate for the frame intterrupt (disadvantage: higher system load) (2) A new facility (maybe via the IOSS) so an ISR can indirectly cause a job re-schedule after the linked list of ISR's has been handled. (The case that the ISR has interrupted the scheduler may not be easy to handle, but it *can* be handled, as other small operating systems like eCos show.) Please note that not many ISR calls are meant to cause entering the scheduler. This has to happen "seldom", e.g. upon completion of an I/O task. The strange thing about QDOS/SMS seem that a much quicker average response time can be achieved by implementing everything within jobs. Example: Imagine I poll an ethernet card for data within job "A". Imagine the system has a low load. This is usually the case if most other jobs are suspended, e.g. waiting for I/O, or have a low priority. Job "B" is a high priority job, suspended waiting for data from the network. Then job "A" will *usually* be active when the last byte of input data from the network has been received. Job "A" can, directly (or indirectly e.g. by the IOSS), release the suspended job "B", and the scheduler will *ususally* activate job "B" *immediately*. Job "B" will process the data, put some response into some send buffer and will be suspended again. This way I easily have an average response time of less than 1 ms an fast systems like Q40/Q60 (already tested practically). Do you see what I mean? A simple job can cause *immediate* job re-scheduling. A properly implemented device driver can not (as it seems). Can please some QDOS/SMS expert tell me I'm wrong ?!? > The SMSQ/E IDE driver (principle: interrupts off) could even make copying > to harddisk interesting. 8-O I have often wondered why. There cant be any physical reason for it such as processing speed or controllers, as other systems - even Msdos - seem to manage much better in this field. SMSQ/E IDE driver seems just "quick and dirty" implementation, the only reason I can see was to save work. The advantage was that Thierry could later implement his CDROM driver without much worry about interaction with SMSQ/E harddisk access on the same controller ;-) All the best Peter
Re: [ql-developers] Job re-scheduling
??? 3/1/2003 9:49:10 ??, ?/? Peter Graf <[EMAIL PROTECTED]> ??: >I see only two possible solutions for QDOS/SMS: > >(1) A higher rate for the frame intterrupt (disadvantage: higher system load) > This would be feasible I think without too much problems for the system. I always believed that the way I/O is implemented might have been good for extremely slow peripheral devices and with media that wasn't exactly reliable (ie making sure that SuperBasic for example did actually save the changes before you could return control to it) but for today's requirements it is totally obsolete. On the other hand and IIRC IOSS in itself is a different scheduler that could be tuned without resorting to drasting measures (ie overhauling the whole system). Please note that drivers (according to everything I have read) do not *NECESSARILY* have to get into Supervisor mode to go about their business(sic!), it's just "strongly advised" (I quote from Adrian Dickens) that they should do so. >Do you see what I mean? A simple job can cause *immediate* job >re-scheduling. A properly implemented device driver can not (as it seems). See above... A properly implemented device driver can be in fact be a simple job (or have I got it all wrong) if the writer chose to, it's just a lot simpler if you don't implement it that way because after that you have to cater for all sorts of possibilities. The "correct" implementation according to my VERY HUMBLE opinion would be to implement the IOSS as a set of two jobs. One vectrored routine (along the lines of what we have now (which would also ensure compatibility)) which would then start the actual scheduler AND driver. According to the type of driver that would be invoked that would close later if it wasn't needed or -in the case of a say ethernet packet driver- keep on working... something like the DETACH command in OS/2 (remember that one?). You set it and forget it in other words... The separate IO scheduler then (for drivers/jobs that were still in use) could be tuned to our desired response. (Or something like that in general). In theory we could have two or three or more IOSS's running at the same time, all invoked from the same vectored routine which would only serve as an entry point. (Not many, many more of course as that would introduce other problems...) Maybe I got it all wrong myself I don't know we're just brainstorming here :-) Phoebus
Re: [ql-developers] Job re-scheduling
Phoebus wrote: See above... A properly implemented device driver can be in fact be a simple job (or have I got it all wrong) if the writer chose to, it's just a lot simpler if you don't implement it that way because after that you have to cater for all sorts of possibilities. The problem in using jobs for handling hardware devices is that a job can never react to asynchronous hardware events unless it is polling, i.e. consuming CPU resources. I may add that my concerns are more or less specific for mid- to high-speed local area networking. Drivers for (hard) disks drives, printers and such can deal with their performance challenges by buffering and can be implemented efficient enough using the given OS facilities. Drivers for duplex serial port connections are also not much affected, because a medium response time of 10 msec is still in the same range as the time needed to tranfer a small amount of data. Note that the specific thing about (ether)networking is that reading the data is not triggered from inside the system (like e.g. a software request for data from harddisk), but absolutely asynchronous and unpredictable from outside. All the best Peter
Re: [ql-developers] Job re-scheduling
??? 3/1/2003 12:58:02 ??, ?/? Peter Graf <[EMAIL PROTECTED]> ??: > >Phoebus wrote: > >> >>See above... A properly implemented device driver can be in fact be a >>simple job (or >>have I got it all wrong) if the writer chose to, it's just a lot simpler >>if you don't >>implement it that way because after that you have to cater for all sorts of >>possibilities. > >The problem in using jobs for handling hardware devices is that a job can >never react to asynchronous hardware events unless it is polling, i.e. >consuming CPU resources. Not necessary if you assume intelligent (ie with their own CPU) peripherals. In the case of a Network card of course that would mean that a special Qx0 card should be designed and/or that in the case of going to a different IO bus (ie PCI) in the future we would be back to square one. Therefore the ONLY other possibilities I see is to update the main scheduler to allow custom timings or some form of independent hardware buffering for devices (to cater for that loss of control while the scheduler moves execution to a different job). Other possible (and exotic) alternatives would be dual processors (and scheduling) etc... >I may add that my concerns are more or less specific for mid- to high-speed >local area networking. Buffering can still be used in a high-speed LAN but what are the limitations on speed anyway? I doubt it that the ISA bus can do the "theoretical" 100Mbit (like my intel card uses). IIRC in the case of the iNTEL Etherexpress, the card buffers input and output packets and then burst-sends them out/in so no loss of frame sync happens. > >Drivers for (hard) disks drives, printers and such can deal with their >performance challenges by buffering and can be implemented efficient enough >using the given OS facilities. Drivers for duplex serial port connections >are also not much affected, because a medium response time of 10 msec is >still in the same range as the time needed to tranfer a small amount of data. > >Note that the specific thing about (ether)networking is that reading the >data is not triggered from inside the system (like e.g. a software request >for data from harddisk), but absolutely asynchronous and unpredictable from >outside. And that's why IIRC even Windows is polling its devices... and also why they were finally able to use more devices / interrupt (which aren't really used anymore)... and also why "Legacy" devices are always so much more responsive from polled ones. maybe a third type of device driver should be devised... one that can act as block or directory, but which is strictly polled. In any case just one device driver of that type wouldn't put such a load on the system (especially if the code for it is written to take advantage of the 040 and 060 high performance extra instructions Phoebus
Re: [ql-developers] Job re-scheduling
Phoebus wrote: >The problem in using jobs for handling hardware devices is that a job can >never react to asynchronous hardware events unless it is polling, i.e. >consuming CPU resources. Not necessary if you assume intelligent (ie with their own CPU) peripherals. [Snip] How should the intelligent peripheral wake up the application running on the QDOS/SMS system other than by the QDOS/SMS scheduler? As long as the scheduler can not react to async hardware events other than in a 20 msec raster, we are stuck at average 10 msec response. I still have problems to believe that QDOS/SMS is really that limited. Every simple job can cause re-scheduling *immediately*, not only in a 20 msec raster! There may be a way how an interrupt-driven device driver can do that also. Maybe some expert knowledge I lack. >I may add that my concerns are more or less specific for mid- to high-speed >local area networking. Buffering can still be used in a high-speed LAN but what are the limitations on speed anyway? I doubt it that the ISA bus can do the "theoretical" 100Mbit (like my intel card uses). IIRC in the case of the iNTEL Etherexpress, the card buffers input and output packets and then burst-sends them out/in so no loss of frame sync happens. At the moment I'm concerned about response, not linear throughput. Every simple Linux process on Q40 can do what I'm asking for. Device driver interrupt-driven, of course. And that's why IIRC even Windows is polling its devices... and also why they were finally able to use more devices / interrupt (which aren't really used anymore)... and also why "Legacy" devices are always so much more responsive from polled ones. maybe a third type of device driver should be devised... one that can act as block or directory, but which is strictly polled. In any case just one device driver of that type wouldn't put such a load on the system (especially if the code for it is written to take advantage of the 040 and 060 high performance extra instructions The device driver can be more than quick enough without polling. It just can't make the OS re-schedule the jobs immediately, so jobs who were blocking for input can't get activated other than in the 20 msec raster. (Unless I'm wrong, and I still hope I'm wrong.) All the best Peter
