Re: [ql-developers] Massive amount of job state transitions and re-scheduling
On 9 Sep 2003 at 1:37, P Witte wrote: [snip] > But does the incoming data need to be processed in any way before > acknowledgement? Why cant the ISR simply receive and buffer the data and > then send the ACK before exiting, leaving any processing to the higher > levels? The reason is that it's part of the TCP processing and can not be done on ethernet packet level. > In our January discussion you mentioned the case of echo. There is nothing > to stop you from implementing time-critical routines, like echo, in the > 'physical layer'. In fact you can take over the whole machine and do as you > please. Not a task where speed is relevant for the user. No point in speeding up ICMP echo only. > he important thing is to split the driver correctly: Time > critical, ie > usually hardware related stuff, and in this case it appears also certain > demands of the TCP/IP protocol (if I understand correctly) are rightly the > provinance of the ISR. If this sort of thing is not clearcut in TCP/IP, then > a messy solution is called for ;) > > Arent you trying to make the OS do something it was never designed to > do? Writing drivers is a programming challenge. The OS is there to help > where it can, but no OS author can anticipate any and every piece of > hardware that is going to be attached to the machine in the future. That is > the job of the driver. (Preferably without each driver author altering the > OS to suit their own needs ;) Somehow I doubt that you need to teach me that writing drivers is a programming challenge or more trivialities and generalities about OS and driver structure ;) I guess you have to accept that QDOS (SMS?) has a principal shortcoming, not an author dependant need, and should be improved. > Afterall, someone did implement TCP/IP on the Spectrum, which neither > multitasks A singletasking TCP/IP implementation is easier not harder. Just in case you didn't notice: My TCP/IP package for QDOS works. I was talking the obstacles of higher data rates. > nor (for all I know) understands interrupts. Of course the Spectrum uses interrupts, and even singletasking TCP/IP needs some sort of timers. All the best Peter
Re: [ql-developers] Massive amount of job state transitions and re-scheduling
On 8 Sep 2003 at 0:53, P Witte wrote: > > Peter Graf writes: > > > Hi Per, > > > > >And Peter, did you try out the suggestions that were made at that time? > > > > Can you be a bit more specific? I remember only one applicable suggestion, > > which was to set a system variable before leaving the ISR. Didn't work, at > > least not under QDOS. > > # By exiting the interrupt handler through the sms.rte function the > # requested re-schedule will be done immediately if possible (i.e. no > # supervisor code was running at that time). Example: > # > # include dev8_smsq_smsq_basekeys > # include dev8_keys_psf > # > # int_handler > # movem.l psf.reg,-(sp) > # > # [blah] > # > # st sys_rshd(a6); Request re-schedule > # move.l sms.rte,a5 ; ...now would be convenient > # jmp (a5) > > etc, as per my mail to this list on 18/01/03 See above. Peter
Re: [ql-developers] Massive amount of job state transitions and re-scheduling
On Sun, 07 Sep 2003 21:53:34 +0200, Peter Graf wrote: > Thierry wrote: > > >Yes, this I know, thanks... I'm perfectly aware of the fragmentation and of > >out of order receipt of TCP packets... That doesn't change the fact you could > >use the fast interrupt to store as many TCP packet as needed (i.e. when they > >come in), into a buffer (organized as a linked list of recieved packets), > >then to transfer the whole lot of packets to the higher level layers of the > >TCP/IP stack at once and every 1/50th of second... > > Obviously correct but useless. Try to understand that the problem in your > approach is latency and can not be solved by buffering, no matter how > efficient buffering is implemented. >From the computer sending packets to a Qx0, the latency would just be seen as a longer route... When you ping a computer on Internet, you have to wait a variable amount of time for the reply, depending on how many routers must be crossed, on how long are the wires (for trans-oceanic links, or worst for satellite links, it's far than neglectable), and how fast and/or busy is the receiver... If the topographic design of the network between a PC and a Q60 should lead to, say, a 200ms delay in the reply, then the TCP/IP implementation on the Q60 would simply add a 20ms latency to this number, but in the end, the sender should still receive its ACKs between 200 and 220ms after the packets are sent... Of course, this supposes that the acknowledgement is -actually- done every 20ms in the Q60, which is -NOT- the case if it's done at the job level (jobs are elected or not, depending on their cumulated priority and are therefore -NOT- running each 20ms unless they are alone in memory)... The code for reassembling the fragmented packets and acknowledging them must be implemented either as a frame interrupt, or (if frame interrupts are still too slow for your taste), by using a polled routine triggered by the Q60 fast interrupt (the one used by the sound system). The struture for the whole TCP/IP stack would then be: 1.- IP packets fetching from the I/F and buffering: - External interrupt handler (best), or fast interrupt polling loop. 2.- IP packets reassembling and acknowledging: - polled task: fast interrupt handler (best) or frame interrupt. 3.- TCP/IP high level protocols: - High priority (127) job. How does this sound ? Thierry.
Re: [ql-developers] Massive amount of job state transitions and re-scheduling
On Tue, Sep 09, 2003 at 01:37:53AM +0100, P Witte wrote: > > Peter Graf writes: > > > >Yes, this I know, thanks... I'm perfectly aware of the fragmentation and > of > > >out of order receipt of TCP packets... That doesn't change the fact you > could > > >use the fast interrupt to store as many TCP packet as needed (i.e. when > they > > >come in), into a buffer (organized as a linked list of recieved packets), > > >then to transfer the whole lot of packets to the higher level layers of > the > > >TCP/IP stack at once and every 1/50th of second... > > > > Obviously correct but useless. Try to understand that the problem in your > > approach is latency and can not be solved by buffering, no matter how > > efficient buffering is implemented. > > > > Simple example: A M$ or Unix machine sends a file to the QDOS machine via > > TCP. It will send one or two packets, then stop and wait for ACK. Further > > packets will only be sent after further ACKs. Your ACKs can only be > > generated in 50 Hz rhythm, so packets will crawl one-by-one in 50 Hz > > rhythm. (Or two-by-two, if you're lucky.) > > But does the incoming data need to be processed in any way before > acknowledgement? Why cant the ISR simply receive and buffer the data and > then send the ACK before exiting, leaving any processing to the higher > levels? my impression is that to do that for TCP you would have to do all of the protocol implementation into the ISR > In our January discussion you mentioned the case of echo. There is nothing > to stop you from implementing time-critical routines, like echo, in the > 'physical layer'. well there is, as long as QDOS won't allow reschedule after interrupts. Echo is supposed to be a normal application and you would not move it into the ISR layer. As I said it isn't hard to change ISR handling in QDOS and I will certainly do it, right now I have other things that need to be done. Richard
Re: [ql-developers] Massive amount of job state transitions and re-scheduling
On 9 Sep 2003 at 14:42, Thierry Godefroy wrote: > > On Sun, 07 Sep 2003 21:53:34 +0200, Peter Graf wrote: > > > Thierry wrote: > > > > >Yes, this I know, thanks... I'm perfectly aware of the fragmentation and of > > >out of order receipt of TCP packets... That doesn't change the fact you could > > >use the fast interrupt to store as many TCP packet as needed (i.e. when they > > >come in), into a buffer (organized as a linked list of recieved packets), > > >then to transfer the whole lot of packets to the higher level layers of the > > >TCP/IP stack at once and every 1/50th of second... > > > > Obviously correct but useless. Try to understand that the problem in your > > approach is latency and can not be solved by buffering, no matter how > > efficient buffering is implemented. > > From the computer sending packets to a Qx0, the latency would just be seen > as a longer route... When you ping a computer on Internet, you have to > wait a variable amount of time for the reply, depending on how many routers > must be crossed, on how long are the wires (for trans-oceanic links, or worst > for satellite links, it's far than neglectable), and how fast and/or busy is > the receiver... > > If the topographic design of the network between a PC and a Q60 should lead > to, say, a 200ms delay in the reply, then the TCP/IP implementation on the > Q60 would simply add a 20ms latency to this number, but in the end, the > sender should still receive its ACKs between 200 and 220ms after the packets > are sent... Sure. That's all obvious but still unrelated to the the given LAN data rate challenge. > Of course, this supposes that the acknowledgement is -actually- done every > 20ms in the Q60, which is -NOT- the case if it's done at the job level (jobs > are elected or not, depending on their cumulated priority and are therefore > -NOT- running each 20ms unless they are alone in memory)... This is still only the trivial view. It is not applicable to the way I've implemented things. Let me give you a (much) simplified example: A top priority (let's call it 50Hz ISR replacement) job that suspends itself long before the 20 ms interval is over (and thereby gives the lower priority jobs their share in the same interval) will practically always be elected to run again after the suspension time (20 ms) is over. You can have 10 other jobs running (most of them will usually block for I/O most of the time) and even a benchmark that consumes all the rest of the CPU power, it still works. There are rare circumstances where indeed the high prio job will not be elected now and then, but under these circumstances it is the best that can happen, because some of the user interface should remain in working condition. It's better to slow down the network or even drop packets under these rare circumstances (let TCP deal with the dropped packets :). Just by the way, the remark with "alone in memory" was completely wrong, because it's irrelevant wether a job is loaded and activated. The point is in which intervals jobs are blocking for IO (or something else). You can easily have the case where several jobs are each and all executed every 20 ms and all do real work. This is not even unusual if the jobs process data from IRQ driven IO, and the IO is slower than the CPU. > The code for reassembling the fragmented packets and acknowledging them > must be implemented either as a frame interrupt, or (if frame interrupts > are still too slow for your taste), Were are not talking a special taste, but the normal 10 Mbit/s Ethernet data rate under TCP. by using a polled routine triggered > by the Q60 fast interrupt (the one used by the sound system). > > The struture for the whole TCP/IP stack would then be: > > 1.- IP packets fetching from the I/F and buffering: > - External interrupt handler (best), or fast interrupt polling loop. > 2.- IP packets reassembling and acknowledging: > - polled task: fast interrupt handler (best) or frame interrupt. > 3.- TCP/IP high level protocols: > - High priority (127) job. > > How does this sound ? I remember that I thought about a similar structure when I was still in the very beginning of this project and had not delt with the details. Firstly your structure would not allow 10 Mbit/s TCP streams in a normal LAN. (Note that level 1 won't generate TCP ACKs.) Second, I wonder what makes you think there were packet acknowledgements on IP level. Third, it is dangerous to let TCP processing run at highest priority. If there is very much TCP traffic (let's for a moment take into acount a wider range of machines than Q60, or malicious traffic) your user interface may stop working. ... The list could go on. I for one have 3 different philosphies: One in case QDOS is fixed and I have a lot of time, the second in case QDOS is fixed and I'm short of time, the third in case QDOS isn't fixed :-) Thierry, I don't search for theoretical advice on OS and drivers str
