RE: Interrupt vs. polling on -current
On Sun, 11 Aug 2002, Maksim Yevmenkin wrote: On Fri, 9 Aug 2002, Maksim Yevmenkin wrote: OS: FreeBSD-current DP1 (dmesg attached) Laptop: Toshiba Tecra 8100 (docked) Hardware: 3Com Bluetooth USB dongle, 3Com Bluetooth PC-CARD Xircom CBT PC-CARD (with 16550A UART) First of all, irq 11 gets shared between PC-CARD controller, USB controller, NIC in docking station (see dmesg). Everything This configuration should be expected to work poorly at best. hmmm... i have a couple of latops here at home, one made by Toshiba and another by IBM and they both have similar configuration. may be such configuration is normal for laptops? Unfortunately it seems to be fairly normal. Warner knows a lot more about this than me (see his reply). I don't even have a laptop. I expect this problem will be fixed by doing more polling for USB devices in a more automatic way. I hoped that more irqs would be standard by now, but instead they seem to be standard on xaly on big SMP systems that don't need them so much. It's systems with lots of interactive and/or real-time deives that need them more. The Xircom card just does not work :( I' getting a lot of silo overflow messages no matter what i try. I checked list archives and source - not much look. Is sio driver totally hopeless? No, but the 3Com driver apparently is. The sio driver wants to have fast interrupts. It can't have them with the irq is shared, so its worst-case interrupt latency for a single serial port is increased from about 50 usec to many msec, depending other interrupt activity in the system (not limited to that for the shared irq except in some configurations). Silo overflows occur at 115200 bps when the latency is more than about 1.5 msec. perhaps, i said it wrong. I only plug *one* PC-CARD at a time, so it only 3com *or* Xircom. So irq11 gets shared between USB, NIC in docking station, PC-CARD controller and Xircom card. That should help a bit. There is still the software problem that the irq wiring for attached devices doesn't change when some devices using the interrupt are detached, so the sio interrupt probably remains shared and non-fast so sio interrupt handling is often delayed by other interrupt activity (if any). Hmmm, there is always some interrupt activity for clocks. Perhaps the largest source of latency is that timeouts have (or not so temporarily borrow) too high a priority in -current. BTW, i see silo overflow messages when i run ppp via null-modem cable. in this configuration i'm using serial port 0 which is on board and hase irq 4 with fast interrupts. What else is active when you get these? This points to bug(s) in the 3Com driver. Perhaps its interrupt handler just runs for too long to determine that irq11's for the serial device are not for it. Running in polled mode decreases yes, and that is what i was thinking too. but now i think it is not only 3com driver's fault. The driver just reads one port and check one bit, if it not set then interrupt is not for it. Yes, this shows that it is not primarily the 3Com driver's fault. I just can't believe that FreeBSD on my Pentium-III/600 can't handle lousy 500-700 interrupts a second from PC-CARD. Can anyone point me into right direction, because i'm obviously doing something wrong here. FreeBSD on a 486/33 can handle about 4 serial interrupts per second to do one character of i/o per interrupt). Pessimizations in -current have only degraded this by a small factor (2?), provided the driver uses fast interrupts. Lose another small factor (2?) for normal interrupts (using normal interrupts only loses a large factor for latency). if my calculations are correct -current should handle about 10,000 interrupt/sec from sio, right? i'm sorry, but it is not what i see here. That's 1 on a 486/33 :-). Bruce To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Interrupt vs. polling on -current
M. Warner Losh wrote: [[ I've read the rest of this thread ]] In message: [EMAIL PROTECTED] Maksim Yevmenkin [EMAIL PROTECTED] writes: : My tests are very simple. I plug USB dongle and one PC-CARD : and try to pump data between them as fast as possible. The : data blocks sizes are between 63 and 1500 bytes. I think that the issue here may be that the USB stack may be interacting badly with the interrupt system. Since the pccard is effectively forced into sharing the interrupt with the USB stack with NEWCARD, that's going to interact very badly, I think. Worse even than if you were multiplexing the xl0 driver and the sio0 driver. you may be right. i have uhci USB controller (i think it is Intel spec.). If i'm not mistaken this chip is not very smart, and every time interrupt happens software must scan queue of pending requests to find out which one has been completed. But of course uhci chip is very cheap :) On the other hand there is a ohci (Microsoft i think) and this chip is better because it does more processing, and, of course costs more. The sio driver is extremely sensitive to the interrupt latency, since the underlying hardware has only a few characters to react before the fifo overflows. At 115200 baud, each character takes 1/11520 seconds (or 86us), and the fifo is set to have only 8 bytes left when the device interrupts. This means that the sio driver can only tolerate about 700us of latency before there are issues. With 500 interrupts per second, that's an interrupt every 2000us, which is about the same order as the maximum latency tolerance. If these interrupts are shared and are taking a while to allow the serial driver to run, then you are almost certain to cause problems. Xircom card is a Bluetooth card with UART interface. Bluetooth link speed is about 700KB/s. I do not think that standard 16550A UART with 16 bytes FIFO can handle such speed. I also can not believe that Xircom would sell such low speed Bluetooth card (115200 bit/s). I think that Xircom card must have a much bigger FIFO in order to handle full speed. W2K, for example, has special driver for Xircom card. For cardbus cards, we're forced to use the pci interrupt and share it. For r2 cards (aka 16-bit cards), we could use an unused ISA interrupt on most (but not all) cardbus bridges, but currently there's no support for that in NEWCARD. that is what i see in W2K. all cards i plug into slots get different irq. PC-CARD controller and USB controller are still share irq 11. I'm also surprised you were able to make it work at all. I've had lots of bad luck when trying to make modems work in recent -current kernels. that is very sad :( thanks, max To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Interrupt vs. polling on -current
On Mon, 12 Aug 2002, Maksim Yevmenkin wrote: Bruce Evans wrote: On Sun, 11 Aug 2002, Maksim Yevmenkin wrote: [...] BTW, i see silo overflow messages when i run ppp via null-modem cable. in this configuration i'm using serial port 0 which is on board and hase irq 4 with fast interrupts. What else is active when you get these? not much: X, user space ppp and i'm doing moderate size transfers via ftp between my current laptop and FreeBSD 4.5 desktop. i also see these errors while cvsup'ing my -current laptop. Some graphics cards can cause sio problems, by using too bus cycles so that sio doesn't get even an average of about 5 i/o cycles per character time that it needs to function. 5 might sound like not many, but it is 5 isa cycles so it takes 6 usec every 87 usec at 115200 bps. 20 devices wanting that much of the bus just won't work. Bruce To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Interrupt vs. polling on -current
On Mon, 12 Aug 2002, Maksim Yevmenkin wrote: Xircom card is a Bluetooth card with UART interface. Bluetooth link speed is about 700KB/s. I do not think that standard 16550A UART with 16 bytes FIFO can handle such speed. I also can not believe that Xircom would sell such low speed Bluetooth card (115200 bit/s). I think that Xircom card must have a much bigger FIFO in order to handle full speed. W2K, for example, has special driver for Xircom card. Big fifos for dumb hardware can also cause by problems by taking a long time to drain, thus increasing latency for other devices. dumb hardware here means almost anything that must be accessed in PIO mode. One of the main reasons why sio uses a fast interrupt handler was that such hardware was common 10-15 years ago (for pre-DMA IDE disks). In Linux the disk driver (sort of) used a fast interrupt handler too, and competition between these handlers caused many silo overflows. Bruce To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Interrupt vs. polling on -current
Bruce Evans wrote: On Mon, 12 Aug 2002, Maksim Yevmenkin wrote: Bruce Evans wrote: On Sun, 11 Aug 2002, Maksim Yevmenkin wrote: [...] BTW, i see silo overflow messages when i run ppp via null-modem cable. in this configuration i'm using serial port 0 which is on board and hase irq 4 with fast interrupts. What else is active when you get these? not much: X, user space ppp and i'm doing moderate size transfers via ftp between my current laptop and FreeBSD 4.5 desktop. i also see these errors while cvsup'ing my -current laptop. Some graphics cards can cause sio problems, by using too bus cycles so that sio doesn't get even an average of about 5 i/o cycles per character time that it needs to function. 5 might sound like not many, but it is 5 isa cycles so it takes 6 usec every 87 usec at 115200 bps. 20 devices wanting that much of the bus just won't work. WHOA! that scares the hell out of me. You sound like any hardware that uses interrupt/bus might hurt poor sio driver pretty badly. There is not much use in computer that has only CPU and serial port :) i'm not trying to blame anybody here, but rather understand what is wrong. BTW, i have s3 savage/mx and neomagic/av video cards in my laptops. thanks, max To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Interrupt vs. polling on -current
On Sat, 10 Aug 2002, Terry Lambert wrote:
Bruce Evans wrote:
No, but the 3Com driver apparently is. The sio driver wants to have
fast interrupts. It can't have them with the irq is shared, so its
worst-case interrupt latency for a single serial port is increased
from about 50 usec to many msec, depending other interrupt activity
in the system (not limited to that for the shared irq except in some
configurations). Silo overflows occur at 115200 bps when the latency
is more than about 1.5 msec.
Anyway to get the code to complain about the sharing of serial
interrupts?
It already complains: from sio.c:
% ret = BUS_SETUP_INTR(device_get_parent(dev), dev, com-irqres,
%INTR_TYPE_TTY | INTR_FAST,
%siointr, com, com-cookie);
% if (ret) {
% ret = BUS_SETUP_INTR(device_get_parent(dev), dev,
%com-irqres, INTR_TYPE_TTY,
%siointr, com, com-cookie);
% if (ret == 0)
% device_printf(dev, unable to activate interrupt in
fast mode - using normal mode\n);
Also, if there is a PCI interrupt for the serial (serial handled
by Northbridge... I'd like to see this, actually), shouldn't it
be capable of sharing *only* fast interrupts somehow? It's an
Yes, this should work. It mainly needs a multiplexer like the old one
for ordinary shared irqs (but even simpler since it doesn't need or
want to change the ipl (soft or hard)) and lots of configuation cruft.
obvious pessimization to mix other interrupts with fast interrupts,
but less obvious what would happen with fast + fast...
It's more than a pessimization; it is impossible by definition, since
sharing fast interrupts (at the same time) makes them non-fast. Another
thing that bus_setup_intr() doesn't do is transparently enough is changing
the setup as devices with different requirements come and go.
FreeBSD on a 486/33 can handle about 4 serial interrupts per second
to do one character of i/o per interrupt). Pessimizations in -current
have only degraded this by a small factor (2?), provided the driver
uses fast interrupts. Lose another small factor (2?) for normal interrupts
(using normal interrupts only loses a large factor for latency).
Any way to fix this, short of don't run -current??
There's also use a fast CPU (almost any new one). Some of the overheads
are related to I/O, so you won't noticed 2x or 4x pessimizations of software
overheads once the I/O overheads are very dominant.
Bruce
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-current in the body of the message
RE: Interrupt vs. polling on -current
On Fri, 9 Aug 2002, Maksim Yevmenkin wrote: OS: FreeBSD-current DP1 (dmesg attached) Laptop: Toshiba Tecra 8100 (docked) Hardware: 3Com Bluetooth USB dongle, 3Com Bluetooth PC-CARD Xircom CBT PC-CARD (with 16550A UART) First of all, irq 11 gets shared between PC-CARD controller, USB controller, NIC in docking station (see dmesg). Everything This configuration should be expected to work poorly at best. hmmm... i have a couple of latops here at home, one made by Toshiba and another by IBM and they both have similar configuration. may be such configuration is normal for laptops? Both laptop/cards seems to work fine in W2K. I do not know much about PC-CARD controllers, but somehow each PC-CARD i plug into slot gets a different interrupt line. [ ... ] The Xircom card just does not work :( I' getting a lot of silo overflow messages no matter what i try. I checked list archives and source - not much look. Is sio driver totally hopeless? No, but the 3Com driver apparently is. The sio driver wants to have fast interrupts. It can't have them with the irq is shared, so its worst-case interrupt latency for a single serial port is increased from about 50 usec to many msec, depending other interrupt activity in the system (not limited to that for the shared irq except in some configurations). Silo overflows occur at 115200 bps when the latency is more than about 1.5 msec. perhaps, i said it wrong. I only plug *one* PC-CARD at a time, so it only 3com *or* Xircom. So irq11 gets shared between USB, NIC in docking station, PC-CARD controller and Xircom card. BTW, i see silo overflow messages when i run ppp via null-modem cable. in this configuration i'm using serial port 0 which is on board and hase irq 4 with fast interrupts. This morning i change 3Com driver to use polling, and, to my extreme surprise it work much, much better now. Also the interrupt load (according to top) has reduced to at least half. I have not noticed any system slow down. So what is up this that? Does that mean that for slow devices like serial ports etc. polling is better? This points to bug(s) in the 3Com driver. Perhaps its interrupt handler just runs for too long to determine that irq11's for the serial device are not for it. Running in polled mode decreases yes, and that is what i was thinking too. but now i think it is not only 3com driver's fault. The driver just reads one port and check one bit, if it not set then interrupt is not for it. the number of interrupts that it looks at (if there are a lot of serial interrupts), and prevents the 3Com interrupt handling from interfering with serial interrupt handling (because timeouts have lower priority than all other interrupts). just like i said, there is *only one* card in the PC-CARD slot and 3Com *USB* dongle. [ ... ] I just can't believe that FreeBSD on my Pentium-III/600 can't handle lousy 500-700 interrupts a second from PC-CARD. Can anyone point me into right direction, because i'm obviously doing something wrong here. FreeBSD on a 486/33 can handle about 4 serial interrupts per second to do one character of i/o per interrupt). Pessimizations in -current have only degraded this by a small factor (2?), provided the driver uses fast interrupts. Lose another small factor (2?) for normal interrupts (using normal interrupts only loses a large factor for latency). if my calculations are correct -current should handle about 10,000 interrupt/sec from sio, right? i'm sorry, but it is not what i see here. thanks, max To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Interrupt vs. polling on -current
[[ I've read the rest of this thread ]] In message: [EMAIL PROTECTED] Maksim Yevmenkin [EMAIL PROTECTED] writes: : My tests are very simple. I plug USB dongle and one PC-CARD : and try to pump data between them as fast as possible. The : data blocks sizes are between 63 and 1500 bytes. I think that the issue here may be that the USB stack may be interacting badly with the interrupt system. Since the pccard is effectively forced into sharing the interrupt with the USB stack with NEWCARD, that's going to interact very badly, I think. Worse even than if you were multiplexing the xl0 driver and the sio0 driver. The sio driver is extremely sensitive to the interrupt latency, since the underlying hardware has only a few characters to react before the fifo overflows. At 115200 baud, each character takes 1/11520 seconds (or 86us), and the fifo is set to have only 8 bytes left when the device interrupts. This means that the sio driver can only tolerate about 700us of latency before there are issues. With 500 interrupts per second, that's an interrupt every 2000us, which is about the same order as the maximum latency tolerance. If these interrupts are shared and are taking a while to allow the serial driver to run, then you are almost certain to cause problems. For cardbus cards, we're forced to use the pci interrupt and share it. For r2 cards (aka 16-bit cards), we could use an unused ISA interrupt on most (but not all) cardbus bridges, but currently there's no support for that in NEWCARD. I'm also surprised you were able to make it work at all. I've had lots of bad luck when trying to make modems work in recent -current kernels. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Interrupt vs. polling on -current
On Fri, 9 Aug 2002, Maksim Yevmenkin wrote: OS: FreeBSD-current DP1 (dmesg attached) Laptop: Toshiba Tecra 8100 (docked) Hardware: 3Com Bluetooth USB dongle, 3Com Bluetooth PC-CARD Xircom CBT PC-CARD (with 16550A UART) First of all, irq 11 gets shared between PC-CARD controller, USB controller, NIC in docking station (see dmesg). Everything This configuration should be expected to work poorly at best. that i plug into PC-CARD slots also get irq 11 (which is normal - i think). Also i have disabled interrupt harvesting. My tests are very simple. I plug USB dongle and one PC-CARD and try to pump data between them as fast as possible. The data blocks sizes are between 63 and 1500 bytes. The Xircom card just does not work :( I' getting a lot of silo overflow messages no matter what i try. I checked list archives and source - not much look. Is sio driver totally hopeless? No, but the 3Com driver apparently is. The sio driver wants to have fast interrupts. It can't have them with the irq is shared, so its worst-case interrupt latency for a single serial port is increased from about 50 usec to many msec, depending other interrupt activity in the system (not limited to that for the shared irq except in some configurations). Silo overflows occur at 115200 bps when the latency is more than about 1.5 msec. The 3Com PC-CARD is a little bit better, but after some time it also starting to drop bytes. This morning i change 3Com driver to use polling, and, to my extreme surprise it work much, much better now. Also the interrupt load (according to top) has reduced to at least half. I have not noticed any system slow down. So what is up this that? Does that mean that for slow devices like serial ports etc. polling is better? This points to bug(s) in the 3Com driver. Perhaps its interrupt handler just runs for too long to determine that irq11's for the serial device are not for it. Running in polled mode decreases the number of interrupts that it looks at (if there are a lot of serial interrupts), and prevents the 3Com interrupt handling from interfering with serial interrupt handling (because timeouts have lower priority than all other interrupts). Does anyone tried to use polling in sio driver? Sure. This was configurable by leaving out the interrupt specification in the device line. Now, leaving out the interrupt spec in the hints line should work for isa devices. I don't know of any way to configure it for pccard devices short of editing the source code. Polling serial devices gives much the same latency problems/silo overflows as not using fast interrupts, but worse since the polling interval is limited by default HZ and the default HZ is too small for it to work except at low serial speeds (HZ = 100 only permits 8250/16550 UARTs to work at 600/9600 bps, respectively). You never want to use polling for serial devices, but it may be required as a hack if there is no usable irq (this sometimes happens for isa devices because isa irqs are not shareable). I just can't believe that FreeBSD on my Pentium-III/600 can't handle lousy 500-700 interrupts a second from PC-CARD. Can anyone point me into right direction, because i'm obviously doing something wrong here. FreeBSD on a 486/33 can handle about 4 serial interrupts per second to do one character of i/o per interrupt). Pessimizations in -current have only degraded this by a small factor (2?), provided the driver uses fast interrupts. Lose another small factor (2?) for normal interrupts (using normal interrupts only loses a large factor for latency). Bruce To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Interrupt vs. polling on -current
Bruce Evans wrote: No, but the 3Com driver apparently is. The sio driver wants to have fast interrupts. It can't have them with the irq is shared, so its worst-case interrupt latency for a single serial port is increased from about 50 usec to many msec, depending other interrupt activity in the system (not limited to that for the shared irq except in some configurations). Silo overflows occur at 115200 bps when the latency is more than about 1.5 msec. Anyway to get the code to complain about the sharing of serial interrupts? Also, if there is a PCI interrupt for the serial (serial handled by Northbridge... I'd like to see this, actually), shouldn't it be capable of sharing *only* fast interrupts somehow? It's an obvious pessimization to mix other interrupts with fast interrupts, but less obvious what would happen with fast + fast... FreeBSD on a 486/33 can handle about 4 serial interrupts per second to do one character of i/o per interrupt). Pessimizations in -current have only degraded this by a small factor (2?), provided the driver uses fast interrupts. Lose another small factor (2?) for normal interrupts (using normal interrupts only loses a large factor for latency). Any way to fix this, short of don't run -current?? Thanks, -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
RE: Interrupt vs. polling on -current
500 - 700 interrupts a second is a lot, depending on how long you stay in the interrupt handler before returning. 1 cent of info Johan -Original Message- From: ext Maksim Yevmenkin [mailto:[EMAIL PROTECTED]] Sent: Friday, August 09, 2002 2:19 PM To: [EMAIL PROTECTED] Subject: Interrupt vs. polling on -current Hackers, Can anyone shed some light on the following problem: OS: FreeBSD-current DP1 (dmesg attached) Laptop: Toshiba Tecra 8100 (docked) Hardware: 3Com Bluetooth USB dongle, 3Com Bluetooth PC-CARD Xircom CBT PC-CARD (with 16550A UART) First of all, irq 11 gets shared between PC-CARD controller, USB controller, NIC in docking station (see dmesg). Everything that i plug into PC-CARD slots also get irq 11 (which is normal - i think). Also i have disabled interrupt harvesting. My tests are very simple. I plug USB dongle and one PC-CARD and try to pump data between them as fast as possible. The data blocks sizes are between 63 and 1500 bytes. The Xircom card just does not work :( I' getting a lot of silo overflow messages no matter what i try. I checked list archives and source - not much look. Is sio driver totally hopeless? The 3Com PC-CARD is a little bit better, but after some time it also starting to drop bytes. This morning i change 3Com driver to use polling, and, to my extreme surprise it work much, much better now. Also the interrupt load (according to top) has reduced to at least half. I have not noticed any system slow down. So what is up this that? Does that mean that for slow devices like serial ports etc. polling is better? Does anyone tried to use polling in sio driver? I just can't believe that FreeBSD on my Pentium-III/600 can't handle lousy 500-700 interrupts a second from PC-CARD. Can anyone point me into right direction, because i'm obviously doing something wrong here. thanks, max To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
