Re: Why is em nic generating interrupts?
On Fri, Oct 10, 2003 at 12:54:55PM -0800, Terry Lambert wrote: If your FXP is not generating any interrupts at all, i think that the polling code in it is probably broken. Is the polling code in -current different to that in -stable? I have a system running 4.6-STABLE (or so) with DEVICE_POLLING and turning kern.polling.enable on/off makes the fxp interrupt count go from 0 to several thousand/sec. The system is primarily routing packets thru various dummynet pipes and natd and is not heavily loaded. Peter ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Why is em nic generating interrupts?
- Original Message - From: Terry Lambert [EMAIL PROTECTED] To: Michael O. Boev [EMAIL PROTECTED]; Terry Lambert [EMAIL PROTECTED] Cc: [EMAIL PROTECTED] Sent: Saturday, October 11, 2003 3:54 AM Subject: RE: Why is em nic generating interrupts? Michael O. Boev wrote: From: Terry Lambert [mailto:[EMAIL PROTECTED] Michael O. Boev wrote: I've got a [uniprocessor 5.1-RELEASE] router machine with fxp and em nics. I've built my kernel with the following included: options DEVICE_POLLING options HZ=2500 and enabled polling in /etc/sysctl.conf. [ ... ] What's happening? Is polling working in my case? If yes, why is vmstat showing interrupts? I see clearly, that fxp's counter doesn't increase, and em's is constantly growing. Is there anyone who knows for sure that em's polling works? You may want to ask Luigi; polling is his code. However, I believe the issue is that polling doesn't start until you take an interrupt, and it stops as soon as there is no more data to process, and waits for the next interrupt. If you were to jack your load way up, you would probably see an increase in interrupts, then them dropping off dramatically. To this dare I object, that there is traffic going through this machine, and fxp0 is NOT generating interrupts, while em IS. So, if the rule above works, they both have to behave in same ways. This is the third time I've received this message. I guess you feel some urgency about receiving a reply? Oh, no )). I even didn't send it for 3 times. Sorry for annoyance. My answer is this: I did the polling code for a number of the drivers, and though it was committed by someone else, I can guarantee you that it will have interrupts, unless the code was implemented incorreectly, since the code is meant to do this polling only under heavy load. I believe you... I just wondered why two drivers behaved differently. ... If your FXP is not generating any interrupts at all, i think that the polling code in it is probably broken. Note: I believe this might be one of the ones I modified, since Luigi's code originally only dealt with Gigabit NICs, though I may be mistaken. If so, my statement that it's broken should be considered to be pretty authoritative. 8-). Then, OK, it is now fxp's polling that probably behaves oddly :)). Since I've ever been only using fxps, I got used to this behaviour. Thus my questions. -- Terry Thanks for thorough explanations! Mike. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Why is em nic generating interrupts?
Michael O. Boev wrote: From: Terry Lambert [mailto:[EMAIL PROTECTED] Michael O. Boev wrote: I've got a [uniprocessor 5.1-RELEASE] router machine with fxp and em nics. I've built my kernel with the following included: options DEVICE_POLLING options HZ=2500 and enabled polling in /etc/sysctl.conf. [ ... ] What's happening? Is polling working in my case? If yes, why is vmstat showing interrupts? I see clearly, that fxp's counter doesn't increase, and em's is constantly growing. Is there anyone who knows for sure that em's polling works? You may want to ask Luigi; polling is his code. However, I believe the issue is that polling doesn't start until you take an interrupt, and it stops as soon as there is no more data to process, and waits for the next interrupt. If you were to jack your load way up, you would probably see an increase in interrupts, then them dropping off dramatically. To this dare I object, that there is traffic going through this machine, and fxp0 is NOT generating interrupts, while em IS. So, if the rule above works, they both have to behave in same ways. This is the third time I've received this message. I guess you feel some urgency about receiving a reply? My answer is this: I did the polling code for a number of the drivers, and though it was committed by someone else, I can guarantee you that it will have interrupts, unless the code was implemented incorreectly, since the code is meant to do this polling only under heavy load. The purpose of going to a polling model is *not* to avoid interrupts, it's to avoid processing data into the input routines when you don't have enough CPU cycles to deal with it before more data comes in. In other words, it's to deal with the situation where you would end up in receiver livelock as a result of taking so many interrupts that you never get an oportunity to process the packets out of the input queue into the per protocol input routines at NETISR, or from the top-of stack (socket buffers) into the user buffers in user space, or run the code in user space to take the data and generate a reply, or in kernel space to send the reply out. In other words, you *only ever* want to do polling when you have sufficient load to make polling worthwhile, and when you do it under low loads, you are actually *wasting* cycles, rather than *saving* them. Polling never *saves* cycles, per se, it only causes incoming data to be dropped (and, presumably, retransmitted) by the other end of the connection. For this, because there is not specific card support for it, and there's no interface entry points to deal with it, even if there were, the DMA engine on the card is *NOT* halted, you *still* eat bus transfer overhead, and it still eats up your PCI bandwidth, and this cost is *incredibly* larger than just interrupt and requeueing overhead. If your FXP is not generating any interrupts at all, i think that the polling code in it is probably broken. Note: I believe this might be one of the ones I modified, since Luigi's code originally only dealt with Gigabit NICs, though I may be mistaken. If so, my statement that it's broken should be considered to be pretty authoritative. 8-). -- Terry ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Why is em nic generating interrupts?
Michael O. Boev wrote: I've got a [uniprocessor 5.1-RELEASE] router machine with fxp and em nics. I've built my kernel with the following included: options DEVICE_POLLING options HZ=2500 and enabled polling in /etc/sysctl.conf. [ ... ] What's happening? Is polling working in my case? If yes, why is vmstat showing interrupts? I see clearly, that fxp's counter doesn't increase, and em's is constantly growing. Is there anyone who knows for sure that em's polling works? You may want to ask Luigi; polling is his code. However, I believe the issue is that polling doesn't start until you take an interrupt, and it stops as soon as there is no more data to process, and waits for the next interrupt. If you were to jack your load way up, you would probably see an increase in interrupts, then them dropping off dramatically. If all else fails, read the source code... 8-). -- Terry ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Why is em nic generating interrupts?
-Original Message- From: Terry Lambert [mailto:[EMAIL PROTECTED] Sent: Thursday, October 09, 2003 5:19 PM To: Michael O. Boev Cc: [EMAIL PROTECTED] Subject: Re: Why is em nic generating interrupts? Michael O. Boev wrote: I've got a [uniprocessor 5.1-RELEASE] router machine with fxp and em nics. I've built my kernel with the following included: options DEVICE_POLLING options HZ=2500 and enabled polling in /etc/sysctl.conf. [ ... ] What's happening? Is polling working in my case? If yes, why is vmstat showing interrupts? I see clearly, that fxp's counter doesn't increase, and em's is constantly growing. Is there anyone who knows for sure that em's polling works? You may want to ask Luigi; polling is his code. However, I believe the issue is that polling doesn't start until you take an interrupt, and it stops as soon as there is no more data to process, and waits for the next interrupt. If you were to jack your load way up, you would probably see an increase in interrupts, then them dropping off dramatically. To this dare I object, that there is traffic going through this machine, and fxp0 is NOT generating interrupts, while em IS. So, if the rule above works, they both have to behave in same ways. If all else fails, read the source code... 8-). )) I've tried to, but... had to ask here. So all is left is to ask Luigi and Intel. -- Terry -- Best wishes, [EMAIL PROTECTED] ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: Why is em nic generating interrupts?
From: Terry Lambert [mailto:[EMAIL PROTECTED] Michael O. Boev wrote: I've got a [uniprocessor 5.1-RELEASE] router machine with fxp and em nics. I've built my kernel with the following included: options DEVICE_POLLING options HZ=2500 and enabled polling in /etc/sysctl.conf. [ ... ] What's happening? Is polling working in my case? If yes, why is vmstat showing interrupts? I see clearly, that fxp's counter doesn't increase, and em's is constantly growing. Is there anyone who knows for sure that em's polling works? You may want to ask Luigi; polling is his code. However, I believe the issue is that polling doesn't start until you take an interrupt, and it stops as soon as there is no more data to process, and waits for the next interrupt. If you were to jack your load way up, you would probably see an increase in interrupts, then them dropping off dramatically. If all else fails, read the source code... 8-). FWIW, this works for me with 4.7. As terry says, you do see a couple of initial interrupts, as below, and then they stop. $ vmstat -i interrupt total rate em0 irq16 4 0 em1 irq17 4 0 ahc0 irq18 5653 0 ahc1 irq19 15 0 em2 irq20 4 0 mux irq21 3 0 sio0 irq41583 0 sio1 irq3 1 0 clk irq0 14545633 2501 rtc irq8 744224128 Total15297124 2631 ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Why is em nic generating interrupts?
On Thursday 09 October 2003 05:57 am, Michael O. Boev wrote: -Original Message- From: Terry Lambert [mailto:[EMAIL PROTECTED] Sent: Thursday, October 09, 2003 5:19 PM To: Michael O. Boev Cc: [EMAIL PROTECTED] Subject: Re: Why is em nic generating interrupts? Michael O. Boev wrote: I've got a [uniprocessor 5.1-RELEASE] router machine with fxp and em nics. I've built my kernel with the following included: options DEVICE_POLLING options HZ=2500 and enabled polling in /etc/sysctl.conf. [ ... ] What's happening? Is polling working in my case? If yes, why is vmstat showing interrupts? I see clearly, that fxp's counter doesn't increase, and em's is constantly growing. Is there anyone who knows for sure that em's polling works? You may want to ask Luigi; polling is his code. However, I believe the issue is that polling doesn't start until you take an interrupt, and it stops as soon as there is no more data to process, and waits for the next interrupt. If you were to jack your load way up, you would probably see an increase in interrupts, then them dropping off dramatically. To this dare I object, that there is traffic going through this machine, and fxp0 is NOT generating interrupts, while em IS. So, if the rule above works, they both have to behave in same ways. If all else fails, read the source code... 8-). )) I've tried to, but... had to ask here. So all is left is to ask Luigi and Intel. I cannot comment on 5.1-R, but I am running DEVICE_POLLING tests today with -current and 2 em NICS and systat -vm shows no interrupts for the IRQs where the NICs are. I can only guess that either 5.1-R has a bug or the polling support was not in the driver at that point. Sam ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to [EMAIL PROTECTED]
