Re: re(4) driver dropping packets when reading NFS files
On Mon, Dec 26, 2011 at 10:55:50PM -0500, Rick Macklem wrote:
Way back in Nov 2010, this thread was related to a problem I
had, where an re(4) { 810xE PCIe 10/100baseTX, according to the
driver } interface dropped received packets, resulting in a
significant impact of NFS performance.
Well, it turns out that a recent (post r224506) commit seems to
have fixed the problem. It hasn't dropped any packets since I
upgraded to a kernel with a r228281 version of if_re.c.
So, good news.
Thanks to those maintaining this driver, rick
ps: If you have a need to know which commit fixed this, I can
probably test variants to find out. Otherwise, I'm just
happy that it's fixed.:-)
Glad to know the issue was fixed. Probably the change made in
r227593 or 227854 might have fixed it.
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
Way back in Nov 2010, this thread was related to a problem I
had, where an re(4) { 810xE PCIe 10/100baseTX, according to the
driver } interface dropped received packets, resulting in a
significant impact of NFS performance.
Well, it turns out that a recent (post r224506) commit seems to
have fixed the problem. It hasn't dropped any packets since I
upgraded to a kernel with a r228281 version of if_re.c.
So, good news.
Thanks to those maintaining this driver, rick
ps: If you have a need to know which commit fixed this, I can
probably test variants to find out. Otherwise, I'm just
happy that it's fixed.:-)
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
By chance, how about disabling RX early interrupt? You can add the following 3 lines of code into re_init_locked(). 2710 /* 2711 * Set the initial RX configuration. 2712 */ 2713 re_set_rxmode(sc); 2714 2715 /* Disable RX early interrupt. */ 2716 cfg = CSR_READ_2(sc, RL_MULTIINTR); ^^^ 2717 cfg = 0xF000; ^^ 2718 CSR_WRITE_2(sc, RL_MULTIINTR, cfg); ^^^ 2719 2720 #ifdef DEVICE_POLLING Afraid it didn't help. I've also discovered that the June 7 kernel only performs better sometimes. I see 500Kbytes/sec read rate on it for most of the test runs. (It does work better sometimes, but maybe the -current kernel would too, if I ran for long enough with it.) I can try some other variations of enabling interrupts in early Dec, rick ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
I've added a counter of how many times re_rxeof() gets called, but then returns without handling any received packets (I think because RL_RDESC_STAT_OWN is set on the first entry it looks at in the rcv. ring.) This count comes out as almost the same as the # of missed frames (see rxe did 0: in the attached stats). So, I think what is happenning about 10% of the time is that re_rxeof() is looking at the ring descriptor before it has been updated and then returns without handling the packet and then it doesn't get called again because the RL_ISR bit has been cleared. When options DEVICE_POLLING is specified, it works ok, because it calls re_rxeof() fairly frequently and it doesn't pay any attention to the RL_ISR bits. That's one of possible theory. See below for another theory. Well, my above theory is bunk. I hacked it so it would continue to run re_int_task() until the receive descriptor was valid and it just hummed away until the next frame was received. Now, I don't know if this is a hardware flaw on this machine or something that can be fixed in software? I know diddly about the current driver I highly doubt it could be hardware issue. You have much more confidence in the hardware guys than I do;-) (see summary below) Another possibility I have in mind is the controller would have reported RL_ISR_RX_OVERRUN but re(4) didn't check that condition. The old data sheet I have does not clearly mention when it is set and what other bits of RL_ISR register is affected from the bit. If RL_ISR_RX_OVERRUN bit is set when there are no more free RX descriptors available to controller and RL_ISR_RX_ERR is not set when RL_ISR_RX_OVERRUN is set, re(4) have ignored that event. Because driver ignored that error interrupt, the next error interrupt would be RL_ISR_FIFO_OFLOW error and this time it would be served in re_rxeof() and will refill RX buffers. However driver would have lost lots of frames received between the time window RL_ISR_RX_OVERRUN error and RL_ISR_FIFO_OFLOW error. RL_ISR_RX_OVERRUN never gets set during the testing I do. If this theory is correct, the attached patch may mitigate the issue. Your re.intr.patch4 had no effect. I've tried a bunch of similar other ones that also had no effect. Otherwise, I can live with options DEVICE_POLLING. This turned out to be my mistake. options DEVICE_POLLING doesn't help. (I think I screwed up by running a test without it enabled and then re-running the test after enabling it. It ran fast because I had primed the cache, oops.) Summary sofar: - almost all variants I've tried result in a missed frame rate of 8-11%, which gives you about 500Kbytes/sec read rate. here are some of the things I tried that didn't change this: - disabling the hardware offload stuff like checksums - changing the size of the receive ring (except making it really small seemed to degrade perf. a little) - enabling use of the I/O map ** - as noted above options DEVICE_POLLING doesn't actually help - enabling RE_TX_MODERATION - changing the value of maxpkts in re_rxeof() smaller seems to make the miss rate slightly higher - when the above is happening, the # of times re_rxeof() finds the first descriptor in the receive ring with the frame still owned by the device is just slightly lower than the # of missed frames reported by the chip's stats. - about half of these interrupts are due to RL_ISR_FIFO_OFLOW and the other half RL_ISR_RX_OK. (As noted above, I never see RL_ISR_RX_OVERRUN.) The only combination that I've come up with that reduces the missed frame rate significantly is the little patch I passed along before. It does 3 things: - disables msi - doesn't clear RL_IMR in re_intr() - doesn't set RL_IMR in re_int_task(), since the bits weren't cleared (This little patch is attached, in case anyone is interested in trying it.) When I run with this variant the missed frame rate drops to less than 2% (1.something) and I get about 5Mbytes/sec read rate on the 100Mbps port. I'm out of ideas w.r.t. what to try. I think it takes someone who knows what actually causes the chip to miss a frame to solve it? I don't know why the little patch I descibe above reduces, but does not eliminate the missed frame problem. (Since the chip seems to know it missed the frame, it smells hardware related to me?) I'd think that RL_ISR_FIFO_OFLOW would mean that it ran out of rcv. buffers, but there's 256 of them (and I tried increasing it to 512 with no effect). Again, I think it takes someone familiar with the hardware to say why this would be happening? Anyhow, I'm out of theories/ideas. I can easily test any patch that seems like it ight help. Thanks for your assistance, rick --- if_re.c.sav 2010-11-07 09:26:33.0 -0500 +++ if_re.c 2010-11-07 09:27:12.0 -0500 @@ -157,7 +157,7 @@ #include miibus_if.h /* Tunables. */ -static int msi_disable = 0;
Re: re(4) driver dropping packets when reading NFS files
I highly doubt it could be hardware issue. Looks like the hardware guys may be off the hook. See below. It's job of bus_dma(9) and I don't think barrier instructions would be helpful here as I don't see out-of-order execution in RX handler. My current hunch is that something that changed between June 7 and June 15 in head/sys has caused the chip to have difficulties doing DMA, resulting in the Fifo overflows and approx. 10% missed frames. Let's kill driver bug. No one reported this kind of issue so far and I guess most users took it granted for the poor performance because they are using low end consumer grade controller. I think your driver is off the hook, too. re0 statistics: Transmit good frames : 101346 Receive good frames : 133390 Tx errors : 0 Rx errors : 0 Rx missed frames : 14394 Rx frame alignment errs : 0 Tx single collisions : 0 Tx multiple collisions : 0 Rx unicast frames : 133378 Rx broadcast frames : 0 Rx multicast frames : 12 Tx aborts : 0 Tx underruns : 0 rxe did 0: 14359 Seeing that someone thought it had worked ok a while back, I decided to try some old kernels I had lying about from head/-current. I found that the one I svn`d on June 7 works well (about 7Mbytes per sec read rate) whereas one svn`d on June 15 had the problem (about 500Kbytes per sec read rate). So what is different between these kernels: - if_re.c is identical - subr_dma.c has a simple change and porting the June 7 one over didn`t make the June 15 one work better - amd64`s busdma_machdep.c is identical so it must be something else. There are a bunch of changes to amd64`s pmap.c, which is why I`ve cc`d Alan, in case he might know if those changes could affect PCIe DMA or similar. Other than that, maybe someone else familiar with the PCIe DMA could look and see if a change done to head between June 7 and 15 might explain it. (and it could be something else, a DMA problem for the chip is just a guess) rick ps: Unfortunately I`ll be on the road for the next month, so I won`t be able to test patches until early Dec. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Sun, Nov 07, 2010 at 07:06:44PM -0500, Rick Macklem wrote: I highly doubt it could be hardware issue. Looks like the hardware guys may be off the hook. See below. It's job of bus_dma(9) and I don't think barrier instructions would be helpful here as I don't see out-of-order execution in RX handler. My current hunch is that something that changed between June 7 and June 15 in head/sys has caused the chip to have difficulties doing DMA, resulting in the Fifo overflows and approx. 10% missed frames. Let's kill driver bug. No one reported this kind of issue so far and I guess most users took it granted for the poor performance because they are using low end consumer grade controller. I think your driver is off the hook, too. re0 statistics: Transmit good frames : 101346 Receive good frames : 133390 Tx errors : 0 Rx errors : 0 Rx missed frames : 14394 Rx frame alignment errs : 0 Tx single collisions : 0 Tx multiple collisions : 0 Rx unicast frames : 133378 Rx broadcast frames : 0 Rx multicast frames : 12 Tx aborts : 0 Tx underruns : 0 rxe did 0: 14359 Seeing that someone thought it had worked ok a while back, I decided to try some old kernels I had lying about from head/-current. I found that the one I svn`d on June 7 works well (about 7Mbytes per sec read rate) whereas one svn`d on June 15 had the problem (about 500Kbytes per sec read rate). So what is different between these kernels: - if_re.c is identical - subr_dma.c has a simple change and porting the June 7 one over didn`t make the June 15 one work better - amd64`s busdma_machdep.c is identical so it must be something else. There are a bunch of changes to amd64`s pmap.c, which is why I`ve cc`d Alan, in case he might know if those changes could affect PCIe DMA or similar. Other than that, maybe someone else familiar with the PCIe DMA could look and see if a change done to head between June 7 and 15 might explain it. (and it could be something else, a DMA problem for the chip is just a guess) If that made difference, all other ethernet controllers would have suffered from the similar issues. rick ps: Unfortunately I`ll be on the road for the next month, so I won`t be able to test patches until early Dec. If you have some spare time please try attach one. I guess fast ethernet controller has smaller FIFO size than that of GigE controller so it is frequently triggered the issue on fast ethernet controller than GigE controllers. I still guess that there are cases that an interrupt is not correctly served such that driver missed a lot of frames. Index: sys/pci/if_rlreg.h === --- sys/pci/if_rlreg.h (revision 214897) +++ sys/pci/if_rlreg.h (working copy) @@ -218,9 +218,10 @@ #define RL_ISR_TX_OK 0x0004 #define RL_ISR_TX_ERR 0x0008 #define RL_ISR_RX_OVERRUN 0x0010 +#define RL_ISR_RX_DESC_UNAVAIL 0x0010 /* C+ only */ #define RL_ISR_PKT_UNDERRUN0x0020 #define RL_ISR_LINKCHG 0x0020 /* 8169 only */ -#define RL_ISR_FIFO_OFLOW 0x0040 /* 8139 only */ +#define RL_ISR_FIFO_OFLOW 0x0040 #define RL_ISR_TX_DESC_UNAVAIL 0x0080 /* C+ only */ #define RL_ISR_SWI 0x0100 /* C+ only */ #define RL_ISR_CABLE_LEN_CHGD 0x2000 @@ -236,12 +237,12 @@ #ifdef RE_TX_MODERATION #define RL_INTRS_CPLUS \ (RL_ISR_RX_OK|RL_ISR_RX_ERR|RL_ISR_TX_ERR| \ - RL_ISR_RX_OVERRUN|RL_ISR_PKT_UNDERRUN|RL_ISR_FIFO_OFLOW|\ + RL_ISR_RX_DESC_UNAVAIL|RL_ISR_PKT_UNDERRUN|RL_ISR_FIFO_OFLOW| \ RL_ISR_PCS_TIMEOUT|RL_ISR_SYSTEM_ERR|RL_ISR_TIMEOUT_EXPIRED) #else #define RL_INTRS_CPLUS \ (RL_ISR_RX_OK|RL_ISR_RX_ERR|RL_ISR_TX_ERR|RL_ISR_TX_OK| \ - RL_ISR_RX_OVERRUN|RL_ISR_PKT_UNDERRUN|RL_ISR_FIFO_OFLOW|\ + RL_ISR_RX_DESC_UNAVAIL|RL_ISR_PKT_UNDERRUN|RL_ISR_FIFO_OFLOW| \ RL_ISR_PCS_TIMEOUT|RL_ISR_SYSTEM_ERR|RL_ISR_TIMEOUT_EXPIRED) #endif @@ -873,9 +874,7 @@ int rl_twist_row; int rl_twist_col; int suspended; /* 0 = normal 1 = suspended */ -#ifdef DEVICE_POLLING int rxcycles; -#endif struct task rl_txtask; struct task rl_inttask; Index: sys/dev/re/if_re.c === --- sys/dev/re/if_re.c (revision 214897) +++ sys/dev/re/if_re.c (working copy) @@ -1860,7 +1860,7 @@ int i, total_len; struct rl_desc *cur_rx; u_int32_t rxstat, rxvlan; - int maxpkt = 16, rx_npkts = 0; + int rx_npkts = 0; RL_LOCK_ASSERT(sc); @@ -1872,7 +1872,7 @@ sc-rl_ldata.rl_rx_list_map, BUS_DMASYNC_POSTREAD |
Re: re(4) driver dropping packets when reading NFS files
If that made difference, all other ethernet controllers would have suffered from the similar issues. Well, some commit done between June 7 and June 15 made a difference, but I have no idea what or why. Also, I had a report of very poor read rate from someone using a bge(4) interface, but I have no reason to believe it is because of the same issue. If you have some spare time please try attach one. I guess fast ethernet controller has smaller FIFO size than that of GigE controller so it is frequently triggered the issue on fast ethernet controller than GigE controllers. I still guess that there are cases that an interrupt is not correctly served such that driver missed a lot of frames. Patch didn't help. If anything it degrading things a bit (stats are attached). rick re0 statistics: Tx frames : 102009 Rx frames : 133318 Tx errors : 0 Rx errors : 0 Rx missed frames : 15246 Rx frame alignment errs : 0 Tx single collisions : 0 Tx multiple collisions : 0 Rx unicast frames : 133306 Rx broadcast frames : 0 Rx multicast frames : 12 Tx aborts : 0 Tx underruns : 0 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Sun, Nov 07, 2010 at 08:14:31PM -0500, Rick Macklem wrote: If that made difference, all other ethernet controllers would have suffered from the similar issues. Well, some commit done between June 7 and June 15 made a difference, but I have no idea what or why. Also, I had a report of very poor read rate from someone using a bge(4) interface, but I have no reason to believe it is because of the same issue. If you have some spare time please try attach one. I guess fast ethernet controller has smaller FIFO size than that of GigE controller so it is frequently triggered the issue on fast ethernet controller than GigE controllers. I still guess that there are cases that an interrupt is not correctly served such that driver missed a lot of frames. Patch didn't help. If anything it degrading things a bit (stats are attached). By chance, how about disabling RX early interrupt? You can add the following 3 lines of code into re_init_locked(). 2710 /* 2711 * Set the initial RX configuration. 2712 */ 2713 re_set_rxmode(sc); 2714 2715 /* Disable RX early interrupt. */ 2716 cfg = CSR_READ_2(sc, RL_MULTIINTR); ^^^ 2717 cfg = 0xF000; ^^ 2718 CSR_WRITE_2(sc, RL_MULTIINTR, cfg); ^^^ 2719 2720 #ifdef DEVICE_POLLING ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Thu, Nov 04, 2010 at 09:31:30PM -0400, Rick Macklem wrote: If the counter was not wrapped, it seem you lost more than 10% out of total RX frames. This is a lot loss and there should be a way to mitigate it. I've attached a patch (to the if_re.c in head, not your patched variant) that works a lot better (about 5Mbytes/sec read rate). To get that, I had to disable msi and not clear the RL_IMR register in re_intr(). I suspect that a packet would be received between when the bits in RL_IMR were cleared and when they were set at the end of re_int_task() and those were getting lost. This patch doesn't completely fix the problem. (I added your stats collecting stuff to the if_re.c in head and attached the result, which still shows some lost packets. One thought is clearing the bits in RL_ISR in re_intr() instead of re_int_task(), but then I can't see a good way to pass the old value of the status reg. through to re_int_task()? Hmm, I still don't understand how the patch mitigates the issue. :-( The patch does not disable interrupts in interrupt handler so taskqueue runs with interrupt enabled. This may ensure not loosing interrupts but it may also generate many interrupts. By chance, did you check number of interrupts generated with/without your patch? I agree. I think all it did was create more interrupts so that re_rxeof() got called more frequently. (see below) The only guess I have at the moment is the writing RL_IMR in interrupt handler at the end of taskqueue might be not immediately reflected so controller can loose interrupts for the time window. Would you try attach patch and let me know it makes any difference? Nope, it didn't make any difference. I've added a counter of how many times re_rxeof() gets called, but then returns without handling any received packets (I think because RL_RDESC_STAT_OWN is set on the first entry it looks at in the rcv. ring.) This count comes out as almost the same as the # of missed frames (see rxe did 0: in the attached stats). So, I think what is happenning about 10% of the time is that re_rxeof() is looking at the ring descriptor before it has been updated and then returns without handling the packet and then it doesn't get called again because the RL_ISR bit has been cleared. When options DEVICE_POLLING is specified, it works ok, because it calls re_rxeof() fairly frequently and it doesn't pay any attention to the RL_ISR bits. Now, I don't know if this is a hardware flaw on this machine or something that can be fixed in software? I know diddly about the current driver setup, but I assume something like this has to happen? - chip (or PCIe handler) forces the DMA of the descriptor change to RAM - then posts the interrupt - and the driver must read the up to date descriptor from RAM -- I notice that volatile isn't used anywhere in the driver. Wouldn't the descriptor ring memory have to be defined volatile somehow? (I don't know how to do this correctly?) So, if you can think of anything that will ensure that re_rxeof() reads up to date descriptor ring entries, please let me know. Otherwise, I can live with options DEVICE_POLLING. Thanks for your help, rick re0 statistics: Transmit good frames : 101346 Receive good frames : 133390 Tx errors : 0 Rx errors : 0 Rx missed frames : 14394 Rx frame alignment errs : 0 Tx single collisions : 0 Tx multiple collisions : 0 Rx unicast frames : 133378 Rx broadcast frames : 0 Rx multicast frames : 12 Tx aborts : 0 Tx underruns : 0 rxe did 0: 14359 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Fri, Nov 05, 2010 at 07:44:56PM -0400, Rick Macklem wrote: On Thu, Nov 04, 2010 at 09:31:30PM -0400, Rick Macklem wrote: If the counter was not wrapped, it seem you lost more than 10% out of total RX frames. This is a lot loss and there should be a way to mitigate it. I've attached a patch (to the if_re.c in head, not your patched variant) that works a lot better (about 5Mbytes/sec read rate). To get that, I had to disable msi and not clear the RL_IMR register in re_intr(). I suspect that a packet would be received between when the bits in RL_IMR were cleared and when they were set at the end of re_int_task() and those were getting lost. This patch doesn't completely fix the problem. (I added your stats collecting stuff to the if_re.c in head and attached the result, which still shows some lost packets. One thought is clearing the bits in RL_ISR in re_intr() instead of re_int_task(), but then I can't see a good way to pass the old value of the status reg. through to re_int_task()? Hmm, I still don't understand how the patch mitigates the issue. :-( The patch does not disable interrupts in interrupt handler so taskqueue runs with interrupt enabled. This may ensure not loosing interrupts but it may also generate many interrupts. By chance, did you check number of interrupts generated with/without your patch? I agree. I think all it did was create more interrupts so that re_rxeof() got called more frequently. (see below) The only guess I have at the moment is the writing RL_IMR in interrupt handler at the end of taskqueue might be not immediately reflected so controller can loose interrupts for the time window. Would you try attach patch and let me know it makes any difference? Nope, it didn't make any difference. I've added a counter of how many times re_rxeof() gets called, but then returns without handling any received packets (I think because RL_RDESC_STAT_OWN is set on the first entry it looks at in the rcv. ring.) This count comes out as almost the same as the # of missed frames (see rxe did 0: in the attached stats). So, I think what is happenning about 10% of the time is that re_rxeof() is looking at the ring descriptor before it has been updated and then returns without handling the packet and then it doesn't get called again because the RL_ISR bit has been cleared. When options DEVICE_POLLING is specified, it works ok, because it calls re_rxeof() fairly frequently and it doesn't pay any attention to the RL_ISR bits. That's one of possible theory. See below for another theory. Now, I don't know if this is a hardware flaw on this machine or something that can be fixed in software? I know diddly about the current driver I highly doubt it could be hardware issue. setup, but I assume something like this has to happen? - chip (or PCIe handler) forces the DMA of the descriptor change to RAM - then posts the interrupt - and the driver must read the up to date descriptor from RAM -- I notice that volatile isn't used anywhere in the driver. Wouldn't the descriptor ring memory have to be defined volatile somehow? (I don't know how to do this correctly?) Because driver check different descriptor location in each loop, adding volatile keyword wouldn't help. You can verify that whether that makes any difference though. Another possibility I have in mind is the controller would have reported RL_ISR_RX_OVERRUN but re(4) didn't check that condition. The old data sheet I have does not clearly mention when it is set and what other bits of RL_ISR register is affected from the bit. If RL_ISR_RX_OVERRUN bit is set when there are no more free RX descriptors available to controller and RL_ISR_RX_ERR is not set when RL_ISR_RX_OVERRUN is set, re(4) have ignored that event. Because driver ignored that error interrupt, the next error interrupt would be RL_ISR_FIFO_OFLOW error and this time it would be served in re_rxeof() and will refill RX buffers. However driver would have lost lots of frames received between the time window RL_ISR_RX_OVERRUN error and RL_ISR_FIFO_OFLOW error. If this theory is correct, the attached patch may mitigate the issue. So, if you can think of anything that will ensure that re_rxeof() reads up to date descriptor ring entries, please let me know. It's job of bus_dma(9) and I don't think barrier instructions would be helpful here as I don't see out-of-order execution in RX handler. Otherwise, I can live with options DEVICE_POLLING. Sorry, I can't live with DEVICE_POLLING on re(4). :-( Let's kill driver bug. No one reported this kind of issue so far and I guess most users took it granted for the poor performance because they are using low end consumer grade controller. Thanks for your help, rick re0 statistics: Transmit good frames : 101346 Receive good frames : 133390 Tx errors : 0 Rx errors : 0 Rx missed frames
Re: re(4) driver dropping packets when reading NFS files
On Fri, Nov 05, 2010 at 07:33:45PM -0700, Pyun YongHyeon wrote:
[...]
If this theory is correct, the attached patch may mitigate the
issue.
Oops, I incorrectly used old code.
Please use this one.
Index: sys/pci/if_rlreg.h
===
--- sys/pci/if_rlreg.h (revision 214844)
+++ sys/pci/if_rlreg.h (working copy)
@@ -873,9 +873,7 @@
int rl_twist_row;
int rl_twist_col;
int suspended; /* 0 = normal 1 = suspended */
-#ifdef DEVICE_POLLING
int rxcycles;
-#endif
struct task rl_txtask;
struct task rl_inttask;
Index: sys/dev/re/if_re.c
===
--- sys/dev/re/if_re.c (revision 214844)
+++ sys/dev/re/if_re.c (working copy)
@@ -1860,7 +1860,7 @@
int i, total_len;
struct rl_desc *cur_rx;
u_int32_t rxstat, rxvlan;
- int maxpkt = 16, rx_npkts = 0;
+ int rx_npkts = 0;
RL_LOCK_ASSERT(sc);
@@ -1872,7 +1872,7 @@
sc-rl_ldata.rl_rx_list_map,
BUS_DMASYNC_POSTREAD | BUS_DMASYNC_POSTWRITE);
- for (i = sc-rl_ldata.rl_rx_prodidx; maxpkt 0;
+ for (i = sc-rl_ldata.rl_rx_prodidx; sc-rxcycles 0;
i = RL_RX_DESC_NXT(sc, i)) {
if ((ifp-if_drv_flags IFF_DRV_RUNNING) == 0)
break;
@@ -2036,7 +2036,7 @@
}
}
}
- maxpkt--;
+ sc-rxcycles--;
if (rxvlan RL_RDESC_VLANCTL_TAG) {
m-m_pkthdr.ether_vtag =
bswap16((rxvlan RL_RDESC_VLANCTL_DATA));
@@ -2058,10 +2058,10 @@
if (rx_npktsp != NULL)
*rx_npktsp = rx_npkts;
- if (maxpkt)
- return (EAGAIN);
+ if (sc-rxcycles)
+ return (0);
- return (0);
+ return (EAGAIN);
}
static void
@@ -2258,8 +2258,11 @@
}
#endif
- if (status (RL_ISR_RX_OK|RL_ISR_RX_ERR|RL_ISR_FIFO_OFLOW))
+ if (status (RL_ISR_RX_OK | RL_ISR_RX_ERR | RL_ISR_FIFO_OFLOW |
+ RL_ISR_RX_OVERRUN)) {
+ sc-rxcycles = sc-rl_ldata.rl_rx_desc_cnt / 2;
rval = re_rxeof(sc, NULL);
+ }
/*
* Some chips will ignore a second TX request issued
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Wed, Nov 03, 2010 at 07:27:20PM -0400, Rick Macklem wrote: I'm more interested in number of dropped frames. See below how to extract that information. I've attached the stats. I'm guessing that the Rx missed frames : 14792 is the culprit. Because that counter is 16bit it's also possible it wrapped multiple times. Could you verify that? Ill look, but since the test was run just after booting the machine, if it wrapped it would mean that it dropped even more packets, I think? From my limited testing, it seems it works as expected. Would you give it try and let me know how well it behaves with NFS? Without DEVICE_POLLING it behaves just like the unpatched one. Hmm, that's strange. Are you sure you rebuilt kernel without polling option? Just disabling polling with ifconfig(8) has no effect to make patch work. Yep. I have two kernel configs and I rebuilt/installed using the one that doesn't have DEVICE_POLLING. (Also, I would have seen about 9Mbytes/sec read rate if I was using the one that has DEVICE_POLLING.) If the counter was not wrapped, it seem you lost more than 10% out of total RX frames. This is a lot loss and there should be a way to mitigate it. Agreed. There is definitely an issue for at least this variant of the chip. Is it fair to assume that, since the chip reports the missed packets, that it thinks that it doesn't have a usable receive buffer at the time the packet is received? (I'm thinking that it couldn't have used up 256 receive buffers when the first drop happens, since I see it early in the tcpdump I took before, so maybe something related to handling of the receive ring? As I said, I'll play with it later to-day and let you know if I learn something more.) rick ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
If the counter was not wrapped, it seem you lost more than 10% out of total RX frames. This is a lot loss and there should be a way to mitigate it. I've attached a patch (to the if_re.c in head, not your patched variant) that works a lot better (about 5Mbytes/sec read rate). To get that, I had to disable msi and not clear the RL_IMR register in re_intr(). I suspect that a packet would be received between when the bits in RL_IMR were cleared and when they were set at the end of re_int_task() and those were getting lost. This patch doesn't completely fix the problem. (I added your stats collecting stuff to the if_re.c in head and attached the result, which still shows some lost packets. One thought is clearing the bits in RL_ISR in re_intr() instead of re_int_task(), but then I can't see a good way to pass the old value of the status reg. through to re_int_task()? The patch doesn't help when msi is enabled and when I played with your patched variant, I got it to hang when RL_IMR wasn't cleared. I've attached the patch and stats. I might play around with it some more tomorrow, rick ps: If you have hardware to test re with, you want to do an NFS mount and then read a large file when nothing else is happening on the machine, to see if you can reproduce the problem. pss: All tests done with a kernel that does not have option DEVICE_POLLING. --- if_re.c.orig 2010-11-03 18:49:29.0 -0400 +++ if_re.c 2010-11-04 21:04:34.0 -0400 @@ -156,7 +156,7 @@ #include miibus_if.h /* Tunables. */ -static int msi_disable = 0; +static int msi_disable = 1; TUNABLE_INT(hw.re.msi_disable, msi_disable); static int prefer_iomap = 0; TUNABLE_INT(hw.re.prefer_iomap, prefer_iomap); @@ -2179,7 +2179,6 @@ status = CSR_READ_2(sc, RL_ISR); if (status == 0x || (status RL_INTRS_CPLUS) == 0) return (FILTER_STRAY); - CSR_WRITE_2(sc, RL_IMR, 0); taskqueue_enqueue_fast(taskqueue_fast, sc-rl_inttask); re0 statistics: Transmit good frames : 83320 Receive good frames : 136158 Tx errors : 0 Rx errors : 0 Rx missed frames : 2666 Rx frame alignment errs : 0 Tx single collisions : 0 Tx multiple collisions : 0 Rx unicast frames : 136157 Rx broadcast frames : 0 Rx multicast frames : 1 Tx aborts : 0 Tx underruns : 0 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Thu, Nov 04, 2010 at 09:31:30PM -0400, Rick Macklem wrote: If the counter was not wrapped, it seem you lost more than 10% out of total RX frames. This is a lot loss and there should be a way to mitigate it. I've attached a patch (to the if_re.c in head, not your patched variant) that works a lot better (about 5Mbytes/sec read rate). To get that, I had to disable msi and not clear the RL_IMR register in re_intr(). I suspect that a packet would be received between when the bits in RL_IMR were cleared and when they were set at the end of re_int_task() and those were getting lost. This patch doesn't completely fix the problem. (I added your stats collecting stuff to the if_re.c in head and attached the result, which still shows some lost packets. One thought is clearing the bits in RL_ISR in re_intr() instead of re_int_task(), but then I can't see a good way to pass the old value of the status reg. through to re_int_task()? Hmm, I still don't understand how the patch mitigates the issue. :-( The patch does not disable interrupts in interrupt handler so taskqueue runs with interrupt enabled. This may ensure not loosing interrupts but it may also generate many interrupts. By chance, did you check number of interrupts generated with/without your patch? The only guess I have at the moment is the writing RL_IMR in interrupt handler at the end of taskqueue might be not immediately reflected so controller can loose interrupts for the time window. Would you try attach patch and let me know it makes any difference? The patch doesn't help when msi is enabled and when I played with your patched variant, I got it to hang when RL_IMR wasn't cleared. I've attached the patch and stats. I might play around with it some more tomorrow, rick Thanks for your work. ps: If you have hardware to test re with, you want to do an NFS mount and then read a large file when nothing else is happening on the machine, to see if you can reproduce the problem. It seems I'm not able to reproduce it on my box(8168B GigE). pss: All tests done with a kernel that does not have option DEVICE_POLLING. Ok, I have to commit statistics counter patch since it seems to help narrowing down driver issue. re0 statistics: Transmit good frames : 83320 Receive good frames : 136158 Tx errors : 0 Rx errors : 0 Rx missed frames : 2666 Rx frame alignment errs : 0 Tx single collisions : 0 Tx multiple collisions : 0 Rx unicast frames : 136157 Rx broadcast frames : 0 Rx multicast frames : 1 Tx aborts : 0 Tx underruns : 0 Index: sys/dev/re/if_re.c === --- sys/dev/re/if_re.c (revision 214692) +++ sys/dev/re/if_re.c (working copy) @@ -2254,6 +2254,7 @@ } CSR_WRITE_2(sc, RL_IMR, RL_INTRS_CPLUS); + CSR_READ_2(sc, RL_IMR); } static int ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
I'm more interested in number of dropped frames. See below how to extract that information. I've attached the stats. I'm guessing that the Rx missed frames : 14792 is the culprit. This was for a read of a fairly large file via NFS over TCP, getting a read rate of about 450Kbytes/sec. (No DEVICE_POLLING option.) (with your patch applied) (It almost looks like it only handles the first received packet, although it appears to be using a receive ring of 64 buffers.) No, re(4) uses 256 TX/RX buffers for RTL810xE controllers. Oops, my mistake. At a quick glance, I had thought rl_type was set to 8139, but I now see it's 8169. Btw, I printed out the hwrev and its a RL_HWREV_8102EL_SPIN1, if that is of any use to you. Ok, here is patch. http://people.freebsd.org/~yongari/re/re.intr.patch The patch has the following changes. o 64bit DMA support for PCIe controllers. o Hardware MAC statistics counter support. You can extract these counters with sysctl dev.re.0.stats=1. You can check the output on console or dmesg. It seems extracting these counters take a lot of time so I didn't try to accumulate the counters. You can see how many frames are dropped from the output. I saw a lot FAE(frame alignment errors) under high RX load and I can't explain how this can happen. This may indicate PHY hardware is poor or it may need DSP fixups. Realtek seems to maintain large set of DSP fixups for each PHY revisions and re(4) does not have the magic code at this moment. o Overhaul MSI interrupt handler such that make it give fairness to TX as well as serving RX. Because re(4) controllers do not have interrupt moderation mechanism, naive interrupt handler can generate more than 125k intrs/sec under high load. Fortunately, Bill implemented TX interrupt moderation with a timer register and it seems to work well on TX path. One drawback of the approach is it will require extra timer register accesses in fast path. There is no second timer register to use in RX path so no RX interrupt moderation is done in driver such that it can generate about 25k intrs/sec under high RX load. However, I think most systems can handle that interrupt load. Note, this feature is activated only when MSI is in use and DEVICE_POLLING is not defined. From my limited testing, it seems it works as expected. Would you give it try and let me know how well it behaves with NFS? Without DEVICE_POLLING it behaves just like the unpatched one. I'm going to look at the driver tomorrow and try some hacks on it, rick re0 statistics: Transmit good frames : 100966 Receive good frames : 133470 Tx errors : 0 Rx errors : 0 Rx missed frames : 14792 Rx frame alignment errs : 0 Tx single collisions : 0 Tx multiple collisions : 0 Rx unicast frames : 133463 Rx broadcast frames : 0 Rx multicast frames : 7 Tx aborts : 0 Tx underruns : 0 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Wed, Nov 03, 2010 at 07:27:20PM -0400, Rick Macklem wrote: I'm more interested in number of dropped frames. See below how to extract that information. I've attached the stats. I'm guessing that the Rx missed frames : 14792 is the culprit. Because that counter is 16bit it's also possible it wrapped multiple times. Could you verify that? This was for a read of a fairly large file via NFS over TCP, getting a read rate of about 450Kbytes/sec. (No DEVICE_POLLING option.) (with your patch applied) (It almost looks like it only handles the first received packet, although it appears to be using a receive ring of 64 buffers.) No, re(4) uses 256 TX/RX buffers for RTL810xE controllers. Oops, my mistake. At a quick glance, I had thought rl_type was set to 8139, but I now see it's 8169. Btw, I printed out the hwrev and its a RL_HWREV_8102EL_SPIN1, if that is of any use to you. Ok, here is patch. http://people.freebsd.org/~yongari/re/re.intr.patch The patch has the following changes. o 64bit DMA support for PCIe controllers. o Hardware MAC statistics counter support. You can extract these counters with sysctl dev.re.0.stats=1. You can check the output on console or dmesg. It seems extracting these counters take a lot of time so I didn't try to accumulate the counters. You can see how many frames are dropped from the output. I saw a lot FAE(frame alignment errors) under high RX load and I can't explain how this can happen. This may indicate PHY hardware is poor or it may need DSP fixups. Realtek seems to maintain large set of DSP fixups for each PHY revisions and re(4) does not have the magic code at this moment. o Overhaul MSI interrupt handler such that make it give fairness to TX as well as serving RX. Because re(4) controllers do not have interrupt moderation mechanism, naive interrupt handler can generate more than 125k intrs/sec under high load. Fortunately, Bill implemented TX interrupt moderation with a timer register and it seems to work well on TX path. One drawback of the approach is it will require extra timer register accesses in fast path. There is no second timer register to use in RX path so no RX interrupt moderation is done in driver such that it can generate about 25k intrs/sec under high RX load. However, I think most systems can handle that interrupt load. Note, this feature is activated only when MSI is in use and DEVICE_POLLING is not defined. From my limited testing, it seems it works as expected. Would you give it try and let me know how well it behaves with NFS? Without DEVICE_POLLING it behaves just like the unpatched one. Hmm, that's strange. Are you sure you rebuilt kernel without polling option? Just disabling polling with ifconfig(8) has no effect to make patch work. I'm going to look at the driver tomorrow and try some hacks on it, rick re0 statistics: Transmit good frames : 100966 Receive good frames : 133470 Tx errors : 0 Rx errors : 0 Rx missed frames : 14792 If the counter was not wrapped, it seem you lost more than 10% out of total RX frames. This is a lot loss and there should be a way to mitigate it. Rx frame alignment errs : 0 Tx single collisions : 0 Tx multiple collisions : 0 Rx unicast frames : 133463 Rx broadcast frames : 0 Rx multicast frames : 7 Tx aborts : 0 Tx underruns : 0 ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Sun, Oct 31, 2010 at 05:46:57PM -0400, Rick Macklem wrote: I recently purchased a laptop that has a re(4) Realtek 8101E/8102E/8103E net chip in it and I find that it is dropping packets like crazy when reading files over an NFS mount. (It seems that bursts of receive traffic cause it, since when I look over wireshark, typically the 2nd packet in a read reply is not received, although it was sent at the other end.) Are you using NFS over UDP? Adding options DEVICE_POLLING helps a lot. (ie. order of magnitude faster reading) Does this hint that interrupts are being lost or delayed too much? Actually I'm not a fan of polling(4) but re(4) controllers might be exceptional one due to controller limitation but order of magnitude faster indicates something is wrong in driver. Anyhow, if anyone has an idea on what the cause/fix might be or are familiar with the driver/hardware, I'd appreciate hearing about it. AFAIK re(4) controllers lacks interrupts moderation so re(4) used to rely on taskqueue to reduce number of interrupts. It was written long time ago by Bill and I'm not sure whether it's still valid for recent PCIe RealTek controllers. One of problem is getting stand-alone PCIe controllers in market and I was not able to buy recent controllers. This is one of reason why re(4) still lacks TSO, jumbo frame and 64bit DMA support for newer controllers. Another problem is RealTek no longer releases data sheet so it's hard to write new features that may present on recent controllers. Recent re(4) controllers started to support small set of hardware MAC statistics counters and that may help to understand how many frames were lost under heavy load. I'll let you know when I have a patch for that. Flow-control may also enhance performance a little bit but it was not implemented yet like most other consumer grade ethernet drivers. But this may change in near future, marius@ is actively working on this so we'll get generic flow-control framework in tree. I'll see what can be done in interrupt handler and I'll let you know when patch is ready. Thanks, rick ps: This laptop is running a low end AMD cpu and I did install amd64 on it, instead of i386, in case that might be relevent? I don't think so. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Sun, Oct 31, 2010 at 05:46:57PM -0400, Rick Macklem wrote: I recently purchased a laptop that has a re(4) Realtek 8101E/8102E/8103E net chip in it and I find that it is dropping packets like crazy when reading files over an NFS mount. (It seems that bursts of receive traffic cause it, since when I look over wireshark, typically the 2nd packet in a read reply is not received, although it was sent at the other end.) Are you using NFS over UDP? The test I referred to was over TCP, which works fine until reading a file and then about the second TCP data segment that is sent by the server isn't received by the client with the re(4). (I had tcpdump capturing on both machines and then compared them using wireshark.) The read does progress slowly, after TCP retransmits the segment that was dropped. The result is a rate of about 10 reads/sec. A test over UDP gets nowhere. You just gets lots of IP fragments timed out when you netstat -s, so it seems to consistently drop a fragment in the read UDP reply. Adding options DEVICE_POLLING helps a lot. (ie. order of magnitude faster reading) Does this hint that interrupts are being lost or delayed too much? Actually I'm not a fan of polling(4) but re(4) controllers might be exceptional one due to controller limitation but order of magnitude faster indicates something is wrong in driver. Yep, I'd agree. I can print out the exact chip device info, but if you don't have data sheets, it may not help. It seems to be a low end chip, since it doesn't support 1Gbps -- closer to an 8139. It might be called an 8036, since that # shows up in the device resources under windoze. AFAIK re(4) controllers lacks interrupts moderation so re(4) used to rely on taskqueue to reduce number of interrupts. It was written long time ago by Bill and I'm not sure whether it's still valid for recent PCIe RealTek controllers. One of problem is getting stand-alone PCIe controllers in market and I was not able to buy recent controllers. This is one of reason why re(4) still lacks TSO, jumbo frame and 64bit DMA support for newer controllers. Another problem is RealTek no longer releases data sheet so it's hard to write new features that may present on recent controllers. Recent re(4) controllers started to support small set of hardware MAC statistics counters and that may help to understand how many frames were lost under heavy load. I'll let you know when I have a patch for that. Flow-control may also enhance performance a little bit but it was not implemented yet like most other consumer grade ethernet drivers. But this may change in near future, marius@ is actively working on this so we'll get generic flow-control framework in tree. It drops a frame as soon as the read starts and there is a burst of more than one. (I can email you the tcpdump captures if you're interested and you won't have to look far into it to see it happen.) It seems to do it consistently and then recovers when the TCP segment is resent, but repeats the fun on the next one. (I'm wondering if it can't support a 64 entry receive ring. I'll try making it smaller and see what happens? Probably won't help, but can't hurt to try:-) I'll see what can be done in interrupt handler and I'll let you know when patch is ready. Thanks, rick ps: This laptop is running a low end AMD cpu and I did install amd64 on it, instead of i386, in case that might be relevent? I don't think so. Ok. I didn't think so, but someone recently mentioned that some drivers for wifi chips don't work for amd64. It actually works fairly well (and quite well with DEVICE_POLLING), except for this issue where it drops received packets when it gets bursts of them. (It almost looks like it only handles the first received packet, although it appears to be using a receive ring of 64 buffers.) Anyhow, I'll keep poking at it and will appreciate any patches/suggestions that you might have. Thanks, rick ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: re(4) driver dropping packets when reading NFS files
On Mon, Nov 01, 2010 at 06:18:13PM -0400, Rick Macklem wrote: On Sun, Oct 31, 2010 at 05:46:57PM -0400, Rick Macklem wrote: I recently purchased a laptop that has a re(4) Realtek 8101E/8102E/8103E net chip in it and I find that it is dropping packets like crazy when reading files over an NFS mount. (It seems that bursts of receive traffic cause it, since when I look over wireshark, typically the 2nd packet in a read reply is not received, although it was sent at the other end.) Are you using NFS over UDP? The test I referred to was over TCP, which works fine until reading a file and then about the second TCP data segment that is sent by the server isn't received by the client with the re(4). (I had tcpdump capturing on both machines and then compared them using wireshark.) Hmm, this is not what I initially thought. re(4) has much more preference on handling RX traffic over TX so it should have received that even though it couldn't send response packets. I'm not sure my patch can mitigate the issue. The read does progress slowly, after TCP retransmits the segment that was dropped. The result is a rate of about 10 reads/sec. A test over UDP gets nowhere. You just gets lots of IP fragments timed out when you netstat -s, so it seems to consistently drop a fragment in the read UDP reply. Adding options DEVICE_POLLING helps a lot. (ie. order of magnitude faster reading) Does this hint that interrupts are being lost or delayed too much? Actually I'm not a fan of polling(4) but re(4) controllers might be exceptional one due to controller limitation but order of magnitude faster indicates something is wrong in driver. Yep, I'd agree. I can print out the exact chip device info, but if you don't have data sheets, it may not help. It seems to be a low end chip, since it doesn't support 1Gbps -- closer to an 8139. It might be called an 8036, since that # shows up in the device resources under windoze. AFAIK re(4) controllers lacks interrupts moderation so re(4) used to rely on taskqueue to reduce number of interrupts. It was written long time ago by Bill and I'm not sure whether it's still valid for recent PCIe RealTek controllers. One of problem is getting stand-alone PCIe controllers in market and I was not able to buy recent controllers. This is one of reason why re(4) still lacks TSO, jumbo frame and 64bit DMA support for newer controllers. Another problem is RealTek no longer releases data sheet so it's hard to write new features that may present on recent controllers. Recent re(4) controllers started to support small set of hardware MAC statistics counters and that may help to understand how many frames were lost under heavy load. I'll let you know when I have a patch for that. Flow-control may also enhance performance a little bit but it was not implemented yet like most other consumer grade ethernet drivers. But this may change in near future, marius@ is actively working on this so we'll get generic flow-control framework in tree. It drops a frame as soon as the read starts and there is a burst of more than one. (I can email you the tcpdump captures if you're interested and you won't have to look far into it to see it happen.) I'm more interested in number of dropped frames. See below how to extract that information. It seems to do it consistently and then recovers when the TCP segment is resent, but repeats the fun on the next one. (I'm wondering if it can't support a 64 entry receive ring. I'll try making it smaller and see what happens? Probably won't help, but can't hurt to try:-) I'll see what can be done in interrupt handler and I'll let you know when patch is ready. Thanks, rick ps: This laptop is running a low end AMD cpu and I did install amd64 on it, instead of i386, in case that might be relevent? I don't think so. Ok. I didn't think so, but someone recently mentioned that some drivers for wifi chips don't work for amd64. All drivers touched by me should work on any architectures. The code is the same so there is no difference. It actually works fairly well (and quite well with DEVICE_POLLING), except for this issue where it drops received packets when it gets bursts of them. Actually this is one of advantage of using interrupts against polling. Interrupts tend to give more fast response. To achieve the similar behavior with polling you should have used high hz. Your test indicates quite opposite result though. (It almost looks like it only handles the first received packet, although it appears to be using a receive ring of 64 buffers.) No, re(4) uses 256 TX/RX buffers for RTL810xE controllers. Anyhow, I'll keep poking at it and will appreciate any patches/suggestions that you might have. Ok, here is patch. http://people.freebsd.org/~yongari/re/re.intr.patch The patch has the following
re(4) driver dropping packets when reading NFS files
I recently purchased a laptop that has a re(4) Realtek 8101E/8102E/8103E net chip in it and I find that it is dropping packets like crazy when reading files over an NFS mount. (It seems that bursts of receive traffic cause it, since when I look over wireshark, typically the 2nd packet in a read reply is not received, although it was sent at the other end.) Adding options DEVICE_POLLING helps a lot. (ie. order of magnitude faster reading) Does this hint that interrupts are being lost or delayed too much? Anyhow, if anyone has an idea on what the cause/fix might be or are familiar with the driver/hardware, I'd appreciate hearing about it. Thanks, rick ps: This laptop is running a low end AMD cpu and I did install amd64 on it, instead of i386, in case that might be relevent? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
