I have encountered something similar while doing CBDMA dev on 2.6.32.x kernel. My e1000 would have horribly slow performance when doing a scp on the C55xx based platform. Never figured out how to reproduce it consistently but didn't connect it with the CBDMA network assist at the time.... I can try to see if it shows up again on 2.6.35.7.....
> -----Original Message----- > From: Williams, Dan J > Sent: Monday, October 04, 2010 1:49 PM > To: Brandeburg, Jesse; Jiang, Dave > Cc: Richard Scobie; [email protected]; Sosnowski, Maciej > Subject: Re: [E1000-devel] IOATDMA and samba > > [ adding Dave, Maciej ] > > On 10/4/2010 1:13 PM, Brandeburg, Jesse wrote: > > > > Dan might be able to further help with the ioatdma portion, this > sounds > > like a possible regression in 4.00 ioatdma? > > Smells that way. I'm traveling internationally this week, but I can > try > to reproduce this on my 5400 next week when I'm back (unless Dave or > Maciej have a chance to take a look). Question below: > > > > > On Sat, 2 Oct 2010, Richard Scobie wrote: > > > >> I have a Tyan S5396 5400 based board, which has been running > >> 2.6.30.10-105.2.23.fc11.x86_64and has a PRO/1000 PT (82571EB) Quad > >> adapter, along with two onboard 80003ES2LAN adapters. > >> > >> This has performed fine with the ioatdma loaded and the BIOS is the > latest. > >> > >> dca service started, version 1.8 > >> ioatdma 0000:00:0f.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 > >> ioatdma 0000:00:0f.0: setting latency timer to 64 > >> ioatdma 0000:00:0f.0: Intel(R) I/OAT DMA Engine found, 4 channels, > >> device version 0x20, driver version 3.64 > >> alloc irq_desc for 62 on cpu 0 node 0 > >> alloc kstat_irqs on cpu 0 node 0 > >> ioatdma 0000:00:0f.0: irq 62 for MSI/MSI-X > >> alloc irq_desc for 63 on cpu 0 node 0 > >> alloc kstat_irqs on cpu 0 node 0 > >> ioatdma 0000:00:0f.0: irq 63 for MSI/MSI-X > >> alloc irq_desc for 64 on cpu 0 node 0 > >> alloc kstat_irqs on cpu 0 node 0 > >> ioatdma 0000:00:0f.0: irq 64 for MSI/MSI-X > >> alloc irq_desc for 65 on cpu 0 node 0 > >> alloc kstat_irqs on cpu 0 node 0 > >> ioatdma 0000:00:0f.0: irq 65 for MSI/MSI-X > >> ioatdma 0000:00:0f.0: DCA is disabled in BIOS > > > > hm, I think this note above is critical that it is missing from the > below. > > > >> I have just updated to 2.6.35.7 stable and with ioatdma loaded, > samba RX > >> performance drops from normal levels, to about 500kB/s with the smbd > >> process at 100% CPU. > > > > does netstat -s show it all to be from retransmits or SACKs? > > I assume smbd normally does not consume 100% cpu. Can you send the > output of: > > perf record -a sleep 5; perf report > > ...while the bad performance is occurring. That might give a clue as > to > what is tanking the performance. > > > > >> dca service started, version 1.12.1 > >> ioatdma: Intel(R) QuickData Technology Driver 4.00 > >> ioatdma 0000:00:0f.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 > >> ioatdma 0000:00:0f.0: setting latency timer to 64 > >> alloc irq_desc for 80 on node -1 > >> alloc kstat_irqs on node -1 > >> ioatdma 0000:00:0f.0: irq 80 for MSI/MSI-X > >> alloc irq_desc for 81 on node -1 > >> alloc kstat_irqs on node -1 > >> ioatdma 0000:00:0f.0: irq 81 for MSI/MSI-X > >> alloc irq_desc for 82 on node -1 > >> alloc kstat_irqs on node -1 > >> ioatdma 0000:00:0f.0: irq 82 for MSI/MSI-X > >> alloc irq_desc for 83 on node -1 > >> alloc kstat_irqs on node -1 > >> ioatdma 0000:00:0f.0: irq 83 for MSI/MSI-X > > > > some of the other messages above are missing too, like DMA engine > found, > > and driver version 3.64 > > > >> > >> > >> If it is not loaded, performance returns to normal levels, but this > >> network RX performance degradation is not seen with rsync. > > > > rsync and samba may be using different sockets sizes which may > prevent > > ioatdma from being utilized to do copy offload. This might explain > rsync > > working. Can you check the stats in > > > > cat /sys/class/dma/dma?chan?/memcpy_count > > > > and see if the stats don't increment for one or the other? > > > > Similarly you can: > > echo $((256*1024)) > /proc/sys/net/ipv4/tcp_dma_copybreak > > ...to effectively disable dma. > > -- > Dan ------------------------------------------------------------------------------ Virtualization is moving to the mainstream and overtaking non-virtualized environment for deploying applications. Does it make network security easier or more difficult to achieve? Read this whitepaper to separate the two and get a better understanding. http://p.sf.net/sfu/hp-phase2-d2d _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
