Thanks Jesse and Dan - comments to both below. I will not be able to access this box until the weekend, so will gather the information requested and anything else that may be useful.
Regards, Richard Dan Williams wrote: > [ adding Dave, Maciej ] > > On 10/4/2010 1:13 PM, Brandeburg, Jesse wrote: >> >> Dan might be able to further help with the ioatdma portion, this sounds >> like a possible regression in 4.00 ioatdma? > > Smells that way. I'm traveling internationally this week, but I can try > to reproduce this on my 5400 next week when I'm back (unless Dave or > Maciej have a chance to take a look). Question below: > >> >> On Sat, 2 Oct 2010, Richard Scobie wrote: >> >>> I have a Tyan S5396 5400 based board, which has been running >>> 2.6.30.10-105.2.23.fc11.x86_64and has a PRO/1000 PT (82571EB) Quad >>> adapter, along with two onboard 80003ES2LAN adapters. >>> >>> This has performed fine with the ioatdma loaded and the BIOS is the latest. >>> >>> dca service started, version 1.8 >>> ioatdma 0000:00:0f.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 >>> ioatdma 0000:00:0f.0: setting latency timer to 64 >>> ioatdma 0000:00:0f.0: Intel(R) I/OAT DMA Engine found, 4 channels, >>> device version 0x20, driver version 3.64 >>> alloc irq_desc for 62 on cpu 0 node 0 >>> alloc kstat_irqs on cpu 0 node 0 >>> ioatdma 0000:00:0f.0: irq 62 for MSI/MSI-X >>> alloc irq_desc for 63 on cpu 0 node 0 >>> alloc kstat_irqs on cpu 0 node 0 >>> ioatdma 0000:00:0f.0: irq 63 for MSI/MSI-X >>> alloc irq_desc for 64 on cpu 0 node 0 >>> alloc kstat_irqs on cpu 0 node 0 >>> ioatdma 0000:00:0f.0: irq 64 for MSI/MSI-X >>> alloc irq_desc for 65 on cpu 0 node 0 >>> alloc kstat_irqs on cpu 0 node 0 >>> ioatdma 0000:00:0f.0: irq 65 for MSI/MSI-X >>> ioatdma 0000:00:0f.0: DCA is disabled in BIOS >> >> hm, I think this note above is critical that it is missing from the below. Although I believe neither 82571EB or 80003ES2LAN make use of DCA. This board does not have an explicit "DCA Enable" feature - just Crystal Beach, which is enabled. >> >>> I have just updated to 2.6.35.7 stable and with ioatdma loaded, samba RX >>> performance drops from normal levels, to about 500kB/s with the smbd >>> process at 100% CPU. >> >> does netstat -s show it all to be from retransmits or SACKs? > > I assume smbd normally does not consume 100% cpu. Can you send the > output of: No, without ioatdma, around 22%. > perf record -a sleep 5; perf report > > ...while the bad performance is occurring. That might give a clue as to > what is tanking the performance. > >> >>> dca service started, version 1.12.1 >>> ioatdma: Intel(R) QuickData Technology Driver 4.00 >>> ioatdma 0000:00:0f.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 >>> ioatdma 0000:00:0f.0: setting latency timer to 64 >>> alloc irq_desc for 80 on node -1 >>> alloc kstat_irqs on node -1 >>> ioatdma 0000:00:0f.0: irq 80 for MSI/MSI-X >>> alloc irq_desc for 81 on node -1 >>> alloc kstat_irqs on node -1 >>> ioatdma 0000:00:0f.0: irq 81 for MSI/MSI-X >>> alloc irq_desc for 82 on node -1 >>> alloc kstat_irqs on node -1 >>> ioatdma 0000:00:0f.0: irq 82 for MSI/MSI-X >>> alloc irq_desc for 83 on node -1 >>> alloc kstat_irqs on node -1 >>> ioatdma 0000:00:0f.0: irq 83 for MSI/MSI-X >> >> some of the other messages above are missing too, like DMA engine found, >> and driver version 3.64 >> >>> >>> >>> If it is not loaded, performance returns to normal levels, but this >>> network RX performance degradation is not seen with rsync. >> >> rsync and samba may be using different sockets sizes which may prevent >> ioatdma from being utilized to do copy offload. This might explain rsync >> working. Can you check the stats in >> >> cat /sys/class/dma/dma?chan?/memcpy_count >> >> and see if the stats don't increment for one or the other? >> > > Similarly you can: > > echo $((256*1024))> /proc/sys/net/ipv4/tcp_dma_copybreak > > ...to effectively disable dma. > > -- > Dan ------------------------------------------------------------------------------ Virtualization is moving to the mainstream and overtaking non-virtualized environment for deploying applications. Does it make network security easier or more difficult to achieve? Read this whitepaper to separate the two and get a better understanding. http://p.sf.net/sfu/hp-phase2-d2d _______________________________________________ E1000-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
