Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
Check to make sure the links are all full-duplex. We started seeing bad performance with the em(4) driver on our HP Proliant 360DL G5's using 1000Mbits. It turned out that switch was setting it's port to half-duplex and the emX interface was following suit. HTH, Patrick joe wrote: On 05/08/2010 01:31 PM, Jack Vogel wrote: Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did pull the server and brought it home so that i can test more things quicker. I am going to try using a raid card instead of the onboard sata ports and see if i still encounter the same problem. I would love any suggestions you may have on where to go from here to figure out where the problem might be. joe ___ freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org I think it might have something to so with the nics / switch, and their features. I brought the box home, plugged into my gb switch, and i am able to FTP data to the server at around 35MB/sec. I dont know what would cause this other than some sort of issue with the the 3 different types of nics and the switch i am using. Any suggestions? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
On 05/08/2010 02:21 PM, Jack Vogel wrote: The cable, its a simple thing but make SURE you try that, a slightly damaged one can do weird things and its quick to check, don't overlook it. Jack On Sat, May 8, 2010 at 10:22 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 01:53 PM, Jack Vogel wrote: I still am not clear on this system, how many ports are on it, and its an 82576? Sounds to me like you've proven its not on the box if you can do fine when its on its own. So change ports in the switch, as I said, change cables, must be something in that environment. Jack On Sat, May 8, 2010 at 10:04 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 01:31 PM, Jack Vogel wrote: Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
On Sun, May 9, 2010 at 2:54 AM, joe j...@hostedcontent.com wrote: On 05/08/2010 02:21 PM, Jack Vogel wrote: The cable, its a simple thing but make SURE you try that, a slightly damaged one can do weird things and its quick to check, don't overlook it. Jack On Sat, May 8, 2010 at 10:22 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 01:53 PM, Jack Vogel wrote: I still am not clear on this system, how many ports are on it, and its an 82576? Sounds to me like you've proven its not on the box if you can do fine when its on its own. So change ports in the switch, as I said, change cables, must be something in that environment. Jack On Sat, May 8, 2010 at 10:04 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 01:31 PM, Jack Vogel wrote: Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
On 05/08/2010 05:08 AM, joe wrote: I have 3 boxes, each with two nics. One nic for the private network and one for the public network. The private network is all on the same vlan. All 6 nics are on the same switch. All connections are 1000tx Full Duplex. I will call the servers Box A, Box B, and Box C. When i FTP data between Box A B i get abou 25MB/sec. When i FTP data from Box C to Box A or B, i get about 20MB/sec. When i FTP data from Box A to C i get 10MB/sec When i FTP data from Box B to C i get 200KB/sec... Can anyone suggest why i might only be getting 200KB when transfering data from Box B to C but not when transferring data from Box A to C? I tried installing ubuntu to see if the problem was still there and it is not. I can FTP data from Box A or B to box C at 10MB/sec, still not the gigabit speeds i should be seeing but not the 200KB/sec i am getting with freebsd. I logged into my switch and it reports that the ports are 1000tx full duplex, the same as what freebsd is reporting. The switch freebsd also report no errors. I really dont know what to do. Nothing anywhere reports showing a problem but obviously there is a problem somewhere. I've tried drivers 1.9.5, 1.8.4, and 1.7.3. If i use 1.8.4 or 1.7.3 i get 50KB/sec transferring data from box B whereas if i use 1.9.5 i get 200-300KB sec. I should be getting like 50MB/sec ;( Any help would be grateful. Or, maybe someone could put me in touch with the person responsible for the intel nic drivers and i can work with them on resolving this issue? Thanks in advance. NameMtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll igb0 1500 Link#1 00:30:48:9f:11:0425853 0 0 18340 0 0 igb0 1500 216.105.91.14 216.105.91.145 19040 - - 18343 - - igb1* 1500 Link#2 00:30:48:9f:11:050 0 00 0 0 lo0 16384 Link#3 72 0 0 72 0 0 lo0 16384 127.0.0.0/8 127.0.0.1 72 - - 72 - - Sysctl info for igb0/1 dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 1.8.4 dev.igb.0.%driver: igb dev.igb.0.%location: slot=0 function=0 dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x15d9 subdevice=0x0100 class=0x02 dev.igb.0.%parent: pci1 dev.igb.0.debug: -1 dev.igb.0.stats: -1 dev.igb.0.flow_control: 3 dev.igb.0.enable_aim: 1 dev.igb.0.low_latency: 128 dev.igb.0.ave_latency: 450 dev.igb.0.bulk_latency: 1200 dev.igb.0.rx_processing_limit: 100 Debug info for igb0 May 8 09:17:57 debbie kernel: igb0: TX(1) Packets sent = 1295 May 8 09:17:57 debbie kernel: igb0: Queue(2) tdh = 106, tdt = 106 May 8 09:17:57 debbie kernel: igb0: TX(2) no descriptors avail event = 0 May 8 09:17:57 debbie kernel: igb0: TX(2) MSIX IRQ Handled = 5353 May 8 09:17:57 debbie kernel: igb0: TX(2) Packets sent = 5450 May 8 09:17:57 debbie kernel: igb0: Queue(3) tdh = 1687, tdt = 1687 May 8 09:17:57 debbie kernel: igb0: TX(3) no descriptors avail event = 0 May 8 09:17:57 debbie kernel: igb0: TX(3) MSIX IRQ Handled = 1335 May 8 09:17:57 debbie kernel: igb0: TX(3) Packets sent = 1354 May 8 09:17:57 debbie kernel: igb0: Queue(0) rdh = 425, rdt = 424 May 8 09:17:57 debbie kernel: igb0: RX(0) Packets received = 16809 May 8 09:17:57 debbie kernel: igb0: RX(0) Split Packets = 12262 May 8 09:17:57 debbie kernel: igb0: RX(0) Byte count = 25894129 May 8 09:17:57 debbie kernel: igb0: RX(0) MSIX IRQ Handled = 43428 May 8 09:17:57 debbie kernel: igb0: RX(0) LRO Queued= 13601 May 8 09:17:57 debbie kernel: igb0: RX(0) LRO Flushed= 9966 May 8 09:17:57 debbie kernel: igb0: Queue(1) rdh = 1700, rdt = 1699 May 8 09:17:57 debbie kernel: igb0: RX(1) Packets received = 1700 May 8 09:17:57 debbie kernel: igb0: RX(1) Split Packets = 937 May 8 09:17:57 debbie kernel: igb0: RX(1) Byte count = 1392691 May 8 09:17:57 debbie kernel: igb0: RX(1) MSIX IRQ Handled = 31888 May 8 09:17:57 debbie kernel: igb0: RX(1) LRO Queued= 1362 May 8 09:17:57 debbie kernel: igb0: RX(1) LRO Flushed= 1351 May 8 09:17:57 debbie kernel: igb0: Queue(2) rdh = 33, rdt = 32 May 8 09:17:57 debbie kernel: igb0: RX(2) Packets received = 6177 May 8 09:17:57 debbie kernel: igb0: RX(2) Split Packets = 1442 May 8 09:17:57 debbie kernel: igb0: RX(2) Byte count = 2518498 May 8 09:17:57 debbie kernel: igb0: RX(2) MSIX IRQ Handled = 36258 May 8 09:17:57 debbie kernel: igb0: RX(2) LRO Queued= 5794 May 8 09:17:57 debbie kernel: igb0: RX(2) LRO Flushed= 5673 May 8 09:17:57 debbie kernel: igb0: Queue(3) rdh = 1418, rdt = 1417 May 8 09:17:57 debbie kernel: igb0: RX(3) Packets received = 1418 May 8 09:17:57 debbie kernel: igb0: RX(3) Split Packets = 939 May 8 09:17:57 debbie kernel: igb0: RX(3) Byte count = 1700710 May 8 09:17:57 debbie kernel: igb0: RX(3) MSIX IRQ Handled = 31399 May 8 09:17:57 debbie kernel: igb0: RX(3) LRO Queued= 1102 May 8 09:17:57 debbie kernel: igb0:
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
On 05/08/2010 05:54 AM, Fabien Thomas wrote: Have you tried to disable TSO / LRO? Fabien I have 3 boxes, each with two nics. One nic for the private network and one for the public network. The private network is all on the same vlan. All 6 nics are on the same switch. All connections are 1000tx Full Duplex. I will call the servers Box A, Box B, and Box C. When i FTP data between Box A B i get abou 25MB/sec. When i FTP data from Box C to Box A or B, i get about 20MB/sec. When i FTP data from Box A to C i get 10MB/sec When i FTP data from Box B to C i get 200KB/sec... Can anyone suggest why i might only be getting 200KB when transfering data from Box B to C but not when transferring data from Box A to C? I tried installing ubuntu to see if the problem was still there and it is not. I can FTP data from Box A or B to box C at 10MB/sec, still not the gigabit speeds i should be seeing but not the 200KB/sec i am getting with freebsd. I logged into my switch and it reports that the ports are 1000tx full duplex, the same as what freebsd is reporting. The switch freebsd also report no errors. I really dont know what to do. Nothing anywhere reports showing a problem but obviously there is a problem somewhere. I've tried drivers 1.9.5, 1.8.4, and 1.7.3. If i use 1.8.4 or 1.7.3 i get 50KB/sec transferring data from box B whereas if i use 1.9.5 i get 200-300KB sec. I should be getting like 50MB/sec ;( Any help would be grateful. Or, maybe someone could put me in touch with the person responsible for the intel nic drivers and i can work with them on resolving this issue? Thanks in advance. NameMtu Network Address Ipkts Ierrs IdropOpkts Oerrs Coll igb0 1500Link#1 00:30:48:9f:11:0425853 0 018340 0 0 igb0 1500 216.105.91.14 216.105.91.145 19040 - -18343 - - igb1* 1500Link#2 00:30:48:9f:11:050 0 00 0 0 lo0 16384Link#3 72 0 0 72 0 0 lo0 16384 127.0.0.0/8 127.0.0.1 72 - - 72 - - Sysctl info for igb0/1 dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 1.8.4 dev.igb.0.%driver: igb dev.igb.0.%location: slot=0 function=0 dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x15d9 subdevice=0x0100 class=0x02 dev.igb.0.%parent: pci1 dev.igb.0.debug: -1 dev.igb.0.stats: -1 dev.igb.0.flow_control: 3 dev.igb.0.enable_aim: 1 dev.igb.0.low_latency: 128 dev.igb.0.ave_latency: 450 dev.igb.0.bulk_latency: 1200 dev.igb.0.rx_processing_limit: 100 Debug info for igb0 May 8 09:17:57 debbie kernel: igb0: TX(1) Packets sent = 1295 May 8 09:17:57 debbie kernel: igb0: Queue(2) tdh = 106, tdt = 106 May 8 09:17:57 debbie kernel: igb0: TX(2) no descriptors avail event = 0 May 8 09:17:57 debbie kernel: igb0: TX(2) MSIX IRQ Handled = 5353 May 8 09:17:57 debbie kernel: igb0: TX(2) Packets sent = 5450 May 8 09:17:57 debbie kernel: igb0: Queue(3) tdh = 1687, tdt = 1687 May 8 09:17:57 debbie kernel: igb0: TX(3) no descriptors avail event = 0 May 8 09:17:57 debbie kernel: igb0: TX(3) MSIX IRQ Handled = 1335 May 8 09:17:57 debbie kernel: igb0: TX(3) Packets sent = 1354 May 8 09:17:57 debbie kernel: igb0: Queue(0) rdh = 425, rdt = 424 May 8 09:17:57 debbie kernel: igb0: RX(0) Packets received = 16809 May 8 09:17:57 debbie kernel: igb0: RX(0) Split Packets = 12262 May 8 09:17:57 debbie kernel: igb0: RX(0) Byte count = 25894129 May 8 09:17:57 debbie kernel: igb0: RX(0) MSIX IRQ Handled = 43428 May 8 09:17:57 debbie kernel: igb0: RX(0) LRO Queued= 13601 May 8 09:17:57 debbie kernel: igb0: RX(0) LRO Flushed= 9966 May 8 09:17:57 debbie kernel: igb0: Queue(1) rdh = 1700, rdt = 1699 May 8 09:17:57 debbie kernel: igb0: RX(1) Packets received = 1700 May 8 09:17:57 debbie kernel: igb0: RX(1) Split Packets = 937 May 8 09:17:57 debbie kernel: igb0: RX(1) Byte count = 1392691 May 8 09:17:57 debbie kernel: igb0: RX(1) MSIX IRQ Handled = 31888 May 8 09:17:57 debbie kernel: igb0: RX(1) LRO Queued= 1362 May 8 09:17:57 debbie kernel: igb0: RX(1) LRO Flushed= 1351 May 8 09:17:57 debbie kernel: igb0: Queue(2) rdh = 33, rdt = 32 May 8 09:17:57 debbie kernel: igb0: RX(2) Packets received = 6177 May 8 09:17:57 debbie kernel: igb0: RX(2) Split Packets = 1442 May 8 09:17:57 debbie kernel: igb0: RX(2) Byte count = 2518498 May 8 09:17:57 debbie kernel: igb0: RX(2) MSIX IRQ Handled = 36258 May 8 09:17:57 debbie kernel: igb0: RX(2) LRO Queued= 5794 May 8 09:17:57 debbie kernel: igb0: RX(2) LRO Flushed= 5673 May 8 09:17:57 debbie kernel: igb0: Queue(3) rdh = 1418, rdt = 1417 May 8 09:17:57 debbie kernel: igb0: RX(3) Packets received = 1418 May 8 09:17:57 debbie kernel: igb0: RX(3) Split Packets = 939 May 8 09:17:57 debbie kernel: igb0: RX(3) Byte count = 1700710 May 8 09:17:57 debbie kernel: igb0: RX(3) MSIX IRQ Handled = 31399 May 8 09:17:57 debbie kernel: igb0: RX(3) LRO
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. Ian -- Ian Freislich ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. Ian -- Ian Freislich I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) These are the only two nics the local computer store has in stock. Anyone know if the TP LINK is supported? I couldnt find anything to suggest it is on google. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did pull the server and brought it home so that i can test more things quicker. I am going to try using a raid card instead of the onboard sata ports and see if i still encounter the same problem. I would love any suggestions you may have on where to go from here to figure out where the problem might be. joe ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did pull the server and brought it home so that i can test more things quicker. I am going to try using a raid card instead of the onboard sata ports and see if i still encounter the same problem. I would love any suggestions you may have on where to go from here to figure out where the problem might be. joe ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
On 05/08/2010 01:31 PM, Jack Vogel wrote: Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did pull the server and brought it home so that i can test more things quicker. I am going to try using a raid card instead of the onboard sata ports and see if i still encounter the same problem. I would love any suggestions you may have on where to go from here to figure out where the problem might be. joe ___ freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org I think it might have something to so with the nics / switch, and their features. I brought the box home, plugged into my gb switch, and i am able to FTP data to the server at around 35MB/sec. I dont know what would cause this other than some sort of issue with the the 3 different types of nics and the switch i am using. Any suggestions? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
I still am not clear on this system, how many ports are on it, and its an 82576? Sounds to me like you've proven its not on the box if you can do fine when its on its own. So change ports in the switch, as I said, change cables, must be something in that environment. Jack On Sat, May 8, 2010 at 10:04 AM, joe j...@hostedcontent.com wrote: On 05/08/2010 01:31 PM, Jack Vogel wrote: Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did pull the server and brought it home so that i can test more things quicker. I am going to try using a raid card instead of the onboard sata ports and see if i still encounter the same problem. I would love any suggestions you may have on where to go from here to figure out where the problem might be. joe ___ freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org I think it might have something to so with the nics / switch, and their features. I brought the box home, plugged into my gb switch, and i am able to FTP data to the server at around 35MB/sec. I dont know what would cause this other than some sort of issue with the the 3 different types of nics and the switch i am using. Any suggestions? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
On 05/08/2010 01:53 PM, Jack Vogel wrote: I still am not clear on this system, how many ports are on it, and its an 82576? Sounds to me like you've proven its not on the box if you can do fine when its on its own. So change ports in the switch, as I said, change cables, must be something in that environment. Jack On Sat, May 8, 2010 at 10:04 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 01:31 PM, Jack Vogel wrote: Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did pull the server and brought it home so that i can test more things quicker. I am going to try using a raid card instead of the onboard sata ports and see if i still encounter the same problem. I would love any suggestions you may have on where to go from here to figure out where the problem might be. joe ___ freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org I think it might have something to so with the nics / switch, and their features. I brought the box home, plugged into my gb switch, and i am able to FTP data to the server at around 35MB/sec. I dont know what would cause this other than some sort of issue with the the 3 different types of nics and the switch i am using. Any suggestions?
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
The cable, its a simple thing but make SURE you try that, a slightly damaged one can do weird things and its quick to check, don't overlook it. Jack On Sat, May 8, 2010 at 10:22 AM, joe j...@hostedcontent.com wrote: On 05/08/2010 01:53 PM, Jack Vogel wrote: I still am not clear on this system, how many ports are on it, and its an 82576? Sounds to me like you've proven its not on the box if you can do fine when its on its own. So change ports in the switch, as I said, change cables, must be something in that environment. Jack On Sat, May 8, 2010 at 10:04 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 01:31 PM, Jack Vogel wrote: Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did pull the server and brought it home so that i can test more things quicker. I am going to try using a raid card instead of the onboard sata ports and see if i still encounter the same problem. I would love any suggestions you may have on where to go from here to figure out where the problem might be. joe ___ freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailto:freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org mailto:freebsd-current-unsubscr...@freebsd.org I think it might have something to so with the nics / switch, and their features. I brought the box home, plugged into my gb
Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
On 05/08/2010 02:21 PM, Jack Vogel wrote: The cable, its a simple thing but make SURE you try that, a slightly damaged one can do weird things and its quick to check, don't overlook it. Jack On Sat, May 8, 2010 at 10:22 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 01:53 PM, Jack Vogel wrote: I still am not clear on this system, how many ports are on it, and its an 82576? Sounds to me like you've proven its not on the box if you can do fine when its on its own. So change ports in the switch, as I said, change cables, must be something in that environment. Jack On Sat, May 8, 2010 at 10:04 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 01:31 PM, Jack Vogel wrote: Looks like something to do with system C, you might isolate it, and try a back to back connection with its NICs, change cables, look at BIOS settings, change the slot the nic is in... All just off the top of my head. Jack On Sat, May 8, 2010 at 9:41 AM, joe j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com mailto:j...@hostedcontent.com wrote: On 05/08/2010 11:17 AM, Ian FREISLICH wrote: joe wrote: On 05/08/2010 06:55 AM, Ian FREISLICH wrote: joe wrote: I have just tried your suggeston and it has no effect for me ;( Do you have another brand of NIC that you can try? At least that will isolate whether it's igb(4) or something else. I will grab a new nic today and try...my options are limited though. Here are the nics i can get my hands on TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter (supported by fbsd?) Based on the RTL8168B chip. Should be supported by the re(4) driver. Intel (EXPI9301CT) Gigabit CT Desktop Adapter (yet another intel nic) i82574L chip. Should be supported by the em(4) driver. I have had good performance in the past with this driver and less than satisfactory performance with the igb(4) driver. That may not be your problem though. Before you go out and buy, have a look at the amount of interrupt time your slow machine spends in 'top' or 'systat -vm'. systat will also show the interrupt rate for each driver, perhaps it's not doing interrupt moderation properly. This will manifest as more than about a 1000 per second. There are loader tunables for the driver to increase the number of transfer descriptors and to tune interrupt moderation. You could try running trafshow (port) on the interface while performing the transfer. Perhaps promiscuous mode will turn off some hardware feature that will improve things. It may however break hardware vlanning as it does on my 82575GB 4 port igb card. Ian -- Ian Freislich I bought those two cards anyways, im in a rush to figure out this problem. That being said i am still encountering the exact same problem regardless on which network card i am running. I am at a complete loss. I am about to try a raid card to see if the problem might lay within the onboard sata ports. I did
