Re: Problems with a quad Realtek NIC
On 10/12/18 21:42, Martin Hanson wrote: >> It is preferable to just include the whole dmesg directly in the mail >> Better still, when it's a "sometimes works" problem, include a "diff -u" >> between the two (the context to show where the lines are added/removed). > > I have pasted a "diff -u" on https://paste FYI... I don't click on links for stuff that should be in-message. I suspect I'm not alone. But yes, I agree with Stuart, sounds like a HW problem. When things come and go without changing, that pretty well screams "hw". Different OSs may work around different hw bugs differently, but it's still a HW bug. In your case, looks like the BIOS isn't initializing the PCI-PCI bridges properly. > +ppb1 at pci1 dev 0 function 0 > vendor "ASMedia", unknown product 0x1184 rev 0x00 > +pci2 at ppb1 bus 2 > +ppb2 at pci2 dev 1 function 0 vendor "ASMedia", unknown product 0x1184 > rev 0x00: not configured by system firmware === > +ppb3 at pci2 dev 3 function 0 vendor "ASMedia", unknown product 0x1184 (man ppb) And yes, while Realteks used to be condemned and insulted, the new network devices on many ARM boards is making Realteks look good. At least their limitations are understood and dealt with well in SW. Most people don't need the absolute best HW. But in your case, you probably want those PCI-PCI bridges configured. :) Nick.
Re: Problems with a quad Realtek NIC
> It is preferable to just include the whole dmesg directly in the mail > Better still, when it's a "sometimes works" problem, include a "diff -u" > between the two (the context to show where the lines are added/removed). I have pasted a "diff -u" on https://paste.debian.net/1047098/ > Very unlikely to be a problem with the NIC driver. From what I've > seen (and having seen other examples of the quality of Jetway's BIOSes) > my money would be on a BIOS bug. I originally suspected it to be a BIOS bug (and please excuse me if I make a wrong assumption here), but I have run a ton of tests booting the same hardware up on Arch Linux, but I haven't seen the problem once. I don't know if could still be a BIOS bug, but I would suspect that if it where that I would see similar problems with the card not showing up. > Can you identify any particular situations where it works or fails? For > example "works after a cold boot but not warm" or something? I have been looking for patterns in the different tests I have run, but haven't found anything. Sometimes it works during a cold boot, other times it doesn't, and the same goes for warm boots. I have also tried disabling and enabling the onboard NIC to see if it might affect the problem, but it doesn't. Just for information, I have another identical box (same motherboard), but with a 4-port Intel NIC (same cheap card-thing from China, but with Intel) that runs flawless, but not only that, it performs super well even with several NIC ports going full speed. It has been running for a couple of years as a firewall/gateway, managing multiple networks (each separated by the 4-port NIC). I have made these as cheap alternatives to the very expensive Soekris. By mistake I got two of the 4-port Realtek cards, but since I got them, I wanted to see if they could run as well. Kindest regards. PS.: Sorry for the crappy Yandex mail client problem.
Re: Problems with a quad Realtek NIC
On 2018-10-12, Aaron Mason wrote: > Wow, never seen a multi-port Realtek NIC before. I have but they're usually single NIC plus switch chip, looks like this is the real thing. > Maybe try the card on another motherboard and see if you get the same > symptoms. agreed if possible. > In any case, it's my understanding (and the maintainers of re(4) are > free to correct me on this) that Realtek NICs are usually a bad idea > for non-desktop applications, since they offload pretty much (if not > entirely) all of its functions to the OS - every time something > sneezes on the connected network, it raises an interrupt to the CPU. > This can lead to pretty craptacular performance and high CPU use under > heavy load. I can only imagine the hailstorm that would ensue if all > four of your NIC ports were running at full tilt. They're not the world's fastest NICs but the re(4) Realteks aren't *terrible* (and the driver is likely not as efficient as it could be.) rl(4) Realteks are *much* less good but people seem to apply Bill Paul's comments in *that* driver across to any nic made by the company... Also a ~5 year old celeron is not one to run 4xGbE full tilt in the first place (though it could be argued it will want any help it can get). > Your best bet for this would be a used quad-port Intel NIC, which can > be had for similar money (but not too much more) and is rather more > self-sufficient than the Realtek cards (and the maintainers of em(4) > are free to correct me there as well). You'll get better line > performance with less CPU stress. If the board has general problems with PCIE cards with bridges that might not fix the problem though. > On Thu, Oct 11, 2018 at 11:49 AM Martin Hanson > wrote: >> >> Hi, I have one of these 4-port Realtek NIC cards: >> https://www.ebay.co.uk/itm/PCIe-PCI-Express-to-4x-Gigabit-Card-4-Port-Ethernet-Network-Adapter-10-100-1000M/252484240577?epid=505371101 >> I >> am running OpenBSD 6.3 stable. During boot the card is seen, but it only >> works occasionally. When it works I can see all the 4 ports using >> "ifconfig" and I can assign IP addresses etc. When it doesn't work >> nothing is shown using "ifconfig". As far as I understand from the "re" >> manpage RTL8168E/8111E is supported. This is my dmesg: >> http://paste.debian.net/1046756/ When >> the card isn't working I also get this: +ppb1 at pci1 dev 0 function 0 It is preferable to just include the whole dmesg directly in the mail Better still, when it's a "sometimes works" problem, include a "diff -u" between the two (the context to show where the lines are added/removed). "pcidump -vxx" in both states might be useful too. (Ideally without non-ascii characters in the mail :-) >> vendor "ASMedia", unknown product 0x1184 rev 0x00 >> +pci2 at ppb1 bus 2 >> +ppb2 at pci2 dev 1 function 0 vendor "ASMedia", unknown product 0x1184 >> rev 0x00: not configured by system firmware >> +ppb3 at pci2 dev 3 function 0 vendor "ASMedia", unknown product 0x1184 >> rev 0x00: not configured by system firmware >> +ppb4 at pci2 dev 5 function 0 vendor "ASMedia", unknown product 0x1184 >> rev 0x00: not configured by system firmware >> +ppb5 at pci2 dev 7 function 0 vendor "ASMedia", unknown product 0x1184 >> rev 0x00: not configured by system firmware >>  Is this a driver issue or something else perhaps? Kindest regards Very unlikely to be a problem with the NIC driver. From what I've seen (and having seen other examples of the quality of Jetway's BIOSes) my money would be on a BIOS bug. Can you identify any particular situations where it works or fails? For example "works after a cold boot but not warm" or something? Personally if I had to use this hardware as a router I'd use a single port card (which will probably work OK) and break the ports out on a managed switch via vlans instead.
Re: Problems with a quad Realtek NIC
Wow, never seen a multi-port Realtek NIC before. Based on your dmesg and the excerpt above, it looks like the issue starts at ppb1 - a PCI-PCI bridge likely on the quad-port NIC based on the vendor being the same as each one below it and the one above it being an Intel ppb(4) - and then cascades through. Looking at the dmesg excerpt below, looks like your card has four separate Realtek RTL8168 NICs each on a separate PCI-PCI bridge, each of which connects to another PCI-PCI bridge, then another before connecting to the one on the mainboard: pci0 at mainbus0 bus 0 [...] ppb0 at pci0 dev 28 function 0 "Intel Bay Trail PCIE" rev 0x0e: msi <-- MAINBOARD pci1 at ppb0 bus 1 ppb1 at pci1 dev 0 function 0 vendor "ASMedia", unknown product 0x1184 rev 0x00 <-- PCIe card pci2 at ppb1 bus 2 ppb2 at pci2 dev 1 function 0 vendor "ASMedia", unknown product 0x1184 rev 0x00: msi pci3 at ppb2 bus 3 re0 at pci3 dev 0 function 0 "Realtek 8168" rev 0x06: RTL8168E/8111E-VL (0x2c80), msi, address 00:e0:4c:68:1a:ea rgephy0 at re0 phy 7: RTL8169S/8110S/8211 PHY, rev. 5 ppb3 at pci2 dev 3 function 0 vendor "ASMedia", unknown product 0x1184 rev 0x00: msi pci4 at ppb3 bus 4 re1 at pci4 dev 0 function 0 "Realtek 8168" rev 0x06: RTL8168E/8111E-VL (0x2c80), msi, address 00:e0:4c:68:1a:eb rgephy1 at re1 phy 7: RTL8169S/8110S/8211 PHY, rev. 5 ppb4 at pci2 dev 5 function 0 vendor "ASMedia", unknown product 0x1184 rev 0x00: msi pci5 at ppb4 bus 5 re2 at pci5 dev 0 function 0 "Realtek 8168" rev 0x06: RTL8168E/8111E-VL (0x2c80), msi, address 00:e0:4c:68:1a:ec rgephy2 at re2 phy 7: RTL8169S/8110S/8211 PHY, rev. 5 ppb5 at pci2 dev 7 function 0 vendor "ASMedia", unknown product 0x1184 rev 0x00: msi pci6 at ppb5 bus 6 re3 at pci6 dev 0 function 0 "Realtek 8168" rev 0x06: RTL8168E/8111E-VL (0x2c80), msi, address 00:e0:4c:68:1a:ed rgephy3 at re3 phy 7: RTL8169S/8110S/8211 PHY, rev. 5 Here's a diagram of the logical layout based on the above: https://imgur.com/AM1sGr8 Maybe try the card on another motherboard and see if you get the same symptoms. In any case, it's my understanding (and the maintainers of re(4) are free to correct me on this) that Realtek NICs are usually a bad idea for non-desktop applications, since they offload pretty much (if not entirely) all of its functions to the OS - every time something sneezes on the connected network, it raises an interrupt to the CPU. This can lead to pretty craptacular performance and high CPU use under heavy load. I can only imagine the hailstorm that would ensue if all four of your NIC ports were running at full tilt. Your best bet for this would be a used quad-port Intel NIC, which can be had for similar money (but not too much more) and is rather more self-sufficient than the Realtek cards (and the maintainers of em(4) are free to correct me there as well). You'll get better line performance with less CPU stress. On Thu, Oct 11, 2018 at 11:49 AM Martin Hanson wrote: > > Hi, I have one of these 4-port Realtek NIC cards: > https://www.ebay.co.uk/itm/PCIe-PCI-Express-to-4x-Gigabit-Card-4-Port-Ethernet-Network-Adapter-10-100-1000M/252484240577?epid=505371101 > I > am running OpenBSD 6.3 stable. During boot the card is seen, but it only > works occasionally. When it works I can see all the 4 ports using > "ifconfig" and I can assign IP addresses etc. When it doesn't work > nothing is shown using "ifconfig". As far as I understand from the "re" > manpage RTL8168E/8111E is supported. This is my dmesg: > http://paste.debian.net/1046756/ When > the card isn't working I also get this: +ppb1 at pci1 dev 0 function 0 > vendor "ASMedia", unknown product 0x1184 rev 0x00 > +pci2 at ppb1 bus 2 > +ppb2 at pci2 dev 1 function 0 vendor "ASMedia", unknown product 0x1184 > rev 0x00: not configured by system firmware > +ppb3 at pci2 dev 3 function 0 vendor "ASMedia", unknown product 0x1184 > rev 0x00: not configured by system firmware > +ppb4 at pci2 dev 5 function 0 vendor "ASMedia", unknown product 0x1184 > rev 0x00: not configured by system firmware > +ppb5 at pci2 dev 7 function 0 vendor "ASMedia", unknown product 0x1184 > rev 0x00: not configured by system firmware >  Is this a driver issue or something else perhaps? Kindest regards -- Aaron Mason - Programmer, open source addict I've taken my software vows - for beta or for worse
