Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
On Thu, 2009-04-30 at 13:31 +0200, Kai Schaetzl wrote: > JohnS wrote on Thu, 30 Apr 2009 02:17:13 -0400: > > > Kai, I read the Xen list and the way your doing it (the last option) > > looks like something I may try for testing in VMs. > > It works fine, I'm converting all my setups to that now. > > > Indeed also I when I installed Xen I had to manually take out peth0 when > > I uninstalled it. > > How did you manually take it down? The problem seems to be that peth0 is > the physical interface now. But I'm not able to take it down as a bridge > nor as a physical interface. And not without breaking the network > connection, anyway. > > Kai Correction, the install above should be "uninstalled". Typo error sorry. But that is right peth0 becomes the interface. JohnStanley ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
On Apr 30, 2009, at 7:31 AM, Kai Schaetzl wrote: > JohnS wrote on Thu, 30 Apr 2009 02:17:13 -0400: > >> Kai, I read the Xen list and the way your doing it (the last option) >> looks like something I may try for testing in VMs. > > It works fine, I'm converting all my setups to that now. > >> Indeed also I when I installed Xen I had to manually take out peth0 >> when >> I uninstalled it. > > How did you manually take it down? The problem seems to be that > peth0 is > the physical interface now. But I'm not able to take it down as a > bridge > nor as a physical interface. And not without breaking the network > connection, anyway. The problem with Xen's network scripts are they assume a bare bones default network setup. If you have anything more then that then I recommend disabling the automated network setup in xend.sxp and manually setting up your bridges which sounds like the conclusion you came to after much discussion. -Ross ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
JohnS wrote on Thu, 30 Apr 2009 02:17:13 -0400: > Kai, I read the Xen list and the way your doing it (the last option) > looks like something I may try for testing in VMs. It works fine, I'm converting all my setups to that now. > Indeed also I when I installed Xen I had to manually take out peth0 when > I uninstalled it. How did you manually take it down? The problem seems to be that peth0 is the physical interface now. But I'm not able to take it down as a bridge nor as a physical interface. And not without breaking the network connection, anyway. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
On Tue, 2009-04-28 at 23:31 +0200, Kai Schaetzl wrote: > JohnS wrote on Mon, 27 Apr 2009 11:09:56 -0400: > > > If you don't mind when you come to an answer would you please let me > > know. I am interested to know. > > I could not find a real solution. I had to go to another way of creating > the network setup for this machine (and maybe others, I will see) and > disable the network-bridge script from xen. That new solution is > documented on the xen-users mailing-list. > > Kai --- Kai, I read the Xen list and the way your doing it (the last option) looks like something I may try for testing in VMs. Although I do want to say Virtual Box does that very same behavior that you first described in your post. When you start up Virtual Box it takes out my eth1 and I am left with eth0. Makes eth1 Brigded if that makes sense to you like Xen was doing. I have to say it really seems like a better solution the way your doing it now. In fact I am going to give it a try also. It may be a little more effort into doing it but the approach is much better. Indeed also I when I installed Xen I had to manually take out peth0 when I uninstalled it. JohnStanley ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
JohnS wrote on Mon, 27 Apr 2009 11:09:56 -0400: > If you don't mind when you come to an answer would you please let me > know. I am interested to know. I could not find a real solution. I had to go to another way of creating the network setup for this machine (and maybe others, I will see) and disable the network-bridge script from xen. That new solution is documented on the xen-users mailing-list. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
Kai Schaetzl wrote on Sun, 26 Apr 2009 00:31:20 +0200: > Thanks for the answers so far. At least confirms that the simple juggling > around of the main network interfaces is normal and to be expected. Simple test, I shut off xend and xendomains and the problem is gone. So, the problem is with the script that xend runs when creating the eth0/peth0 network bridge for the domUs. I'll move this to the xen-users list. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
JohnS wrote on Sat, 25 Apr 2009 16:32:06 -0400: > You *must* specify the HWADDR field in the ifcfg-* files in order to > have persistent ethernet naming. And that is what I always do. Never done it another way. You may have overlooked that part in my message where I state that it works without a problem despite this juggling around until I add a virtual interface to eth0. I'll try tomorrow adding HWADDR to eth0:1 as well, but I think this will fail. I guess I will have to turn off in the BIOS or remove eth2 and maybe eth1 as well tomorrow and run some more tests with just one adapter and then add to it. I hope I can switch off eth2 in the BIOS somehow. I would hate to remove it as it is below the SAS adapter and the many SATA cables. Thanks for the answers so far. At least confirms that the simple juggling around of the main network interfaces is normal and to be expected. Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
At Sat, 25 Apr 2009 16:32:06 -0400 CentOS mailing list wrote: > > > On Sat, 2009-04-25 at 14:52 -0500, Les Mikesell wrote: > > JohnS wrote: > > > On Sat, 2009-04-25 at 20:33 +0200, Kai Schaetzl wrote: > > >> I have a strange problem on one machine where eth0 gets killed when I > > >> add > > >> a virtual interface. It's got something to do with the NIC ordering or > > >> with the xen network script having a problem with multiple NICs and > > >> virtual interfaces. I could need some help/comments on this. > > >> > > >> Some history: > > >> I added a NIC (chip identifies as Realtek Semiconductor Co., Ltd. > > >> RTL8111/8168B PCI Express Gigabit Ethernet) to a Dell R200 server. > > >> CentOS 5.3 with Xen 3.3.1 (gitco repo). > > > > > > see this: > > > http://linux.dell.com/files/whitepapers/nic-enum-whitepaper-v3.pdf > > > > > > This is a known issue with all Poweredge Servers. It will give you an > > > explanation and workaround for it. > > > > I don't think there is anything unique to Dells about this. The kernel > > essentially randomizes device naming on everything. Dell just took the > > trouble to document it. > --- > > Also: https://bugzilla.redhat.com/show_bug.cgi?id=491432 > Seems to apply to Kais case. > > You *must* specify the HWADDR field in the ifcfg-* files in order to > have persistent ethernet naming. Was the way I done it on dell hardware > and it states that on the Bug Report. On ALL RedHat flavored distros (even eith 2.4 kernels), I *always* specificed the HWADDR field in the ifcfg-* files. I *think* the RedHat installers generally always set this field during installation as well. At least as early as RH 7. or RH 9, which would be when I first was dealing with machines with more than one NIC. > > JohnStanley > > > ___ > CentOS mailing list > CentOS@centos.org > http://lists.centos.org/mailman/listinfo/centos > > -- Robert Heller -- 978-544-6933 Deepwoods Software-- Download the Model Railroad System http://www.deepsoft.com/ -- Binaries for Linux and MS-Windows hel...@deepsoft.com -- http://www.deepsoft.com/ModelRailroadSystem/ ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
JohnS wrote: > On Sat, 2009-04-25 at 14:52 -0500, Les Mikesell wrote: >> JohnS wrote: >>> On Sat, 2009-04-25 at 20:33 +0200, Kai Schaetzl wrote: I have a strange problem on one machine where eth0 gets killed when I add a virtual interface. It's got something to do with the NIC ordering or with the xen network script having a problem with multiple NICs and virtual interfaces. I could need some help/comments on this. Some history: I added a NIC (chip identifies as Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet) to a Dell R200 server. CentOS 5.3 with Xen 3.3.1 (gitco repo). >>> >>> see this: >>> http://linux.dell.com/files/whitepapers/nic-enum-whitepaper-v3.pdf >>> >>> This is a known issue with all Poweredge Servers. It will give you an >>> explanation and workaround for it. >> I don't think there is anything unique to Dells about this. The kernel >> essentially randomizes device naming on everything. Dell just took the >> trouble to document it. > --- > > Also: https://bugzilla.redhat.com/show_bug.cgi?id=491432 > Seems to apply to Kais case. > > You *must* specify the HWADDR field in the ifcfg-* files in order to > have persistent ethernet naming. Was the way I done it on dell hardware > and it states that on the Bug Report. I've had my ifcfg-* files renamed to ifcfg-*.bak files and ignored completely when moving drives, even among identical hardware. It's no fun when shipping to remote locations where the on-site people don't know much about linux. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
On Sat, 2009-04-25 at 14:52 -0500, Les Mikesell wrote: > JohnS wrote: > > On Sat, 2009-04-25 at 20:33 +0200, Kai Schaetzl wrote: > >> I have a strange problem on one machine where eth0 gets killed when I add > >> a virtual interface. It's got something to do with the NIC ordering or > >> with the xen network script having a problem with multiple NICs and > >> virtual interfaces. I could need some help/comments on this. > >> > >> Some history: > >> I added a NIC (chip identifies as Realtek Semiconductor Co., Ltd. > >> RTL8111/8168B PCI Express Gigabit Ethernet) to a Dell R200 server. > >> CentOS 5.3 with Xen 3.3.1 (gitco repo). > > > > see this: > > http://linux.dell.com/files/whitepapers/nic-enum-whitepaper-v3.pdf > > > > This is a known issue with all Poweredge Servers. It will give you an > > explanation and workaround for it. > > I don't think there is anything unique to Dells about this. The kernel > essentially randomizes device naming on everything. Dell just took the > trouble to document it. --- Also: https://bugzilla.redhat.com/show_bug.cgi?id=491432 Seems to apply to Kais case. You *must* specify the HWADDR field in the ifcfg-* files in order to have persistent ethernet naming. Was the way I done it on dell hardware and it states that on the Bug Report. JohnStanley ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
On Sat, 2009-04-25 at 14:52 -0500, Les Mikesell wrote: > JohnS wrote: > > On Sat, 2009-04-25 at 20:33 +0200, Kai Schaetzl wrote: > >> I have a strange problem on one machine where eth0 gets killed when I add > >> a virtual interface. It's got something to do with the NIC ordering or > >> with the xen network script having a problem with multiple NICs and > >> virtual interfaces. I could need some help/comments on this. > >> > >> Some history: > >> I added a NIC (chip identifies as Realtek Semiconductor Co., Ltd. > >> RTL8111/8168B PCI Express Gigabit Ethernet) to a Dell R200 server. > >> CentOS 5.3 with Xen 3.3.1 (gitco repo). > > > > see this: > > http://linux.dell.com/files/whitepapers/nic-enum-whitepaper-v3.pdf > > > > This is a known issue with all Poweredge Servers. It will give you an > > explanation and workaround for it. > > I don't think there is anything unique to Dells about this. The kernel > essentially randomizes device naming on everything. Dell just took the > trouble to document it. >From what I understand this was only with dell hardware that this was happening and they submitted a patch to red hat. Also it is the only hardware I have encountered with the problem also. There could be others. What's more is Kai says he's running 5.3 but the fix should be in that kernel. What I do wonder is if when the centos kernel was built, was it included? Maybe the CentOS Kernel builder could let us know? 2.6.19-rc3 and higher are supposed to have the fix? It is however a strange thing when you encounter it. I pulled my hair for a long time. Last thing is he has this problem on a R200 and from memory those were not a problem. Could be this is something new? He could check for a BIOS Revision if there is one. JohnStanley ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
JohnS wrote: > On Sat, 2009-04-25 at 20:33 +0200, Kai Schaetzl wrote: >> I have a strange problem on one machine where eth0 gets killed when I add >> a virtual interface. It's got something to do with the NIC ordering or >> with the xen network script having a problem with multiple NICs and >> virtual interfaces. I could need some help/comments on this. >> >> Some history: >> I added a NIC (chip identifies as Realtek Semiconductor Co., Ltd. >> RTL8111/8168B PCI Express Gigabit Ethernet) to a Dell R200 server. >> CentOS 5.3 with Xen 3.3.1 (gitco repo). > > see this: > http://linux.dell.com/files/whitepapers/nic-enum-whitepaper-v3.pdf > > This is a known issue with all Poweredge Servers. It will give you an > explanation and workaround for it. I don't think there is anything unique to Dells about this. The kernel essentially randomizes device naming on everything. Dell just took the trouble to document it. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] eth0 killed when adding virtual interface and multiple NICs are present
On Sat, 2009-04-25 at 20:33 +0200, Kai Schaetzl wrote: > I have a strange problem on one machine where eth0 gets killed when I add > a virtual interface. It's got something to do with the NIC ordering or > with the xen network script having a problem with multiple NICs and > virtual interfaces. I could need some help/comments on this. > > Some history: > I added a NIC (chip identifies as Realtek Semiconductor Co., Ltd. > RTL8111/8168B PCI Express Gigabit Ethernet) to a Dell R200 server. > CentOS 5.3 with Xen 3.3.1 (gitco repo). see this: http://linux.dell.com/files/whitepapers/nic-enum-whitepaper-v3.pdf This is a known issue with all Poweredge Servers. It will give you an explanation and workaround for it. JohnStanley ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] eth0 killed when adding virtual interface and multiple NICs are present
I have a strange problem on one machine where eth0 gets killed when I add a virtual interface. It's got something to do with the NIC ordering or with the xen network script having a problem with multiple NICs and virtual interfaces. I could need some help/comments on this. Some history: I added a NIC (chip identifies as Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet) to a Dell R200 server. CentOS 5.3 with Xen 3.3.1 (gitco repo). eth0 and eth1 are the built-in NICs, this is then eth2 (or it should be). Works. Everything is fine until I add a virtual interface to eth0 and reboot. I can add eth0:1 at runtime just fine. But if I let it stay in network-scripts and boot the whole eth0 is killed (doesn't show up in ifconfig and doesn't work). A network restart brings it up as if nothing is wrong. I first thought it might have something to do with the fact that eth0 is actually a bridge on Xen > 3.2 and tried the same config on another machine and there it works. It's not the exact same xen version, not 64bit and it's got only 1 NIC. So there are differences, but it seems to rule out the bridge as a cause. I then checked the logs more thoroughly and found that CentOS changes the NIC initialization order at boot-time. Without the third NIC it's eth0=NIC1 and eth1=NIC2 (as shown on the chassis). But with the third NIC it's most often that one that goes first. Here's a typical excerpt from messages. tigon/tg3 is the driver for the internal NICs which normally were on eth0 and eth1. Apr 25 19:00:59 c4 kernel: eth0: RTL8168b/8111b at 0xc2022000, 00:21:27:c9:d1:f5, XID 3800 IRQ 16 Apr 25 19:00:59 c4 kernel: eth1: Tigon3 [partno(BCM95721) rev 4201 PHY (5750)] (PCI Express) 10/100/1000Base-T Ethernet 00:1e:c9:fe:fb:ab Apr 25 19:00:59 c4 kernel: eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] WireSpeed[1] TSOcap[1] Apr 25 19:00:59 c4 kernel: eth1: dma_rwctrl[7618] dma_mask[64-bit] Apr 25 19:00:59 c4 kernel: eth2: Tigon3 [partno(BCM95721) rev 4201 PHY (5750)] (PCI Express) 10/100/1000Base-T Ethernet 00:1e:c9:fe:fb:ac Apr 25 19:00:59 c4 kernel: eth2: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] WireSpeed[1] TSOcap[1] Apr 25 19:00:59 c4 kernel: eth2: dma_rwctrl[7618] dma_mask[64-bit] Apr 25 19:00:59 c4 kernel: tg3: eth0: Link is up at 1000 Mbps, full duplex. Apr 25 19:00:59 c4 kernel: tg3: eth0: Flow control is on for TX and on for RX. Apr 25 19:00:59 c4 kernel: r8169: eth2: link up Apr 25 19:00:59 c4 kernel: r8169: eth2: link up Apr 25 19:01:01 c4 ntpd[2461]: Listening on interface eth2, 192.168.2.4#123 Enabled Apr 25 19:01:01 c4 ntpd[2461]: Listening on interface eth0, 192.168.1.24#123 Enabled Apr 25 19:01:01 c4 ntpd[2461]: Listening on interface eth1, 192.168.2.3#123 Enabled Apr 25 19:01:08 c4 uxmon: c4.net: started monitoring: lo eth2 eth0 eth1 Apr 25 19:01:18 c4 kernel: tg3: peth0: Link is up at 1000 Mbps, full duplex. Apr 25 19:01:18 c4 kernel: tg3: peth0: Flow control is on for TX and on for RX. Apr 25 19:01:18 c4 kernel: device peth0 entered promiscuous mode Apr 25 19:01:18 c4 kernel: type=1700 audit(1240678878.244:3): dev=peth0 prom=256 old_prom=0 auid=4294967295 ses=4294967295 Apr 25 19:01:18 c4 kernel: eth0: topology change detected, propagating Apr 25 19:01:18 c4 kernel: eth0: port 1(peth0) entering forwarding state Repeated booting sometimes gives me a different order, e.g. the two tigon come first, but this is rare. Well, it seems this wasn't a problem until I added a virtual interface to eth0. When the eth interfaces are brought up the system seems to reenumerate the eth numbering according to the HWADDR matches and thus eth0=NIC1 and so on. As soon as I add a virtual interface to eth0 this breaks and all of eth0 is killed. At least that's what I figure. So, the next obvious question is: How can I set a fixed order, so that NIC1 is always brought up first as eth0? I'm not sure if this would fix it, though. I have done too few reboots yet, but it seems that at least once I got the "correct" initialization order but eth0 got killed, anyway. So, it might not be the order but still something in the Xen script which happens only when multiple NICs are present and a virtual interface is added. Any thoughts so far? Kai -- Kai Schätzl, Berlin, Germany Get your web at Conactive Internet Services: http://www.conactive.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
