[Lxc-users] lxcbr0 MAC addr issue
It is a fact that the bridge takes the lowest MAC address from the attached ports for the host port. See for example http://backreference.org/2010/07/28/linux-bridge-mac-addresses-and-dynamic-ports/ Thus if a container is restarted, the host port can potentially change its MAC address and containers will have a stale ARP cache. This of course causes problem for communication container-host. Tested the workaround mentioned in the link but then I got problem with network manager on a later Ubuntu version. Then I tried using a dummy container and reusing its MAC addr for the host port. Works but... Now my question, could not lxc (at boot) setup a fixed MAC addr for the host port? Thanks, Hans -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [Spam-Wahrscheinlichkeit=45] lxcbr0 MAC addr issue
Dear Hans, this is a FAQ here but -- as you already found -- not basically caused by LXC. The software bridge will always choose the lowest MAC of the attached devices or hold an explicit assigned (from the set of currently assigned devices) as long as possible. In your case you either may set the MAC of the outgoing NIC or set the MAC of the veth's from a range which is above common hardware. There are more or less reserved ranges used for this by us here and in similar projects. Guido -Original Message- From: Hans Feldt [mailto:hans.fe...@ericsson.com] Sent: Wednesday, June 05, 2013 8:23 AM To: lxc-users@lists.sourceforge.net Subject: [Spam-Wahrscheinlichkeit=45][Lxc-users] lxcbr0 MAC addr issue It is a fact that the bridge takes the lowest MAC address from the attached ports for the host port. See for example http://backreference.org/2010/07/28/linux-bridge-mac-addresses-and-dynamic-ports/ Thus if a container is restarted, the host port can potentially change its MAC address and containers will have a stale ARP cache. This of course causes problem for communication container-host. Tested the workaround mentioned in the link but then I got problem with network manager on a later Ubuntu version. Then I tried using a dummy container and reusing its MAC addr for the host port. Works but... Now my question, could not lxc (at boot) setup a fixed MAC addr for the host port? Thanks, Hans -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] [Spam-Wahrscheinlichkeit=45] lxcbr0 MAC addr issue
-Original Message- From: Jäkel, Guido [mailto:g.jae...@dnb.de] Sent: den 5 juni 2013 09:26 To: Hans Feldt; 'lxc-users@lists.sourceforge.net' Subject: RE: [Spam-Wahrscheinlichkeit=45][Lxc-users] lxcbr0 MAC addr issue Dear Hans, this is a FAQ here but -- as you already found -- not basically caused by LXC. The software bridge will always choose the lowest MAC of the attached devices or hold an explicit assigned (from the set of currently assigned devices) as long as possible. In your case you either may set the MAC of the outgoing NIC or set the MAC of the veth's from a range which is above common hardware. There are more or less reserved ranges used for this by us here and in similar projects. Setting it to the MAC of the outgoing NIC is that safe or can it cause any problems? The MACs of the veth's are automatically set by lxc so what do you mean with set the MAC of the veth's from a range? Those reserved ranges, do you have a pointer? I just thought it would be nice if lxc at boot could set this up automagically for all of us... Thanks a lot, Hans Guido -Original Message- From: Hans Feldt [mailto:hans.fe...@ericsson.com] Sent: Wednesday, June 05, 2013 8:23 AM To: lxc-users@lists.sourceforge.net Subject: [Spam-Wahrscheinlichkeit=45][Lxc-users] lxcbr0 MAC addr issue It is a fact that the bridge takes the lowest MAC address from the attached ports for the host port. See for example http://backreference.org/2010/07/28/linux-bridge-mac-addresses-and- dyna mic-ports/ Thus if a container is restarted, the host port can potentially change its MAC address and containers will have a stale ARP cache. This of course causes problem for communication container-host. Tested the workaround mentioned in the link but then I got problem with network manager on a later Ubuntu version. Then I tried using a dummy container and reusing its MAC addr for the host port. Works but... Now my question, could not lxc (at boot) setup a fixed MAC addr for the host port? Thanks, Hans -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
[Lxc-users] lxc containers as backup 'replicas'
Following the pretty successful tests** I've been making of using lxc containers I'd be grateful for some advice on using lxc containers as backp 'replicas' of running machines, to bring up in case the main host fails. **(I've been on the list to discuss routing problems which I've solved temporarily by turning off kernel ethernet filtering -- still an issue in discussion). There are 4 parts to a running machine that I will need to replicate to a lxc container, which I intend to do nightly. These are: 1. etc configuration (we back config files up through etckeeper) 2. binaries (we're happy with a dpkg -l listings here) 3. run-time config for web apps (we back up through a file backup) 4. database backup (backed up via log shipping) I'd be grateful to know if it is possible to sync 1. and 3. into the container when it is not running. In other words, to simply update the config files in /var/lib/lxc/container/rootfs/etc, for example? On another point I'd also like to know of the recommended way of using another mount point for lxc containers and the dpkg cache. For example, I wish to hold my containers in /dev/sdb/ mounted on /containers. Should I symlink /var/lib/lxc/ to this mount point? Finally I'd be grateful to learn of people's experiences with btrfs for snapshotting and managing containers. I personally use it for my laptop backups, but my host server is on a 3.2.0-4-amd64 kernel which is pretty old by btrfs standards. Rory -- Rory Campbell-Lange r...@campbell-lange.net -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc containers as backup 'replicas'
Quoting Rory Campbell-Lange (r...@campbell-lange.net): Following the pretty successful tests** I've been making of using lxc containers I'd be grateful for some advice on using lxc containers as backp 'replicas' of running machines, to bring up in case the main host fails. **(I've been on the list to discuss routing problems which I've solved temporarily by turning off kernel ethernet filtering -- still an issue in discussion). There are 4 parts to a running machine that I will need to replicate to a lxc container, which I intend to do nightly. These are: 1. etc configuration (we back config files up through etckeeper) 2. binaries (we're happy with a dpkg -l listings here) 3. run-time config for web apps (we back up through a file backup) 4. database backup (backed up via log shipping) I'd be grateful to know if it is possible to sync 1. and 3. into the container when it is not running. In other words, to simply update the config files in /var/lib/lxc/container/rootfs/etc, for example? If the source is still running then I suppose depending on the application some of the source files could be in a transient, inconsistent state. On another point I'd also like to know of the recommended way of using another mount point for lxc containers and the dpkg cache. For example, I wish to hold my containers in /dev/sdb/ mounted on /containers. Should I symlink /var/lib/lxc/ to this mount point? If you're on a new enough lxc (i.e. 0.9.0) I'd recommend using lxcpath. You can set 'lxcpath = /srv/lxc' in /etc/lxc/lxc.conf, then all containers will be created and run from /srv/lxc instead of /var/lib/lxc. Or you can just add '-P /srv/lxc' to all lxc-* commands. Finally I'd be grateful to learn of people's experiences with btrfs for snapshotting and managing containers. I personally use it for my laptop backups, but my host server is on a 3.2.0-4-amd64 kernel which is pretty old by btrfs standards. yeah I don't know that I'd trust it under 3.2. I think 3.5 is where it stopped losing data for me. But best to run some tests. When it failed me, it generally did so after one or two subvolume commands. -serge -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc containers as backup 'replicas'
On Wed, Jun 5, 2013 at 6:50 PM, Rory Campbell-Lange r...@campbell-lange.net wrote: There are 4 parts to a running machine that I will need to replicate to a lxc container, which I intend to do nightly. These are: 1. etc configuration (we back config files up through etckeeper) 2. binaries (we're happy with a dpkg -l listings here) 3. run-time config for web apps (we back up through a file backup) 4. database backup (backed up via log shipping) I'd be grateful to know if it is possible to sync 1. and 3. into the container when it is not running. In other words, to simply update the config files in /var/lib/lxc/container/rootfs/etc, for example? Should be possible. However, personally I'd just forget for a moment that the backup will be run on lxc and do the same things I'd do on a normal machine. In my case, I'd use zfs snapshot and send|receive (yes, you can use zfs for root). In your case it'd probably be rsync or whatever you're happy with. On another point I'd also like to know of the recommended way of using another mount point for lxc containers and the dpkg cache. For example, I wish to hold my containers in /dev/sdb/ mounted on /containers. Should I symlink /var/lib/lxc/ to this mount point? I'm pretty sure there were problems wiith that on some versions on lxc (can't remember the exact details, sorry). A bind mount would probably be safer. Finally I'd be grateful to learn of people's experiences with btrfs for snapshotting and managing containers. I personally use it for my laptop backups, but my host server is on a 3.2.0-4-amd64 kernel which is pretty old by btrfs standards. Is there a particular requirement for that version of kernel? In RHEL/Centos/Ubuntu you can often use prebuilt latest vanilla kernel with only minimum change required (although the distro won't offically support it, obviously). If you're stuck with kernel 3.2 then I'd say use zfs. The devs take extra care to make sure it works well on RHEL6 (with its ancient 2.6.32 kernel), and should work on all kernel from that version up to 3.9. -- Fajar -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxcbr0 MAC addr issue
Now my question, could not lxc (at boot) setup a fixed MAC addr for the host port? Yeah, given how bad this was for libvirt/qemu I'm surprised I've not seen this happen in lxc - but I haven't, and noone else has reported it. Since you mention lxcbr0, I assume you're using ubuntu? Until we (somehow - netcf?) implement a distro-agnostic lxc bridge upstream, this is not an upstream issue, but an ubuntu package issue. Have you actually seen a hang as a result of this? If so, then please open a bug at https://bugs.launchpad.net/ubuntu/+source/lxc/+filebug as it should be trivially fixable in /etc/init/lxc-net.conf. -serge -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxcbr0 MAC addr issue
On Wed, 2013-06-05 at 07:40 -0500, Serge Hallyn wrote: Now my question, could not lxc (at boot) setup a fixed MAC addr for the host port? Yeah, given how bad this was for libvirt/qemu I'm surprised I've not seen this happen in lxc - but I haven't, and noone else has reported it. Actually, this has come up on both the -devel list and here, the last time about a year and a half ago: On the -devel list... Subject: Set high byte of mac addresses for host veth devices to 0xfe It had a patch and referenced an open bug associated with it: random veth device MAC addresses cause bridge problems - ID: 3411497 https://sourceforge.net/tracker/index.php?func=detailaid=3411497group_id=163076atid=826303 Christian Seiler included a proposed patch with his original posting to the -devel list back in November of 2011 where he set the high order byte to FE for a private locally administered MAC and some discussion ensued. After a couple of bug fixes, it was Acked it on 01/03/2012. It looks like it was applied. Right around line 3109 of src/lxc/conf.c: static int setup_private_host_hw_addr(char *veth1) ... ifr.ifr_hwaddr.sa_data[0] = 0xfe; err = ioctl(sockfd, SIOCSIFHWADDR, ifr); close(sockfd); The bug is still open. Is this not working? What version did it go into? Someone should probably close that bug with an annotation. Looks to have been fixed in the 0.8.0 release. Seems to be working for me: ip addr ls ... 7: vethNoBHvN: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc pfifo_fast master lxcbr0 state UP qlen 1000 link/ether fe:57:fa:34:4c:35 brd ff:ff:ff:ff:ff:ff inet6 fe80::fc57:faff:fe34:4c35/64 scope link valid_lft forever preferred_lft forever 9: vethFmslRq: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc pfifo_fast master lxcbr0 state UP qlen 1000 link/ether fe:a8:39:d6:c2:ee brd ff:ff:ff:ff:ff:ff inet6 fe80::fca8:39ff:fed6:c2ee/64 scope link valid_lft forever preferred_lft forever 11: vethhw7OZw: BROADCAST,MULTICAST,UP,LOWER_UP mtu 1500 qdisc pfifo_fast master lxcbr0 state UP qlen 1000 link/ether fe:6b:8e:9d:f2:4f brd ff:ff:ff:ff:ff:ff inet6 fe80::fc6b:8eff:fe9d:f24f/64 scope link valid_lft forever preferred_lft forever So this shouldn't be happening if all of the host side MAC addresses are beginning with fe:. What version of the lxc tools is in play here? 0.7.5 dates back to August of 2011 and would have predated that fix. There's still a lot of distros with 0.7.5 still running around out there. It was also discussed here under these threads here in this forum from around the time time frame (about a year and a half ago): Subject: New LXC Creation Script: lxc-ubuntu-x Subject: seeing a network pause when starting and stopping LXCs - how do I stop this ? Yes, I had seen the bug show up. It causes the network to hang, but only for a half a minute or less as the ARP caches settle. If you have frequent starting and stopping of containers, it can be a PITA. My solution was to assign a high vendor field (top 3 bytes) using some safe vendor value that was above my host card vendor value. I seem to recall some discussion a few years ago about also trying to get an official MAC vendor code assigned to this project in one of the earlier discussions. I don't think that went anywhere and I'm having a problem finding it now. I do remember participating in that discussion as well. I seem to recall that it was decided not to do this even though some other projects such as VMware and, maybe, VirtualBox did chose this route. Since you mention lxcbr0, I assume you're using ubuntu? Until we (somehow - netcf?) implement a distro-agnostic lxc bridge upstream, this is not an upstream issue, but an ubuntu package issue. Have you actually seen a hang as a result of this? If so, then please open a bug at https://bugs.launchpad.net/ubuntu/+source/lxc/+filebug as it should be trivially fixable in /etc/init/lxc-net.conf. See above. Opened on 09/19/2011 and still open. You'll find I commented in the bug the same day. There's been other discussions in several forum, including including these. I believe it has also come up on the OpenVZ forum (and is where a lot of my earlier information originated) and has been mentioned in some of the kernel lists, probably the net list. There is a reason it works that way, mostly because the bridge has to have an address and it has to be an address on the bridge so the kernel can know what's on the bridge and what's foreign to the bridge in order to manage local routing. So, it has to pick something. But, if you delete an interface from the bridge and the bridge is using that address, it has to pick another. The rule the followed was to always choose the lowest. The only downside to assigning a static bridge address to the bridge is the chance of that associated interface ever being deleted from the bridge. I think you also loose that static state if certain bridge
Re: [Lxc-users] lxcbr0 MAC addr issue
Quoting Michael H. Warfield (m...@wittsend.com): On Wed, 2013-06-05 at 07:40 -0500, Serge Hallyn wrote: Now my question, could not lxc (at boot) setup a fixed MAC addr for the host port? Yeah, given how bad this was for libvirt/qemu I'm surprised I've not seen this happen in lxc - but I haven't, and noone else has reported it. Actually, this has come up on both the -devel list and here, the last time about a year and a half ago: On the -devel list... Subject: Set high byte of mac addresses for host veth devices to 0xfe It had a patch and referenced an open bug associated with it: random veth device MAC addresses cause bridge problems - ID: 3411497 https://sourceforge.net/tracker/index.php?func=detailaid=3411497group_id=163076atid=826303 Christian Seiler included a proposed patch with his original posting to the -devel list back in November of 2011 where he set the high order byte to FE for a private locally administered MAC and some discussion ensued. After a couple of bug fixes, it was Acked it on 01/03/2012. It looks like it was applied. Right around line 3109 of src/lxc/conf.c: static int setup_private_host_hw_addr(char *veth1) ... ifr.ifr_hwaddr.sa_data[0] = 0xfe; err = ioctl(sockfd, SIOCSIFHWADDR, ifr); close(sockfd); yes and it does this. The point is that lxcbr0 is not tied to any physical nic. So the first container you start, however high the macaddr is, lxcbr0 takes its mac. If the next container gets a lower macaddr, lxcbr0's macaddr drops. Right now I have: lxcbr0Link encap:Ethernet HWaddr fe:02:72:77:79:ff vethtdjU5K Link encap:Ethernet HWaddr fe:02:72:77:79:ff -serge -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxcbr0 MAC addr issue
On Wed, 2013-06-05 at 09:30 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): On Wed, 2013-06-05 at 07:40 -0500, Serge Hallyn wrote: Now my question, could not lxc (at boot) setup a fixed MAC addr for the host port? Yeah, given how bad this was for libvirt/qemu I'm surprised I've not seen this happen in lxc - but I haven't, and noone else has reported it. Actually, this has come up on both the -devel list and here, the last time about a year and a half ago: On the -devel list... Subject: Set high byte of mac addresses for host veth devices to 0xfe It had a patch and referenced an open bug associated with it: random veth device MAC addresses cause bridge problems - ID: 3411497 https://sourceforge.net/tracker/index.php?func=detailaid=3411497group_id=163076atid=826303 Christian Seiler included a proposed patch with his original posting to the -devel list back in November of 2011 where he set the high order byte to FE for a private locally administered MAC and some discussion ensued. After a couple of bug fixes, it was Acked it on 01/03/2012. It looks like it was applied. Right around line 3109 of src/lxc/conf.c: static int setup_private_host_hw_addr(char *veth1) ... ifr.ifr_hwaddr.sa_data[0] = 0xfe; err = ioctl(sockfd, SIOCSIFHWADDR, ifr); close(sockfd); yes and it does this. The point is that lxcbr0 is not tied to any physical nic. So the first container you start, however high the macaddr is, lxcbr0 takes its mac. If the next container gets a lower macaddr, lxcbr0's macaddr drops. Right now I have: lxcbr0Link encap:Ethernet HWaddr fe:02:72:77:79:ff vethtdjU5K Link encap:Ethernet HWaddr fe:02:72:77:79:ff Oh, yeah... I'm always using bridged mode with the host ethernet address on the bridge, so that's never a problem. You're right. The NAT mode is problematical there because it's not anchored to a physical interface. I think someone, at one point in those earlier threads, was suggesting that they were using a dummy interface or a dummy container as a place holder to lock the interface address for that reason. I just happen to have enough IPv4 addresses, personally, that I bridge everything and never really need to NAT anything where I can avoid it. :-P Of course, in a server environment, where you are hosting a farm of virtual servers, you would almost always want global public addresses on the servers, I would imagine. You're still going to have the problem that, if you shut down the container that the bridge is using for the address, the address is going to shift, static or not. The address of the bridge must be an address of an interface on the bridge or you are going to have routing problems. That was made clear in some discussion on some of the kernel mailing lists. How do you deal with that then? Do you designate a container that must never be shut down or the bridge hangs? You could load the dummy module and bridge a dummy interface to the bridge. That would guarantee a MAC address lower than the fe: private addresses of the bridge and would be cheaper than a dummy container. I've done that before a long time ago. (We should still close that old bug, however.) -serge -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
[Lxc-users] lxcbr0 MAC addr issue
yes and it does this. The point is that lxcbr0 is not tied to any physical nic. So the first container you start, however high the macaddr is, lxcbr0 takes its mac. If the next container gets a lower macaddr, lxcbr0's macaddr drops. This lxcbr0 is special to Ubuntu, right? And if not to a physical NIC, to what is this bridge connected to on the host? -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxcbr0 MAC addr issue
On Wed, 2013-06-05 at 15:17 +, Jäkel, Guido wrote: yes and it does this. The point is that lxcbr0 is not tied to any physical nic. So the first container you start, however high the macaddr is, lxcbr0 takes its mac. If the next container gets a lower macaddr, lxcbr0's macaddr drops. This lxcbr0 is special to Ubuntu, right? And if not to a physical NIC, to what is this bridge connected to on the host? Not to the best of my knowledge. It should be a simple bridge. What do you get for this command? brctl show A bridge doesn -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxcbr0 MAC addr issue
Crap... Bumped the keyboard and this one got away from me prematurely. On Wed, 2013-06-05 at 11:23 -0400, Michael H. Warfield wrote: On Wed, 2013-06-05 at 15:17 +, Jäkel, Guido wrote: yes and it does this. The point is that lxcbr0 is not tied to any physical nic. So the first container you start, however high the macaddr is, lxcbr0 takes its mac. If the next container gets a lower macaddr, lxcbr0's macaddr drops. This lxcbr0 is special to Ubuntu, right? And if not to a physical NIC, to what is this bridge connected to on the host? Not to the best of my knowledge. It should be a simple bridge. What do you get for this command? brctl show A bridge doesn A bridge doesn't have to be attach to a device. A bridge is its own logical entity in the kernel to which you may attach devices. You can not attach a bridge to something else. You can only attach something else to the bridge. There's a difference. In the case of a NATing configuration, you set up a bridge (name it whatever you want) and attach the containers to it. Then you use the NAT modules to route between the bridge and the external interface while NATing the addresses. I use lxcbr0 on my Fedora hosts. It's just a bridge. I could see where Ubuntu might have some preconfigured setups for this purpose where I have to set them up by hand in Fedora. That's just a matter of the distro specific support scripts. Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxcbr0 MAC addr issue
On Wed, 2013-06-05 at 06:23 +, Hans Feldt wrote: It is a fact that the bridge takes the lowest MAC address from the attached ports for the host port. See for example http://backreference.org/2010/07/28/linux-bridge-mac-addresses-and-dynamic-ports/ Thus if a container is restarted, the host port can potentially change its MAC address and containers will have a stale ARP cache. This of course causes problem for communication container-host. Tested the workaround mentioned in the link but then I got problem with network manager on a later Ubuntu version. Then I tried using a dummy container and reusing its MAC addr for the host port. Works but... Now my question, could not lxc (at boot) setup a fixed MAC addr for the host port? There's a gotcha in there. You can not set an arbitrary MAC address on a bridge. It can only be the MAC address of an attached interface. It has to do with how packets are routed down in the kernel and determining if a packet is to be handled locally on the bridge or not. It also may have some ties in to the spanning tree algorithm protocol logic (whether you are using STP or have it enabled or not). If you set it to a fixed MAC address of a container, you can't stop or reboot that container without losing that static assignment on the bridge. A dummy container is one option, if you don't have a host hardware interface connected to the bridge. But you need one with a MAC address lower than any of the others. Another alternative is to use a dummy interface... modprobe dummy brctl addif lxcbr0 dummy0 The dummy0 interface doesn't even need to be up or have an IP address assigned to it. Since the container host-local addresses will all be private (fe:...) and the dummy0 interface will have something lower, you should be good to go. Thanks, Hans Regards, Mike -- Michael H. Warfield (AI4NB) | (770) 985-6132 | m...@wittsend.com /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/ NIC whois: MHW9 | An optimist believes we live in the best of all PGP Key: 0x674627FF| possible worlds. A pessimist is sure of it! signature.asc Description: This is a digitally signed message part -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxcbr0 MAC addr issue
Quoting Michael H. Warfield (m...@wittsend.com): Crap... Bumped the keyboard and this one got away from me prematurely. On Wed, 2013-06-05 at 11:23 -0400, Michael H. Warfield wrote: On Wed, 2013-06-05 at 15:17 +, Jäkel, Guido wrote: yes and it does this. The point is that lxcbr0 is not tied to any physical nic. So the first container you start, however high the macaddr is, lxcbr0 takes its mac. If the next container gets a lower macaddr, lxcbr0's macaddr drops. This lxcbr0 is special to Ubuntu, right? And if not to a physical NIC, to what is this bridge connected to on the host? Not to the best of my knowledge. It should be a simple bridge. What do you get for this command? brctl show A bridge doesn A bridge doesn't have to be attach to a device. A bridge is its own logical entity in the kernel to which you may attach devices. You can not attach a bridge to something else. You can only attach something else to the bridge. There's a difference. In the case of a NATing configuration, you set up a bridge (name it whatever you want) and attach the containers to it. Then you use the NAT modules to route between the bridge and the external interface while NATing the addresses. I use lxcbr0 on my Fedora hosts. It's just a bridge. I could see where Ubuntu might have some preconfigured setups for this purpose where I have to set them up by hand in Fedora. That's just a matter of the distro specific support scripts. Right. And we *could* attach a dummy device with mac starting with something lower. BUT I just did some testing, and even as I watch lxcbr0's addr go down when starting a new container, my ssh to the container which had the higher macaddr doesn't hiccough. Perhaps it'll be a problem when connected from an outside host (through port forwarding). In that case I'll happily do the dummy if hack. But it seems possible that it just isn't needed. (And since the iptables rule is --to-destination an ip address, I'm thinking it won't be) -serge -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
[Lxc-users] [PATCH] lxclock and lxccontainer: switch from flock to fcntl
flock is not supported on nfs. fcntl is at least supported on newer
(v3 and above) nfs.
Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com
---
src/lxc/lxccontainer.c | 22 +-
src/lxc/lxclock.c | 16 ++--
2 files changed, 31 insertions(+), 7 deletions(-)
diff --git a/src/lxc/lxccontainer.c b/src/lxc/lxccontainer.c
index 2edf749..cf5252b 100644
--- a/src/lxc/lxccontainer.c
+++ b/src/lxc/lxccontainer.c
@@ -23,6 +23,7 @@
#include sys/types.h
#include sys/wait.h
#include errno.h
+#include fcntl.h
#include sched.h
#include config.h
#include lxc.h
@@ -39,7 +40,6 @@
#include lxc/utils.h
#include lxc/monitor.h
#include sched.h
-#include fcntl.h
#include arpa/inet.h
#include ifaddrs.h
@@ -66,6 +66,8 @@ int ongoing_create(struct lxc_container *c)
int len = strlen(c-config_path) + strlen(c-name) + 10;
char *path = alloca(len);
int fd, ret;
+ struct flock lk;
+
ret = snprintf(path, len, %s/%s/partial, c-config_path, c-name);
if (ret 0 || ret = len) {
ERROR(Error writing partial pathname);
@@ -82,8 +84,12 @@ int ongoing_create(struct lxc_container *c)
process_unlock();
return 0;
}
- if ((ret = flock(fd, LOCK_EX | LOCK_NB)) == -1
- errno == EWOULDBLOCK) {
+ lk.l_type = F_WRLCK;
+ lk.l_whence = SEEK_SET;
+ lk.l_start = 0;
+ lk.l_len = 0;
+ lk.l_pid = -1;
+ if (fcntl(fd, F_GETLK, lk) == 0 lk.l_pid != -1) {
// create is still ongoing
close(fd);
process_unlock();
@@ -101,6 +107,8 @@ int create_partial(struct lxc_container *c)
int len = strlen(c-config_path) + strlen(c-name) + 10;
char *path = alloca(len);
int fd, ret;
+ struct flock lk;
+
ret = snprintf(path, len, %s/%s/partial, c-config_path, c-name);
if (ret 0 || ret = len) {
ERROR(Error writing partial pathname);
@@ -108,12 +116,16 @@ int create_partial(struct lxc_container *c)
}
if (process_lock())
return -1;
- if ((fd=open(path, O_CREAT | O_EXCL, 0755)) 0) {
+ if ((fd=open(path, O_RDWR | O_CREAT | O_EXCL, 0755)) 0) {
SYSERROR(Erorr creating partial file);
process_unlock();
return -1;
}
- if (flock(fd, LOCK_EX) 0) {
+ lk.l_type = F_WRLCK;
+ lk.l_whence = SEEK_SET;
+ lk.l_start = 0;
+ lk.l_len = 0;
+ if (fcntl(fd, F_SETLKW, lk) 0) {
SYSERROR(Error locking partial file %s, path);
close(fd);
process_unlock();
diff --git a/src/lxc/lxclock.c b/src/lxc/lxclock.c
index 4bbe873..d004cc5 100644
--- a/src/lxc/lxclock.c
+++ b/src/lxc/lxclock.c
@@ -23,6 +23,7 @@
#include stdio.h
#include errno.h
#include unistd.h
+#include fcntl.h
#include lxc/utils.h
#include lxc/log.h
#include lxc/lxccontainer.h
@@ -111,6 +112,7 @@ out:
int lxclock(struct lxc_lock *l, int timeout)
{
int ret = -1, saved_errno = errno;
+ struct flock lk;
switch(l-type) {
case LXC_LOCK_ANON_SEM:
@@ -152,7 +154,11 @@ int lxclock(struct lxc_lock *l, int timeout)
goto out;
}
}
- ret = flock(l-u.f.fd, LOCK_EX);
+ lk.l_type = F_WRLCK;
+ lk.l_whence = SEEK_SET;
+ lk.l_start = 0;
+ lk.l_len = 0;
+ ret = fcntl(l-u.f.fd, F_SETLKW, lk);
process_unlock();
if (ret == -1)
saved_errno = errno;
@@ -167,6 +173,7 @@ out:
int lxcunlock(struct lxc_lock *l)
{
int ret = 0, saved_errno = errno;
+ struct flock lk;
switch(l-type) {
case LXC_LOCK_ANON_SEM:
@@ -179,7 +186,12 @@ int lxcunlock(struct lxc_lock *l)
case LXC_LOCK_FLOCK:
process_lock();
if (l-u.f.fd != -1) {
- if ((ret = flock(l-u.f.fd, LOCK_UN)) 0)
+ lk.l_type = F_UNLCK;
+ lk.l_whence = SEEK_SET;
+ lk.l_start = 0;
+ lk.l_len = 0;
+ ret = fcntl(l-u.f.fd, F_SETLK, lk);
+ if (ret 0)
saved_errno = errno;
close(l-u.f.fd);
l-u.f.fd = -1;
--
1.7.9.5
--
How ServiceNow helps IT people transform IT departments:
1. A cloud service to automate IT design, transition and operations
2. Dashboards that offer high-level views of enterprise services
3. A single system of record for all IT processes
http://p.sf.net/sfu/servicenow-d2d-j
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
Re: [Lxc-users] lxcbr0 MAC addr issue
On Wed, 2013-06-05 at 11:26 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): Crap... Bumped the keyboard and this one got away from me prematurely. On Wed, 2013-06-05 at 11:23 -0400, Michael H. Warfield wrote: On Wed, 2013-06-05 at 15:17 +, Jäkel, Guido wrote: yes and it does this. The point is that lxcbr0 is not tied to any physical nic. So the first container you start, however high the macaddr is, lxcbr0 takes its mac. If the next container gets a lower macaddr, lxcbr0's macaddr drops. This lxcbr0 is special to Ubuntu, right? And if not to a physical NIC, to what is this bridge connected to on the host? Not to the best of my knowledge. It should be a simple bridge. What do you get for this command? brctl show A bridge doesn A bridge doesn't have to be attach to a device. A bridge is its own logical entity in the kernel to which you may attach devices. You can not attach a bridge to something else. You can only attach something else to the bridge. There's a difference. In the case of a NATing configuration, you set up a bridge (name it whatever you want) and attach the containers to it. Then you use the NAT modules to route between the bridge and the external interface while NATing the addresses. I use lxcbr0 on my Fedora hosts. It's just a bridge. I could see where Ubuntu might have some preconfigured setups for this purpose where I have to set them up by hand in Fedora. That's just a matter of the distro specific support scripts. Right. And we *could* attach a dummy device with mac starting with something lower. BUT I just did some testing, and even as I watch lxcbr0's addr go down when starting a new container, my ssh to the container which had the higher macaddr doesn't hiccough. Hmmm... It would be interesting to see what you get from arp -a on the host and the container before and after that. It would also be interesting to see what happens if you ssh to a container with the higher address first and then bring up a container with a lower mac address and see if it impacts the existing connection. It really all depends on how the host is managing the arp table and, if the MAC address changes, how does the bridge change impact the arp table. In the case of ssh'ing to a container from the host, the host would still have the correct arp entry for the container which would facilitate the delivery of the initial SYN. The container would have an incorrect entry but that should correct itself as soon as that new packet arrives connecting from the host, invalidating the containers arp table entry, I would think. It should also correct the MAC table entries for the bridge (used by the STP) if it originates from the interface that changed. It works the same way over an eitherswitch externally. The moment a packet is sent with a new from MAC address, the switch (bridge) remembers what port (attachment) that MAC address was last seen on and enters it into it's mac switch table. What would be interesting to know is how the container sees it, since his arp table entry for the host is then wrong if he initiates a packet first, after the change. It could be that the tap/bridge connection from container to host bridge would clear that up quickly. Perhaps it'll be a problem when connected from an outside host (through port forwarding). In that case I'll happily do the dummy if hack. But it seems possible that it just isn't needed. (And since the iptables rule is --to-destination an ip address, I'm thinking it won't be) Yeah, since the host exposed IP and MAC are not changing, it shouldn't be, in the external case with NAT as you describe. The external systems will only be referencing the external IP address for which the MAC doesn't change and the ARP tables are perfectly coherent, externally. So NAT w/ port forwarding to a container should be covered. It's where the MAC address of the host interface changes (because it's attached to a bridge) where you get into trouble in the external case. But, we have that case covered now with the fixes that are in-place since 0.8.0 (assuming the OP is on a recent enough version of the lxc tools). So both of those should be covered. That really just leaves the host - container case (container - container should be a trivial non-issue). Then I think it's an issue of which side of the bridge is issuing the first packet after the MAC address changes and what does the bridge logic do about it. Seems like a corner case to me. I also don't know what role STP has in this game. On my fedora system, my lxcbr0 bridge has STP disabled but the virbr0 bridge (used by libvirt and setup by default) has it enabled and I have no idea why. It's possible that these symptoms could vary depending on that setting. I've definitely heard it mentioned before. Oh, and this is likely to get really ugly with IPv6. With the host
Re: [Lxc-users] [PATCH] lxclock and lxccontainer: switch from flock to fcntl
Hi Serge,
On Wed, Jun 5, 2013 at 1:56 PM, Serge Hallyn serge.hal...@ubuntu.com wrote:
flock is not supported on nfs. fcntl is at least supported on newer
(v3 and above) nfs.
Applied on top of lxc-0.9.0.0~staging~20130605-1508 and it is now
working ok here.
Thank you Serge!
Best Regards,
Norberto
Signed-off-by: Serge Hallyn serge.hal...@ubuntu.com
---
src/lxc/lxccontainer.c | 22 +-
src/lxc/lxclock.c | 16 ++--
2 files changed, 31 insertions(+), 7 deletions(-)
diff --git a/src/lxc/lxccontainer.c b/src/lxc/lxccontainer.c
index 2edf749..cf5252b 100644
--- a/src/lxc/lxccontainer.c
+++ b/src/lxc/lxccontainer.c
@@ -23,6 +23,7 @@
#include sys/types.h
#include sys/wait.h
#include errno.h
+#include fcntl.h
#include sched.h
#include config.h
#include lxc.h
@@ -39,7 +40,6 @@
#include lxc/utils.h
#include lxc/monitor.h
#include sched.h
-#include fcntl.h
#include arpa/inet.h
#include ifaddrs.h
@@ -66,6 +66,8 @@ int ongoing_create(struct lxc_container *c)
int len = strlen(c-config_path) + strlen(c-name) + 10;
char *path = alloca(len);
int fd, ret;
+ struct flock lk;
+
ret = snprintf(path, len, %s/%s/partial, c-config_path, c-name);
if (ret 0 || ret = len) {
ERROR(Error writing partial pathname);
@@ -82,8 +84,12 @@ int ongoing_create(struct lxc_container *c)
process_unlock();
return 0;
}
- if ((ret = flock(fd, LOCK_EX | LOCK_NB)) == -1
- errno == EWOULDBLOCK) {
+ lk.l_type = F_WRLCK;
+ lk.l_whence = SEEK_SET;
+ lk.l_start = 0;
+ lk.l_len = 0;
+ lk.l_pid = -1;
+ if (fcntl(fd, F_GETLK, lk) == 0 lk.l_pid != -1) {
// create is still ongoing
close(fd);
process_unlock();
@@ -101,6 +107,8 @@ int create_partial(struct lxc_container *c)
int len = strlen(c-config_path) + strlen(c-name) + 10;
char *path = alloca(len);
int fd, ret;
+ struct flock lk;
+
ret = snprintf(path, len, %s/%s/partial, c-config_path, c-name);
if (ret 0 || ret = len) {
ERROR(Error writing partial pathname);
@@ -108,12 +116,16 @@ int create_partial(struct lxc_container *c)
}
if (process_lock())
return -1;
- if ((fd=open(path, O_CREAT | O_EXCL, 0755)) 0) {
+ if ((fd=open(path, O_RDWR | O_CREAT | O_EXCL, 0755)) 0) {
SYSERROR(Erorr creating partial file);
process_unlock();
return -1;
}
- if (flock(fd, LOCK_EX) 0) {
+ lk.l_type = F_WRLCK;
+ lk.l_whence = SEEK_SET;
+ lk.l_start = 0;
+ lk.l_len = 0;
+ if (fcntl(fd, F_SETLKW, lk) 0) {
SYSERROR(Error locking partial file %s, path);
close(fd);
process_unlock();
diff --git a/src/lxc/lxclock.c b/src/lxc/lxclock.c
index 4bbe873..d004cc5 100644
--- a/src/lxc/lxclock.c
+++ b/src/lxc/lxclock.c
@@ -23,6 +23,7 @@
#include stdio.h
#include errno.h
#include unistd.h
+#include fcntl.h
#include lxc/utils.h
#include lxc/log.h
#include lxc/lxccontainer.h
@@ -111,6 +112,7 @@ out:
int lxclock(struct lxc_lock *l, int timeout)
{
int ret = -1, saved_errno = errno;
+ struct flock lk;
switch(l-type) {
case LXC_LOCK_ANON_SEM:
@@ -152,7 +154,11 @@ int lxclock(struct lxc_lock *l, int timeout)
goto out;
}
}
- ret = flock(l-u.f.fd, LOCK_EX);
+ lk.l_type = F_WRLCK;
+ lk.l_whence = SEEK_SET;
+ lk.l_start = 0;
+ lk.l_len = 0;
+ ret = fcntl(l-u.f.fd, F_SETLKW, lk);
process_unlock();
if (ret == -1)
saved_errno = errno;
@@ -167,6 +173,7 @@ out:
int lxcunlock(struct lxc_lock *l)
{
int ret = 0, saved_errno = errno;
+ struct flock lk;
switch(l-type) {
case LXC_LOCK_ANON_SEM:
@@ -179,7 +186,12 @@ int lxcunlock(struct lxc_lock *l)
case LXC_LOCK_FLOCK:
process_lock();
if (l-u.f.fd != -1) {
- if ((ret = flock(l-u.f.fd, LOCK_UN)) 0)
+ lk.l_type = F_UNLCK;
+ lk.l_whence = SEEK_SET;
+ lk.l_start = 0;
+ lk.l_len = 0;
+ ret = fcntl(l-u.f.fd, F_SETLK, lk);
+ if (ret 0)
saved_errno = errno;
close(l-u.f.fd);
l-u.f.fd = -1;
--
1.7.9.5
--
How ServiceNow helps IT people transform
Re: [Lxc-users] lxc containers as backup 'replicas'
On 05/06/13, Serge Hallyn (serge.hal...@ubuntu.com) wrote: Quoting Rory Campbell-Lange (r...@campbell-lange.net): On another point I'd also like to know of the recommended way of using another mount point for lxc containers and the dpkg cache. For example, I wish to hold my containers in /dev/sdb/ mounted on /containers. Should I symlink /var/lib/lxc/ to this mount point? If you're on a new enough lxc (i.e. 0.9.0) I'd recommend using lxcpath. You can set 'lxcpath = /srv/lxc' in /etc/lxc/lxc.conf, then all containers will be created and run from /srv/lxc instead of /var/lib/lxc. Or you can just add '-P /srv/lxc' to all lxc-* commands. I'm on Debian Wheezy which has 0.8.0~rc1-8+deb7u1. Looks like I should use the -P flag. Finally I'd be grateful to learn of people's experiences with btrfs for snapshotting and managing containers. I personally use it for my laptop backups, but my host server is on a 3.2.0-4-amd64 kernel which is pretty old by btrfs standards. yeah I don't know that I'd trust it under 3.2. I think 3.5 is where it stopped losing data for me. But best to run some tests. When it failed me, it generally did so after one or two subvolume commands. Cheers for those notes. Rory -- Rory Campbell-Lange r...@campbell-lange.net -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc containers as backup 'replicas'
On 05/06/13, Fajar A. Nugraha (l...@fajar.net) wrote: On Wed, Jun 5, 2013 at 6:50 PM, Rory Campbell-Lange r...@campbell-lange.net wrote: I'd be grateful to know if it is possible to sync 1. and 3. into the container when it is not running. In other words, to simply update the config files in /var/lib/lxc/container/rootfs/etc, for example? ... However, personally I'd just forget for a moment that the backup will be run on lxc and do the same things I'd do on a normal machine. In my case, I'd use zfs snapshot and send|receive (yes, you can use zfs for root). In your case it'd probably be rsync or whatever you're happy with. Are there any files that shouldn't percolate between a normal running server's /etc/ and one in an lxc container? On another point I'd also like to know of the recommended way of using another mount point for lxc containers and the dpkg cache. For example, I wish to hold my containers in /dev/sdb/ mounted on /containers. Should I symlink /var/lib/lxc/ to this mount point? I'm pretty sure there were problems wiith that on some versions on lxc (can't remember the exact details, sorry). A bind mount would probably be safer. do you mean the exivalent of 'mount /dev/sdb1 /var/lib/lxc/' ? Finally I'd be grateful to learn of people's experiences with btrfs for snapshotting and managing containers. I personally use it for my laptop backups, but my host server is on a 3.2.0-4-amd64 kernel which is pretty old by btrfs standards. Is there a particular requirement for that version of kernel? In RHEL/Centos/Ubuntu you can often use prebuilt latest vanilla kernel with only minimum change required (although the distro won't offically support it, obviously). If you're stuck with kernel 3.2 then I'd say use zfs. The devs take extra care to make sure it works well on RHEL6 (with its ancient 2.6.32 kernel), and should work on all kernel from that version up to 3.9. I'm on Debian stable and I like being there for production machines (even though this is a backup machine). I'm not sure about the availability of a 3.8+ kernel on Debian. I'm tempted by zfs but worried about its likely cohabitation -- licence-wise -- over time with the kernel. Thanks for your comments Rory -- Rory Campbell-Lange r...@campbell-lange.net -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxcbr0 MAC addr issue
Quoting Michael H. Warfield (m...@wittsend.com): On Wed, 2013-06-05 at 11:26 -0500, Serge Hallyn wrote: Quoting Michael H. Warfield (m...@wittsend.com): Crap... Bumped the keyboard and this one got away from me prematurely. On Wed, 2013-06-05 at 11:23 -0400, Michael H. Warfield wrote: On Wed, 2013-06-05 at 15:17 +, Jäkel, Guido wrote: yes and it does this. The point is that lxcbr0 is not tied to any physical nic. So the first container you start, however high the macaddr is, lxcbr0 takes its mac. If the next container gets a lower macaddr, lxcbr0's macaddr drops. This lxcbr0 is special to Ubuntu, right? And if not to a physical NIC, to what is this bridge connected to on the host? Not to the best of my knowledge. It should be a simple bridge. What do you get for this command? brctl show A bridge doesn A bridge doesn't have to be attach to a device. A bridge is its own logical entity in the kernel to which you may attach devices. You can not attach a bridge to something else. You can only attach something else to the bridge. There's a difference. In the case of a NATing configuration, you set up a bridge (name it whatever you want) and attach the containers to it. Then you use the NAT modules to route between the bridge and the external interface while NATing the addresses. I use lxcbr0 on my Fedora hosts. It's just a bridge. I could see where Ubuntu might have some preconfigured setups for this purpose where I have to set them up by hand in Fedora. That's just a matter of the distro specific support scripts. Right. And we *could* attach a dummy device with mac starting with something lower. BUT I just did some testing, and even as I watch lxcbr0's addr go down when starting a new container, my ssh to the container which had the higher macaddr doesn't hiccough. Hmmm... It would be interesting to see what you get from arp -a on Uh, well - with one container started, I get ? (10.0.3.182) at 00:16:3e:1a:33:6c [ether] on lxcbr0 static.65.5.9.176.clients.your-server.de (176.9.5.65) at 00:26:88:76:1c:05 [ether] on eth0 Note that lxcbr0 and the container's veth are HWaddr fe:ab:34:b2:2f:cc. So I guess the reason it doesn't matter is that lxcbr0 is not an endpoint for anything. The endpoint is always the other veth endpoint. the host and the container before and after that. It would also be interesting to see what happens if you ssh to a container with the higher address first and then bring up a container with a lower mac address and see if it impacts the existing connection. (I did both.) -serge -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] Routing issues
Quoting Rory Campbell-Lange (r...@campbell-lange.net): On 04/06/13, Michael H. Warfield (m...@wittsend.com) wrote: I'd be grateful to know if anyone has some firewall (iptables) advice for allowing traffic to the container? I expect to run another firewall on the container itself. That's probably your FORWARD chain there. Set that policy to ACCEPT and flush all the rules from the FORWARD chain like this: iptables -P FORWARD ACCEPT iptables -F FORWARD FORWARD chain is going to affect packets forwarded over the host's bridge to the containers. The INPUT and OUTPUT chains will affect the packets coming in and going out from the local host's OS interfaces. Depending on your distro, track down your persistent rule storage and make those changes permanent. Fedora prior to firewalld (here we go again), RedHat, and RH derivatives (CentOS et al) are generally in /etc/sysconfig/iptables unless you've also installed one of the sundry firewall toolkits. Ubuntu, I'm not so sure about. I'm using Debian, and I'm using a simple ufw firewall on the host server at present. The iptables -L output is here: http://pastebin.com/QzQKRDX0 I don't have any trouble with the firewall restarting. Thanks very much Rory I think 'ufw status' information will probably be more helpful. -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
Re: [Lxc-users] lxc containers as backup 'replicas'
On Thu, Jun 6, 2013 at 2:57 AM, Rory Campbell-Lange r...@campbell-lange.net wrote: In my case, I'd use zfs snapshot and send|receive (yes, you can use zfs for root). In your case it'd probably be rsync or whatever you're happy with. Are there any files that shouldn't percolate between a normal running server's /etc/ and one in an lxc container? Newer versions of Ubuntu has pretty tight lxc integration, in that if you use an existing root on lxc, it would mostly just work. The exceptions are mostly stuff that you'd need to take care of even on normal servers, like networking config. On another point I'd also like to know of the recommended way of using another mount point for lxc containers and the dpkg cache. For example, I wish to hold my containers in /dev/sdb/ mounted on /containers. Should I symlink /var/lib/lxc/ to this mount point? I'm pretty sure there were problems wiith that on some versions on lxc (can't remember the exact details, sorry). A bind mount would probably be safer. do you mean the exivalent of 'mount /dev/sdb1 /var/lib/lxc/' ? mount --bind /containers /var/lib/lxc Finally I'd be grateful to learn of people's experiences with btrfs for snapshotting and managing containers. I personally use it for my laptop backups, but my host server is on a 3.2.0-4-amd64 kernel which is pretty old by btrfs standards. Is there a particular requirement for that version of kernel? In RHEL/Centos/Ubuntu you can often use prebuilt latest vanilla kernel with only minimum change required (although the distro won't offically support it, obviously). If you're stuck with kernel 3.2 then I'd say use zfs. The devs take extra care to make sure it works well on RHEL6 (with its ancient 2.6.32 kernel), and should work on all kernel from that version up to 3.9. I'm on Debian stable and I like being there for production machines (even though this is a backup machine). I'm not sure about the availability of a 3.8+ kernel on Debian. One way to test is to use latest kernel from Debian testing, which should be 3.9. That is, if you're REALLY set on using btrfs I'm tempted by zfs but worried about its likely cohabitation -- licence-wise -- over time with the kernel. More details about this should be on zfsonlinux's list, but suffice to say that ZoL is distributed separately from the kernel (similar to nvidia's kernel module), and that is unlikely to change. Looking at its current devs and contributors, personally I'm pretty sure ZoL will still be around for quite a while. -- Fajar -- How ServiceNow helps IT people transform IT departments: 1. A cloud service to automate IT design, transition and operations 2. Dashboards that offer high-level views of enterprise services 3. A single system of record for all IT processes http://p.sf.net/sfu/servicenow-d2d-j ___ Lxc-users mailing list Lxc-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/lxc-users
