Re: [Lxc-users] Converting existing CentOS 6.x to container within Ubuntu 12.04 - can that be simple?

2012-11-01 Thread Whit Blauvelt
\On Sun, Oct 28, 2012 at 04:50:07PM +0700, Fajar A. Nugraha wrote:

 Did your search brought you to
 http://wiki.1tux.org/wiki/Lxc/Installation/Guest/Centos/6 ? :D

Did not, and that's a very nice recipe. 

My current question is if there's an available bridging scheme that will
work in my context. The host is an ESXi VMware VM (currently CentOS 6, but
could be Ubuntu 12.04 if helpful). The CentOS 6 guest on that host needs to
end up with a unique IP on the VMware LAN. VMware does not work unless it
can assign the host's IP by dhcp, and at least so far in my experiments will
not do that if I set the host to use bro0 rather than eth0.

Once the host is up, I can add additional LAN IPs to eth0 without problem.
What's not clear is which, if any, of the bridging schemes for the LXC guest
on the host can enable that guest to take its own IP on the LAN.

Why am I trying such a silly trick? Because I have some perfectly good KVM
VMs, but no tool that can convert them to VDOs to put on VMware - the
existing tools satisfy the common demand, which sanely is to get VMs off of
VMware and onto KVM, not the other way around. But my client is committed to
a cloud provider with an ancient, creaky VMware beneath a crippled user
interface.

So _if_ I can take the LXC guest creation recipe above - which is even
easier to follow that it looks at first glance - and then manage the right
bridging trick with it, this will be far more efficient than configuring
VMware VMs from scratch to duplicate the existing, highly-configured KVM
VMs. It could even enable combining some of the less stressed KVM VMs onto
single LXC hosts on VMware, to cut back a bit on the cloud fees.

But ... can it be done? Looking at this page,
http://wiki.1tux.org/wiki/Ubuntu/Bridge, it's not clear if it can be. In
KVM, I always just set up a real bridge on the host - the thing it seems I
can't do in this VMware setting. In all cases, the LXC guests need to end up
with a LAN IP on which they can be addressed from other systems, but not
necessarily the LXC host.

Thanks,
Whit


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] Converting existing CentOS 6.x to container within Ubuntu 12.04 - can that be simple?

2012-11-01 Thread Fajar A. Nugraha
On Thu, Nov 1, 2012 at 8:47 PM, Whit Blauvelt w...@transpect.com wrote:

 The host is an ESXi VMware VM (currently CentOS 6, but
 could be Ubuntu 12.04 if helpful).

Ubuntu will definitely be easier. It has new-enough lxc version, plus
you won't have selinux fiasco (search the list archive for details).

 The CentOS 6 guest on that host needs to
 end up with a unique IP on the VMware LAN.

Sure. Easy.

 But ... can it be done? Looking at this page,
 http://wiki.1tux.org/wiki/Ubuntu/Bridge, it's not clear if it can be.

Sure you can.

http://wiki.1tux.org/wiki/Ubuntu/Bridge#Bridging_a_real_network_interface

It that setup, both the host (the example assumes ubuntu host) and the
containers which uses that bridge will be on the same LAN, getting IP
address from the same DHCP server (whatever that is) on that network.

-- 
Fajar

--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] Converting existing CentOS 6.x to container within Ubuntu 12.04 - can that be simple?

2012-11-01 Thread Whit Blauvelt
Fajar,

Thanks for the quick response. I've gotten a bit farther with VMware. It
will allow br0 to be the interface on its guest - it just can't assign that
by dhcp. But when I get the invocation right for a static assignment, it
takes. It had been seeming that br0 for the host interface just wasn't an
option under VMware.

Is a real bridge the only way to get an unique IP on the same LAN for the
LXC guest, or would some combination of assigning a second IP to the LXC
host, and then DNAT'ing ans SNAT'ing to it, work reasonably well? I just ask
in case at some point the cloud provider sees that we're violating their TOS
with the static IP assignment. Most probably they won't.

Whit


--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] Converting existing CentOS 6.x to container within Ubuntu 12.04 - can that be simple?

2012-11-01 Thread Fajar A. Nugraha
On Thu, Nov 1, 2012 at 9:37 PM, Whit Blauvelt w...@transpect.com wrote:
 Fajar,

 Thanks for the quick response. I've gotten a bit farther with VMware. It
 will allow br0 to be the interface on its guest - it just can't assign that
 by dhcp. But when I get the invocation right for a static assignment, it
 takes. It had been seeming that br0 for the host interface just wasn't an
 option under VMware.

It shouldn't matter.

Vmware shouldn't care what the interface name is under the guest OS.
Or what you do with it (bridge or not).

If you have an Ubuntu host, with a wired ethernet interface (e.g.
eth0), then you can create a bridge on top of that. It doesn't matter
whether the ubuntu host is actually native (on real server), a KVM
guest, or a vmware guest.

... of course there are probably some settings (even on real server +
switch) that limits only ONE MAC address on a single port. In this
case you can't use bridge. Again, it doesn't matter whether it's
vmware or not. It's a matter whether the switch (real or virtual,
doesn't matter) allows more than one MAC on a single port.


 Is a real bridge the only way to get an unique IP on the same LAN for the
 LXC guest,

That's the easy way.

 or would some combination of assigning a second IP to the LXC
 host, and then DNAT'ing ans SNAT'ing to it, work reasonably well?

That's the hard way. If you use this one you'd probably want to use a
host-only bridge (e.g.
http://wiki.1tux.org/wiki/Ubuntu/Bridge#Bridge_with_IP_address) and
setup NAT rules manually.

-- 
Fajar

--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] [lxc-devel] [GIT] lxc branch, master, updated. 7f99e339363d9f005c9386f60a1d8c0953c85053

2012-11-01 Thread Michael H. Warfield
On Thu, 2012-11-01 at 21:20 +0100, Daniel Baumann wrote:
 On 11/01/2012 09:08 PM, Michael H. Warfield wrote:
  I know, I KNOW this is an 11th hour request.  Can we please get Serge's
  autodev stuff into this release?  Please?

 release early, release often?

 just release current git as 0.8.0 now, and the one with the autofoo as 
 0.8.1 soon after that?

That would be ideal but we've been sitting at 0.8.0rc2 for something
like 3-1/2 months now.  I know Daniel (the other Daniel, the Daniel) has
been incredibly busy.  I have no objection to getting this out the door
as 0.8.0 with a fast bump to 0.8.1 for the systemd stuff, but another
several months is not good.  Can we get this fast bump?  We'll be
staring Fedora 18 in the face by then.  The working versions of Fedora
are no longer in support and we've got more distros adopting systemd.

 -- 
 Address:Daniel Baumann, Donnerbuehlweg 3, CH-3012 Bern
 Email:  daniel.baum...@progress-technologies.net
 Internet:   http://people.progress-technologies.net/~daniel.baumann/

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  m...@wittsend.com
   /\/\|=mhw=|\/\/  | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9  | An optimist believes we live in the best of all
 PGP Key: 0x674627FF| possible worlds.  A pessimist is sure of it!


signature.asc
Description: This is a digitally signed message part
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_sfd2d_oct___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] [lxc-devel] [GIT] lxc branch, master, updated. 7f99e339363d9f005c9386f60a1d8c0953c85053

2012-11-01 Thread Serge Hallyn
Quoting Michael H. Warfield (m...@wittsend.com):
 On Thu, 2012-11-01 at 21:20 +0100, Daniel Baumann wrote:
  On 11/01/2012 09:08 PM, Michael H. Warfield wrote:
   I know, I KNOW this is an 11th hour request.  Can we please get Serge's
   autodev stuff into this release?  Please?
 
  release early, release often?
 
  just release current git as 0.8.0 now, and the one with the autofoo as 
  0.8.1 soon after that?
 
 That would be ideal but we've been sitting at 0.8.0rc2 for something
 like 3-1/2 months now.  I know Daniel (the other Daniel, the Daniel) has
 been incredibly busy.  I have no objection to getting this out the door
 as 0.8.0 with a fast bump to 0.8.1 for the systemd stuff, but another
 several months is not good.  Can we get this fast bump?  We'll be
 staring Fedora 18 in the face by then.  The working versions of Fedora
 are no longer in support and we've got more distros adopting systemd.

I think this will end up slated for 0.9.0 (which we're hoping will be
soon), but in any case I went ahead and created a branch at
git://github.com/hallyn/lxc called upstream.nov1.2012.autodev, with
an autodev patch on top of Daniel's latest push.

I quickly tried my hand at fixing the error you had with /dev/ttyN.  I
haven't tested that bit.  I will not be able to be online at all from
now until weekend or monday, so if it needs more tweaks please feel
free to 'just fix it'.

(there are also some todos in the commit msg - if we're going to wait
for 0.9.0 then I can handle those later, and port the patch on top of
the 100 additional patches queued in github.com/lxc/lxc#staging)

-serge

--
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] Converting existing CentOS 6.x to container within Ubuntu 12.04 - can that be simple?

2012-11-01 Thread Fajar A. Nugraha
On Fri, Nov 2, 2012 at 3:21 AM, Whit Blauvelt w...@transpect.com wrote:
 obscured login: root
 Password: init: tty (/dev/tty2) main process (131) terminated with status 1
 init: tty (/dev/tty2) main process ended, respawning
 init: tty (/dev/tty3) main process (133) terminated with status 1
 init: tty (/dev/tty3) main process ended, respawning
 init: tty (/dev/tty4) main process (135) terminated with status 1
 init: tty (/dev/tty4) main process ended, respawning
 init: tty (/dev/tty5) main process (137) terminated with status 1
 init: tty (/dev/tty5) main process ended, respawning
 init: tty (/dev/tty6) main process (139) terminated with status 1
 init: tty (/dev/tty6) main process ended, respawning


I use this: lxc.tty = 1. which means, only tty1 is active :)

You could either:
- delete /dev/tty[2-6], or
- use lxc.tty = 6. Haven't test this though.

I prefer the first one. The tty's are used only when use
lxc-console, and by default it connects to tty1, so IMHO there's no
point in having tty = 2.

-- 
Fajar

--
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] [lxc-devel] [GIT] lxc branch, master, updated. 7f99e339363d9f005c9386f60a1d8c0953c85053

2012-11-01 Thread Michael H. Warfield
On Thu, 2012-11-01 at 22:44 +0100, Serge Hallyn wrote:
 Quoting Michael H. Warfield (m...@wittsend.com):
  On Thu, 2012-11-01 at 21:20 +0100, Daniel Baumann wrote:
   On 11/01/2012 09:08 PM, Michael H. Warfield wrote:
I know, I KNOW this is an 11th hour request.  Can we please get Serge's
autodev stuff into this release?  Please?
  
   release early, release often?
  
   just release current git as 0.8.0 now, and the one with the autofoo as 
   0.8.1 soon after that?
  
  That would be ideal but we've been sitting at 0.8.0rc2 for something
  like 3-1/2 months now.  I know Daniel (the other Daniel, the Daniel) has
  been incredibly busy.  I have no objection to getting this out the door
  as 0.8.0 with a fast bump to 0.8.1 for the systemd stuff, but another
  several months is not good.  Can we get this fast bump?  We'll be
  staring Fedora 18 in the face by then.  The working versions of Fedora
  are no longer in support and we've got more distros adopting systemd.

 I think this will end up slated for 0.9.0 (which we're hoping will be
 soon), but in any case I went ahead and created a branch at
 git://github.com/hallyn/lxc called upstream.nov1.2012.autodev, with
 an autodev patch on top of Daniel's latest push.

 I quickly tried my hand at fixing the error you had with /dev/ttyN.  I
 haven't tested that bit.  I will not be able to be online at all from
 now until weekend or monday, so if it needs more tweaks please feel
 free to 'just fix it'.

Problem.  Works for the systemd containers but not for my older
containers.  I get this...

[root@forest Plover]# cat 2012-10-30-18:17:46.log
  lxc-start 1351635466.998 ERRORlxc_conf - Operation not permitted - 
error 1 creating /usr/lib64/lxc/rootfs/dev/tty6

  lxc-start 1351635466.999 ERRORlxc_conf - failed to setup the ttys for 
'Plover'
  lxc-start 1351635466.999 ERRORlxc_start - failed to setup the 
container
  lxc-start 1351635466.999 ERRORlxc_sync - invalid sequence number 1. 
expected 2
  lxc-start 1351635466.999 ERRORlxc_start - failed to spawn 'Plover'

Alcove (the systemd container) was the first one started so it may be an
ordinal thing or it may be a systemd thing.  But it's a problem.

 (there are also some todos in the commit msg - if we're going to wait
 for 0.9.0 then I can handle those later, and port the patch on top of
 the 100 additional patches queued in github.com/lxc/lxc#staging)

 -serge

Regards,
Mike
-- 
Michael H. Warfield (AI4NB) | (770) 985-6132 |  m...@wittsend.com
   /\/\|=mhw=|\/\/  | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9  | An optimist believes we live in the best of all
 PGP Key: 0x674627FF| possible worlds.  A pessimist is sure of it!


signature.asc
Description: This is a digitally signed message part
--
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] [lxc-devel] [GIT] lxc branch, master, updated. 7f99e339363d9f005c9386f60a1d8c0953c85053

2012-11-01 Thread Serge Hallyn
Quoting Michael H. Warfield (m...@wittsend.com):
 On Thu, 2012-11-01 at 22:44 +0100, Serge Hallyn wrote:
  Quoting Michael H. Warfield (m...@wittsend.com):
   On Thu, 2012-11-01 at 21:20 +0100, Daniel Baumann wrote:
On 11/01/2012 09:08 PM, Michael H. Warfield wrote:
 I know, I KNOW this is an 11th hour request.  Can we please get 
 Serge's
 autodev stuff into this release?  Please?
   
release early, release often?
   
just release current git as 0.8.0 now, and the one with the autofoo as 
0.8.1 soon after that?
   
   That would be ideal but we've been sitting at 0.8.0rc2 for something
   like 3-1/2 months now.  I know Daniel (the other Daniel, the Daniel) has
   been incredibly busy.  I have no objection to getting this out the door
   as 0.8.0 with a fast bump to 0.8.1 for the systemd stuff, but another
   several months is not good.  Can we get this fast bump?  We'll be
   staring Fedora 18 in the face by then.  The working versions of Fedora
   are no longer in support and we've got more distros adopting systemd.
 
  I think this will end up slated for 0.9.0 (which we're hoping will be
  soon), but in any case I went ahead and created a branch at
  git://github.com/hallyn/lxc called upstream.nov1.2012.autodev, with
  an autodev patch on top of Daniel's latest push.
 
  I quickly tried my hand at fixing the error you had with /dev/ttyN.  I
  haven't tested that bit.  I will not be able to be online at all from
  now until weekend or monday, so if it needs more tweaks please feel
  free to 'just fix it'.
 
 Problem.  Works for the systemd containers but not for my older
 containers.  I get this...
 
 [root@forest Plover]# cat 2012-10-30-18:17:46.log
   lxc-start 1351635466.998 ERRORlxc_conf - Operation not permitted - 
 error 1 creating /usr/lib64/lxc/rootfs/dev/tty6
 
   lxc-start 1351635466.999 ERRORlxc_conf - failed to setup the ttys 
 for 'Plover'
   lxc-start 1351635466.999 ERRORlxc_start - failed to setup the 
 container
   lxc-start 1351635466.999 ERRORlxc_sync - invalid sequence number 1. 
 expected 2
   lxc-start 1351635466.999 ERRORlxc_start - failed to spawn 'Plover'
 
 Alcove (the systemd container) was the first one started so it may be an
 ordinal thing or it may be a systemd thing.  But it's a problem.

Hm, perhaps the container doesn't have mknod?

I've updated the git tree to not fail when mknod is denied.  It should
just spit out an error, and presumably another one when it next tries
to bind mount the pty onto it, but that's ok.

Does git://github.com/hallyn/lxc#upstream.nov1.2012.autodev work better
now?  (I'm logging off now, so if not I probably won't know for some time)

-serge

--
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] Converting existing CentOS 6.x to container within Ubuntu 12.04 - can that be simple?

2012-11-01 Thread Whit Blauvelt
On Fri, Nov 02, 2012 at 04:46:19AM +0700, Fajar A. Nugraha wrote:
 
 I use this: lxc.tty = 1. which means, only tty1 is active :)
 
 You could either:
 - delete /dev/tty[2-6], or
 - use lxc.tty = 6. Haven't test this though.
 
 I prefer the first one. The tty's are used only when use
 lxc-console, and by default it connects to tty1, so IMHO there's no
 point in having tty = 2.

Went the route of deleting all of dev, then recreating the specific parts in
the document. At this point it lxc-starts, and due to my having cribbed this
line from a doc elsewhere for the config file 

lxc.network.ipv4 = 10.196.58.117

it comes up with that IP. Still needs work on the routing and netmask but I
can reconfigure that by hand for the moment.

What it does do is connect nicely to the host, on 10.196.58.116, and vice
versa. What it doesn't do, unlike the host, is connect to anywhere else,
despite being set up with identical routes. 

Now, on other VMs here I've tested if VMware will allow multiple IPs per
machine. It does, when they're both just assigned to the base machine, and
so would have the same MAC. On the host ip_forward = 1 so that's not it. I
wonder if the trick would be to have the guest have the _same_ MAC as the
host rather than an arbitrary one? So ...

Ah if I do that, then VMware manages to force both the IP belonging to the
host and the appropriate routing tables onto it. That's funny. But it can't
be recovered from. Changing the IP doesn't result in it being able to
connect even to the host. 

I'm thinking the VMware restriction - apparently that it come from a known
MAC - may be incompatible with using br0. So I guess I have to work out the
DNAT/SNAT formula, and assign the 2nd IP for the guest on the host level and
then forward it through.

I really do appreciate the advice and encouragement, Fajar.

Whit

--
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] Converting existing CentOS 6.x to container within Ubuntu 12.04 - can that be simple?

2012-11-01 Thread Fajar A. Nugraha
On Fri, Nov 2, 2012 at 5:30 AM, Whit Blauvelt w...@transpect.com wrote:
 I'm thinking the VMware restriction - apparently that it come from a known
 MAC - may be incompatible with using br0. So I guess I have to work out the
 DNAT/SNAT formula, and assign the 2nd IP for the guest on the host level and
 then forward it through.

Think of this from another perspective: other linux virtualizations
(e.g. KVM and xen) also use the same network configuration (bridge to
a physical network device) by default. So while xen (when using PV)
can run under vmware, your particular setup would prevent it from
getting network connectivity too. Just saying that it's not lxc's
fault :)

Having said that, I recall some VPS providers enforcing the same
limitation, so your condition is quite common. Please share whatever
ended up working for you so others can benefit from it as well.

-- 
Fajar

--
LogMeIn Central: Instant, anywhere, Remote PC access and management.
Stay in control, update software, and manage PCs from one command center
Diagnose problems and improve visibility into emerging IT issues
Automate, monitor and manage. Do more in less time with Central
http://p.sf.net/sfu/logmein12331_d2d
___
Lxc-users mailing list
Lxc-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/lxc-users


Re: [Lxc-users] [lxc-devel] [GIT] lxc branch, master, updated. 7f99e339363d9f005c9386f60a1d8c0953c85053

2012-11-01 Thread Michael H. Warfield
On Thu, 2012-11-01 at 19:17 -0400, Michael H. Warfield wrote:
 On Thu, 2012-11-01 at 23:28 +0100, Serge Hallyn wrote:
  Quoting Michael H. Warfield (m...@wittsend.com):
   On Thu, 2012-11-01 at 22:44 +0100, Serge Hallyn wrote:
Quoting Michael H. Warfield (m...@wittsend.com):
 On Thu, 2012-11-01 at 21:20 +0100, Daniel Baumann wrote:
  On 11/01/2012 09:08 PM, Michael H. Warfield wrote:
   I know, I KNOW this is an 11th hour request.  Can we please get 
   Serge's
   autodev stuff into this release?  Please?
 
  release early, release often?
 
  just release current git as 0.8.0 now, and the one with the autofoo 
  as 
  0.8.1 soon after that?
 
 That would be ideal but we've been sitting at 0.8.0rc2 for something
 like 3-1/2 months now.  I know Daniel (the other Daniel, the Daniel) 
 has
 been incredibly busy.  I have no objection to getting this out the 
 door
 as 0.8.0 with a fast bump to 0.8.1 for the systemd stuff, but another
 several months is not good.  Can we get this fast bump?  We'll be
 staring Fedora 18 in the face by then.  The working versions of Fedora
 are no longer in support and we've got more distros adopting systemd.
   
I think this will end up slated for 0.9.0 (which we're hoping will be
soon), but in any case I went ahead and created a branch at
git://github.com/hallyn/lxc called upstream.nov1.2012.autodev, with
an autodev patch on top of Daniel's latest push.
   
I quickly tried my hand at fixing the error you had with /dev/ttyN.  I
haven't tested that bit.  I will not be able to be online at all from
now until weekend or monday, so if it needs more tweaks please feel
free to 'just fix it'.
   
   Problem.  Works for the systemd containers but not for my older
   containers.  I get this...
   
   [root@forest Plover]# cat 2012-10-30-18:17:46.log
 lxc-start 1351635466.998 ERRORlxc_conf - Operation not 
   permitted - error 1 creating /usr/lib64/lxc/rootfs/dev/tty6
   
 lxc-start 1351635466.999 ERRORlxc_conf - failed to setup the 
   ttys for 'Plover'
 lxc-start 1351635466.999 ERRORlxc_start - failed to setup the 
   container
 lxc-start 1351635466.999 ERRORlxc_sync - invalid sequence 
   number 1. expected 2
 lxc-start 1351635466.999 ERRORlxc_start - failed to spawn 
   'Plover'
   
   Alcove (the systemd container) was the first one started so it may be an
   ordinal thing or it may be a systemd thing.  But it's a problem.
  \
  Hm, perhaps the container doesn't have mknod?

 They all should have, but I will investigate.  Those devices would have
 existed in the static file system with /dev.  Could it be a problem with
 the device already existing in the /dev directory?

Ok...  Now this is just bloody weird.  I do not understand this.

Yes the containers come up.  But...

Here's what shows up in the detached container's log...

[root@forest Audience]# cat 2012-10-30-18:52:41.log
  lxc-start 1351637562.011 ERRORlxc_conf - Operation not permitted - 
error creating /usr/lib64/lxc/rootfs/dev/tty6

Now wait a minute...  What about 1, 2, 3, 4, and 5???  They succeeded
but 6 failed?  How does that make any sense.  In the container...

crw-rw-rw- 1 root root   5,   0 Apr 13  2006 tty
crw--w 1 root tty  136,  16 Oct 30  2012 tty1
crw--w 1 root tty  136,  17 Oct 30  2012 tty2
crw--w 1 root tty  136,  18 Oct 30  2012 tty3
crw--w 1 root tty  136,  19 Oct 30  2012 tty4
crw--w 1 root tty  136,  20 Oct 30  2012 tty5
crw--w 1 root tty  136,  21 Oct 30  2012 tty6

Ok...  That's probably from a couple of days ago.  But no error messages
for the others and they are not freshly made either...  That was a
CentOS 5 container.

Trying it with another Fedora container but removed the tty? entries.
No errors.  Hmmm...  Wait...  Another problem...  Container Plover...

[mhw@plover ~]$ who
mhw  pts/92012-10-30 19:47 (forest.ip6.wittsend.com)
[mhw@plover ~]$ sudo -s
sudo: sorry, you must have a tty to run sudo

What?

Sigh...

No problem in Alcove (F17):

[mhw@canyon mhw]$ ssh alcove.ip6.wittsend.com
Last login: Wed Oct 31 01:51:39 2012 from canyon.ip6.wittsend.com
[mhw@alcove ~]$ sudo -s
[root@alcove mhw]# 

Back to Audience (CentOS 5) and removed /dev/tty?:

No errors as seen before.

Successfully created tty1-6.

Sudo works.

WTH?

There's something wrong here.  Audience and Plover do not have autodev
enabled.  Why has this changed?  Shouldn't that be under the autodev
switch as well?

  I've updated the git tree to not fail when mknod is denied.  It should
  just spit out an error, and presumably another one when it next tries
  to bind mount the pty onto it, but that's ok.

  Does git://github.com/hallyn/lxc#upstream.nov1.2012.autodev work better
  now?  (I'm logging off now, so if not I probably won't know for some time)
 
 Working now.  All containers are up.
 
 When you get back to