[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2014-01-15 Thread Stéphane Graber
Marking as invalid for LXC since this is a kernel bug. ** Changed in: lxc (Ubuntu) Status: Confirmed = Invalid -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to lxc in Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title:

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2014-01-15 Thread Stéphane Graber
Marking as invalid for LXC since this is a kernel bug. ** Changed in: lxc (Ubuntu) Status: Confirmed = Invalid -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-10-04 Thread Serge Hallyn
@arges does the kernel patch set you sent to kernel-team@ yesterday help the testcase you put in comment #22? Someone just pointed out he got this [54545.873460] unregister_netdevice: waiting for lo to become free. Usage count = 1 in a 3.8 kernel when using macvlan in a container. It seems to

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-10-04 Thread Serge Hallyn
@arges does the kernel patch set you sent to kernel-team@ yesterday help the testcase you put in comment #22? Someone just pointed out he got this [54545.873460] unregister_netdevice: waiting for lo to become free. Usage count = 1 in a 3.8 kernel when using macvlan in a container. It seems to

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-10-03 Thread Chris J Arges
I can achieve a similar hung task by using the following script: #!/bin/bash -x NUM=7999 for (( i=0; i${NUM}; i++ )) do ip netns add foobar$i done for (( i=0; i${NUM}; i++ )) do ip netns delete foobar$i done for (( i=0; i${NUM}; i++ )) do ip netns add foobar$i done for (( i=0; i${NUM};

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-10-03 Thread Chris J Arges
I can achieve a similar hung task by using the following script: #!/bin/bash -x NUM=7999 for (( i=0; i${NUM}; i++ )) do ip netns add foobar$i done for (( i=0; i${NUM}; i++ )) do ip netns delete foobar$i done for (( i=0; i${NUM}; i++ )) do ip netns add foobar$i done for (( i=0; i${NUM};

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-10-01 Thread Joseph Salisbury
** Changed in: linux (Ubuntu) Status: Confirmed = Incomplete -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to lxc in Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters uninterruptible sleep To manage

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-10-01 Thread Joseph Salisbury
** Changed in: linux (Ubuntu) Status: Confirmed = Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters uninterruptible sleep To manage notifications

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-19 Thread Stefan Bader
This looks to be the third case we hit with the same symptom (and likely for yet another reason). The complicating issue is that lxc containers make use of net namespaces and if those are released, references to I think the netdevice structures, are temporarily moved over to the loopback device.

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-19 Thread Stefan Bader
This looks to be the third case we hit with the same symptom (and likely for yet another reason). The complicating issue is that lxc containers make use of net namespaces and if those are released, references to I think the netdevice structures, are temporarily moved over to the loopback device.

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-17 Thread Joseph Salisbury
Thanks for testing the upstream kernel. One additional question. Was there a previous kernel version that did not exhibit this issue? It would be good to know if this is a regression. If it is, we can perform a bisect to identify the commit that introduced it. -- You received this bug

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-17 Thread Pavel Bennett
The last one I'm aware of that did not exhibit this issue was 3.5.0-27. I wish I had a simpler repro though, since on our system it takes 10-15 hours of heavy processing to hit the uninterruptible sleeps. Could it be tracked by looking at the state of the OS? Every new lxc- start ends up hanging

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-17 Thread Joseph Salisbury
Thanks for testing the upstream kernel. One additional question. Was there a previous kernel version that did not exhibit this issue? It would be good to know if this is a regression. If it is, we can perform a bisect to identify the commit that introduced it. -- You received this bug

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-17 Thread Pavel Bennett
The last one I'm aware of that did not exhibit this issue was 3.5.0-27. I wish I had a simpler repro though, since on our system it takes 10-15 hours of heavy processing to hit the uninterruptible sleeps. Could it be tracked by looking at the state of the OS? Every new lxc- start ends up hanging

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-05 Thread Pavel Bennett
** Tags added: kernel-bug-exists-upstream -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to lxc in Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters uninterruptible sleep To manage notifications about this bug

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-05 Thread Pavel Bennett
Managed to repro with v3.10-saucy last night. What do you guys suspect it could be? I'm keeping the server in this state for now if you'd like me to gather some data. -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to lxc in Ubuntu.

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-05 Thread Pavel Bennett
** Tags added: kernel-bug-exists-upstream -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters uninterruptible sleep To manage notifications about this bug go to:

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-05 Thread Pavel Bennett
Managed to repro with v3.10-saucy last night. What do you guys suspect it could be? I'm keeping the server in this state for now if you'd like me to gather some data. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-03 Thread Pavel Bennett
Changing to Confirmed as per instructions in comment #7 ** Changed in: linux Status: Incomplete = Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to lxc in Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title:

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-03 Thread Joseph Salisbury
** Changed in: linux (Ubuntu) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to lxc in Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters uninterruptible sleep To manage

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-03 Thread Joseph Salisbury
Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.10 kernel[0]. If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'. If the mainline kernel does not fix

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-03 Thread Pavel Bennett
Changing to Confirmed as per instructions in comment #7 ** Changed in: linux Status: Incomplete = Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-03 Thread Joseph Salisbury
** Changed in: linux (Ubuntu) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters uninterruptible sleep To manage notifications about this

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-03 Thread Joseph Salisbury
Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v3.10 kernel[0]. If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'. If the mainline kernel does not fix

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
Reproduced even with lxc-stop. dmesg: [178420.689704] unregister_netdevice: waiting for lo to become free. Usage count = 1 [178430.919783] unregister_netdevice: waiting for lo to become free. Usage count = 1 [178441.149854] unregister_netdevice: waiting for lo to become free. Usage count = 1

Re: [Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Serge Hallyn
Thanks for the info affects: linux ** Also affects: linux Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to lxc in Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
This should help. kern.log: Jul 2 05:41:32 server1 kernel: [136565.201601] device vethbJ4JsM left promiscuous mode Jul 2 05:41:32 server1 kernel: [136565.201603] vmbr: port 5(vethbJ4JsM) entered disabled state Jul 2 05:41:38 server1 kernel: [136570.551496] vmbr: port 2(veth49SiBX) entered

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Joseph Salisbury
** Changed in: linux Importance: Undecided = High ** Also affects: linux (Ubuntu) Importance: Undecided Status: New ** Changed in: linux (Ubuntu) Importance: Undecided = High ** Tags added: kernel-da-key raring -- You received this bug notification because you are a member of

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
apport information ** Tags added: apport-collected ** Description changed: After running and terminating around 6000 containers overnight, something happened on my box that is affecting every new LXC container I try to start. The DEBUG log file looks like: lxc-start

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
Looks like apport was missing some module to gather what it wanted. Let me know if this info would be valuable and I can re-run it. ** Changed in: lxc (Ubuntu) Status: Incomplete = Confirmed -- You received this bug notification because you are a member of Ubuntu Server Team, which is

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
Attaching a better apport file after installing the missing dependency. I will hide the ones from earlier as this will contain the same data and more. ** Attachment added: apport.lxc.txt https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1196295/+attachment/3722586/+files/apport.lxc.txt --

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
Reproduced even with lxc-stop. dmesg: [178420.689704] unregister_netdevice: waiting for lo to become free. Usage count = 1 [178430.919783] unregister_netdevice: waiting for lo to become free. Usage count = 1 [178441.149854] unregister_netdevice: waiting for lo to become free. Usage count = 1

Re: [Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Serge Hallyn
Thanks for the info affects: linux ** Also affects: linux Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1196295 Title: lxc-start enters

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
This should help. kern.log: Jul 2 05:41:32 server1 kernel: [136565.201601] device vethbJ4JsM left promiscuous mode Jul 2 05:41:32 server1 kernel: [136565.201603] vmbr: port 5(vethbJ4JsM) entered disabled state Jul 2 05:41:38 server1 kernel: [136570.551496] vmbr: port 2(veth49SiBX) entered

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Joseph Salisbury
** Changed in: linux Importance: Undecided = High ** Also affects: linux (Ubuntu) Importance: Undecided Status: New ** Changed in: linux (Ubuntu) Importance: Undecided = High ** Tags added: kernel-da-key raring -- You received this bug notification because you are a member of

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
apport information ** Tags added: apport-collected ** Description changed: After running and terminating around 6000 containers overnight, something happened on my box that is affecting every new LXC container I try to start. The DEBUG log file looks like: lxc-start

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
Looks like apport was missing some module to gather what it wanted. Let me know if this info would be valuable and I can re-run it. ** Changed in: lxc (Ubuntu) Status: Incomplete = Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-07-02 Thread Pavel Bennett
Attaching a better apport file after installing the missing dependency. I will hide the ones from earlier as this will contain the same data and more. ** Attachment added: apport.lxc.txt https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/1196295/+attachment/3722586/+files/apport.lxc.txt --

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-06-30 Thread Pavel Bennett
Also, in dmesg: [54545.873460] unregister_netdevice: waiting for lo to become free. Usage count = 1 [54556.103535] unregister_netdevice: waiting for lo to become free. Usage count = 1 [54566.333609] unregister_netdevice: waiting for lo to become free. Usage count = 1 [54576.563664]

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-06-30 Thread Pavel Bennett
Some basic environment details. I can post more if requested. Ubuntu Server 13.04 64-bit $ uname -r 3.8.0-25-generic $ dpkg -l | grep lxc ii liblxc00.9.0-0ubuntu3.3 amd64Linux Containers userspace tools (library) ii lxc

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-06-30 Thread Pavel Bennett
Also, in dmesg: [54545.873460] unregister_netdevice: waiting for lo to become free. Usage count = 1 [54556.103535] unregister_netdevice: waiting for lo to become free. Usage count = 1 [54566.333609] unregister_netdevice: waiting for lo to become free. Usage count = 1 [54576.563664]

[Bug 1196295] Re: lxc-start enters uninterruptible sleep

2013-06-30 Thread Pavel Bennett
Some basic environment details. I can post more if requested. Ubuntu Server 13.04 64-bit $ uname -r 3.8.0-25-generic $ dpkg -l | grep lxc ii liblxc00.9.0-0ubuntu3.3 amd64Linux Containers userspace tools (library) ii lxc