[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-31 Thread nutznboltz
This report (in German)
https://forge.univention.org/bugzilla/show_bug.cgi?id=23258 suggests
that when the guest VM is reset by a reboot the clock jumps backward in
time.

Two patches were suggested, both being necessary together:

KVM: x86: Fix kvmclock bug
backported to Natty Aug 11, 2011 but not to linux-lts-backport-natty-2.6.38 
yet!!!
http://patchwork.ozlabs.org/patch/109436/
http://lkml.org/lkml/2011/5/5/436
http://bugs.launchpad.net/bugs/714335

KVM: x86: Fix a possible backwards warp of kvmclock
http://kerneltrap.org/mailarchive/linux-kernel/2010/8/20/4609037
http://choon.net/forum/read.php?21,76764


** Bug watch added: forge.univention.org/bugzilla/ #23258
   https://forge.univention.org/bugzilla/show_bug.cgi?id=23258

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-31 Thread nutznboltz
This patch is missing from linux-lts-backport-natty-2.6.38
http://patchwork.ozlabs.org/patch/109436/

diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index 980317a..0556e05 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -2100,8 +2100,8 @@  void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
if (check_tsc_unstable()) {
kvm_x86_ops-adjust_tsc_offset(vcpu, -tsc_delta);
vcpu-arch.tsc_catchup = 1;
-   kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
}
+   kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
if (vcpu-cpu != cpu)
kvm_migrate_timers(vcpu);
vcpu-cpu = cpu;

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-31 Thread nutznboltz
Please help apply patch http://patchwork.ozlabs.org/patch/109436/ to
linux-lts-backport-natty-2.6.38 thanks

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-31 Thread nutznboltz
** Changed in: libvirt (Ubuntu)
   Status: Incomplete = Confirmed

** Description changed:

+ To reproduce:
+ 
+ apt-get source linux-image-2.6.38-12-generic
+ 
+ examine linux-lts-backport-natty-2.6.38/arch/x86/kvm/x86.c
+ 
+ See missing patch:
+ 
+ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
+ index 980317a..0556e05 100644
+ --- a/arch/x86/kvm/x86.c
+ +++ b/arch/x86/kvm/x86.c
+ @@ -2100,8 +2100,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
+if (check_tsc_unstable()) {
+ kvm_x86_ops-adjust_tsc_offset(vcpu, -tsc_delta);
+ vcpu-arch.tsc_catchup = 1;
+ - kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
+}
+ + kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
+if (vcpu-cpu != cpu)
+ kvm_migrate_timers(vcpu);
+vcpu-cpu = cpu;
+ 
+ =
+ 
  While attempting to use virsh commands this error occurs:
  
  Oct 27 08:52:22 boron libvirtd: 08:52:23.000: 25851: error :
  qemuDomainObjBeginJobWithDriver:453 : Timed out during operation: cannot
  acquire state change lock
  
  While the error state is present certain commands like virsh shutdown
  and virsh destroy elicit that error message and fail to work
  completely I found that if I kill the kvm guest processes
  
  $ kill -9 $(ps ax | grep '/usr/bin/kvm.*guestname' | awk '{print $1}')
  
  Then virsh start guestname would work.
  
  I'm not certain that the number of VM guests is the issue, however, this
  happens more frequently to me on systems with larger numbers of guests.
  We have one with 14 guests and another with 53.
  
- There seems to be a corresponding Red Hat bug at: 
+ There seems to be a corresponding Red Hat bug at:
  https://bugzilla.redhat.com/show_bug.cgi?id=676205
  
  ProblemType: Bug
  DistroRelease: Ubuntu 11.04
  Package: libvirt-bin 0.8.8-1ubuntu6.5
  ProcVersionSignature: Ubuntu 2.6.38-12.51-generic 2.6.38.8
  Uname: Linux 2.6.38-12-generic x86_64
  Architecture: amd64
  Date: Thu Oct 27 09:41:23 2011
  InstallationMedia: Ubuntu 10.10 Maverick Meerkat - Release amd64 (20101007)
  ProcEnviron:
-  LANGUAGE=en_US:en
-  PATH=(custom, user)
-  LANG=en_US.UTF-8
-  SHELL=/bin/bash
+  LANGUAGE=en_US:en
+  PATH=(custom, user)
+  LANG=en_US.UTF-8
+  SHELL=/bin/bash
  SourcePackage: libvirt
  UpgradeStatus: Upgraded to natty on 2011-05-26 (153 days ago)

** Description changed:

  To reproduce:
  
+ On Lucid run:
  apt-get source linux-image-2.6.38-12-generic
  
  examine linux-lts-backport-natty-2.6.38/arch/x86/kvm/x86.c
  
  See missing patch:
  
  diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
  index 980317a..0556e05 100644
  --- a/arch/x86/kvm/x86.c
  +++ b/arch/x86/kvm/x86.c
  @@ -2100,8 +2100,8 @@ void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
-if (check_tsc_unstable()) {
- kvm_x86_ops-adjust_tsc_offset(vcpu, -tsc_delta);
- vcpu-arch.tsc_catchup = 1;
+    if (check_tsc_unstable()) {
+ kvm_x86_ops-adjust_tsc_offset(vcpu, -tsc_delta);
+ vcpu-arch.tsc_catchup = 1;
  - kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
-}
+    }
  + kvm_make_request(KVM_REQ_CLOCK_UPDATE, vcpu);
-if (vcpu-cpu != cpu)
- kvm_migrate_timers(vcpu);
-vcpu-cpu = cpu;
+    if (vcpu-cpu != cpu)
+ kvm_migrate_timers(vcpu);
+    vcpu-cpu = cpu;
  
  =
  
  While attempting to use virsh commands this error occurs:
  
  Oct 27 08:52:22 boron libvirtd: 08:52:23.000: 25851: error :
  qemuDomainObjBeginJobWithDriver:453 : Timed out during operation: cannot
  acquire state change lock
  
  While the error state is present certain commands like virsh shutdown
  and virsh destroy elicit that error message and fail to work
  completely I found that if I kill the kvm guest processes
  
  $ kill -9 $(ps ax | grep '/usr/bin/kvm.*guestname' | awk '{print $1}')
  
  Then virsh start guestname would work.
  
  I'm not certain that the number of VM guests is the issue, however, this
  happens more frequently to me on systems with larger numbers of guests.
  We have one with 14 guests and another with 53.
  
  There seems to be a corresponding Red Hat bug at:
  https://bugzilla.redhat.com/show_bug.cgi?id=676205
  
  ProblemType: Bug
  DistroRelease: Ubuntu 11.04
  Package: libvirt-bin 0.8.8-1ubuntu6.5
  ProcVersionSignature: Ubuntu 2.6.38-12.51-generic 2.6.38.8
  Uname: Linux 2.6.38-12-generic x86_64
  Architecture: amd64
  Date: Thu Oct 27 09:41:23 2011
  InstallationMedia: Ubuntu 10.10 Maverick Meerkat - Release amd64 (20101007)
  ProcEnviron:
   LANGUAGE=en_US:en
   PATH=(custom, user)
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  SourcePackage: libvirt
  UpgradeStatus: Upgraded to natty on 2011-05-26 (153 days ago)

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:

[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-31 Thread nutznboltz
** Tags removed: natty running-unity
** Tags added: lucid testcase

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-31 Thread Serge Hallyn
** Also affects: linux (Ubuntu)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-31 Thread nutznboltz
This is a bug in the kernel not libvirt.

** Changed in: libvirt (Ubuntu)
   Status: Confirmed = Invalid

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-31 Thread nutznboltz
*** This bug is a duplicate of bug 795717 ***
https://bugs.launchpad.net/bugs/795717

** This bug has been marked a duplicate of bug 795717
   32bit rhel and centos 5.(5|6) hangs on boot on natty

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-30 Thread nutznboltz
Probably related thread: kvm linux guest hanging for minutes at a time
http://www.spinics.net/lists/kvm/msg59479.html

Notable in that thread:

a post about using -F with trace-cmd too: 
http://www.linux-kvm.org/page/Tracing
http://www.spinics.net/lists/kvm/msg59716.html

a post which states: When booting with clocksouce=hpet, the system does not 
hang.
http://www.spinics.net/lists/kvm/msg59837.html

a post which states: Using the wrong clock easily causes hangs. The system 
schedules a wakeup in 3 ms, wrong clock causes it to wakeup in 3 years, you get 
a hang for (3 years - 3 ms). 
http://www.spinics.net/lists/kvm/msg59884.html

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-30 Thread nutznboltz
I have one that rebooted this morning and is spinning

attaching gdb shows very little due to lack of symbol tables:

(gdb) where
#0  0x7f671aab3ff3 in select () from /lib/libc.so.6

strace shows a loop:

Process 971 attached - interrupt to quit
select(16, [7 10 13 14 15], [], [], {0, 995193}) = 1 (in [14], left {0, 970413})
read(14, 
\16\0\0\0\0\0\0\0\376\377\377\377\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\0\0\0\0..., 
128) = 128
rt_sigaction(SIGALRM, NULL, {0x4a9900, ~[KILL STOP RTMIN RT_1], SA_RESTORER, 
0x7f671d28c8f0}, 8) = 0
write(8, \1\0\0\0\0\0\0\0, 8) = 8
read(14, 0x7fff53785d20, 128)   = -1 EAGAIN (Resource temporarily 
unavailable)
timer_gettime(0x5, {it_interval={0, 0}, it_value={0, 0}}) = 0
timer_settime(0x5, 0, {it_interval={0, 0}, it_value={0, 25}}, NULL) = 0
timer_gettime(0x5, {it_interval={0, 0}, it_value={0, 185840}}) = 0
select(16, [7 10 13 14 15], [], [], {1, 0}) = 1 (in [7], left {0, 97})
read(7, \1\0\0\0\0\0\0\0, 512)= 8
select(16, [7 10 13 14 15], [], [], {1, 0}) = 1 (in [14], left {0, 97})
read(14, 
\16\0\0\0\0\0\0\0\376\377\377\377\0\0\0\0\0\0\0\0\0\0\0\0\5\0\0\0\0\0\0\0..., 
128) = 128
rt_sigaction(SIGALRM, NULL, {0x4a9900, ~[KILL STOP RTMIN RT_1], SA_RESTORER, 
0x7f671d28c8f0}, 8) = 0
write(8, \1\0\0\0\0\0\0\0, 8) = 8
read(14, 0x7fff53785d20, 128)   = -1 EAGAIN (Resource temporarily 
unavailable)
timer_gettime(0x5, {it_interval={0, 0}, it_value={0, 0}}) = 0
timer_settime(0x5, 0, {it_interval={0, 0}, it_value={0, 28902469}}, NULL) = 0
(repeats)

$ sudo lsof -p 971
COMMAND PID USER   FD   TYPE DEVICESIZE/OFF NODE 
NAME
kvm 971 libvirt-qemu  cwdDIR  259,04096  128 /
kvm 971 libvirt-qemu  rtdDIR  259,04096  128 /
kvm 971 libvirt-qemu  txtREG  259,0 3217592 16817685 
/usr/bin/qemu-system-x86_64
kvm 971 libvirt-qemu  memREG  259,0   18520 33597583 
/usr/lib/sasl2/libplain.so.2.0.23
kvm 971 libvirt-qemu  memREG  259,0   34904 33597580 
/usr/lib/sasl2/libntlm.so.2.0.23
kvm 971 libvirt-qemu  memREG  259,0   43296 33595849 
/lib/libcrypt-2.11.1.so
kvm 971 libvirt-qemu  memREG  259,0   18520 33597534 
/usr/lib/sasl2/liblogin.so.2.0.23
kvm 971 libvirt-qemu  memREG  259,0 1622304 33596048 
/lib/libcrypto.so.0.9.8
kvm 971 libvirt-qemu  memREG  259,0   51752 33597531 
/usr/lib/sasl2/libdigestmd5.so.2.0.23
kvm 971 libvirt-qemu  memREG  259,0   22624 33597528 
/usr/lib/sasl2/libcrammd5.so.2.0.23
kvm 971 libvirt-qemu  memREG  259,0   18528 33597524 
/usr/lib/sasl2/libanonymous.so.2.0.23
kvm 971 libvirt-qemu  memREG  259,0 1494424  187 
/usr/lib/libdb-4.8.so
kvm 971 libvirt-qemu  memREG  259,0   22464 33597518 
/usr/lib/sasl2/libsasldb.so.2.0.23
kvm 971 libvirt-qemu  memREG  259,0   26840  8762051 
/usr/lib/libogg.so.0.6.0
kvm 971 libvirt-qemu  memREG  259,0  18258456392 
/usr/lib/libvorbis.so.0.4.3
kvm 971 libvirt-qemu  memREG  259,0 1943600  8762070 
/usr/lib/libvorbisenc.so.2.0.6
kvm 971 libvirt-qemu  memREG  259,0  306744  8762063 
/usr/lib/libFLAC.so.8.2.0
kvm 971 libvirt-qemu  memREG  259,0   97256 33595855 
/lib/libnsl-2.11.1.so
kvm 971 libvirt-qemu  memREG  259,0   22560   530121 
/usr/lib/libXdmcp.so.6.0.0
kvm 971 libvirt-qemu  memREG  259,0   14488   530127 
/usr/lib/libXau.so.6.0.0
kvm 971 libvirt-qemu  memREG  259,0  256768 33607491 
/lib/libdbus-1.so.3.4.0
kvm 971 libvirt-qemu  memREG  259,0  397304  9012660 
/usr/lib/libsndfile.so.1.0.21
kvm 971 libvirt-qemu  memREG  259,0   40720 34176892 
/lib/libwrap.so.0.7.6
kvm 971 libvirt-qemu  memREG  259,0   63864   467747 
/usr/lib/libXi.so.6.1.0
kvm 971 libvirt-qemu  memREG  259,0   73600   533951 
/usr/lib/libXext.so.6.4.0
kvm 971 libvirt-qemu  memREG  259,0   14344 33597467 
/lib/libgpg-error.so.0.4.0
kvm 971 libvirt-qemu  memREG  259,0   10224 33597288 
/lib/libkeyutils-1.2.so
kvm 971 libvirt-qemu  memREG  259,0   31168 1063 
/usr/lib/libkrb5support.so.0.1
kvm 971 libvirt-qemu  memREG  259,0   14584 33595836 
/lib/libcom_err.so.2.1
kvm 971 libvirt-qemu  memREG  259,0  154048 1076 
/usr/lib/libk5crypto.so.3.1
kvm 971 libvirt-qemu  memREG  259,0  803192 1073 
/usr/lib/libkrb5.so.3.3
kvm 971 libvirt-qemu  memREG  259,0  113072   530136 
/usr/lib/libxcb.so.1.1.0
kvm 971 

[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-28 Thread nutznboltz
I've been able to narrow this down a little more.  The number of VM
guests is not a factor, this has happened on a VM host with only one
guest.  The VM guest kernel does not seem to matter either, even VM
guests which are Linux distros other than Ubuntu are affected.

It seems that the /usr/bin/kvm process goes into an infinite loop during
VM guest shutdown.

I'm still investigating this further.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread nutznboltz
-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread nutznboltz
Attached the output of

for p in `pidof libvirtd`; do
 echo -n $p: 
 cat /proc/$p/cmdline
 cat /proc/$p/cgroup
 cat /proc/$p/status
   done
   for p in `pidof kvm`; do
  echo -n $p: 
 cat /proc/$p/cmdline
 cat /proc/$p/cgroup
 cat /proc/$p/status
   done

** Attachment added: LP-882579-pidof.txt
   
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+attachment/2575222/+files/LP-882579-pidof.txt

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread Dave Walker
This really needs a reproducible test case.  Are you able to offer any
further guidance on how you managed to encounter this?

Thanks.

** Changed in: libvirt (Ubuntu)
   Importance: Undecided = Medium

** Changed in: libvirt (Ubuntu)
   Status: Confirmed = Incomplete

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread Serge Hallyn
Thanks for filing this new bug.

(I think it is in fact a dup of bug 734777, but it's not certain)

I'm afraid I'll simply have to try to reproduce this on a maverick
system and debug with gdb, as per the linked redhat bug, upstream is not
sure what fixed the problem between 0.8.8 and 0.9.0.

If it is feasible for you to upgrade to oneiric, the newer version of
libvirt there should bypass this problem, however that certainly is not
a satisfactory solution, especially as this appears to be hitting
multiple users.

** Changed in: libvirt (Ubuntu)
   Status: New = Confirmed

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread nutznboltz
While the VM guests were hung they were using 100% of the CPU.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread nutznboltz
If this issue was seen on Maverick it is still an issue with Natty.

I could try running tests on an Oneiric system too.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread nutznboltz
The system with 53 VM guests:
ProLiant DL585 G5, Four CPU sockets each with Quad-Core AMD Opteron(tm) 
Processor 8382

The system with 14 VM guests:
Sun Fire X4600 M2, Eight CPU sockets each with Dual-Core AMD Opteron(tm) 
Processor 8220

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread nutznboltz
Today two VM hosts running kernel 2.6.38-11, one with with 14 guests and
another with 53 had scheduled reboots of the many of their guests.

On the VM host with 14 guests, 11 rebooted. On the one with 53 guests,
48 rebooted.

VM guests that automatically rebooted this morning at 6:30 AM via at(8)
hung instead of rebooted.

All of the VM guests were rebooting from kernel 2.6.38-11 into 2.6.38-12

Not all of the VM guests on the hosts were scheduled to be rebooted.

virsh shutdown and virsh destroy appeared to always hang.

After using kill -9 on the PID of the VM guest virsh start for that
guest worked every time.

Initially on the VM hosts virsh list would output some VM names then
hang.

After enough VM guests were reset via kill -9 (but before all of them
were) virsh list became able to run to completion.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


Re: [Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread Serge Hallyn
Quoting nutznboltz (882...@bugs.launchpad.net):
 While the VM guests were hung they were using 100% of the CPU.

Is 'virsh destroy' and 'virsh shutdown' what always triggers this?

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread nutznboltz
Using kill -9 directly on the /usr/bin/kvm process of the hung VM
guest that was using 100% CPU did restore the ability to use virsh
start on the guest.  That is consistent with the comment #26 by Daniel
Berrange in the Red Hat ticket about The QEMU process has hung.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread nutznboltz
If you look at the last comment in the Red Hat bug report:
https://bugzilla.redhat.com/show_bug.cgi?id=676205#c26

{{ 1. The QEMU process has hung.

QEMU won't respond to monitor commands. The API call making the first
monitor command will wait forever, any subsequent API calls issuing monitor
commands will timeout after ~30 seconds with this libvirt error message.

This is expected behaviour when QEMU has hung. }}

I think this might be the issue I am experiencing.


** Bug watch added: Red Hat Bugzilla #676205
   https://bugzilla.redhat.com/show_bug.cgi?id=676205

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs


[Bug 882579] Re: Some Commands return: Timed out during operation: cannot acquire state change lock

2011-10-27 Thread nutznboltz
I don't think virsh shutdown and virsh destroy are triggers.

I think when the guest rebooted something went wrong and subsequent
attempts to use virsh shutdown and virsh destroy fail.

This morning the en-mass VM guest reboot was triggered by cron jobs
inside the guests.

I tried unsuccessfully to reproduce this on a VM host that only has a
single VM guest.  I'm still investigating if having several VM guests
reboot at the same time is the trigger or not.

-- 
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to libvirt in Ubuntu.
https://bugs.launchpad.net/bugs/882579

Title:
  Some Commands return: Timed out during operation: cannot acquire
  state change lock

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/882579/+subscriptions

-- 
Ubuntu-server-bugs mailing list
Ubuntu-server-bugs@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs