On Mon, 2011-11-14 at 20:43 +0100, Alessandro Ghedini wrote:
> On Mon, Nov 14, 2011 at 12:25:36PM +0100, Frederik Himpe wrote:
> > On Mon, 2011-06-06 at 11:11 +0200, Alessandro Ghedini wrote:
> >
> > > On Mon, Jun 06, 2011 at 10:12:36AM +0200, Frederik Himpe wrote:
> > > > It seems that since I installed ulatencyd, virtinst cannot create VM's
> > > > anymore
> > > > and complains that it cannot create a cgroup for the VM instance with
> > > > the
> > > > error: no such file or directory. Restarting libvirt-bin service, makes
> > > > it work
> > > > again.
> > > >
> > > > A similar bug occured in Fedora because systemd was deleting libvirt's
> > > > cgroups.
> > > > :
> > > >
> > > > https://bugzilla.redhat.com/show_bug.cgi?id=678555
> > >
> > > I've never used virtinst, could you please provide a step-by-step
> > > procedure
> > > to reproduce the bug?
> >
> > I don't know how to reliably reproduce it, it just happens now and then.
> >
> > Currently I am suffering from this problem. I try to start a VM defined
> > in libvirt:
> > # virsh start Ubuntu
> > error: Failed to start domain Ubuntu
> > error: Unable to create cgroup for Ubuntu: No such file or directory
> >
> > Notice that libvirt was moved under grp_31597, so now libvirt does not
> > find its cgroup anymore:
> >
> > $ ulatency
> > /sys/fs/cgroup/cpu
> > └─┬»cpu«
> > [...]
> >
> > So I restart libvirtd, which will recreate the needed cgroup:
> >
> > # ulatency
> > /sys/fs/cgroup/cpu
> > └─┬»cpu«
> > [...]
> >
> > Now I can again correctly start the VM. After starting the VM with
> > # virsh start Ubuntu
> >
> > active looks like this:
> > │ ├─┬»active«
> > [...]
> >
> > So I guess ulatency should never move the libvirt cgroup.
>
> I see. I think adjusting the ulatencyd scheduler would work (I'm not sure
> _how_ though), but, if I understood this right, simply starting ulatencyd
> before libvirtd at boot time would work too, and wouldn't require you to
> change the default scheduler configuration. That can be done by adding the
> line:
>
> # X-Start-Before: libvirtd
>
> to /etc/init.d/ulatencyd (after Required-Stop) and then running "insserv" as
> root. Would you mind checking if this works? So that I can push the change
> to the package in the archive.
That will not help I think, because the problem does not only happen
when booting the system.
Today, I encountered again the problem of the libvirt/qemu group not
existing at the expected location anymore. I restarted the libvirt
service. However, libvirt still could not find the group. ulatency now
gives this output:
/sys/fs/cgroup/cpu
└─┬»cpu«
├ 6 migration/0
├─┬»s_ul«
│ └ 15778 ulatencyd
├─┬»sys_essential«
│ └ 5783 Xorg
├─┬»sys_bg«
│ └ 18953 cron
├─┬»usr_1000«
│ ├─┬»grp_26849«
│ │ └ 26849 gcalctool
│ ├─┬»grp_5421«
│ │ └ 8470 plugin-containe
│ ├─┬»active«
│ │ ├ 5421 firefox-bin
│ │ ├ 6271 python
│ │ ├ 9125 evolution
│ │ └ 17782 python
│ ├─┬»grp_15204«
│ │ └ 15204 ssh
│ ├─┬»grp_5455«
│ │ └ 5457 pcscd
│ ├─┬»grp_6189«
│ │ └ 6189 gnome-power-man
│ ├─┬»grp_15288«
│ │ ├ 15288 libreoffice
│ │ ├ 15290 oosplash.bin
│ │ └ 15306 soffice.bin
│ ├─┬»grp_27612«
│ │ └ 27612 bash
│ ├─┬»grp_27604«
│ │ └ 27604 bash
│ ├─┬»grp_6257«
│ │ └ 6257 update-notifier
│ ├─┬»grp_16540«
│ │ └ 16540 bash
│ ├─┬»grp_10931«
│ │ └ 10931 bash
│ ├─┬»grp_9073«
│ │ └ 9073 bash
│ ├─┬»grp_6261«
│ │ └ 6261 nm-applet
│ ├─┬»grp_9773«
│ │ └ 9773 bash
│ ├─┬»grp_9769«
│ │ └ 9769 bash
│ ├─┬»grp_4369«
│ │ └ 4369 bash
│ ├─┬»grp_1844«
│ │ └ 1844 bash
│ ├─┬»grp_6187«
│ │ └ 6187 gnome-settings-
│ ├─┬»grp_27916«
│ │ ├ 27916 bash
│ │ └─┬»libvirt«
│ │ └─┬»qemu«
│ ├─┬»grp_20865«
│ │ └ 20865 bash
│ ├─┬»grp_6373«
│ │ └ 6373 sh
│ ├─┬»grp_6535«
│ │ └ 6535 bash
│ ├─┬»grp_6256«
│ │ └ 6256 tracker-miner-f
│ ├─┬»grp_6274«
│ │ └ 6274 parcellite
│ ├─┬»grp_6250«
│ │ └ 6250 python
│ ├─┬»grp_6254«
│ │ └ 6254 gdu-notificatio
│ ├─┬»grp_6262«
│ │ └ 6262 tracker-store
│ ├─┬»grp_6249«
│ │ └ 6249 bluetooth-apple
│ ├─┬»grp_6252«
│ │ └ 6252 zeitgeist-datah
│ ├─┬»grp_6284«
│ │ └ 6312 gconf-helper
│ ├─┬»grp_6287«
│ │ └ 6287 gnome-screensav
│ ├─┬»grp_6246«
│ │ ├ 6246 bonobo-activati
│ │ ├ 6294 sensors-applet
│ │ ├ 6298 multiload-apple
│ │ └ 6300 python
│ ├─┬»grp_6275«
│ │ └ 6275 gnome-volume-co
│ ├─┬»ui«
│ │ ├ 6230 compiz
│ │ ├ 6237 gnome-panel
│ │ ├ 6244 nautilus
│ │ └ 6374 compiz-decorato
│ ├─┬»grp_6228«
│ │ └ 6228 compiz-ccp
│ ├─┬»grp_6175«
│ │ └ 6175 dbus-launch
│ ├─┬»grp_6176«
│ │ ├ 6176 dbus-daemon
│ │ ├ 6181 gconfd-2
│ │ ├ 6205 dconf-service
│ │ ├ 6210 gvfsd
│ │ ├ 6224 gvfs-gdu-volume
│ │ ├ 6240 gvfs-afc-volume
│ │ ├ 6243 gvfs-gphoto2-vo
│ │ ├ 6270 zeitgeist-daemo
│ │ ├ 6510 geoclue-master
│ │ ├ 6530 geoclue-manual
│ │ ├ 6588 geoclue-hostip
│ │ ├ 6600 cat
│ │ ├ 6650 gvfsd-trash
│ │ ├ 6658 gvfsd-burn
│ │ ├ 6674 gvfsd-metadata
│ │ ├ 20095 gvfsd-http
│ │ └ 26305 e-addressbook-f
│ ├─┬»grp_6172«
│ │ └ 6172 ssh-agent
│ ├─┬»grp_5877«
│ │ └ 5877 x-session-manag
│ └─┬»grp_2498«
│ └ 5853 gnome-keyring-d
├─┬»sys_idle«
│ └ 2478 preload
├─┬»usr_65534«
│ └─┬»grp_2798«
│ └ 2799 dnsmasq
├─┬»rt_tasks«
│ ├ 2 kthreadd
│ ├ 3 ksoftirqd/0
│ ├ 13 cpuset
│ ├ 14 khelper
│ ├ 15 netns
│ ├ 16 sync_supers
│ ├ 17 bdi-default
│ ├ 18 kintegrityd
│ ├ 19 kblockd
│ ├ 21 khungtaskd
│ ├ 22 kswapd0
│ ├ 23 ksmd
│ ├ 24 khugepaged
│ ├ 25 fsnotify_mark
│ ├ 26 crypto
│ ├ 132 khubd
│ ├ 135 ata_sff
│ ├ 172 firewire
│ ├ 180 scsi_eh_0
│ ├ 181 scsi_eh_1
│ ├ 182 scsi_eh_2
│ ├ 183 scsi_eh_3
│ ├ 184 scsi_eh_4
│ ├ 185 scsi_eh_5
│ ├ 259 kdmflush
│ ├ 267 kdmflush
│ ├ 296 jbd2/dm-0-8
│ ├ 297 ext4-dio-unwrit
│ ├ 493 kpsmoused
│ ├ 558 cfg80211
│ ├ 645 iwlagn
│ ├ 653 pccardd
│ ├ 785 hd-audio0
│ ├ 1234 l2cap
│ ├ 1357 kvm-irqfd-clean
│ ├ 1553 kdmflush
│ ├ 1652 jbd2/sda5-8
│ ├ 1653 ext4-dio-unwrit
│ ├ 1656 xfs_mru_cache
│ ├ 1657 xfslogd
│ ├ 1658 xfsdatad
│ ├ 1659 xfsconvertd
│ ├ 1660 xfsbufd/dm-2
│ ├ 1661 xfsaild/dm-2
│ ├ 1812 rpciod
│ ├ 1814 nfsiod
│ ├ 2024 flush-254:0
│ ├ 2307 jfsIO
│ ├ 2308 jfsCommit
│ ├ 2309 jfsCommit
│ ├ 2310 jfsSync
│ ├ 2535 krfcommd
│ ├ 4502 kauditd
│ ├ 6286 pulseaudio
│ ├ 8399 flush-254:2
│ ├ 11593 kworker/0:1
│ ├ 14250 migration/1
│ ├ 14252 ksoftirqd/1
│ ├ 14253 watchdog/1
│ ├ 14282 kworker/u:45
│ ├ 14284 kworker/u:47
│ ├ 14722 hci0
│ ├ 17667 kworker/u:0
│ ├ 17833 kworker/0:8
│ ├ 17875 kworker/1:9
│ ├ 17880 kworker/0:16
│ ├ 18425 kworker/1:0
│ ├ 18885 watchdog/0
│ └ 26856 rtkit-daemon
└─┬»sys_daemon«
├ 1 init
├ 881 rsyslogd
├ 1789 rpcbind
├ 1804 rpc.statd
├ 1821 rpc.idmapd
├ 1939 zfs-fuse
├ 2457 dbus-daemon
├ 2468 acpi_fakekeyd
├ 2492 modem-manager
├ 2495 wpa_supplicant
├ 2500 gdm3
├ 2526 bluetoothd
├ 2868 console-kit-dae
├ 2963 master
├ 3010 smartd
├ 3740 getty
├ 3741 getty
├ 3742 getty
├ 3743 getty
├ 3744 getty
├ 4476 mount.ntfs
├ 5347 udisks-daemon
├ 5348 udisks-daemon
├ 5782 gdm-simple-slav
├ 5827 gdm-session-wor
├ 7299 cupsd
├ 9112 dbus-launch
├ 9113 dbus-daemon
├ 14897 dhclient
├ 14919 dhclient
├ 15071 qmgr
├ 15268 getty
├ 15472 chronyd
├ 16089 su
├ 16097 bash
├ 17473 pcscd
├ 17514 udevd
├ 17730 pickup
├ 18683 su
├ 18693 bash
├ 18708 libvirtd
├ 18802 udevd
├ 18957 atd
├ 20640 python
├ 22186 fail2ban-server
├ 22210 polkitd
├ 24406 sshd
├ 27651 NetworkManager
├ 29383 avahi-daemon
├ 29384 avahi-daemon
├ 30238 upowerd
├ 31644 su
├ 31654 bash
├ 32125 acpid
├ 32743 udevd
└─┬»libvirt«
└─┬»lxc«
So it looks like the libvirt/qemu group was moved to grp_27916, while
libvirt/lxc was still at the right location. I am not sure when these
move happened exactly.
I restarted ulatencyd after that, and after that, the all libvirt groups
were even completely gone.
> Also, for the long-term, I may also ask upstream a more user-friendly way
> to ignore specific processes so that similar situations may be resolved too.
I think ulatencyd should be taught to never touch the cgroup named
libvirt and all its subgroups and processes. Note that ulatencyd can
move the libvirtd process itself, but not the libvirt cgroup and all its
children.
Actually, it looks like ulatencyd even completely removes all empty
cgroups at startup and I guess this should probably never be done,
except if they are groups created by ulatencyd itself.
--
Frederik Himpe <[email protected]>
--
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]
