[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-22 Thread James Hunt
** Also affects: systemd-shim (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cgmanager in Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs specifying cgroup

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-22 Thread James Hunt
** Changed in: upstart Status: In Progress = Invalid ** Changed in: upstart Importance: Critical = Undecided -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cgmanager in Ubuntu. https://bugs.launchpad.net/bugs/1357252

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-22 Thread James Hunt
** Also affects: systemd-shim (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs specifying cgroup fail to start

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-22 Thread James Hunt
After a lot of testing, I think I've found the cause - when lightdm starts the session init, logind is supposed to setup the cgroups. But that isn't happening. Normally, logind would do that by asking systemd, but in our case it talks to systemd-shim. sytemd-shim then asks cgmanager to create

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-22 Thread James Hunt
** Changed in: upstart Status: In Progress = Invalid ** Changed in: upstart Importance: Critical = Undecided -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-22 Thread James Hunt
** Changed in: systemd-shim (Ubuntu) Assignee: (unassigned) = Stéphane Graber (stgraber) ** Summary changed: - Upstart jobs specifying cgroup fail to start occasionally + systemd-shim fails to handle cgmanager being unavailable -- You received this bug notification because you are a

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-21 Thread James Hunt
We definitely have a race here. I can now reliably create either a broken system or a working system simply by delaying the start of cgmanager slightly: - With the cgmanager startup delay, delay everything works correctly. - With no cgmanager startup delay, the session init ends up in the root

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-21 Thread Łukasz Zemczak
** Tags removed: lt-whitelisted -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs specifying cgroup fail to start occasionally To manage notifications about this bug go to:

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-21 Thread James Hunt
This still needs further investigation bug I think what's happening is that cgmanager is racing with dbus. And since other jobs refer to cgmanager they end up starting too early, resulting in the possibility that the session init starts too early (before it's been cgroup- constrained). If

Re: [Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-21 Thread Serge Hallyn
On a N4, I did (quoted from irc by jodh) 'adb shell', su - phablet, create ~/.config/upstart/cgroup.conf containing http://paste.ubuntu.com/8108361/ and try 'start cgroup' and see what happens it appears correct. I also added a 'start on started cgmanager', rebooted the

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-21 Thread Stéphane Graber
Also just to clarify some of the above comments. cgmanager does offer a dbus API but it does that on a private socket, it DOESN'T use the dbus system bus or dbus-daemon in any way, so there's absolutely no reason why cgmanager should ever wait for dbus to be started. -- You received this bug

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-20 Thread James Hunt
** Changed in: upstart Status: New = In Progress ** Also affects: linux (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cgmanager in Ubuntu.

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-20 Thread James Hunt
** No longer affects: linux (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cgmanager in Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs specifying cgroup fail to start occasionally To manage

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-20 Thread James Hunt
** Patch added: Latest cgmanager patch that applies to lp:ubuntu/cgmanager. https://bugs.launchpad.net/ubuntu-app-launch/+bug/1357252/+attachment/4182420/+files/cgmanager.diff -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-20 Thread James Hunt
Hi Serge, Attached are upstart session job output and cgmanager output showing good (desktop utopic in kvm) and bad (device) runs: Note the following: - The session init in both cases is making the call on cgmanager as a non-priv user in an identical way:

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-20 Thread James Hunt
The simplest way to see the problem (the following works as expected on a desktop system, creating the cgroup and resulting in no error output): $ id uid=32011(phablet) gid=32011(phablet)

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-20 Thread James Hunt
** Changed in: upstart Status: New = In Progress ** Also affects: linux (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title:

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-20 Thread James Hunt
** No longer affects: linux (Ubuntu) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs specifying cgroup fail to start occasionally To manage notifications about this bug

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread Jean-Baptiste Lallement
I reproduced this issue with 0.4+14.10.20140808.2.is.0.4+14.10.20140808.2-0ubuntu1 The output of the 'cat' commands are attached in _proc_1357252.out When it happens the following lines are displayed in: = unity8-dash.log= Trying to open application:///dialer-app.desktop = unity8.log =

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread Timo Jyrinki
It turns out the revert wasn't a revert and needs another really.is.revert version: https://launchpadlibrarian.net/182632645/ubuntu-app- launch_0.4%2B14.10.20140808.2-0ubuntu1_0.4%2B14.10.20140808.2.is.0.4%2B14.10.20140808.2-0ubuntu1.diff.gz -- You received this bug notification because you are

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread Timo Jyrinki
And this one is the real revert: https://launchpad.net/ubuntu/+source/ubuntu-app-launch/0.4+14.10.20140808.3.is.0.4+14.10.20140605.3-0ubuntu1 It will be on image #200 and onwards. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu.

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread James Hunt
The 'may not create under /run/cgmanager/fs/freezer' errors are caused by the client (session init) cgmanager request falling through may_access() in cgmanager. Here's some debug first from upstart attempting to start a cgroup job by first creating the cgroup: init:cgroup_create:1172:

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread James Hunt
Patch for serge that applies to lp:ubuntu/cgmanager and which was used to collect output in #20. ** Patch added: cgmanager-debug.diff https://bugs.launchpad.net/ubuntu-app-launch/+bug/1357252/+attachment/4181741/+files/cgmanager-debug.diff ** Tags removed: patch -- You received this bug

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread Łukasz Zemczak
** Tags added: lt-whitelisted -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs specifying cgroup fail to start occasionally To manage notifications about this bug go to:

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread James Hunt
There is definitely an upstart bug here as I have a child that has failed to setup cgroups successfully and is spinning, attempting to notify the parent via JOB_PROCESS_ERROR_CGROUP_SETUP. The only reason it is spinning could be that the other end of the pipe is closed, but that shouldn't be

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread Ubuntu Foundations Team Bug Bot
** Tags added: patch -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs specifying cgroup fail to start occasionally To manage notifications about this bug go to:

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread Serge Hallyn
Hi James, regarding comment #20, 1. access is being denied for uid 32011 to get O_RDWR to /run/cgmanager/fs/freezer. I'd expect that to be denied. 2. In either case, could you add to your may_access debug statements a printout of the sb.st_uid, sb.st_gid, and st.mode? -- You received this

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-19 Thread Leo Arias
** Tags added: qa-daily-testing regression -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs specifying cgroup fail to start occasionally To manage notifications about this

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-18 Thread Parameswaran Sivatharman
The debug enabled logs for cgmanager (from /var/log/upstart/cgmanager.lg) is http://paste.ubuntu.com/8080145/plain/ -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-18 Thread James Hunt
** Changed in: upstart Assignee: (unassigned) = James Hunt (jamesodhunt) ** Changed in: upstart Importance: Undecided = Critical -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title:

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-18 Thread James Hunt
@psivaa: thanks - please can you attach the full log as this seems to be just the error lines? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: Upstart jobs specifying cgroup fail to

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-18 Thread Stéphane Graber
On an affected system, can you please post the following: - cat /proc/mounts - cat /proc/cgroups - cat /proc/self/cgroup - cat /proc/$(pidof cgmanager)/cgroup - cat /proc/$(pidof cgmanager)/mounts - find /proc/$(pidof cgmanager)/root/run/cgmanager/ -- You received this bug notification

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-18 Thread James Hunt
I have so far been unable to recreate any cgroup failures at the job level: Upstart is correctly handling the requests and passing them to cgmanager resulting in job processes being contained as expected (even under heavy load). I've just run 'phablet-test-run ubuntuuitoolkit' and this resulted

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-18 Thread Serge Hallyn
@James - sorry, didn't see comment #13 (it takes more than 2 mins to post a comment from here :) So I would suspect UAL is truncating a path. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-18 Thread Serge Hallyn
In the logfile the cgroup name character string seems to always be cut off at 127. I tested cgmanager and cgproxy with creating and moving tasks into cgroups with longer names, but that succeeded. Is it possible that upstart is cutting off the name at 127? -- You received this bug notification

[Bug 1357252] Re: Upstart jobs specifying cgroup fail to start occasionally

2014-08-18 Thread Launchpad Bug Tracker
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: cgmanager (Ubuntu) Status: New = Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1357252 Title: