[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
I ran some quick tests last night and everything looked good. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
Hi Andreas, any update on this? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
Andreas, can you provide any insight on when we might be able to expect this to make it into the repos? I'm not sure how long that process takes. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
Yeah, my pleasure. I updated a server with your packages yesterday and have been testing since, including having a loop run overnight. I've yet to see it get stuck in the sched_yield() loop. So far everything looks much better. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
We have the following packages installed that match the ones you updated so I will just be installing your updates to these: [dfinn@ps-backup-db05 PS SERVER 0 ~]$ dpkg -l|grep -i ruby2 ii libruby2.3:amd64 2.3.1-2~16.04.12 amd64Libraries necessary to run Ruby 2.3 ii ruby2.3 2.3.1-2~16.04.12 amd64Interpreter of object-oriented scripting language Ruby ii ruby2.3-dev:amd64 2.3.1-2~16.04.12 amd64Header files for compiling extension modules for the Ruby 2.3 I was able to get it to happen twice in somewhere around 100 tries so this isn't a great test but I'm going to update one system and continually run it on there. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
Do you want me to install all the .deb packages at that URL? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
OK, I'm playing with that ruby script mentioned above now. I've run it several times across several servers and it's only crashed one, leaving a hung process using 100% CPU. I'm trying to get a feel for how often it does this before I patch a test system. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
I could try the ruby script mentioned in message #22 of this thread: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876377 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
Yes, I will work on that. The only tricky part is that I don't know of a way to manually reproduce this. It seems to happen randomly to us and sometimes infrequently. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] Re: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
Yes, I could test it on a dev box -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1834072] [NEW] Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu.
Public bug reported: Ubuntu 16.04 ruby 2.3.1-2~16.04.12 kernel 4.4.0-148-generic We've noticed an issue across multiple servers where puppet agent will seem to get stuck and consume 100% CPU for days or weeks on end until manually killed. root@ps-prod-jenkins-qa-ui02:~# ps aux|grep -i puppe[t] root 1412 0.0 0.2 143716 38680 ?Ssl Jun11 0:39 /usr/bin/ruby /usr/bin/puppet agent root 34884 74.4 0.3 286848 53724 ?Rs Jun23 1141:44 puppet agent: applying configuration root 111481 94.1 0.3 288572 54996 ?Rs Jun18 8642:32 puppet agent: applying configuration root 128479 54.8 0.3 286744 53596 ?Rs 10:30 250:17 puppet agent: applying configuration Strace shows it in a sched_yield() loop: root@ps-prod-jenkins-qa-ui02:~# strace -p 34884 -c strace: Process 34884 attached ^Cstrace: Process 34884 detached % time seconds usecs/call callserrors syscall -- --- --- - - 100.000.002130 0123189 sched_yield -- --- --- - - 100.000.002130123189 total Some googling shows this is a common issue which was supposedly fixed/backported to ruby 2.3: https://tickets.puppetlabs.com/browse/PA-1743 https://bugs.ruby-lang.org/issues/13794 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=876377 The following open Ubuntu bugs look to be having the same issue and suggest that this fix made it into Debian but never made it into Ubuntu: https://bugs.launchpad.net/ubuntu/+source/pcs/+bug/1783161 https://bugs.launchpad.net/ubuntu/+source/pcs/+bug/1772998 ** Affects: ruby2.3 (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1834072 Title: Puppet agent using 100% CPU, in sched_yield() loop. Looks like an issue with ruby2.3 which has been fixed but not yet made it into Ubuntu. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/ruby2.3/+bug/1834072/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
