Issue #11381 has been updated by Jo Rhett.

Eric Shamow wrote:
> Jo Rhett wrote:
> > Yes, there are a half-dozen RPMS, each of which is so convoluted that it 
> > will require a week's worth of evaluation to determine what if anything 
> > each of these people added/changed in each of the various patches.  Not a 
> > single one of these is a clean extract, apply a few patches, install.  They 
> > are instead massive updates with thousands of lines of localization.  The 
> > ruby srpm at karan.org 
> > 
> > * does not compile on centos 5
> > * downloads ruby 1.8.7 and then also a large branch of the ruby trunk and 
> > builds a hybrid thing half from each
> 
> Are you referring specifically to the kbsingh package?  Examining the SRPM, 
> it isn't nearly as messy as that.  It pulls down a small tk branch from trunk 
> based on work done by the Fedora packagers around Ruby's tk module.  

Yes, because Tk windowing system is sooooo important to ruby used for 
automation.  It's just incremental junk.

> The rest of the patches are clearly delineated, explained, and well 
> commented.  And frankly, this is what is necessary to get Ruby running 
> properly on the fairly old CentOS 5 infrastructure.
...
> The fact that you've managed to get Ruby compiled doesn't mean that you are 
> on a well-tested or functional environment.  The well-documented patches in 
> that SRPM resolve many of these issues.  You are welcome to run off of your 
> own native compile, and work through these issues as you hit them.  Trusting 
> community patches is part of participating in the open source community.

You're missing a fairly serious point.  The SRPM does not work.  It would be an 
entirely different thing if it compiled, but it fails to compile on an up to 
date centos release, as well as older releases.  So it's not an issue of 
"evaluating changes of a working thing", but rather "the centos maintainer 
can't manage to put together something which works on EL5 platform, and I need 
to go through all of these patches and figure out what broke it"

> I'm sorry to hear that.  I think when the crunch is off, if you review this 
> thread, you'll find that multiple high-profile community members offered you 
> some ways out of your situation.  If you choose to consider one of them, I 
> think you'll find Puppet to be a stable and performant tool.  Unfortunately, 
> we can only make managing Operating Systems easier a chunk at a time, and 
> replacing the vagaries of packaging is not something we're tackling at this 
> point in time.

This entire thread says "upgrade ruby" and there isn't a simple suggestion made 
which provides a working method to do so.  The fact that absolutely nobody has 
a working method at hand to upgrade ruby cleanly makes a fairly strong argument 
against this.  And honestly, I haven't seen any real analysis of the issue, so 
much as just "upgrade, it will solve all your problems" -- and we both know how 
that goes more often than not.
 
> I don't mean to be rude or flippant in saying this, but welcome to Linux.  I 
> understand from other posts that you are returning from a vendor-controlled 
> OS... in that case you are re-entering a community-driven OS, and that model 
> is quite different.  

FreeBSD?  hahaha.  It's a lot more community driven than Linux, which has both 
a strong kernel core and market shapers like RedHat.  But FreeBSD people do the 
work to ensure that the packages/ports work together, and complaints that ports 
don't compile/build correctly are taken seriously.  In contrast, even with the 
commercial core of Linux, there are dozens of random packages out there with 
nothing in common and so far, none of them work.

> If you are unwilling to trust community patches in the Ruby RPMs assembled by 
> the lead of the CentOS project itself, I wonder why you are willing to trust 
> the rest of CentOS without line-by-line checking it against RHEL, and then 
> against the original sources.  Both RHEL and CentOS add considerable patching 
> and vendor code to the Linux kernel.  Do you go through this process with 
> every package?

If I take this statement seriously, it would appear that you really, truly fail 
to understand basic production values.  The company has a commitment to CentOS, 
and everything is currently working well on the carefully tested and evaluated 
versions of CentOS and various other software that we are running.  But you are 
suggesting that we stop evaluating or testing software, use a bunch of random 
patches (which aren't working at the moment) and throw caution to the wind.  
That's very cute for your home computer, but it's a terrible idea in a 
production environment.

> Again, I hope you'll read back these posts and discover that numerous 
> solutions and suggestions have been made to buy you some time to properly 
> examine your situation and determine an appropriate course of action, and we 
> are of course willing to assist you should you choose to do so.

"Numerous" implies "more than 1".  Let's go back and find *ANY* solution and 
suggestion other than "upgrade ruby" -- without a single suggestion as to where 
to find a working ruby RPM or SRPM.

Note: yes there were some pointers to Passenger tuning, which I already had 
read and already had done before I initially deployed Puppet under Passenger.  
I included my tuning parameters and math calculations used as the basis, to 
which no further suggestions were made.  There was a suggestion about a tool to 
gather more information, which I have downloaded and in fact built an RPM and 
installed everywhere already ;-)  -- helpful for analysis, but that doesn't 
count as a solution.
----------------------------------------
Bug #11381: puppetmaster death spiral under passenger -- document the needs!
https://projects.puppetlabs.com/issues/11381

Author: Jo Rhett
Status: Needs More Information
Priority: Normal
Assignee: Jo Rhett
Category: passenger
Target version: 
Affected Puppet version: 2.6.12
Keywords: 
Branch: 


Having run a cfengine master server that handled 25k clients, I guess I should 
feel spoiled.  But the apparent system requirements for puppetmaster are 
phenomenal.  With a mere 500 nodes we have a dedicated machine with 4 cores, 8 
GB of memory and 6GB of swap, and yet puppetmaster goes into a death spiral 
daily.  There is nothing on this host other than apache, passenger and 
puppetmaster. (and nrpe/nagios test to ensure puppet client is running)

This is what top looks like when it happens:

<pre>
top - 01:18:06 up 1 day,  1:53,  2 users,  load average: 185.70, 148.74, 77.73
Tasks: 379 total, 181 running, 198 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us, 99.8%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.1%hi,  0.1%si,  0.0%st
Mem:   8174508k total,  8132764k used,    41744k free,      524k buffers
Swap:  6094840k total,  6094840k used,        0k free,    19784k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
                                           
 7938 puppet    18   0  216m 100m  648 R 43.0  1.3   0:02.65 ruby               
                                           
31786 puppet    19   0  215m 107m 1724 R 34.1  1.3   2:46.71 ruby               
                                           
  364 root      15   0     0    0    0 S 13.2  0.0   1:21.89 pdflush            
                                           
 7868 puppet    19   0  217m 102m  648 R 11.4  1.3   0:05.21 ruby               
                                           
 8028 root      15   0     0    0    0 S 11.4  0.0   0:21.73 pdflush            
                                           
 7804 puppet    19   0  212m  96m  648 R 11.1  1.2   0:02.38 ruby               
                                           
 7802 puppet    18   0  243m 131m  840 R  7.4  1.6   0:06.40 ruby               
                                           
 7692 puppet    19   0  212m  16m  648 R  7.1  0.2   0:06.10 ruby               
                                           
 7573 puppet    18   0  210m  12m  648 R  6.1  0.2   0:13.12 ruby               
                                           
 7900 puppet    18   0  225m 111m  648 R  6.1  1.4   0:05.88 ruby               
                                           
 7926 puppet    19   0  215m 105m  648 R  6.1  1.3   0:03.42 ruby               
                                           
 7941 puppet    18   0  181m  79m  648 R  6.1  1.0   0:02.68 ruby               
                                           
 7561 puppet    18   0  200m  21m  648 R  5.8  0.3   0:13.21 ruby               
                                           
 7792 puppet    18   0  222m 113m  940 R  4.9  1.4   0:11.08 ruby               
                                           
 8113 root      19   0  102m  896  608 R  4.9  0.0   0:01.40 crond              
                                           
 7902 puppet    18   0  209m 100m  852 R  4.3  1.3   0:04.42 ruby               
                                           
 7429 puppet    18   0  207m  25m  648 R  4.0  0.3   0:10.24 ruby               
                                           
31816 puppet    19   0  225m 117m 1652 R  4.0  1.5   2:28.63 ruby               
                                           
 7685 puppet    18   0  210m  19m  648 R  3.7  0.2   0:10.95 ruby               
                                           
 7918 puppet    18   0  215m 101m  648 R  3.7  1.3   0:03.52 ruby               
                                           
 8121 root      18   0 60476 1144  800 R  3.4  0.0   0:00.73 sshd               
                                           
31825 puppet    18   0  220m 110m 1652 R  3.4  1.4   2:54.23 ruby               
                                           
 7417 puppet    19   0  198m  30m  648 R  3.1  0.4   0:10.72 ruby               
                                           
 7459 puppet    19   0  206m  17m  648 R  3.1  0.2   0:08.91 ruby               
                                           
 7479 puppet    19   0  199m  17m  648 R  3.1  0.2   0:09.01 ruby               
                                           
 7570 puppet    18   0  205m  19m  648 R  3.1  0.2   0:14.22 ruby               
                                           
 7576 puppet    19   0  212m  12m  648 R  3.1  0.2   0:08.61 ruby               
                                           
 7585 puppet    19   0  207m  18m  648 R  3.1  0.2   0:07.44 ruby               
                                           
 7589 puppet    19   0  204m  14m  648 R  3.1  0.2   0:07.00 ruby               
                                           
 7593 puppet    19   0  181m  81m 1548 R  3.1  1.0   0:37.07 ruby               
                                           
 7620 puppet    19   0  210m  17m  648 R  3.1  0.2   0:07.81 ruby               
                                           
 7625 puppet    19   0  209m  21m  648 R  3.1  0.3   0:08.22 ruby               
                                           
 7652 puppet    18   0  164m  10m  648 R  3.1  0.1   0:03.61 ruby               
                                           
 7656 puppet    19   0  213m  35m  648 R  3.1  0.5   0:18.16 ruby               
                                           
 7669 puppet    19   0  204m  23m  648 R  3.1  0.3   0:10.32 ruby               
                                           
 7672 puppet    19   0  207m  14m  648 R  3.1  0.2   0:06.61 ruby               
                                           
 7676 puppet    20   0  205m  17m  648 R  3.1  0.2   0:07.71 ruby               
                                           
 7708 puppet    18   0  208m  16m  648 R  3.1  0.2   0:04.46 ruby               
                                           
 7739 puppet    19   0  221m  14m  648 R  3.1  0.2   0:04.93 ruby               
                                           
 7743 puppet    19   0  212m  34m  648 R  3.1  0.4   0:04.51 ruby               
                                           
 7747 puppet    19   0  207m  25m  648 R  3.1  0.3   0:08.15 ruby               
                                           
 7794 puppet    19   0  213m  41m  648 R  3.1  0.5   0:07.06 ruby               
                                           
 7842 puppet    18   0  211m 100m  648 R  3.1  1.3   0:06.48 ruby               
                                           
 7850 puppet    19   0  212m  96m  852 R  3.1  1.2   0:05.51 ruby               
                                           
 7852 puppet    19   0  212m  95m  648 R  3.1  1.2   0:01.68 ruby               
                                           
 7855 puppet    19   0  209m  97m  924 R  3.1  1.2   0:10.06 ruby               
                                           
 7872 puppet    19   0  214m  97m  852 R  3.1  1.2   0:08.38 ruby   
</pre>

1. Passenger clients are limited to 20.  Where did all these other ruby 
instances come from?  (there is no other ruby code on the system)

2. Why is it willing to spawn until system death?  How can I limit this?

CentOS 5.7 with ruby 1.8.5 and all puppet packages from yum.puppetlabs.com
Passenger 3.0.11 at the moment but we first saw this with passenger 2.2 and 
upgraded without any change in behavior.



-- 
You have received this notification because you have either subscribed to it, 
or are involved in it.
To change your notification preferences, please click here: 
http://projects.puppetlabs.com/my/account

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to 
[email protected].
For more options, visit this group at 
http://groups.google.com/group/puppet-bugs?hl=en.

Reply via email to