On Mon, 2013-03-04 at 06:09 -0800, jcbollinger wrote: > > > On Sunday, March 3, 2013 3:53:29 AM UTC-6, Brice Figureau wrote: > On 01/03/13 22:24, r.yeo wrote: > > Passenger settings are currently - > > > > PassengerHighPerformance on > > PassengerUseGlobalQueue on > > > > PassengerMaxPoolSize 3 > > This parameter means you're allowing only 3 puppet clients to > talk to > your master at any time. > > I can understand why you've set it up this low, because > essentially a > puppet master is a CPU bound process, so your master host > might not be > able to compile more than 3 catalog at one time. > > > I think you're missing the key characteristics of the problem: > 1. The error does not occur during catalog compilation. That's > fast (enough) in all the failure cases. Instead, the failure > occurs during retrieval of 'sourced' files.
I know that the problems appears while trying to get a file resource. What I was saying is that with such low concurrency allowed on the master side, there are great chances to have large queue forming in the case of several lengthy operations (like catalog compilation), and thus slow metadata retrieval for the other poor node. Let's take a simple example to show what I really meant: Concurrency is set at 3, with 4 nodes. A checks in B checks in a couple of seconds after C checks in a couple of seconds after D has to wait. Now assuming all hosts take the same time to compile, and the machine is able to provide enough CPU time for 3 compilation at the same time, then: A will gets its catalog D can now run, because a free "concurrency" slot is available A will now apply its catalog A requests for file metadata will wait for at least B to finish its catalog execution, because the concurrency is at 3. Rinse & repeat > 1. On failing nodes, all the file retrievals exhibit the same > delay. > 2. File retrieval delays are always within a few milliseconds of > 75s. > If the problem arose from congestion at the master then the delays > would be less consistent from file to file and from run to run, and > they would sometimes affect catalog compilation. Well, I've seen stranger things to happen :) Only the OP can tell us if there is queueing or not, but with this limited concurrency and depending on the number of checking in clients that's certainly something to check. -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-users?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
