Re: [Puppet Users] Re: puppet master and puppet client running very slow

2011-07-05 Thread Peter Meier
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

  File: 1539.03

This indicates clearly a problem in serving files, so does the other
error you posted.

As others asked: Are you still running with Webrick (means default
puppetmaster without setting up passenger/mongrel)?  And how many
clients are connecting to the server how frequently?

~pete
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4SpsUACgkQbwltcAfKi3+AeQCfVKw/DBKcMIlCzMfGK7gF0xpK
0BYAn3ph1iWV8YjO5WxuEzm7bhucvAU+
=Aj9t
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



[Puppet Users] Re: puppet master and puppet client running very slow

2011-07-05 Thread sanjiv.singh
Hi Peter,
thanks for quick reply.
ya i am using Webrick default with puppetmaster.

As per performance guild line,for servicing more nodes it is
recommended to move on passenger/mongrel. surely i will do this.

There are around 100+ servers in my environment.but at most puppet
master serving to 10-12 nodes (varies) concurrently.


Thanks,
Sanjiv Singh
Impetus Info-tech.

On Jul 5, 10:53 am, Peter Meier peter.me...@immerda.ch wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

               File: 1539.03

 This indicates clearly a problem in serving files, so does the other
 error you posted.

 As others asked: Are you still running with Webrick (means default
 puppetmaster without setting up passenger/mongrel)?  And how many
 clients are connecting to the server how frequently?

 ~pete
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.11 (GNU/Linux)
 Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/

 iEYEARECAAYFAk4SpsUACgkQbwltcAfKi3+AeQCfVKw/DBKcMIlCzMfGK7gF0xpK
 0BYAn3ph1iWV8YjO5WxuEzm7bhucvAU+
 =Aj9t
 -END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



[Puppet Users] Re: puppet master and puppet client running very slow

2011-07-05 Thread sanjiv.singh
hi all,
I am using  environment option in /etc/puppet/puppet.conf for
supporting  various releases.
like
.
.
[release1]
modulepath = /usr/local/applications/release1
[release2]
modulepath = /usr/local/applications/release2
[release3]
modulepath = /usr/local/applications/release3
.
.
As i added new release [] with addition of few files/templates and
jars,
it seems following issue coming with only new release [].

what can be various way with puppet , if i wanna to do alot file
serving for puppet client ?


thanks,
Sanjiv Singh
Impetus Infotech


On Jul 5, 12:50 pm, sanjiv.singh sanjiv.si...@impetus.co.in wrote:
 Hi Peter,
 thanks for quick reply.
 ya i am using Webrick default with puppetmaster.

 As per performance guild line,for servicing more nodes it is
 recommended to move on passenger/mongrel. surely i will do this.

 There are around 100+ servers in my environment.but at most puppet
 master serving to 10-12 nodes (varies) concurrently.

 Thanks,
 Sanjiv Singh
 Impetus Info-tech.

 On Jul 5, 10:53 am, Peter Meier peter.me...@immerda.ch wrote:







  -BEGIN PGP SIGNED MESSAGE-
  Hash: SHA1

                File: 1539.03

  This indicates clearly a problem in serving files, so does the other
  error you posted.

  As others asked: Are you still running with Webrick (means default
  puppetmaster without setting up passenger/mongrel)?  And how many
  clients are connecting to the server how frequently?

  ~pete
  -BEGIN PGP SIGNATURE-
  Version: GnuPG v1.4.11 (GNU/Linux)
  Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/

  iEYEARECAAYFAk4SpsUACgkQbwltcAfKi3+AeQCfVKw/DBKcMIlCzMfGK7gF0xpK
  0BYAn3ph1iWV8YjO5WxuEzm7bhucvAU+
  =Aj9t
  -END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Re: puppet master and puppet client running very slow

2011-07-05 Thread Ken Barber
FYI - for performance I would upgrade Puppet (there were performance
improvements in newer revisions), and use something like
Apache/Passenger as a delivery mechanism.

One question - does this happen each time you run it on the same box?
If there is no new changes - is the performance still terrible or is
it only for systems where there is a new change?

I'm just pondering also - did you introduce a new large file? I'm just
wondering if this time is spent doing an md5 sum of a huge file for
file {} comparison. You see the file provider will do an md5
comparison to make sure a a file that already exists doesn't get
installed again. If the file is large - then it takes a long time.
Does this slowness happen even when it appears no changes need to be
made? If you think a file is a culprit you can emulate the md5
behaviour by running 'time md5sum bigassfile.iso' to see if it takes a
long time.

Otherwise its the delivery of such a large file ... which might be the
case, and you definitely should switch out your stack.

I see you have 211 resources or so ... so its not the quantity of
files here that is the problem I would guess.

 There are around 100+ servers in my environment.but at most puppet
 master serving to 10-12 nodes (varies) concurrently.

Webrick at best can manage 1 node concurrently properly. So the minute
you fire up more then 1 it will slow down as it tries to serve each
client when available. Switch to Apache/Mongrel :-).

ken.

On Tue, Jul 5, 2011 at 8:58 AM, sanjiv.singh sanjiv.si...@impetus.co.in wrote:
 hi all,
 I am using  environment option in /etc/puppet/puppet.conf for
 supporting  various releases.
 like
 .
 .
 [release1]
    modulepath = /usr/local/applications/release1
 [release2]
    modulepath = /usr/local/applications/release2
 [release3]
    modulepath = /usr/local/applications/release3
 .
 .
 As i added new release [] with addition of few files/templates and
 jars,
 it seems following issue coming with only new release [].

 what can be various way with puppet , if i wanna to do alot file
 serving for puppet client ?


 thanks,
 Sanjiv Singh
 Impetus Infotech


 On Jul 5, 12:50 pm, sanjiv.singh sanjiv.si...@impetus.co.in wrote:
 Hi Peter,
 thanks for quick reply.
 ya i am using Webrick default with puppetmaster.

 As per performance guild line,for servicing more nodes it is
 recommended to move on passenger/mongrel. surely i will do this.

 There are around 100+ servers in my environment.but at most puppet
 master serving to 10-12 nodes (varies) concurrently.

 Thanks,
 Sanjiv Singh
 Impetus Info-tech.

 On Jul 5, 10:53 am, Peter Meier peter.me...@immerda.ch wrote:







  -BEGIN PGP SIGNED MESSAGE-
  Hash: SHA1

                File: 1539.03

  This indicates clearly a problem in serving files, so does the other
  error you posted.

  As others asked: Are you still running with Webrick (means default
  puppetmaster without setting up passenger/mongrel)?  And how many
  clients are connecting to the server how frequently?

  ~pete
  -BEGIN PGP SIGNATURE-
  Version: GnuPG v1.4.11 (GNU/Linux)
  Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/

  iEYEARECAAYFAk4SpsUACgkQbwltcAfKi3+AeQCfVKw/DBKcMIlCzMfGK7gF0xpK
  0BYAn3ph1iWV8YjO5WxuEzm7bhucvAU+
  =Aj9t
  -END PGP SIGNATURE-

 --
 You received this message because you are subscribed to the Google Groups 
 Puppet Users group.
 To post to this group, send email to puppet-users@googlegroups.com.
 To unsubscribe from this group, send email to 
 puppet-users+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/puppet-users?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



[Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread sanjiv.singh


On Jul 4, 6:12 pm, Daniel Maher dma...@milestonelab.com wrote:
 On 07/04/2011 02:46 PM, sanjiv.singh wrote:

  hi all,
  I have configured puppet 0.25.4 setup.

  since last few days , puppet master and client running very slow .It
  seems puppet client taking soo much time in taking configuration from
  master (compilation manifest ) ,specially for those node(puppet
  client ) for which large manifest class  to be compiled and to be
  deployed .

  what can i do to over come this issue ?

 What sorts of steps have you already taken in order to investigate the
 problem?  Does the Puppetmaster have enough RAM?  It is otherwise
 overloaded from a system load perspective?  What about those two factors
 on the client(s)?  In other words, have you eliminated all of the
 standard system-based considerations?

 If so, I would recommend running the client manually with test and debug
 enabled, and taking a look at the output there.  As well, if it's
 possible, you may wish to do the same for the master daemon - that will
 give you a much better idea of where along the process any potential
 hangs are occurring.

 Good luck !

 --
 Daniel Maher
 makin' plans now to live on Mars 'cuz I got Earth on lock.



Hi Daniel,
I tried  to run puppet client
/usr/sbin/puppetd --test --debug


there are more entries as following :
.
.
.
.
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson


its throwing same entries for long time (in minutes), prior it take
few seconds to get catalog from server and get deployed.

and

even when i fired  puppetrun on master.
it takes long time to trigger puppet client (listening to master).

what cuold be the issue?


thanks,
Sanjiv singh


-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread Ken Barber
How big are these manifests (ie. number of files, total lines for all
files that get imported)? For example - are you pulling in a large
file of all your nodes or something? I recall there has been problems
in the past with parsing large files - its fixed now though.

The obvious suggestion is to upgrade :-).

ken.

On Mon, Jul 4, 2011 at 2:35 PM, sanjiv.singh sanjiv.si...@impetus.co.in wrote:


 On Jul 4, 6:12 pm, Daniel Maher dma...@milestonelab.com wrote:
 On 07/04/2011 02:46 PM, sanjiv.singh wrote:

  hi all,
  I have configured puppet 0.25.4 setup.

  since last few days , puppet master and client running very slow .It
  seems puppet client taking soo much time in taking configuration from
  master (compilation manifest ) ,specially for those node(puppet
  client ) for which large manifest class  to be compiled and to be
  deployed .

  what can i do to over come this issue ?

 What sorts of steps have you already taken in order to investigate the
 problem?  Does the Puppetmaster have enough RAM?  It is otherwise
 overloaded from a system load perspective?  What about those two factors
 on the client(s)?  In other words, have you eliminated all of the
 standard system-based considerations?

 If so, I would recommend running the client manually with test and debug
 enabled, and taking a look at the output there.  As well, if it's
 possible, you may wish to do the same for the master daemon - that will
 give you a much better idea of where along the process any potential
 hangs are occurring.

 Good luck !

 --
 Daniel Maher
 makin' plans now to live on Mars 'cuz I got Earth on lock.



 Hi Daniel,
 I tried  to run puppet client
 /usr/sbin/puppetd --test --debug


 there are more entries as following :
 .
 .
 .
 .
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson


 its throwing same entries for long time (in minutes), prior it take
 few seconds to get catalog from server and get deployed.

 and

 even when i fired  puppetrun on master.
 it takes long time to trigger puppet client (listening to master).

 what cuold be the issue?


 thanks,
 Sanjiv singh


 --
 You received this message because you are subscribed to the Google Groups 
 Puppet Users group.
 To post to this group, send email to puppet-users@googlegroups.com.
 To unsubscribe from this group, send email to 
 puppet-users+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/puppet-users?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



[Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread sanjiv.singh
Hi Ken,

my manifest contains  number of file (flat and templete ) and for
pulling some jars on client.

prior few days , same manifest took 2-3 minutes , now its taking  20
minutes on all puppet clients.

it seems(not sure), puppet client is not bottleneck .
puppet master is an issue , bcuz all puppet client facing same
problem.

and
there is sufficient RAM and storage on puppet master.

and

puppet master hardly serving few (4-5) puppet client at a time in my
environment.

feel free to ask , if some in put require to solve issue

thanks,
Sanjiv Singh

On Jul 4, 6:46 pm, Ken Barber k...@puppetlabs.com wrote:
 How big are these manifests (ie. number of files, total lines for all
 files that get imported)? For example - are you pulling in a large
 file of all your nodes or something? I recall there has been problems
 in the past with parsing large files - its fixed now though.

 The obvious suggestion is to upgrade :-).

 ken.







 On Mon, Jul 4, 2011 at 2:35 PM, sanjiv.singh sanjiv.si...@impetus.co.in 
 wrote:

  On Jul 4, 6:12 pm, Daniel Maher dma...@milestonelab.com wrote:
  On 07/04/2011 02:46 PM, sanjiv.singh wrote:

   hi all,
   I have configured puppet 0.25.4 setup.

   since last few days , puppet master and client running very slow .It
   seems puppet client taking soo much time in taking configuration from
   master (compilation manifest ) ,specially for those node(puppet
   client ) for which large manifest class  to be compiled and to be
   deployed .

   what can i do to over come this issue ?

  What sorts of steps have you already taken in order to investigate the
  problem?  Does the Puppetmaster have enough RAM?  It is otherwise
  overloaded from a system load perspective?  What about those two factors
  on the client(s)?  In other words, have you eliminated all of the
  standard system-based considerations?

  If so, I would recommend running the client manually with test and debug
  enabled, and taking a look at the output there.  As well, if it's
  possible, you may wish to do the same for the master daemon - that will
  give you a much better idea of where along the process any potential
  hangs are occurring.

  Good luck !

  --
  Daniel Maher
  makin' plans now to live on Mars 'cuz I got Earth on lock.

  Hi Daniel,
  I tried  to run puppet client
  /usr/sbin/puppetd --test --debug

  there are more entries as following :
  .
  .
  .
  .
  debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
  yaml; using pson
  debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
  yaml; using pson
  debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
  yaml; using pson

  its throwing same entries for long time (in minutes), prior it take
  few seconds to get catalog from server and get deployed.

  and

  even when i fired  puppetrun on master.
  it takes long time to trigger puppet client (listening to master).

  what cuold be the issue?

  thanks,
  Sanjiv singh

  --
  You received this message because you are subscribed to the Google Groups 
  Puppet Users group.
  To post to this group, send email to puppet-users@googlegroups.com.
  To unsubscribe from this group, send email to 
  puppet-users+unsubscr...@googlegroups.com.
  For more options, visit this group 
  athttp://groups.google.com/group/puppet-users?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread Ken Barber
 my manifest contains  number of file (flat and templete ) and for
 pulling some jars on client.

I guess I was looking to find out the specifics of how big these
manifests and templates are in total on the nodes you are having
specific performance issues with. You say 'big manifests' but this is
a relative statement :-).

If 'nothing has changed' (please confirm this with your svn/git
history) then it must be something else other then content or puppet
itself 

Without seeing your content its a hard one to trace via a mailing
list, there could be a network issue and some function you are trying
to execute could be lagging or having trouble resolving dns etc. Try
running your puppetmasterd in the foreground using something like:

puppetmasterd --no-daemonize --debug --trace --logdest console

And as you do a single run, see if it gives you any clues as to what
its blocking on during those 20 minutes.

ken.

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



[Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread sanjiv.singh


On Jul 4, 7:36 pm, Ken Barber k...@puppetlabs.com wrote:
  my manifest contains  number of file (flat and templete ) and for
  pulling some jars on client.

 I guess I was looking to find out the specifics of how big these
 manifests and templates are in total on the nodes you are having
 specific performance issues with. You say 'big manifests' but this is
 a relative statement :-).

 If 'nothing has changed' (please confirm this with your svn/git
 history) then it must be something else other then content or puppet
 itself 

 Without seeing your content its a hard one to trace via a mailing
 list, there could be a network issue and some function you are trying
 to execute could be lagging or having trouble resolving dns etc. Try
 running your puppetmasterd in the foreground using something like:

 puppetmasterd --no-daemonize --debug --trace --logdest console

 And as you do a single run, see if it gives you any clues as to what
 its blocking on during those 20 minutes.

 ken.

Hi Ken,
thanks for quick reply.

as per yr suggestion ,

sudo /usr/sbin/puppetmasterd --no-daemonize --debug --trace --logdest
console

fired in  puppet master
there is noting  seems wrong in debug  trace, only taking time to
do stuff.

these is some clue for you that may help you to find out root cause.
i have configured puppet master to log in file /var/log/puppet/
masterhttp.log

.
.
.
.
.
[2011-07-04 10:57:18] DEBUG close: 10.122.130.1:45575
[2011-07-04 10:57:24] DEBUG Puppet::Network::HTTP::WEBrickREST is
invoked.
[2011-07-04 10:57:24] 209.251.184.24 - - [04/Jul/2011:10:57:24 EDT]
GET/production/file_metadata/production/tomcat-web.xml HTTP/1.1 200
391
[2011-07-04 10:57:24] - - /production/file_metadata/production/tomcat-
web.xml
[2011-07-04 10:57:24] DEBUG close: 209.251.184.24:20132
[2011-07-04 10:57:24] DEBUG accept: 10.122.130.1:8767
[2011-07-04 10:56:33] ERROR OpenSSL::SSL::SSLError:
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
44:in `accept'
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
44:in `listen'
/usr/lib/ruby/1.8/webrick/server.rb:173:in `call'
/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
42:in `listen'
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
41:in `initialize'
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
41:in `new'
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
41:in `listen'
/usr/lib/ruby/1.8/thread.rb:135:in `synchronize'
/usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
38:in `listen'
/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:131:in
`listen'
/usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:146:in
`start'
/usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:128:in `start'
/usr/lib/ruby/site_ruby/1.8/puppet/application/
puppetmasterd.rb:122:in `main'
/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in
`send'
/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in
`run_command'
/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'
/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:306:in
`exit_on_fail'
/usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'
/usr/sbin/puppetmasterd:66

there are in log  repetted error trace for each file puppet master
serving to client.


thanks,
Sanjiv singh

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



[Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread Luke Bigum
Sanjiv,

How long does your master think that catalog compilation is taking?
Look for these lines:

Jul  4 15:13:53 puppetmaster puppet-master[26138]: Compiled catalog
for puppetclient in environment live in 0.12 seconds

And then the time for the agents applying that catalog:

Jul  4 15:13:55 puppetclient puppet-agent[24198]: Finished catalog run
in 1.22 seconds


-Luke

On Jul 4, 4:13 pm, sanjiv.singh sanjiv.si...@impetus.co.in wrote:
 On Jul 4, 7:36 pm, Ken Barber k...@puppetlabs.com wrote:



   my manifest contains  number of file (flat and templete ) and for
   pulling some jars on client.

  I guess I was looking to find out the specifics of how big these
  manifests and templates are in total on the nodes you are having
  specific performance issues with. You say 'big manifests' but this is
  a relative statement :-).

  If 'nothing has changed' (please confirm this with your svn/git
  history) then it must be something else other then content or puppet
  itself 

  Without seeing your content its a hard one to trace via a mailing
  list, there could be a network issue and some function you are trying
  to execute could be lagging or having trouble resolving dns etc. Try
  running your puppetmasterd in the foreground using something like:

  puppetmasterd --no-daemonize --debug --trace --logdest console

  And as you do a single run, see if it gives you any clues as to what
  its blocking on during those 20 minutes.

  ken.

 Hi Ken,
 thanks for quick reply.

 as per yr suggestion ,

 sudo /usr/sbin/puppetmasterd --no-daemonize --debug --trace --logdest
 console

 fired in  puppet master
 there is noting  seems wrong in debug  trace, only taking time to
 do stuff.

 these is some clue for you that may help you to find out root cause.
 i have configured puppet master to log in file /var/log/puppet/
 masterhttp.log

 .
 .
 .
 .
 .
 [2011-07-04 10:57:18] DEBUG close: 10.122.130.1:45575
 [2011-07-04 10:57:24] DEBUG Puppet::Network::HTTP::WEBrickREST is
 invoked.
 [2011-07-04 10:57:24] 209.251.184.24 - - [04/Jul/2011:10:57:24 EDT]
 GET/production/file_metadata/production/tomcat-web.xml HTTP/1.1 200
 391
 [2011-07-04 10:57:24] - - /production/file_metadata/production/tomcat-
 web.xml
 [2011-07-04 10:57:24] DEBUG close: 209.251.184.24:20132
 [2011-07-04 10:57:24] DEBUG accept: 10.122.130.1:8767
 [2011-07-04 10:56:33] ERROR OpenSSL::SSL::SSLError:
         /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
 44:in `accept'
         /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
 44:in `listen'
         /usr/lib/ruby/1.8/webrick/server.rb:173:in `call'
         /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
         /usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
         /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
         /usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
         /usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
         /usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
         /usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
         /usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
         /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
 42:in `listen'
         /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
 41:in `initialize'
         /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
 41:in `new'
         /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
 41:in `listen'
         /usr/lib/ruby/1.8/thread.rb:135:in `synchronize'
         /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
 38:in `listen'
         /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:131:in
 `listen'
         /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:146:in
 `start'
         /usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:128:in `start'
         /usr/lib/ruby/site_ruby/1.8/puppet/application/
 puppetmasterd.rb:122:in `main'
         /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in
 `send'
         /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in
 `run_command'
         /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'
         /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:306:in
 `exit_on_fail'
         /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'
         /usr/sbin/puppetmasterd:66

 there are in log  repetted error trace for each file puppet master
 serving to client.

 thanks,
 Sanjiv singh

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



[Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread sanjiv.singh


Hi luke,

thanks for quick reply.

as i m able to see that puppet master taking around 7-8 seconds to
compiler catalog
and puppet client taking around 1106 second to apply catalog .

seems strange , because some manifest and configuration (on both
master and client) working fine with dozens of servers concurrently.
Since Sunday getting strange behavior

feel free to ask, if some more input required to resolve d issue.

thanks,
Sanjiv Singh


On Jul 4, 8:32 pm, Luke Bigum luke.bi...@lmax.com wrote:
 Sanjiv,

 How long does your master think that catalog compilation is taking?
 Look for these lines:

 Jul  4 15:13:53 puppetmaster puppet-master[26138]: Compiled catalog
 for puppetclient in environment live in 0.12 seconds

 And then the time for the agents applying that catalog:

 Jul  4 15:13:55 puppetclient puppet-agent[24198]: Finished catalog run
 in 1.22 seconds


 -Luke

 On Jul 4, 4:13 pm, sanjiv.singh sanjiv.si...@impetus.co.in wrote:







  On Jul 4, 7:36 pm, Ken Barber k...@puppetlabs.com wrote:

my manifest contains  number of file (flat and templete ) and for
pulling some jars on client.

   I guess I was looking to find out the specifics of how big these
   manifests and templates are in total on the nodes you are having
   specific performance issues with. You say 'big manifests' but this is
   a relative statement :-).

   If 'nothing has changed' (please confirm this with your svn/git
   history) then it must be something else other then content or puppet
   itself 

   Without seeing your content its a hard one to trace via a mailing
   list, there could be a network issue and some function you are trying
   to execute could be lagging or having trouble resolving dns etc. Try
   running your puppetmasterd in the foreground using something like:

   puppetmasterd --no-daemonize --debug --trace --logdest console

   And as you do a single run, see if it gives you any clues as to what
   its blocking on during those 20 minutes.

   ken.

  Hi Ken,
  thanks for quick reply.

  as per yr suggestion ,

  sudo /usr/sbin/puppetmasterd --no-daemonize --debug --trace --logdest
  console

  fired in  puppet master
  there is noting  seems wrong in debug  trace, only taking time to
  do stuff.

  these is some clue for you that may help you to find out root cause.
  i have configured puppet master to log in file /var/log/puppet/
  masterhttp.log

  .
  .
  .
  .
  .
  [2011-07-04 10:57:18] DEBUG close: 10.122.130.1:45575
  [2011-07-04 10:57:24] DEBUG Puppet::Network::HTTP::WEBrickREST is
  invoked.
  [2011-07-04 10:57:24] 209.251.184.24 - - [04/Jul/2011:10:57:24 EDT]
  GET/production/file_metadata/production/tomcat-web.xml HTTP/1.1 200
  391
  [2011-07-04 10:57:24] - - /production/file_metadata/production/tomcat-
  web.xml
  [2011-07-04 10:57:24] DEBUG close: 209.251.184.24:20132
  [2011-07-04 10:57:24] DEBUG accept: 10.122.130.1:8767
  [2011-07-04 10:56:33] ERROR OpenSSL::SSL::SSLError:
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  44:in `accept'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  44:in `listen'
          /usr/lib/ruby/1.8/webrick/server.rb:173:in `call'
          /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
          /usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
          /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
          /usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
          /usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
          /usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
          /usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
          /usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  42:in `listen'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  41:in `initialize'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  41:in `new'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  41:in `listen'
          /usr/lib/ruby/1.8/thread.rb:135:in `synchronize'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  38:in `listen'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:131:in
  `listen'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:146:in
  `start'
          /usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:128:in `start'
          /usr/lib/ruby/site_ruby/1.8/puppet/application/
  puppetmasterd.rb:122:in `main'
          /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in
  `send'
          /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in
  `run_command'
          /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'
          /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:306:in
  `exit_on_fail'
          /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'
          /usr/sbin/puppetmasterd:66

  there are in log  repetted error trace for 

Re: [Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread Ken Barber
Can you try the puppet run with the setting --summarize and show us
the result ... (not sure if that flag was round then).

ken.

On Mon, Jul 4, 2011 at 5:12 PM, sanjiv.singh sanjiv.si...@impetus.co.in wrote:


 Hi luke,

 thanks for quick reply.

 as i m able to see that puppet master taking around 7-8 seconds to
 compiler catalog
 and puppet client taking around 1106 second to apply catalog .

 seems strange , because some manifest and configuration (on both
 master and client) working fine with dozens of servers concurrently.
 Since Sunday getting strange behavior

 feel free to ask, if some more input required to resolve d issue.

 thanks,
 Sanjiv Singh


 On Jul 4, 8:32 pm, Luke Bigum luke.bi...@lmax.com wrote:
 Sanjiv,

 How long does your master think that catalog compilation is taking?
 Look for these lines:

 Jul  4 15:13:53 puppetmaster puppet-master[26138]: Compiled catalog
 for puppetclient in environment live in 0.12 seconds

 And then the time for the agents applying that catalog:

 Jul  4 15:13:55 puppetclient puppet-agent[24198]: Finished catalog run
 in 1.22 seconds


 -Luke

 On Jul 4, 4:13 pm, sanjiv.singh sanjiv.si...@impetus.co.in wrote:







  On Jul 4, 7:36 pm, Ken Barber k...@puppetlabs.com wrote:

my manifest contains  number of file (flat and templete ) and for
pulling some jars on client.

   I guess I was looking to find out the specifics of how big these
   manifests and templates are in total on the nodes you are having
   specific performance issues with. You say 'big manifests' but this is
   a relative statement :-).

   If 'nothing has changed' (please confirm this with your svn/git
   history) then it must be something else other then content or puppet
   itself 

   Without seeing your content its a hard one to trace via a mailing
   list, there could be a network issue and some function you are trying
   to execute could be lagging or having trouble resolving dns etc. Try
   running your puppetmasterd in the foreground using something like:

   puppetmasterd --no-daemonize --debug --trace --logdest console

   And as you do a single run, see if it gives you any clues as to what
   its blocking on during those 20 minutes.

   ken.

  Hi Ken,
  thanks for quick reply.

  as per yr suggestion ,

  sudo /usr/sbin/puppetmasterd --no-daemonize --debug --trace --logdest
  console

  fired in  puppet master
  there is noting  seems wrong in debug  trace, only taking time to
  do stuff.

  these is some clue for you that may help you to find out root cause.
  i have configured puppet master to log in file /var/log/puppet/
  masterhttp.log

  .
  .
  .
  .
  .
  [2011-07-04 10:57:18] DEBUG close: 10.122.130.1:45575
  [2011-07-04 10:57:24] DEBUG Puppet::Network::HTTP::WEBrickREST is
  invoked.
  [2011-07-04 10:57:24] 209.251.184.24 - - [04/Jul/2011:10:57:24 EDT]
  GET/production/file_metadata/production/tomcat-web.xml HTTP/1.1 200
  391
  [2011-07-04 10:57:24] - - /production/file_metadata/production/tomcat-
  web.xml
  [2011-07-04 10:57:24] DEBUG close: 209.251.184.24:20132
  [2011-07-04 10:57:24] DEBUG accept: 10.122.130.1:8767
  [2011-07-04 10:56:33] ERROR OpenSSL::SSL::SSLError:
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  44:in `accept'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  44:in `listen'
          /usr/lib/ruby/1.8/webrick/server.rb:173:in `call'
          /usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
          /usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
          /usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
          /usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
          /usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
          /usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
          /usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
          /usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  42:in `listen'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  41:in `initialize'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  41:in `new'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  41:in `listen'
          /usr/lib/ruby/1.8/thread.rb:135:in `synchronize'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/http/webrick.rb:
  38:in `listen'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:131:in
  `listen'
          /usr/lib/ruby/site_ruby/1.8/puppet/network/server.rb:146:in
  `start'
          /usr/lib/ruby/site_ruby/1.8/puppet/daemon.rb:128:in `start'
          /usr/lib/ruby/site_ruby/1.8/puppet/application/
  puppetmasterd.rb:122:in `main'
          /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in
  `send'
          /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:226:in
  `run_command'
          /usr/lib/ruby/site_ruby/1.8/puppet/application.rb:217:in `run'
          

Re: [Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread Patrick

On Jul 4, 2011, at 6:35 AM, sanjiv.singh wrote:

 
 
 On Jul 4, 6:12 pm, Daniel Maher dma...@milestonelab.com wrote:
 On 07/04/2011 02:46 PM, sanjiv.singh wrote:
 
 hi all,
 I have configured puppet 0.25.4 setup.
 
 since last few days , puppet master and client running very slow .It
 seems puppet client taking soo much time in taking configuration from
 master (compilation manifest ) ,specially for those node(puppet
 client ) for which large manifest class  to be compiled and to be
 deployed .
 
 what can i do to over come this issue ?
 
 What sorts of steps have you already taken in order to investigate the
 problem?  Does the Puppetmaster have enough RAM?  It is otherwise
 overloaded from a system load perspective?  What about those two factors
 on the client(s)?  In other words, have you eliminated all of the
 standard system-based considerations?
 
 If so, I would recommend running the client manually with test and debug
 enabled, and taking a look at the output there.  As well, if it's
 possible, you may wish to do the same for the master daemon - that will
 give you a much better idea of where along the process any potential
 hangs are occurring.
 
 Good luck !
 
 --
 Daniel Maher
 makin' plans now to live on Mars 'cuz I got Earth on lock.
 
 
 
 Hi Daniel,
 I tried  to run puppet client
 /usr/sbin/puppetd --test --debug
 
 
 there are more entries as following :
 .
 .
 .
 .
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 
 
 its throwing same entries for long time (in minutes), prior it take
 few seconds to get catalog from server and get deployed.

I'm pretty sure those entries you are looking at are file resources.  In 
general, file resources scale very poorly with webrick.  Even without webrick, 
they still tend to perform poorly.

My advice:
Stop using webrick
Take a look at recuse=remote if you use recursive file copies a lot
Make sure you're not swapping and that you have enough ram
Consider copying big files or directories with a lot of files with a package or 
tar+wget.

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



Re: [Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread Peter Meier
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

 as i m able to see that puppet master taking around 7-8 seconds to
 compiler catalog
 and puppet client taking around 1106 second to apply catalog .

run the client with --summarize and see on which resource types it is
spending most of the time, then you can probably figure out what is wrong.

also analyzing the reports with something like the dashboard or the
foreman can help to trace these issues.

Certainly something within your environment or its dependencies has
changed. Puppet is heavily calling out to tools like facter, package
managers, service scripts etc. which all can easily affect your puppet runs.

~pete
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4R+mcACgkQbwltcAfKi3/d/QCeODSqHvF8OxULMcbw6gfuEOT9
noMAn2b0DySrqrZ27kA6Xfvb1WaVWKdX
=pHX/
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
Puppet Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.



[Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread sanjiv.singh


On Jul 4, 10:37 pm, Peter Meier peter.me...@immerda.ch wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

  as i m able to see that puppet master taking around 7-8 seconds to
  compiler catalog
  and puppet client taking around 1106 second to apply catalog .

 run the client with --summarize and see on which resource types it is
 spending most of the time, then you can probably figure out what is wrong.

 also analyzing the reports with something like the dashboard or the
 foreman can help to trace these issues.

 Certainly something within your environment or its dependencies has
 changed. Puppet is heavily calling out to tools like facter, package
 managers, service scripts etc. which all can easily affect your puppet runs.

 ~pete
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.11 (GNU/Linux)
 Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/

 iEYEARECAAYFAk4R+mcACgkQbwltcAfKi3/d/QCeODSqHvF8OxULMcbw6gfuEOT9
 noMAn2b0DySrqrZ27kA6Xfvb1WaVWKdX
 =pHX/
 -END PGP SIGNATURE-


Hi Peter,

thanks for quick reply .

as per yr suggestion , i have tried  --summarize option on both puppet
master and client.

here is all the details,


(on puppet master)

 sudo /usr/sbin/puppetrun -c generic::pb11-8Maintenance --summarize
generic::pb11-8Maintenance: pb11-8.puppet.com
Triggering pb11-8.puppet.com
pb11-8.puppet.com finished with exit code 0
Finished


but , it took around 5-6 minutes only to trigger  puppet client(listen
for puppet master).


(on puppet client )
  sudo /usr/sbin/puppetd --test --debug --summarize
.
.
.
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
memcached.properties]/content: Executing 'diff -u /var/lib/dmb/conf/
memcached.properties /tmp/puppet-diffing.8146.0'
--- /var/lib/dmb/conf/memcached.properties  2011-07-04
12:55:13.0 -0400
+++ /tmp/puppet-diffing.8146.0  2011-07-04 23:13:08.0 -0400
@@ -1,4 +1,4 @@
-memcached.servers=10.122.215.42:11211,10.122.215.41:11211
+memcached.servers=pb11-9:10.122.215.42,pb11-8:10.122.215.41:11211
 memcached.weights=2
 memcached.init.connections=5
 memcached.min.connections=5
debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
memcached.properties]: Changing checksum,content
debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
memcached.properties]: 2 change(s)
debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
memcached.properties]/checksum: Replacing /var/lib/dmb/conf/
memcached.properties checksum {md5}c428ee9bfadf4893267d39585ecac0fd
with {md5}496f78a690517e9d4a3a05c7248de0b2
notice: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
memcached.properties]/checksum: checksum changed '{md5}
c428ee9bfadf4893267d39585ecac0fd' to '{md5}
496f78a690517e9d4a3a05c7248de0b2'
info: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
memcached.properties]: Filebucketed /var/lib/dmb/conf/
memcached.properties to puppet with sum
496f78a690517e9d4a3a05c7248de0b2
debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
memcached.properties]/checksum: Replacing /var/lib/dmb/conf/
memcached.properties checksum {md5}496f78a690517e9d4a3a05c7248de0b2
with {md5}c428ee9bfadf4893267d39585ecac0fd
notice: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
memcached.properties]/content: content changed '{md5}
496f78a690517e9d4a3a05c7248de0b2' to 'unknown checksum'

..
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
yaml; using pson
.
.
.
.
.
debug: Time for triggering 1 events to 

[Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread sanjiv.singh
some times puppet client fails with following...

debug: Loaded state in 0.01 seconds
debug: Using cached certificate for ca
debug: Using cached certificate for pb11-9.domain.local
debug: Using cached certificate_revocation_list for ca
debug: catalog supports formats: b64_zlib_yaml marshal pson raw yaml;
using pson
err: Could not retrieve catalog from remote server: Could not intern
from pson: expected value in object at 'File[/var/lib/'!
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run

thanks,
Sanjiv Singh

On Jul 5, 8:35 am, sanjiv.singh sanjiv.si...@impetus.co.in wrote:
 On Jul 4, 10:37 pm, Peter Meier peter.me...@immerda.ch wrote:









  -BEGIN PGP SIGNED MESSAGE-
  Hash: SHA1

   as i m able to see that puppet master taking around 7-8 seconds to
   compiler catalog
   and puppet client taking around 1106 second to apply catalog .

  run the client with --summarize and see on which resource types it is
  spending most of the time, then you can probably figure out what is wrong.

  also analyzing the reports with something like the dashboard or the
  foreman can help to trace these issues.

  Certainly something within your environment or its dependencies has
  changed. Puppet is heavily calling out to tools like facter, package
  managers, service scripts etc. which all can easily affect your puppet runs.

  ~pete
  -BEGIN PGP SIGNATURE-
  Version: GnuPG v1.4.11 (GNU/Linux)
  Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/

  iEYEARECAAYFAk4R+mcACgkQbwltcAfKi3/d/QCeODSqHvF8OxULMcbw6gfuEOT9
  noMAn2b0DySrqrZ27kA6Xfvb1WaVWKdX
  =pHX/
  -END PGP SIGNATURE-

 Hi Peter,

 thanks for quick reply .

 as per yr suggestion , i have tried  --summarize option on both puppet
 master and client.

 here is all the details,

 (on puppet master)

  sudo /usr/sbin/puppetrun -c generic::pb11-8Maintenance --summarize

 generic::pb11-8Maintenance: pb11-8.puppet.com
 Triggering pb11-8.puppet.com
 pb11-8.puppet.com finished with exit code 0
 Finished

 but , it took around 5-6 minutes only to trigger  puppet client(listen
 for puppet master).

 (on puppet client )
   sudo /usr/sbin/puppetd --test --debug --summarize
 .
 .
 .
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
 memcached.properties]/content: Executing 'diff -u /var/lib/dmb/conf/
 memcached.properties /tmp/puppet-diffing.8146.0'
 --- /var/lib/dmb/conf/memcached.properties      2011-07-04
 12:55:13.0 -0400
 +++ /tmp/puppet-diffing.8146.0  2011-07-04 23:13:08.0 -0400
 @@ -1,4 +1,4 @@
 -memcached.servers=10.122.215.42:11211,10.122.215.41:11211
 +memcached.servers=pb11-9:10.122.215.42,pb11-8:10.122.215.41:11211
  memcached.weights=2
  memcached.init.connections=5
  memcached.min.connections=5
 debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
 memcached.properties]: Changing checksum,content
 debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
 memcached.properties]: 2 change(s)
 debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
 memcached.properties]/checksum: Replacing /var/lib/dmb/conf/
 memcached.properties checksum {md5}c428ee9bfadf4893267d39585ecac0fd
 with {md5}496f78a690517e9d4a3a05c7248de0b2
 notice: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
 memcached.properties]/checksum: checksum changed '{md5}
 c428ee9bfadf4893267d39585ecac0fd' to '{md5}
 496f78a690517e9d4a3a05c7248de0b2'
 info: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
 memcached.properties]: Filebucketed /var/lib/dmb/conf/
 memcached.properties to puppet with sum
 496f78a690517e9d4a3a05c7248de0b2
 debug: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
 memcached.properties]/checksum: Replacing /var/lib/dmb/conf/
 memcached.properties checksum {md5}496f78a690517e9d4a3a05c7248de0b2
 with {md5}c428ee9bfadf4893267d39585ecac0fd
 notice: //templatebasedapplicationconfigs/File[/var/lib/dmb/conf/
 memcached.properties]/content: content changed '{md5}
 496f78a690517e9d4a3a05c7248de0b2' to 'unknown checksum'

 ..
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata supports formats: b64_zlib_yaml marshal pson raw
 yaml; using pson
 debug: file_metadata 

Re: [Puppet Users] Re: puppet master and puppet client running very slow

2011-07-04 Thread vagn scott

Is the client healthy?
File system not full?
No errors in log files, like a disk might be failing?
Load average reasonable?

Any network changes?
Do you have a switch plugged into itself?
what does tcpdump show?  Normal activity?

--
vagn

On 07/05/2011 12:11 AM, sanjiv.singh wrote:

some times puppet client fails with following...

debug: Loaded state in 0.01 seconds
debug: Using cached certificate for ca
debug: Using cached certificate for pb11-9.domain.local
debug: Using cached certificate_revocation_list for ca
debug: catalog supports formats: b64_zlib_yaml marshal pson raw yaml;
using pson
err: Could not retrieve catalog from remote server: Could not intern
from pson: expected value in object at 'File[/var/lib/'!
warning: Not using cache on failed catalog
err: Could not retrieve catalog; skipping run

thanks,
Sanjiv Singh
   


--
You received this message because you are subscribed to the Google Groups Puppet 
Users group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to 
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/puppet-users?hl=en.