Re: [Puppet Users] Outofmemory error for puppetserver process
tus >>>>> ◠puppetserver.service - LSB: puppetserver >>>>>Loaded: loaded (/etc/init.d/puppetserver) >>>>>Active: active (running) since Thu 2016-05-26 23:50:01 AEST; 4min >>>>> 55s ago >>>>> Docs: man:systemd-sysv-generator(8) >>>>> Process: 4397 ExecStart=/etc/init.d/puppetserver start (code=exited, >>>>> status=0/SUCCESS) >>>>>CGroup: /system.slice/puppetserver.service >>>>>└─4403 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p >>>>> -Djava.security.egd=/dev/urandom -Xms512m -Xmx512m -XX:MaxPermSize=512m >>>>> -cp >>>>> /opt/puppetlabs/server/apps/puppetserver/puppet-ser... >>>>> >>>>> May 26 23:49:19 jim-Ubuntu1504 systemd[1]: Starting LSB: >>>>> puppetserver... >>>>> May 26 23:50:01 jim-Ubuntu1504 systemd[1]: Started LSB: puppetserver. >>>>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>>>> cat /etc/default/puppetserver >>>>> ### >>>>> # Init settings for puppetserver >>>>> ### >>>>> >>>>> # Location of your Java binary (version 7 or higher) >>>>> JAVA_BIN="/usr/bin/java" >>>>> >>>>> # Modify this if you'd like to change the memory allocation, enable >>>>> JMX, etc >>>>> JAVA_ARGS="-Xms512m -Xmx512m -XX:MaxPermSize=512m" >>>>> >>>>> # These normally shouldn't need to be edited if using OS packages >>>>> USER="puppet" >>>>> GROUP="puppet" >>>>> INSTALL_DIR="/opt/puppetlabs/server/apps/puppetserver" >>>>> CONFIG="/etc/puppetlabs/puppetserver/conf.d" >>>>> BOOTSTRAP_CONFIG="/etc/puppetlabs/puppetserver/bootstrap.cfg" >>>>> SERVICE_STOP_RETRIES=60 >>>>> >>>>> # START_TIMEOUT can be set here to alter the default startup timeout in >>>>> # seconds. This is used in System-V style init scripts only, and will >>>>> have no >>>>> # effect in systemd. >>>>> # START_TIMEOUT=120 >>>>> root@jim-Ubuntu1504 >>>>> :/etc/puppetlabs/code/environments/production/manifests# >>>>> >>>>> Cheers >>>>> Jim >>>>> >>>>> On Wednesday, 25 May 2016 19:56:43 UTC+10, Trevor Vaughan wrote: >>>>>> >>>>>> Hi Jim, >>>>>> >>>>>> Your logs should be in /var/log/puppetserver/puppetserver.log. >>>>>> >>>>>> Also, I'm guessing that you need to change the Xmx java option in >>>>>> your puppetserver start script. >>>>>> >>>>>> Trevor >>>>>> >>>>>> On Tue, May 24, 2016 at 8:51 AM, Jim wrote: >>>>>> >>>>>>> Any idea which log file to look at when I run the service >>>>>>> puppetserver start command? >>>>>>> >>>>>>> >>>>>>> On Tuesday, 24 May 2016 22:44:03 UTC+10, Jim wrote: >>>>>>>> >>>>>>>> Hi Johan, >>>>>>>> >>>>>>>> I did try 512m option not just 256m before posting. and it didn't >>>>>>>> work. The link which you have provided talks basically about >>>>>>>> >>>>>>>> 1. Changing parameters for java options >>>>>>>> 2. max active instances(Its commented out by default) >>>>>>>> 3.num-cpus configuration. I did recursive grep but couldn't find >>>>>>>> it. Puppetlabs says that they ship it anyway with low value so I don't >>>>>>>> think that should be cause or concerb. >>>>>>>> >>>>>>>> I would probably get hold of logs and see if I can find any >>>>>>>> valuable information. >>>>>>>> >>>>>>>> Kind Regards, >>>>>>>> Jim >>>>>>>> >>>>>>>> On Tuesday, 24 May 2016 15:39:05 UTC+10, Johan De Wit wrote: >>>>>>>>> >>>>>>>>> Hi Jim, >>>>>>>>> >>
Re: [Puppet Users] Outofmemory error for puppetserver process
t;>> JAVA_ARGS="-Xms512m -Xmx512m -XX:MaxPermSize=512m" >>>> >>>> # These normally shouldn't need to be edited if using OS packages >>>> USER="puppet" >>>> GROUP="puppet" >>>> INSTALL_DIR="/opt/puppetlabs/server/apps/puppetserver" >>>> CONFIG="/etc/puppetlabs/puppetserver/conf.d" >>>> BOOTSTRAP_CONFIG="/etc/puppetlabs/puppetserver/bootstrap.cfg" >>>> SERVICE_STOP_RETRIES=60 >>>> >>>> # START_TIMEOUT can be set here to alter the default startup timeout in >>>> # seconds. This is used in System-V style init scripts only, and will >>>> have no >>>> # effect in systemd. >>>> # START_TIMEOUT=120 >>>> >>>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>>> >>>> >>>> Cheers >>>> Jim >>>> >>>> On Wednesday, 25 May 2016 19:56:43 UTC+10, Trevor Vaughan wrote: >>>>> >>>>> Hi Jim, >>>>> >>>>> Your logs should be in /var/log/puppetserver/puppetserver.log. >>>>> >>>>> Also, I'm guessing that you need to change the Xmx java option in your >>>>> puppetserver start script. >>>>> >>>>> Trevor >>>>> >>>>> On Tue, May 24, 2016 at 8:51 AM, Jim wrote: >>>>> >>>>>> Any idea which log file to look at when I run the service >>>>>> puppetserver start command? >>>>>> >>>>>> >>>>>> On Tuesday, 24 May 2016 22:44:03 UTC+10, Jim wrote: >>>>>>> >>>>>>> Hi Johan, >>>>>>> >>>>>>> I did try 512m option not just 256m before posting. and it didn't >>>>>>> work. The link which you have provided talks basically about >>>>>>> >>>>>>> 1. Changing parameters for java options >>>>>>> 2. max active instances(Its commented out by default) >>>>>>> 3.num-cpus configuration. I did recursive grep but couldn't find it. >>>>>>> Puppetlabs says that they ship it anyway with low value so I don't >>>>>>> think >>>>>>> that should be cause or concerb. >>>>>>> >>>>>>> I would probably get hold of logs and see if I can find any valuable >>>>>>> information. >>>>>>> >>>>>>> Kind Regards, >>>>>>> Jim >>>>>>> >>>>>>> On Tuesday, 24 May 2016 15:39:05 UTC+10, Johan De Wit wrote: >>>>>>>> >>>>>>>> Hi Jim, >>>>>>>> >>>>>>>> >>>>>>>> You should find some answers here : >>>>>>>> https://docs.puppet.com/puppetserver/2.3/tuning_guide.html >>>>>>>> >>>>>>>> >>>>>>>> per jruby instance, we assign 512mb tot the VM, so 256mb is a >>>>>>>> little too less. >>>>>>>> >>>>>>>> >>>>>>>> hth >>>>>>>> >>>>>>>> >>>>>>>> Johan >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -Original message- >>>>>>>> *From:* Jim >>>>>>>> *Sent:* Tuesday 24th May 2016 4:54 >>>>>>>> *To:* Puppet Users >>>>>>>> *Subject:* [Puppet Users] Outofmemory error for puppetserver >>>>>>>> process >>>>>>>> >>>>>>>> Hey Guys, >>>>>>>> >>>>>>>> I am wondering what is the problem with the following puppetserver >>>>>>>> process and how to have it resolved. >>>>>>>> >>>>>>>> Interesting thing is puppetserver is still running ! and is able to >>>>>>>> communicate it with agents so I am not sure what is the problem ! >>>>>>>> >>>>>>>> I still have 2GB of free memory on my VM >>>>>>>> >>>>>>>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# &
Re: [Puppet Users] Outofmemory error for puppetserver process
6-05-26 23:49:59,868 INFO [async-dispatch-2] [p.t.s.w.jetty9-core] >>> webserver config overridden for key 'ssl-crl-path' >>> 2016-05-26 23:50:00,249 INFO [async-dispatch-2] [p.t.s.w.jetty9-core] >>> Starting web server. >>> 2016-05-26 23:50:00,260 INFO [async-dispatch-2] [o.e.j.s.Server] >>> jetty-9.2.z-SNAPSHOT >>> 2016-05-26 23:50:00,360 INFO [async-dispatch-2] >>> [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@62f22fd >>> {/puppet-ca,null,AVAILABLE} >>> 2016-05-26 23:50:00,360 INFO [async-dispatch-2] >>> [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@dc925ec >>> {/puppet,null,AVAILABLE} >>> 2016-05-26 23:50:00,360 INFO [async-dispatch-2] >>> [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@55c827e3 >>> {/puppet-admin-api,null,AVAILABLE} >>> 2016-05-26 23:50:00,360 INFO [async-dispatch-2] >>> [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@3b82279 >>> {/,null,AVAILABLE} >>> 2016-05-26 23:50:00,472 INFO [async-dispatch-2] >>> [o.e.j.s.ServerConnector] Started ServerConnector@2fcb900d >>> {SSL-HTTP/1.1}{0.0.0.0:8140} >>> 2016-05-26 23:50:00,473 INFO [async-dispatch-2] [o.e.j.s.Server] >>> Started @41120ms >>> 2016-05-26 23:50:00,486 INFO [async-dispatch-2] [p.s.m.master-service] >>> Puppet Server has successfully started and is now ready to handle requests >>> 2016-05-26 23:50:00,488 INFO [async-dispatch-2] >>> [p.s.l.legacy-routes-service] The legacy routing service has successfully >>> started and is now ready to handle requests >>> 2016-05-26 23:50:00,602 INFO [clojure-agent-send-off-pool-0] >>> [p.d.version-check] Newer version 2.4.0 is available! Visit >>> https://github.com/puppetlabs/puppetserver/tree/stable for details. >>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>> service puppetserver status >>> ◠puppetserver.service - LSB: puppetserver >>>Loaded: loaded (/etc/init.d/puppetserver) >>>Active: active (running) since Thu 2016-05-26 23:50:01 AEST; 4min 55s >>> ago >>> Docs: man:systemd-sysv-generator(8) >>> Process: 4397 ExecStart=/etc/init.d/puppetserver start (code=exited, >>> status=0/SUCCESS) >>>CGroup: /system.slice/puppetserver.service >>>└─4403 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p >>> -Djava.security.egd=/dev/urandom -Xms512m -Xmx512m -XX:MaxPermSize=512m -cp >>> /opt/puppetlabs/server/apps/puppetserver/puppet-ser... >>> >>> May 26 23:49:19 jim-Ubuntu1504 systemd[1]: Starting LSB: puppetserver... >>> May 26 23:50:01 jim-Ubuntu1504 systemd[1]: Started LSB: puppetserver. >>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>> cat /etc/default/puppetserver >>> ### >>> # Init settings for puppetserver >>> ### >>> >>> # Location of your Java binary (version 7 or higher) >>> JAVA_BIN="/usr/bin/java" >>> >>> # Modify this if you'd like to change the memory allocation, enable JMX, >>> etc >>> JAVA_ARGS="-Xms512m -Xmx512m -XX:MaxPermSize=512m" >>> >>> # These normally shouldn't need to be edited if using OS packages >>> USER="puppet" >>> GROUP="puppet" >>> INSTALL_DIR="/opt/puppetlabs/server/apps/puppetserver" >>> CONFIG="/etc/puppetlabs/puppetserver/conf.d" >>> BOOTSTRAP_CONFIG="/etc/puppetlabs/puppetserver/bootstrap.cfg" >>> SERVICE_STOP_RETRIES=60 >>> >>> # START_TIMEOUT can be set here to alter the default startup timeout in >>> # seconds. This is used in System-V style init scripts only, and will >>> have no >>> # effect in systemd. >>> # START_TIMEOUT=120 >>> root@jim-Ubuntu1504 >>> :/etc/puppetlabs/code/environments/production/manifests# >>> >>> Cheers >>> Jim >>> >>> On Wednesday, 25 May 2016 19:56:43 UTC+10, Trevor Vaughan wrote: >>>> >>>> Hi Jim, >>>> >>>> Your logs should be in /var/log/puppetserver/puppetserver.log. >>>> >>>> Also, I'm guessing that you need to change the Xmx java option in your >>>> puppetserver start script. >>>> >>>> Trevor >>>> >>>> On Tue, May 24, 2016 at 8:51 AM, Jim wrote: >>>> &
Re: [Puppet Users] Outofmemory error for puppetserver process
2] >> [o.e.j.s.h.ContextHandler] Started >> o.e.j.s.h.ContextHandler@3b82279{/,null,AVAILABLE} >> 2016-05-26 23:50:00,472 INFO [async-dispatch-2] >> [o.e.j.s.ServerConnector] Started ServerConnector@2fcb900d{SSL-HTTP/1.1}{ >> 0.0.0.0:8140} >> 2016-05-26 23:50:00,473 INFO [async-dispatch-2] [o.e.j.s.Server] Started >> @41120ms >> 2016-05-26 23:50:00,486 INFO [async-dispatch-2] [p.s.m.master-service] >> Puppet Server has successfully started and is now ready to handle requests >> 2016-05-26 23:50:00,488 INFO [async-dispatch-2] >> [p.s.l.legacy-routes-service] The legacy routing service has successfully >> started and is now ready to handle requests >> 2016-05-26 23:50:00,602 INFO [clojure-agent-send-off-pool-0] >> [p.d.version-check] Newer version 2.4.0 is available! Visit >> https://github.com/puppetlabs/puppetserver/tree/stable for details. >> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >> service puppetserver status >> ◠puppetserver.service - LSB: puppetserver >>Loaded: loaded (/etc/init.d/puppetserver) >>Active: active (running) since Thu 2016-05-26 23:50:01 AEST; 4min 55s >> ago >> Docs: man:systemd-sysv-generator(8) >> Process: 4397 ExecStart=/etc/init.d/puppetserver start (code=exited, >> status=0/SUCCESS) >>CGroup: /system.slice/puppetserver.service >>└─4403 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p >> -Djava.security.egd=/dev/urandom -Xms512m -Xmx512m -XX:MaxPermSize=512m -cp >> /opt/puppetlabs/server/apps/puppetserver/puppet-ser... >> >> May 26 23:49:19 jim-Ubuntu1504 systemd[1]: Starting LSB: puppetserver... >> May 26 23:50:01 jim-Ubuntu1504 systemd[1]: Started LSB: puppetserver. >> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >> cat /etc/default/puppetserver >> ### >> # Init settings for puppetserver >> ### >> >> # Location of your Java binary (version 7 or higher) >> JAVA_BIN="/usr/bin/java" >> >> # Modify this if you'd like to change the memory allocation, enable JMX, >> etc >> JAVA_ARGS="-Xms512m -Xmx512m -XX:MaxPermSize=512m" >> >> # These normally shouldn't need to be edited if using OS packages >> USER="puppet" >> GROUP="puppet" >> INSTALL_DIR="/opt/puppetlabs/server/apps/puppetserver" >> CONFIG="/etc/puppetlabs/puppetserver/conf.d" >> BOOTSTRAP_CONFIG="/etc/puppetlabs/puppetserver/bootstrap.cfg" >> SERVICE_STOP_RETRIES=60 >> >> # START_TIMEOUT can be set here to alter the default startup timeout in >> # seconds. This is used in System-V style init scripts only, and will >> have no >> # effect in systemd. >> # START_TIMEOUT=120 >> >> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >> >> Cheers >> Jim >> >> On Wednesday, 25 May 2016 19:56:43 UTC+10, Trevor Vaughan wrote: >>> >>> Hi Jim, >>> >>> Your logs should be in /var/log/puppetserver/puppetserver.log. >>> >>> Also, I'm guessing that you need to change the Xmx java option in your >>> puppetserver start script. >>> >>> Trevor >>> >>> On Tue, May 24, 2016 at 8:51 AM, Jim wrote: >>> >>>> Any idea which log file to look at when I run the service puppetserver >>>> start command? >>>> >>>> >>>> On Tuesday, 24 May 2016 22:44:03 UTC+10, Jim wrote: >>>>> >>>>> Hi Johan, >>>>> >>>>> I did try 512m option not just 256m before posting. and it didn't >>>>> work. The link which you have provided talks basically about >>>>> >>>>> 1. Changing parameters for java options >>>>> 2. max active instances(Its commented out by default) >>>>> 3.num-cpus configuration. I did recursive grep but couldn't find it. >>>>> Puppetlabs says that they ship it anyway with low value so I don't think >>>>> that should be cause or concerb. >>>>> >>>>> I would probably get hold of logs and see if I can find any valuable >>>>> information. >>>>> >>>>> Kind Regards, >>>>> Jim >>>>> >>>>> On Tuesday, 24 May 2016 15:39:05 UTC+10, Johan De Wit wrote: >>>>>> >>>>>> Hi Jim
Re: [Puppet Users] Outofmemory error for puppetserver process
2:44:03 UTC+10, Jim wrote: >>>> >>>> Hi Johan, >>>> >>>> I did try 512m option not just 256m before posting. and it didn't work. >>>> The link which you have provided talks basically about >>>> >>>> 1. Changing parameters for java options >>>> 2. max active instances(Its commented out by default) >>>> 3.num-cpus configuration. I did recursive grep but couldn't find it. >>>> Puppetlabs says that they ship it anyway with low value so I don't think >>>> that should be cause or concerb. >>>> >>>> I would probably get hold of logs and see if I can find any valuable >>>> information. >>>> >>>> Kind Regards, >>>> Jim >>>> >>>> On Tuesday, 24 May 2016 15:39:05 UTC+10, Johan De Wit wrote: >>>>> >>>>> Hi Jim, >>>>> >>>>> >>>>> You should find some answers here : >>>>> https://docs.puppet.com/puppetserver/2.3/tuning_guide.html >>>>> >>>>> >>>>> per jruby instance, we assign 512mb tot the VM, so 256mb is a little >>>>> too less. >>>>> >>>>> >>>>> hth >>>>> >>>>> >>>>> Johan >>>>> >>>>> >>>>> >>>>> -Original message- >>>>> *From:* Jim >>>>> *Sent:* Tuesday 24th May 2016 4:54 >>>>> *To:* Puppet Users >>>>> *Subject:* [Puppet Users] Outofmemory error for puppetserver process >>>>> >>>>> Hey Guys, >>>>> >>>>> I am wondering what is the problem with the following puppetserver >>>>> process and how to have it resolved. >>>>> >>>>> Interesting thing is puppetserver is still running ! and is able to >>>>> communicate it with agents so I am not sure what is the problem ! >>>>> >>>>> I still have 2GB of free memory on my VM >>>>> >>>>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>>>> ps -aef|grep -i puppet|grep -v grep >>>>> root 742 1 0 09:39 ?00:00:02 >>>>> /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent >>>>> --no-daemonize >>>>> root 1369 1 0 09:39 ?00:00:08 >>>>> /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/mcollectived >>>>> --config=/etc/puppetlabs/mcollective/server.cfg >>>>> --pidfile=/var/run/puppetlabs/mcollective.pid --daemonize >>>>> puppet8044 1 0 09:53 ?00:01:09 /usr/bin/java >>>>> -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom >>>>> -Xms256M >>>>> -Xmx256M -XX:MaxPermSize=256m -cp >>>>> /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar >>>>> clojure.main -m puppetlabs.trapperkeeper.main --config >>>>> /etc/puppetlabs/puppetserver/conf.d -b >>>>> /etc/puppetlabs/puppetserver/bootstrap.cfg >>>>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>>>> free -g >>>>> total used free sharedbuffers >>>>> cached >>>>> Mem: 3 1 2 0 0 >>>>> 0 >>>>> -/+ buffers/cache: 1 2 >>>>> Swap:3 0 3 >>>>> root@jim-Ubuntu1504 >>>>> :/etc/puppetlabs/code/environments/production/manifests# >>>>> >>>>> >>>>> As you can see I have set puppetserver memory to as low as 256MB. I >>>>> tried to change it to 512MB as well but still it reports "OutOfMemory" so >>>>> kept it on 256MB anyway. >>>>> >>>>> Here is the output which confirms that puppetserver is running. >>>>> >>>>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>>>> service puppetserver status >>>>> â— puppetserver.service - LSB: puppetserver >>>>>Loaded: loaded (/etc/init.d/puppetserver) >>>>>Active: active (running) since Tue 2016-05-24 12:53:06 AEST; 16s ago >>>>> Docs: man:systemd-sysv-generator(8) >>>>> Process: 10986 ExecStart=/etc
Re: [Puppet Users] Outofmemory error for puppetserver process
pet-ca,null,AVAILABLE} 2016-05-26 23:50:00,360 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@dc925ec{/puppet,null,AVAILABLE} 2016-05-26 23:50:00,360 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@55c827e3{/puppet-admin-api,null,AVAILABLE} 2016-05-26 23:50:00,360 INFO [async-dispatch-2] [o.e.j.s.h.ContextHandler] Started o.e.j.s.h.ContextHandler@3b82279{/,null,AVAILABLE} 2016-05-26 23:50:00,472 INFO [async-dispatch-2] [o.e.j.s.ServerConnector] Started ServerConnector@2fcb900d{SSL-HTTP/1.1}{0.0.0.0:8140} 2016-05-26 23:50:00,473 INFO [async-dispatch-2] [o.e.j.s.Server] Started @41120ms 2016-05-26 23:50:00,486 INFO [async-dispatch-2] [p.s.m.master-service] Puppet Server has successfully started and is now ready to handle requests 2016-05-26 23:50:00,488 INFO [async-dispatch-2] [p.s.l.legacy-routes-service] The legacy routing service has successfully started and is now ready to handle requests 2016-05-26 23:50:00,602 INFO [clojure-agent-send-off-pool-0] [p.d.version-check] Newer version 2.4.0 is available! Visit https://github.com/puppetlabs/puppetserver/tree/stable for details. root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# service puppetserver status ◠puppetserver.service - LSB: puppetserver Loaded: loaded (/etc/init.d/puppetserver) Active: active (running) since Thu 2016-05-26 23:50:01 AEST; 4min 55s ago Docs: man:systemd-sysv-generator(8) Process: 4397 ExecStart=/etc/init.d/puppetserver start (code=exited, status=0/SUCCESS) CGroup: /system.slice/puppetserver.service └─4403 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -Xms512m -Xmx512m -XX:MaxPermSize=512m -cp /opt/puppetlabs/server/apps/puppetserver/puppet-ser... May 26 23:49:19 jim-Ubuntu1504 systemd[1]: Starting LSB: puppetserver... May 26 23:50:01 jim-Ubuntu1504 systemd[1]: Started LSB: puppetserver. root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# cat /etc/default/puppetserver ### # Init settings for puppetserver ### # Location of your Java binary (version 7 or higher) JAVA_BIN="/usr/bin/java" # Modify this if you'd like to change the memory allocation, enable JMX, etc JAVA_ARGS="-Xms512m -Xmx512m -XX:MaxPermSize=512m" # These normally shouldn't need to be edited if using OS packages USER="puppet" GROUP="puppet" INSTALL_DIR="/opt/puppetlabs/server/apps/puppetserver" CONFIG="/etc/puppetlabs/puppetserver/conf.d" BOOTSTRAP_CONFIG="/etc/puppetlabs/puppetserver/bootstrap.cfg" SERVICE_STOP_RETRIES=60 # START_TIMEOUT can be set here to alter the default startup timeout in # seconds. This is used in System-V style init scripts only, and will have no # effect in systemd. # START_TIMEOUT=120 root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# Cheers Jim On Wednesday, 25 May 2016 19:56:43 UTC+10, Trevor Vaughan wrote: > > Hi Jim, > > Your logs should be in /var/log/puppetserver/puppetserver.log. > > Also, I'm guessing that you need to change the Xmx java option in your > puppetserver start script. > > Trevor > > On Tue, May 24, 2016 at 8:51 AM, Jim > > wrote: > >> Any idea which log file to look at when I run the service puppetserver >> start command? >> >> >> On Tuesday, 24 May 2016 22:44:03 UTC+10, Jim wrote: >>> >>> Hi Johan, >>> >>> I did try 512m option not just 256m before posting. and it didn't work. >>> The link which you have provided talks basically about >>> >>> 1. Changing parameters for java options >>> 2. max active instances(Its commented out by default) >>> 3.num-cpus configuration. I did recursive grep but couldn't find it. >>> Puppetlabs says that they ship it anyway with low value so I don't think >>> that should be cause or concerb. >>> >>> I would probably get hold of logs and see if I can find any valuable >>> information. >>> >>> Kind Regards, >>> Jim >>> >>> On Tuesday, 24 May 2016 15:39:05 UTC+10, Johan De Wit wrote: >>>> >>>> Hi Jim, >>>> >>>> >>>> You should find some answers here : >>>> https://docs.puppet.com/puppetserver/2.3/tuning_guide.html >>>> >>>> >>>> per jruby instance, we assign 512mb tot the VM, so 256mb is a little >>>> too less. >>>> >>>> >>>> hth >>>> >>>> >>>> Johan >>>> >>>> >>>> >>>> -Original message- >>>&
Re: [Puppet Users] Outofmemory error for puppetserver process
Hi Jim, Your logs should be in /var/log/puppetserver/puppetserver.log. Also, I'm guessing that you need to change the Xmx java option in your puppetserver start script. Trevor On Tue, May 24, 2016 at 8:51 AM, Jim wrote: > Any idea which log file to look at when I run the service puppetserver > start command? > > > On Tuesday, 24 May 2016 22:44:03 UTC+10, Jim wrote: >> >> Hi Johan, >> >> I did try 512m option not just 256m before posting. and it didn't work. >> The link which you have provided talks basically about >> >> 1. Changing parameters for java options >> 2. max active instances(Its commented out by default) >> 3.num-cpus configuration. I did recursive grep but couldn't find it. >> Puppetlabs says that they ship it anyway with low value so I don't think >> that should be cause or concerb. >> >> I would probably get hold of logs and see if I can find any valuable >> information. >> >> Kind Regards, >> Jim >> >> On Tuesday, 24 May 2016 15:39:05 UTC+10, Johan De Wit wrote: >>> >>> Hi Jim, >>> >>> >>> You should find some answers here : >>> https://docs.puppet.com/puppetserver/2.3/tuning_guide.html >>> >>> >>> per jruby instance, we assign 512mb tot the VM, so 256mb is a little too >>> less. >>> >>> >>> hth >>> >>> >>> Johan >>> >>> >>> >>> -Original message- >>> *From:* Jim >>> *Sent:* Tuesday 24th May 2016 4:54 >>> *To:* Puppet Users >>> *Subject:* [Puppet Users] Outofmemory error for puppetserver process >>> >>> Hey Guys, >>> >>> I am wondering what is the problem with the following puppetserver >>> process and how to have it resolved. >>> >>> Interesting thing is puppetserver is still running ! and is able to >>> communicate it with agents so I am not sure what is the problem ! >>> >>> I still have 2GB of free memory on my VM >>> >>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>> ps -aef|grep -i puppet|grep -v grep >>> root 742 1 0 09:39 ?00:00:02 >>> /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent >>> --no-daemonize >>> root 1369 1 0 09:39 ?00:00:08 >>> /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/mcollectived >>> --config=/etc/puppetlabs/mcollective/server.cfg >>> --pidfile=/var/run/puppetlabs/mcollective.pid --daemonize >>> puppet8044 1 0 09:53 ?00:01:09 /usr/bin/java >>> -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -Xms256M >>> -Xmx256M -XX:MaxPermSize=256m -cp >>> /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar >>> clojure.main -m puppetlabs.trapperkeeper.main --config >>> /etc/puppetlabs/puppetserver/conf.d -b >>> /etc/puppetlabs/puppetserver/bootstrap.cfg >>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>> free -g >>> total used free sharedbuffers cached >>> Mem: 3 1 2 0 0 0 >>> -/+ buffers/cache: 1 2 >>> Swap:3 0 3 >>> root@jim-Ubuntu1504 >>> :/etc/puppetlabs/code/environments/production/manifests# >>> >>> >>> As you can see I have set puppetserver memory to as low as 256MB. I >>> tried to change it to 512MB as well but still it reports "OutOfMemory" so >>> kept it on 256MB anyway. >>> >>> Here is the output which confirms that puppetserver is running. >>> >>> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >>> service puppetserver status >>> ◠puppetserver.service - LSB: puppetserver >>>Loaded: loaded (/etc/init.d/puppetserver) >>>Active: active (running) since Tue 2016-05-24 12:53:06 AEST; 16s ago >>> Docs: man:systemd-sysv-generator(8) >>> Process: 10986 ExecStart=/etc/init.d/puppetserver start (code=exited, >>> status=0/SUCCESS) >>>CGroup: /system.slice/puppetserver.service >>>└─10992 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p >>> -Djava.security.egd=/dev/urandom -Xms256M -Xmx256M -XX:MaxPermSize=256m -cp >>> /opt/puppetlabs/server/apps/puppetserver/puppet-se... >>> >>> May 24 12:52:22 j
Re: [Puppet Users] Outofmemory error for puppetserver process
Any idea which log file to look at when I run the service puppetserver start command? On Tuesday, 24 May 2016 22:44:03 UTC+10, Jim wrote: > > Hi Johan, > > I did try 512m option not just 256m before posting. and it didn't work. > The link which you have provided talks basically about > > 1. Changing parameters for java options > 2. max active instances(Its commented out by default) > 3.num-cpus configuration. I did recursive grep but couldn't find it. > Puppetlabs says that they ship it anyway with low value so I don't think > that should be cause or concerb. > > I would probably get hold of logs and see if I can find any valuable > information. > > Kind Regards, > Jim > > On Tuesday, 24 May 2016 15:39:05 UTC+10, Johan De Wit wrote: >> >> Hi Jim, >> >> >> You should find some answers here : >> https://docs.puppet.com/puppetserver/2.3/tuning_guide.html >> >> >> per jruby instance, we assign 512mb tot the VM, so 256mb is a little too >> less. >> >> >> hth >> >> >> Johan >> >> >> >> -Original message- >> *From:* Jim >> *Sent:* Tuesday 24th May 2016 4:54 >> *To:* Puppet Users >> *Subject:* [Puppet Users] Outofmemory error for puppetserver process >> >> Hey Guys, >> >> I am wondering what is the problem with the following puppetserver >> process and how to have it resolved. >> >> Interesting thing is puppetserver is still running ! and is able to >> communicate it with agents so I am not sure what is the problem ! >> >> I still have 2GB of free memory on my VM >> >> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >> ps -aef|grep -i puppet|grep -v grep >> root 742 1 0 09:39 ?00:00:02 >> /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent >> --no-daemonize >> root 1369 1 0 09:39 ?00:00:08 >> /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/mcollectived >> --config=/etc/puppetlabs/mcollective/server.cfg >> --pidfile=/var/run/puppetlabs/mcollective.pid --daemonize >> puppet8044 1 0 09:53 ?00:01:09 /usr/bin/java >> -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -Xms256M >> -Xmx256M -XX:MaxPermSize=256m -cp >> /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar >> clojure.main -m puppetlabs.trapperkeeper.main --config >> /etc/puppetlabs/puppetserver/conf.d -b >> /etc/puppetlabs/puppetserver/bootstrap.cfg >> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >> free -g >> total used free sharedbuffers cached >> Mem: 3 1 2 0 0 0 >> -/+ buffers/cache: 1 2 >> Swap:3 0 3 >> >> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >> >> >> As you can see I have set puppetserver memory to as low as 256MB. I tried >> to change it to 512MB as well but still it reports "OutOfMemory" so kept it >> on 256MB anyway. >> >> Here is the output which confirms that puppetserver is running. >> >> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >> service puppetserver status >> ◠puppetserver.service - LSB: puppetserver >>Loaded: loaded (/etc/init.d/puppetserver) >>Active: active (running) since Tue 2016-05-24 12:53:06 AEST; 16s ago >> Docs: man:systemd-sysv-generator(8) >> Process: 10986 ExecStart=/etc/init.d/puppetserver start (code=exited, >> status=0/SUCCESS) >>CGroup: /system.slice/puppetserver.service >>└─10992 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p >> -Djava.security.egd=/dev/urandom -Xms256M -Xmx256M -XX:MaxPermSize=256m -cp >> /opt/puppetlabs/server/apps/puppetserver/puppet-se... >> >> May 24 12:52:22 jim-Ubuntu1504 systemd[1]: Starting LSB: puppetserver... >> May 24 12:53:06 jim-Ubuntu1504 systemd[1]: Started LSB: puppetserver. >> >> root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# >> >> Any advice would be much appreciated. >> >> Cheers >> Jim >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to puppet-users...@googlegroups.com. >> To view this discus
Re: [Puppet Users] Outofmemory error for puppetserver process
Hi Johan, I did try 512m option not just 256m before posting. and it didn't work. The link which you have provided talks basically about 1. Changing parameters for java options 2. max active instances(Its commented out by default) 3.num-cpus configuration. I did recursive grep but couldn't find it. Puppetlabs says that they ship it anyway with low value so I don't think that should be cause or concerb. I would probably get hold of logs and see if I can find any valuable information. Kind Regards, Jim On Tuesday, 24 May 2016 15:39:05 UTC+10, Johan De Wit wrote: > > Hi Jim, > > > You should find some answers here : > https://docs.puppet.com/puppetserver/2.3/tuning_guide.html > > > per jruby instance, we assign 512mb tot the VM, so 256mb is a little too > less. > > > hth > > > Johan > > > > -Original message- > *From:* Jim > > *Sent:* Tuesday 24th May 2016 4:54 > *To:* Puppet Users > > *Subject:* [Puppet Users] Outofmemory error for puppetserver process > > Hey Guys, > > I am wondering what is the problem with the following puppetserver process > and how to have it resolved. > > Interesting thing is puppetserver is still running ! and is able to > communicate it with agents so I am not sure what is the problem ! > > I still have 2GB of free memory on my VM > > root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# > ps -aef|grep -i puppet|grep -v grep > root 742 1 0 09:39 ?00:00:02 > /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent > --no-daemonize > root 1369 1 0 09:39 ?00:00:08 > /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/mcollectived > --config=/etc/puppetlabs/mcollective/server.cfg > --pidfile=/var/run/puppetlabs/mcollective.pid --daemonize > puppet8044 1 0 09:53 ?00:01:09 /usr/bin/java > -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -Xms256M > -Xmx256M -XX:MaxPermSize=256m -cp > /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar > clojure.main -m puppetlabs.trapperkeeper.main --config > /etc/puppetlabs/puppetserver/conf.d -b > /etc/puppetlabs/puppetserver/bootstrap.cfg > root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# > free -g > total used free sharedbuffers cached > Mem: 3 1 2 0 0 0 > -/+ buffers/cache: 1 2 > Swap:3 0 3 > > root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# > > > As you can see I have set puppetserver memory to as low as 256MB. I tried > to change it to 512MB as well but still it reports "OutOfMemory" so kept it > on 256MB anyway. > > Here is the output which confirms that puppetserver is running. > > root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# > service puppetserver status > ◠puppetserver.service - LSB: puppetserver >Loaded: loaded (/etc/init.d/puppetserver) >Active: active (running) since Tue 2016-05-24 12:53:06 AEST; 16s ago > Docs: man:systemd-sysv-generator(8) > Process: 10986 ExecStart=/etc/init.d/puppetserver start (code=exited, > status=0/SUCCESS) >CGroup: /system.slice/puppetserver.service >└─10992 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p > -Djava.security.egd=/dev/urandom -Xms256M -Xmx256M -XX:MaxPermSize=256m -cp > /opt/puppetlabs/server/apps/puppetserver/puppet-se... > > May 24 12:52:22 jim-Ubuntu1504 systemd[1]: Starting LSB: puppetserver... > May 24 12:53:06 jim-Ubuntu1504 systemd[1]: Started LSB: puppetserver. > > root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# > > Any advice would be much appreciated. > > Cheers > Jim > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to puppet-users...@googlegroups.com . > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-users/60c8e923-10ca-4d01-a09c-7e87a5036fdb%40googlegroups.com > > <https://groups.google.com/d/msgid/puppet-users/60c8e923-10ca-4d01-a09c-7e87a5036fdb%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/7473f693-bbb1-47a2-b694-b59c5a8e21da%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
RE: [Puppet Users] Outofmemory error for puppetserver process
Hi Jim, You should find some answers here : https://docs.puppet.com/puppetserver/2.3/tuning_guide.html per jruby instance, we assign 512mb tot the VM, so 256mb is a little too less. hth Johan -Original message- From: Jim Sent: Tuesday 24th May 2016 4:54 To: Puppet Users Subject: [Puppet Users] Outofmemory error for puppetserver process Hey Guys, I am wondering what is the problem with the following puppetserver process and how to have it resolved. Interesting thing is puppetserver is still running ! and is able to communicate it with agents so I am not sure what is the problem ! I still have 2GB of free memory on my VM root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# ps -aef|grep -i puppet|grep -v grep root 742 1 0 09:39 ? 00:00:02 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent --no-daemonize root 1369 1 0 09:39 ? 00:00:08 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/mcollectived --config=/etc/puppetlabs/mcollective/server.cfg --pidfile=/var/run/puppetlabs/mcollective.pid --daemonize puppet 8044 1 0 09:53 ? 00:01:09 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -Xms256M -Xmx256M -XX:MaxPermSize=256m -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d -b /etc/puppetlabs/puppetserver/bootstrap.cfg root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# free -g total used free shared buffers cached Mem: 3 1 2 0 0 0 -/+ buffers/cache: 1 2 Swap: 3 0 3 root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# As you can see I have set puppetserver memory to as low as 256MB. I tried to change it to 512MB as well but still it reports "OutOfMemory" so kept it on 256MB anyway. Here is the output which confirms that puppetserver is running. root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# service puppetserver status ◠puppetserver.service - LSB: puppetserver Loaded: loaded (/etc/init.d/puppetserver) Active: active (running) since Tue 2016-05-24 12:53:06 AEST; 16s ago Docs: man:systemd-sysv-generator(8) Process: 10986 ExecStart=/etc/init.d/puppetserver start (code=exited, status=0/SUCCESS) CGroup: /system.slice/puppetserver.service └─10992 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -Xms256M -Xmx256M -XX:MaxPermSize=256m -cp /opt/puppetlabs/server/apps/puppetserver/puppet-se... May 24 12:52:22 jim-Ubuntu1504 systemd[1]: Starting LSB: puppetserver... May 24 12:53:06 jim-Ubuntu1504 systemd[1]: Started LSB: puppetserver. root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# Any advice would be much appreciated. Cheers Jim -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com <mailto:puppet-users+unsubscr...@googlegroups.com> . To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/60c8e923-10ca-4d01-a09c-7e87a5036fdb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/zarafa.5743e8e3.4d32.16d28aed05678ec2%40zarafa.open-future.be. For more options, visit https://groups.google.com/d/optout.
[Puppet Users] Outofmemory error for puppetserver process
Hey Guys, I am wondering what is the problem with the following puppetserver process and how to have it resolved. Interesting thing is puppetserver is still running ! and is able to communicate it with agents so I am not sure what is the problem ! I still have 2GB of free memory on my VM root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# ps -aef|grep -i puppet|grep -v grep root 742 1 0 09:39 ?00:00:02 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/puppet agent --no-daemonize root 1369 1 0 09:39 ?00:00:08 /opt/puppetlabs/puppet/bin/ruby /opt/puppetlabs/puppet/bin/mcollectived --config=/etc/puppetlabs/mcollective/server.cfg --pidfile=/var/run/puppetlabs/mcollective.pid --daemonize puppet8044 1 0 09:53 ?00:01:09 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -Xms256M -Xmx256M -XX:MaxPermSize=256m -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d -b /etc/puppetlabs/puppetserver/bootstrap.cfg root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# free -g total used free sharedbuffers cached Mem: 3 1 2 0 0 0 -/+ buffers/cache: 1 2 Swap:3 0 3 root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# As you can see I have set puppetserver memory to as low as 256MB. I tried to change it to 512MB as well but still it reports "OutOfMemory" so kept it on 256MB anyway. Here is the output which confirms that puppetserver is running. root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# service puppetserver status ◠puppetserver.service - LSB: puppetserver Loaded: loaded (/etc/init.d/puppetserver) Active: active (running) since Tue 2016-05-24 12:53:06 AEST; 16s ago Docs: man:systemd-sysv-generator(8) Process: 10986 ExecStart=/etc/init.d/puppetserver start (code=exited, status=0/SUCCESS) CGroup: /system.slice/puppetserver.service └─10992 /usr/bin/java -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/dev/urandom -Xms256M -Xmx256M -XX:MaxPermSize=256m -cp /opt/puppetlabs/server/apps/puppetserver/puppet-se... May 24 12:52:22 jim-Ubuntu1504 systemd[1]: Starting LSB: puppetserver... May 24 12:53:06 jim-Ubuntu1504 systemd[1]: Started LSB: puppetserver. root@jim-Ubuntu1504:/etc/puppetlabs/code/environments/production/manifests# Any advice would be much appreciated. Cheers Jim -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/60c8e923-10ca-4d01-a09c-7e87a5036fdb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.