It might be that PuppetDB is running out of heap? Check /var/log/puppetdb for a file 'puppetdb-oom.hprof' for an indiciation this is happening.
You can find instructions for how to adjust your heap space here: https://docs.puppetlabs.com/puppetdb/2.2/configure.html#configuring-the-java-heap-size ken. On Mon, Feb 16, 2015 at 11:23 AM, James Green <[email protected]> wrote: > We have a puppet-master box with the following installed: > > root@puppet-master:/var/log/puppetdb# dpkg -l | grep puppet > ii facter 2.4.1-1puppetlabs1 all > Ruby module for collecting simple facts about a host operating system > ii hiera 1.3.4-1puppetlabs1 all > A simple pluggable Hierarchical Database. > ii puppet 3.7.4-1puppetlabs1 all > Centralized configuration management - agent startup and compatibility > scripts > ii puppet-common 3.7.4-1puppetlabs1 all > Centralized configuration management > ii puppetdb 2.2.2-1puppetlabs1 all > PuppetDB Centralized Storage. > ii puppetdb-terminus 2.2.2-1puppetlabs1 all > Connect Puppet to PuppetDB by setting up a terminus for PuppetDB. > ii puppetlabs-release 1.0-11 all > "Package to install Puppet Labs gpg key and apt repo" > ii puppetmaster-common 3.7.4-1puppetlabs1 all > Puppet master common scripts > ii puppetmaster-passenger 3.7.4-1puppetlabs1 all > Centralised configuration management - master setup to run under mod > passenger > > Occasionally puppetdb is found to no longer be running. The end of the log > says that it is replacing facts. Then syslog shows puppet-master is unable > to "replace-facts" as the connection to puppet-db is refused. The start of > the log when we boot it again states: > > 2015-02-16 10:34:28,202 INFO [p.t.s.w.jetty9-core] Removing buggy security > provider SunPKCS11-NSS version 1.7 > 2015-02-16 10:34:28,537 INFO [p.t.s.w.jetty9-service] Initializing web > server. > 2015-02-16 10:34:28,604 INFO [p.t.s.w.jetty9-service] Starting web server. > 2015-02-16 10:34:28,606 INFO [o.e.j.s.Server] jetty-9.1.z-SNAPSHOT > 2015-02-16 10:34:28,638 INFO [o.e.j.s.ServerConnector] Started > ServerConnector@395ac93f{HTTP/1.1}{localhost:8080} > 2015-02-16 10:34:28,732 INFO [o.e.j.s.ServerConnector] Started > ServerConnector@298d54c9{SSL-HTTP/1.1}{0.0.0.0:8081} > 2015-02-16 10:34:28,787 INFO [c.p.p.c.services] PuppetDB version 2.2.2 > 2015-02-16 10:34:28,792 WARN [c.j.b.BoneCPConfig] JDBC username was not set > in config! > 2015-02-16 10:34:28,792 WARN [c.j.b.BoneCPConfig] JDBC password was not set > in config! > 2015-02-16 10:34:31,650 INFO [c.p.p.s.migrate] There are no pending > migrations > 2015-02-16 10:34:31,650 WARN [c.p.p.s.migrate] Unable to install optimal > indexing > > We are unable to create optimal indexes for your database. > For maximum index performance, we recommend using PostgreSQL 9.3 or > greater. > > 2015-02-16 10:34:31,654 INFO [c.p.p.c.services] Starting broker > 2015-02-16 10:34:31,899 INFO [o.a.a.s.k.MessageDatabase] KahaDB is version > 4 > 2015-02-16 10:34:31,931 INFO [o.a.a.s.k.MessageDatabase] Recovering from > the journal ... > 2015-02-16 10:34:31,931 INFO [o.a.a.s.k.MessageDatabase] Recovery replayed > 2 operations from the journal in 0.026 seconds. > 2015-02-16 10:34:32,455 INFO [c.p.p.c.services] Starting 12 command > processor threads > 2015-02-16 10:34:32,471 WARN [c.j.b.BoneCPConfig] JDBC username was not set > in config! > 2015-02-16 10:34:32,473 WARN [c.j.b.BoneCPConfig] JDBC password was not set > in config! > 2015-02-16 10:34:32,479 INFO [c.p.p.c.services] Starting query server > 2015-02-16 10:34:32,496 WARN [o.e.j.s.h.ContextHandler] Empty contextPath > 2015-02-16 10:34:32,500 INFO [o.e.j.s.h.ContextHandler] Started > o.e.j.s.h.ContextHandler@41ec3132{/,null,AVAILABLE} > 2015-02-16 10:34:32,515 INFO [c.p.p.c.services] Starting sweep of stale > reports (threshold: 14 days) > 2015-02-16 10:34:32,530 INFO [c.p.p.c.services] Finished sweep of stale > reports (threshold: 14 days) > 2015-02-16 10:34:32,531 INFO [c.p.p.c.services] Starting database garbage > collection > 2015-02-16 10:34:32,752 INFO [c.p.p.c.services] Finished database garbage > collection > > And then we're back to replacing facts again. > > Any ideas where we should go from here? > > Thanks, > > James > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-users/CAMH6%2BazCSD-BPD%2ByAO1jV_36bExqhVJ98DBBPk3s3ex4iDVNvg%40mail.gmail.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/CAE4bNTmTwMH%2B8ZU54R%3D-W-LuD%2Bz2qsk8BN16773G_2-9QXyBcQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
