Just out of curiosity, what is your catalog duplication rate? On Tuesday, October 29, 2013 3:26:20 AM UTC+1, David Mesler wrote: > > I reconfigured postgres based on the recommendations from pgtune and your > document. I still had a lot of agent timeouts and eventually after running > overnight the command queue on the puppetdb server was over 4000. Maybe I > need a box with traditional RAID and a lot of spindles instead of the SSD. > Or maybe I need a cluster of postgres servers (if that's possible), I don't > know. The puppetdb docs said a laptop with a consumer grade SSD was enough > for 5000 virtual nodes so I was optimistic this would be a simple setup. Oh > well. > > On Thursday, October 24, 2013 1:02:55 PM UTC-4, Ken Barber wrote: >> >> pgtune is probably a good place to start: >> https://github.com/gregs1104/pgtune ... available as an rpm/deb on the >> more popular distros I believe. >> >> Also, this is probably very premature, but I have a draft doc with >> notes for how to tune your DB for PuppetDB: >> >> >> https://docs.google.com/document/d/1hpFbh2q0WmxAvwfWRlurdaEF70fLc6oZtdktsCq2UFU/edit?usp=sharing >> >> >> Use at your own risk, as it hasn't been completely vetted. Happy to >> get any feedback on this, as I plan on making this part of our >> endorsed documentation. >> >> Also ... there is an index that lately has been causing people >> problems 'idx_catalog_resources_tags_gin'. You might want to try >> dropping it to see if it improves performances (thanks to Erik Dalen >> and his colleagues for that one): >> >> DROP INDEX idx_catalog_resources_tags_gin; >> >> It is easily restored if it doesn't help ... but may take some time to >> build: >> >> CREATE INDEX idx_catalog_resources_tags_gin >> ON catalog_resources >> USING gin >> (tags COLLATE pg_catalog."default"); >> >> ken. >> >> On Thu, Oct 24, 2013 at 4:55 PM, David Mesler <[email protected]> >> wrote: >> > Hello, I'm currently trying to deploy puppetdb to my environment but >> I'm >> > having difficulties and am unsure on how to proceed. >> > I have 1300+ nodes checking in at 15 minute intervals (3.7 million >> resources >> > in the population). The load is spread across 6 puppet masters. I >> > requisitioned what I thought would be a powerful enough machine for the >> > puppetdb/postgres server. A machine with 128GB of RAM, 16 physical cpu >> > cores, and a 500GB ssd for the database. I can point one or two of my >> puppet >> > masters at puppetdb with reasonable enough performance, but anymore and >> > commands start stacking up in the puppetdb command queue and agents >> start >> > timing out. (Actually, even with just one puppet master using puppetdb >> I >> > still have occasional agent timeouts.) Is one postgres server not going >> to >> > cut it? Do I need to look into clustering? I'm sure some of you must >> run >> > puppetdb in larger environments than this, any tips? >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups >> > "Puppet Users" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to [email protected]. >> > To post to this group, send email to [email protected]. >> > Visit this group at http://groups.google.com/group/puppet-users. >> > For more options, visit https://groups.google.com/groups/opt_out. >> >
-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/34a832fd-dcbb-4ffe-ae99-3e0ae80f24cc%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
