Running for 7 hours now. Has exported ~15-20% of the data. I'm intrigued to see what I end up with.
On Tuesday, October 7, 2014 1:19:04 PM UTC-7, Wyatt Alt wrote: > > Thanks Jony. > > I've loaded up an embedded database to comparable capacity and while > export isn't quick, it's not nearly as slow as you're experiencing. > > Fro your process list the PDB appears to be running with a max heap size > (Xmx) of 1024m. Perhaps increasing this could make a difference? > > The export process appears to be using 192m, but I think that should be > using the same JAVA_ARGS as PuppetDB itself and so could be a bug. > > Wyatt > > > On Tue, Oct 7, 2014 at 11:03 AM, JonY <[email protected] <javascript:>> > wrote: > >> https://gist.github.com/ce60c590a0531c0b09cd.git >> >> >> # rpm -qa | grep puppet >> puppet-server-3.7.1-1.el6.noarch >> puppetlabs-release-6-11.noarch >> puppetdb-terminus-2.2.0-1.el6.noarch >> puppet-3.7.1-1.el6.noarch >> vim-puppet-2.7.20-1.el6.rf.noarch >> puppetdb-2.2.0-1.el6.noarch >> >> Am storing 30 days of data. Yes - it's a fair bit more then the default. >> >> >> >> >> On Tuesday, October 7, 2014 9:00:17 AM UTC-7, Wyatt Alt wrote: >>> >>> Hey JonY, >>> >>> Sounds interesting. What version of PuppetDB are you using? Do you >>> have reports, facts, and catalogs, or only some of those? Can you paste >>> your config.ini? >>> >>> Also can you give the output of >>> >>> ps aux |grep java >>> top -n1 >>> free >>> >>> in a gist maybe? >>> >>> Wyatt >>> >>> >>> >>> >>> >>> On Tue, Oct 7, 2014 at 8:30 AM, JonY <[email protected]> wrote: >>> >>>> (ok - terrribly unprofessional title.. I get it). >>>> >>>> I'm trying to 'do the right thing' and move from the embedded DB to >>>> Postgres. Following the instructions I figured I would dump out the >>>> contents of the embedded DB and import this into Postgres. >>>> >>>> So I start 'puppetdb export --outfile <someplace>'. >>>> >>>> Pour some coffee. >>>> >>>> Watch it dump one node. >>>> >>>> Have some more coffee. >>>> >>>> Wait. >>>> >>>> One more node. >>>> >>>> Hours pass. Glaciers melt. >>>> >>>> 3+ hours into the process and it has dumped 10 nodes (of > 100). At >>>> this rate I'm looking at about 1.5 days to get this done. Really? >>>> >>>> System has 16 cores, 32Gb of RAM.. barely running at idle. Tell me I >>>> missed some critical parameter. >>>> >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Puppet Users" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit https://groups.google.com/d/ >>>> msgid/puppet-users/1fbe251a-2c25-42ea-9529-d01bbf561980% >>>> 40googlegroups.com >>>> <https://groups.google.com/d/msgid/puppet-users/1fbe251a-2c25-42ea-9529-d01bbf561980%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-users/9b735460-1722-48dc-9ad5-f34f554567f4%40googlegroups.com >> >> <https://groups.google.com/d/msgid/puppet-users/9b735460-1722-48dc-9ad5-f34f554567f4%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/032a81c4-adc6-431e-91a8-2037a87e9d53%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
