Update: 45 hours - ~75% complete On Tuesday, October 7, 2014 1:32:02 PM UTC-7, JonY wrote: > > Running for 7 hours now. Has exported ~15-20% of the data. > > I'm intrigued to see what I end up with. > > > On Tuesday, October 7, 2014 1:19:04 PM UTC-7, Wyatt Alt wrote: >> >> Thanks Jony. >> >> I've loaded up an embedded database to comparable capacity and while >> export isn't quick, it's not nearly as slow as you're experiencing. >> >> Fro your process list the PDB appears to be running with a max heap size >> (Xmx) of 1024m. Perhaps increasing this could make a difference? >> >> The export process appears to be using 192m, but I think that should be >> using the same JAVA_ARGS as PuppetDB itself and so could be a bug. >> >> Wyatt >> >> >> On Tue, Oct 7, 2014 at 11:03 AM, JonY <[email protected]> wrote: >> >>> https://gist.github.com/ce60c590a0531c0b09cd.git >>> >>> >>> # rpm -qa | grep puppet >>> puppet-server-3.7.1-1.el6.noarch >>> puppetlabs-release-6-11.noarch >>> puppetdb-terminus-2.2.0-1.el6.noarch >>> puppet-3.7.1-1.el6.noarch >>> vim-puppet-2.7.20-1.el6.rf.noarch >>> puppetdb-2.2.0-1.el6.noarch >>> >>> Am storing 30 days of data. Yes - it's a fair bit more then the default. >>> >>> >>> >>> >>> On Tuesday, October 7, 2014 9:00:17 AM UTC-7, Wyatt Alt wrote: >>>> >>>> Hey JonY, >>>> >>>> Sounds interesting. What version of PuppetDB are you using? Do you >>>> have reports, facts, and catalogs, or only some of those? Can you paste >>>> your config.ini? >>>> >>>> Also can you give the output of >>>> >>>> ps aux |grep java >>>> top -n1 >>>> free >>>> >>>> in a gist maybe? >>>> >>>> Wyatt >>>> >>>> >>>> >>>> >>>> >>>> On Tue, Oct 7, 2014 at 8:30 AM, JonY <[email protected]> wrote: >>>> >>>>> (ok - terrribly unprofessional title.. I get it). >>>>> >>>>> I'm trying to 'do the right thing' and move from the embedded DB to >>>>> Postgres. Following the instructions I figured I would dump out the >>>>> contents of the embedded DB and import this into Postgres. >>>>> >>>>> So I start 'puppetdb export --outfile <someplace>'. >>>>> >>>>> Pour some coffee. >>>>> >>>>> Watch it dump one node. >>>>> >>>>> Have some more coffee. >>>>> >>>>> Wait. >>>>> >>>>> One more node. >>>>> >>>>> Hours pass. Glaciers melt. >>>>> >>>>> 3+ hours into the process and it has dumped 10 nodes (of > 100). At >>>>> this rate I'm looking at about 1.5 days to get this done. Really? >>>>> >>>>> System has 16 cores, 32Gb of RAM.. barely running at idle. Tell me I >>>>> missed some critical parameter. >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Puppet Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>> msgid/puppet-users/1fbe251a-2c25-42ea-9529-d01bbf561980% >>>>> 40googlegroups.com >>>>> <https://groups.google.com/d/msgid/puppet-users/1fbe251a-2c25-42ea-9529-d01bbf561980%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Puppet Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/puppet-users/9b735460-1722-48dc-9ad5-f34f554567f4%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/puppet-users/9b735460-1722-48dc-9ad5-f34f554567f4%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >>
-- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-users/b0bd8590-0649-4188-a325-fe41168f96bc%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
