I understood, what I wanted to say is that I run a cron job every day for the 2 weeks prune.. and I have less than 2 weeks data in the database right now since I moved it recently to a new server.
I'm running a optimize task, the resource_statuses is really big. On Wed, Jul 24, 2013 at 3:21 PM, Ramin K <[email protected]> wrote: > Doesn't matter, it's the size of the data and the amount you're > deleting. > > Let's say that your reports:prune upto=2 unit=wk is ultimately going to > delete 100k records because you've never pruned the database before. > > reports:prune upto=2 unit=wk, 100k, time: 3600s > > By breaking it up, it'll run faster because large deletes can cause poor > performance in Mysql. > > reports:prune upto=9 unit=wk, deletes: 9k, time: 280s > reports:prune upto=8 unit=wk, deletes: 11k, time: 250s > reports:prune upto=7 unit=wk, deletes: 10k, time: 251s > reports:prune upto=6 unit=wk, deletes: 12k, time: 239s > reports:prune upto=5 unit=wk, deletes: 11k, time: 205s > reports:prune upto=4 unit=wk, deletes: 11k, time: 180s > reports:prune upto=2 unit=wk, deletes: 22k, time: 200s > > Running it nightly with a fully prune data set. > reports:prune upto=2 unit=wk, deletes: 1.5k, time: 25s > > ** I made up all these numbers up, but the performance ratio should be > roughly correct based on past experience. ymmv. ** > > Also the Rake task will load the id of the records it plans to delete into > memory. This can be quite large and is another reason to break it into > smaller chunks. > > Ramin > > > On 7/24/2013 2:08 PM, Felipe Salum wrote: > >> I actually do for 2 weeks. >> >> rake RAILS_ENV=production reports:prune upto=2 unit=wk >> >> >> >> On Wed, Jul 24, 2013 at 2:03 PM, Ramin K <[email protected] >> <mailto:ramin-list@badapple.**net <[email protected]>>> wrote: >> >> On 7/24/2013 1:49 PM, Felipe Salum wrote: >> >> I had a friend helping me to debug and looks like it is taking >> forever >> to delete the entries from resource status table (a lot of >> entries). >> >> Is that something we can improve ? >> >> >> env RAILS_ENV=production rake reports:prune upto=5 unit=day >> env RAILS_ENV=production rake reports:prune:orphaned >> >> You can change the RAILS_ENV and number of day, weeks, months, etc >> to match your system and its needs. >> >> Sounds like you've never pruned so it's going to be slow going. Try >> walking it forward like so. >> >> env RAILS_ENV=production rake reports:prune upto=3 unit=mon >> env RAILS_ENV=production rake reports:prune upto=1 unit=mon >> env RAILS_ENV=production rake reports:prune upto=2 unit=wk >> env RAILS_ENV=production rake reports:prune upto=5 unit=day >> >> Don't forget to add this to your crontab with the retentions >> settings that best fit your system. >> >> Ramin >> >> -- >> You received this message because you are subscribed to a topic in >> the Google Groups "Puppet Users" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/__**topic/puppet-users/7-__** >> sjp1Ef1p0/unsubscribe<https://groups.google.com/d/__topic/puppet-users/7-__sjp1Ef1p0/unsubscribe> >> <https://groups.google.com/d/**topic/puppet-users/7-** >> sjp1Ef1p0/unsubscribe<https://groups.google.com/d/topic/puppet-users/7-sjp1Ef1p0/unsubscribe> >> >. >> >> To unsubscribe from this group and all its topics, send an email to >> puppet-users+unsubscribe@__goo**glegroups.com<http://googlegroups.com> >> >> <mailto:puppet-users%**[email protected]<puppet-users%[email protected]> >> **>. >> >> To post to this group, send email to [email protected] >> <mailto:puppet-users@**googlegroups.com<[email protected]> >> >. >> Visit this group at >> http://groups.google.com/__**group/puppet-users<http://groups.google.com/__group/puppet-users> >> >> <http://groups.google.com/**group/puppet-users<http://groups.google.com/group/puppet-users> >> >. >> For more options, visit >> https://groups.google.com/__**groups/opt_out<https://groups.google.com/__groups/opt_out> >> >> <https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >> >. >> >> >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Puppet Users" group. >> To unsubscribe from this group and stop receiving emails from it, send >> >> an email to >> puppet-users+unsubscribe@**googlegroups.com<puppet-users%[email protected]> >> . >> To post to this group, send email to [email protected]. >> Visit this group at >> http://groups.google.com/**group/puppet-users<http://groups.google.com/group/puppet-users> >> . >> For more options, visit >> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >> . >> >> >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "Puppet Users" group. > To unsubscribe from this topic, visit https://groups.google.com/d/** > topic/puppet-users/7-**sjp1Ef1p0/unsubscribe<https://groups.google.com/d/topic/puppet-users/7-sjp1Ef1p0/unsubscribe> > . > To unsubscribe from this group and all its topics, send an email to > puppet-users+unsubscribe@**googlegroups.com<puppet-users%[email protected]> > . > To post to this group, send email to [email protected]. > Visit this group at > http://groups.google.com/**group/puppet-users<http://groups.google.com/group/puppet-users> > . > For more options, visit > https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> > . > > > -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/puppet-users. For more options, visit https://groups.google.com/groups/opt_out.
