As each report from the old_reports table is saved to the new reports schema, it is deleted from the old_reports table. This means you can do this migration in chunks and resume when it's convenient. Recent reports are migrated first under the assumption that more recent data is more relevant. This move is done transactionally so that interrupting the migration won't leave the database in an inconsistent state.
Reviewed-by: Nick Lewis Signed-off-by: Matt Robinson <[email protected]> --- Local-branch: ticket/next/5535 lib/tasks/schematize_reports.rake | 33 +++++++++++++++++++++++++++++++++ 1 files changed, 33 insertions(+), 0 deletions(-) create mode 100644 lib/tasks/schematize_reports.rake diff --git a/lib/tasks/schematize_reports.rake b/lib/tasks/schematize_reports.rake new file mode 100644 index 0000000..436792b --- /dev/null +++ b/lib/tasks/schematize_reports.rake @@ -0,0 +1,33 @@ +namespace :reports do + desc 'Migrate old reports to the new reports schema in reverse chronological order' + task :schematize => :environment do + + class OldReport < ActiveRecord::Base + end + + old_report_count = OldReport.count + + puts + puts "Beginning to migrate #{old_report_count} reports" + puts "Type Ctrl+c at any time to interrupt the migration" + puts "Restarting the migration will resume where you left off" + puts + + require "#{RAILS_ROOT}/lib/progress_bar" + pbar = ProgressBar.new("Migrating:", old_report_count, STDOUT) + + while OldReport.count > 0 do + # Doing records in groups of 10_000 since finding all with millions at once takes forever and eats memory + OldReport.find(:all, :limit => 10_000, :order => "time desc").each do |report| + ActiveRecord::Base.transaction do + Report.create_from_yaml(report.report) + report.destroy + end + pbar.inc + end + end + pbar.finish + + end +end + -- 1.7.3.1 -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en.
