Oh that's perfect and works awesome, thank you! On Sunday, October 25, 2015 at 8:20:52 AM UTC-7, Jeremy Evans wrote: > > On Sunday, October 25, 2015 at 8:10:59 AM UTC-7, Adarsh Pandit wrote: >> >> Hello! >> >> I am updating a large number of records (~11k) with responses from an >> external API. >> >> For context, each Company has many Applicants. Applicant has a >> `job_application` JSONB column. I am using Postgres 9.4.5, the >> `sequel-rails` gem, and the most recent Sequel. >> >> I'm running the below and seeing surprising behavior, so the code is: >> >> Applicant.where(job_application: nil).count # => 11,000 >> >> company.applicants_dataset.where(job_application: nil).use_cursor.each >> do |applicant| >> job_application = get_applications_from_external_api(applicant) >> applicant.update(job_application: job_application) >> >> end >> >> # Exit early after a few records update >> Applicant.where(job_application: nil).count # => 11,000 >> >> >> >> I'm seeing rollback of EVERY update made to the `job_application` JSONB >> column in Applicant if the block exits with an error. >> >> My config in `application.rb` is >> >> config.sequel.after_connect = proc do >> Sequel::Model.db.extension :pg_json >> Sequel.extension :pg_json_ops >> Sequel::Model.plugin :timestamps, update_on_create: true >> end >> >> >> >> Feels like this whole thing is run as a transaction, but I have not set >> it do do so anywhere (other than in the `spec_helper`). >> I know this batch update will periodically fail due to API vagaries - any >> help on saving the results if there is an error? >> > > You probably want to pass :hold=>true to use_cursor, otherwise use_cursor > will use a transaction. See > http://sequel.jeremyevans.net/rdoc-adapters/classes/Sequel/Postgres/Dataset.html#method-i-use_cursor > > Thanks, > Jeremy >
-- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/sequel-talk. For more options, visit https://groups.google.com/d/optout.
