Joe Van Dyk <j...@tanga.com> wrote: > Just sent this via gmail. Any ideas why I got this error?
It contained an HTML portion which Mailman couldn't strip or handle. Forwarding to the list for now[1] > ---------- Forwarded message ---------- > From: Joe Van Dyk <j...@tanga.com> > To: unicorn list <mongrel-unicorn@rubyforge.org> > Cc: > Date: Tue, 19 Nov 2013 15:29:27 -0800 > Subject: Zero-downtime deploys, signals, ruby-pg > Hi folks, > > I think the upgrade to ruby-pg 0.16 broke the zero-downtime deployments on > unicorn. > > See mailing list post here: > https://groups.google.com/d/msg/ruby-pg/5_ylGmog1S4/uvQ3EDl7MUUJ > > I think the problem is that unicorn sends a signal to the worker thread > when it's about to be shutdown. ruby-pg receives that signal and > immediately cancels any actively-running queries. Leading to exceptions > like: > > PG::QueryCanceled: ERROR: canceling statement due to user request : UPDATE > "orders" SET... > from deep inside Sequel or ActiveRecord. > > Michael Granger said it's a unicorn problem (apparently). Unfortunately, there's no easy solution in unicorn. ruby-pg is correct in interrupting the running query since it doesn't know if the interrupt signal is for graceful unicorn shutdown or if the user is thinking: "must stop query now or all data is destroyed!" So I suspect ruby-pg favors the panicking user who accidentally issued an UPDATE or DELETE without a proper WHERE clause :) As far as workarounds on the unicorn/rack-side go: 1) retry on PG::QueryCanceled queries, this is analogous to the EINTR retries Ruby does for most interrupted syscalls. However, this takes time and you need to be careful in case you forgot transactions and what not. 2) Avoid SIGQUIT on the unicorn master completely, just send SIGKILL to the master. The workers will slowly realize the master is dead and die gracefully. This could increase RAM usage. _______________________________________________ Unicorn mailing list - mongrel-unicorn@rubyforge.org http://rubyforge.org/mailman/listinfo/mongrel-unicorn Do not quote signatures (like this one) or top post when replying
