Michael, Thanks for this - I have since changed the way we are restarting the unicorn servers after a deploy by changing capistrano task to do:
in :sequence, wait: 30 We have 4 backends and the above will restart them sequentially, waiting 30s (which I think should be more than enough time), however, I still get the following latency spikes after a deploy: http://goo.gl/tYnLUJ This is what the individual servers look like for the same time interval: http://goo.gl/x7KcKq On Tue, Mar 3, 2015 at 2:32 PM, Michael Fischer <[email protected]> wrote: > If the response times are falling a minute or so after the reload, I'd > chalk it up to a cold CPU cache. You will probably want to stagger your > reloads across backends to minimize the impact. > > --Michael > > On Tue, Mar 3, 2015 at 2:24 PM, Sarkis Varozian <[email protected]> > wrote: > >> We have a rails application with the following unicorn.rb: >> http://goo.gl/qZ5NLn >> >> When we deploy to the application, a USR2 signal is sent to the unicorn >> master which spins up a new master and we use the before_fork in the >> unicorn.rb config above to send signals to the old master as the new >> workers come online. >> >> I've been trying to debug a weird issue that manifests as "Request >> Queueing" in our Newrelic APM. The graph shows what happens after a >> deployment (represented by the vertical lines). Here is the graph: >> http://goo.gl/iFZPMv . As you see from the graph, it is inconsistent - >> there is always a latency spike - however, at times Request Queueing is >> higher than previous deploys. >> >> Any ideas on what exactly is going on here? Any suggestions on >> tools/profilers to use to get to the bottom of this? Should we expect this >> to happen on each deploy? >> >> Thanks, >> >> -- >> *Sarkis Varozian* >> [email protected] >> >> >> > -- *Sarkis Varozian* [email protected]
