On Wednesday, May 9, 2018 at 10:57:03 AM UTC-7, [email protected] wrote: > > ### Complete Description of Issue > > Sequel Version: 4.49.0 > > We've had a read replica (configured via the db `:servers` option) for a > long time, but recently we added a second additional server (in total 3) > and have started seeing a large increase in `ThreadError: deadlock;` > issues. We're not entirely sure why - and digging into it now, but figured > we'd post in here in case it was immediately clear to anyone else. > > It's possible connections to this newest database are slightly slower > since it's not in the AWS network as is the rest of our infrastructure. The > errors predominantly happen on queries to the replica or Citus, but > sometimes also happen in large transactions on our primary node. > > ### Full Backtrace of Exception (if any) > ``` > ThreadError: deadlock; recursive locking > - 14 non-project frames > 1 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/threaded.rb" > line 282 in synchronize > 2 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/threaded.rb" > line 282 in sync > 3 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/extensions/connection_validator.rb" > > line 88 in disconnect_connection > 4 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/sharded_threaded.rb" > > line 353 in remove > 5 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/sharded_threaded.rb" > > line 336 in release > 6 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/sharded_threaded.rb" > > line 138 in block in hold > 7 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/threaded.rb" > line 282 in block in sync > 8 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/threaded.rb" > line 282 in synchronize > 9 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/threaded.rb" > line 282 in sync > 10 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/sharded_threaded.rb" > > line 138 in ensure in hold > 11 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/connection_pool/sharded_threaded.rb" > > line 138 in hold > 12 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/database/connecting.rb" > line 301 in synchronize > 13 > File > "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/adapters/postgres.rb" line > 327 in execute > 14 > File "/usr/local/bundle/gems/sequel-4.49.0/lib/sequel/dataset/actions.rb" > line 1135 in execute > ``` > > Thanks for the great product Jeremy, we're big Sequel fans! >
This definitely looks like a issue in the connection_validator extension, and it is still an issue in the current version. It's also an issue in the connection_expiration extension. Simplest fix would be to use a separate mutex in those extensions, but I think there may be a deeper issue in the threaded connection pool that needs to be fixed. I'll work on this right away, I should have it fixed by tomorrow. Thanks, Jeremy -- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/sequel-talk. For more options, visit https://groups.google.com/d/optout.
