I wanted to thank you also - I was in the process of upgrading a Rails app from Passenger 5.3.3 to 6.0.2 when a health-check endpoint started issuing timeouts.
This thread confirmed my analysis. FWIW, in case it helps someone later: This app config/initializers/sequel.rb creates the connection: SEQUEL_DB = Sequel.connect(sequel_config) SEQUEL_DB.extension :pg_array Just adding this below fixes the issue: if defined?(PhusionPassenger) Rails.logger.info "Passenger detected - disconnecting during app preload" # NOTE: after the fork occurs, the next operation on SEQUEL_DB will re-open it gently SEQUEL_DB.disconnect end (message first posted, then deleted & reposted to avoid a spammy url link on rails logger info) -- Thibaut Le mardi 9 avril 2019 12:54:06 UTC+2, Gustavo Sobral a écrit : > > Thanks a lot, Jeremy and James! I did the changes in my app and I'm > monitoring to see if new exceptions will happen, so far so good. I oversaw > these configurations. > > On Tuesday, April 9, 2019 at 8:22:40 AM UTC+2, James Silberbauer wrote: >> >> >> >> On 2019/04/09 02:04, Jeremy Evans wrote: >> >> On Monday, April 8, 2019 at 1:22:48 PM UTC-7, James Silberbauer wrote: >>> >>> >>> On 2019/04/08 20:51, Jeremy Evans wrote: >>> >>> On Monday, April 8, 2019 at 6:57:50 AM UTC-7, Gustavo Sobral wrote: >>>> >>>> Hi, >>>> >>>> I have a Sinatra app with Sequel and Postgres adapter running in a >>>> Phusion Passenger server and I'm facing around 50~60 >>>> Sequel::DatabaseDisconnectError exceptions a day. These exceptions have >>>> mainly three messages on it: >>>> >>>> PG::UnableToSend: SSL error: decryption failed or bad record mac >>>> PG::UnableToSend: SSL SYSCALL error: EOF detected >>>> PG::UnableToSend: PQconsumeInput() SSL error: decryption failed or bad >>>> record mac >>>> >>>> This is my file responsible for initializing Sequel, I've already tried >>>> without success to use the connection_validation extension: >>>> >>>> # frozen_string_literal: true >>>> >>>> # TZ settings >>>> Sequel.application_timezone = 'Amsterdam' >>>> Sequel.database_timezone = :utc >>>> >>>> # Json serializer plugin >>>> Sequel::Model.plugin :json_serializer >>>> >>>> # Global 'DB' handle is a Sequel convention >>>> db_config = YAML.load_file(File.join(Dir.pwd, 'config/database.yml'))[ >>>> ENV['RACK_ENV']] >>>> DB = Sequel.connect(db_config) >>>> >>>> # Validate that connections checked out from the pool are still valid, >>>> before yielding them for use >>>> # >>>> http://sequel.jeremyevans.net/rdoc-plugins/files/lib/sequel/extensions/connection_validator_rb.html >>>> DB.extension(:connection_validator >>>> DB.pool.connection_validation_timeout = -1 >>>> >>>> Can anybody shed some light what can possibly be going on here? I'm >>>> using the free version of Phusion Passenger with single-threaded, >>>> multi-processed concurrency model. >>>> >>> >>> That does look like it should work. The only reason I think it wouldn't >>> is if the disconnects are happening on a connection that has been validated >>> and already been checked out. In that case, there is nothing that Sequel >>> can do about the issue. When the exception is raised, Sequel will remove >>> the connection from the pool, so that a new connection can be established. >>> You could try wrapping everything in a transaction and using the :retry_on >>> option, but a better approach would be to find out why the disconnections >>> are happening in the first place and fix them. >>> >>> Thanks, >>> Jeremy >>> >>> >>> Hi >>> >>> I have a similar issue which I have not yet been able to reproduce >>> in-house. >>> This is not exactly the same issue, but I mention it here as it may help >>> in some way. >>> >>> This is a Roda app using Sequel which raises the exception: >>> "Sequel::DatabaseDisconnectError: PG::ConnectionBad: PQconsumeInput() >>> server closed the connection unexpectedly This probably means the server >>> terminated abnormally before or while processing the request.". >>> >>> This only happens on servers that are running the app in Phusion >>> Passenger. >>> It also only seems to happen after using Capistrano to deploy the app. >>> (probably because the deploy triggers a passenger reload-app call) >>> The disconnect error will come up a short while after deploy and it will >>> only happen once. >>> Immediately after deploy I am able to interact with the app without >>> triggering the exception, but some time later a different user will trigger >>> the exception. >>> >>> The exception is always raised in the same place - for every request >>> there is a check to see if the ip address is in a table which identifies >>> certain devices by ip address. >>> This is the first database query in all requests. >>> >>> This would appear to point to something going wrong when passenger >>> reloads a stale process. >>> I have not yet had time to try to reproduce this locally, so I don't >>> have any further insight at this stage. >>> >> >> Based on the reports, I think in both cases, this may be caused because >> you didn't disconnect before forking. If you aren't manually issuing a >> disconnect call before forking, and you are using preload_app, that is the >> probable cause. >> >> See: >> >> >> http://sequel.jeremyevans.net/rdoc/files/doc/code_order_rdoc.html#label-Disconnect+If+Using+Forking+Webserver+with+Code+Preloading >> >> >> https://www.phusionpassenger.com/library/indepth/ruby/spawn_methods/#am-i-responsible-for-reestablishing-database-connections-after-the-preloader-has-forked-a-child-process >> >> Thanks, >> Jeremy >> >> >> Thank you Jeremy, I was not catering for this, so I'm pretty sure that's >> the cause. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sequel-talk" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/sequel-talk. >> For more options, visit https://groups.google.com/d/optout. >> >> >> -- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/sequel-talk. To view this discussion on the web visit https://groups.google.com/d/msgid/sequel-talk/86fd0ac8-685d-44d0-8229-eacfcfcd1260%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
