On Fri, Jul 21, 2023 at 8:58 AM 'Michael Scrivo' via sequel-talk < [email protected]> wrote:
> Hey Jeremy, > > We currently use Sequel with Sharded Connection Pooling (5-10 threads > usually) in our codebase, connecting to postgres instances, along with some > select usages of Async blocks using the Async gem. All is working great, > but recently I wanted to try enabling the fiber_concurrency extension to > see if that would squeeze out any additional performance gains. As soon as > I enabled it, some of our tests broke, specifically in scenarios where an > Async block was used in the context of a model. I started seeing errors > like this: > > 0.0s warn: Async::Task [oid=0x27ee8] [ec=0x27efc] [pid=4388] > [2023-07-21 08:48:12 -0700] > | Task may have ended with unhandled exception. > | NoMethodError: undefined method `foo' for nil:NilClass > > where foo was a method on some model called within an Async block. The > really strange thing, in one case, is that it only threw the error if > calling refresh on the model right before hand. ie given this test > scernario: > > models = create models > do some updates on the models > models.each(&refresh) > Model.some_function_with_async_block(models) > > dies with above exception where foo was called within the async block in > some_function_with_async_block. > > However, if I remove the call to refresh, it all works fine! > > In another case, there was a a model that had a function with an async > block, and in that block it called another model to get some records. With > fiber_concurrency on, that dataset was returning nothing when called inside > the block. In this case, it didn't need to be in the async block, so moving > it outside is a fine solution, but these issues are a bit scary to say the > least. > > Do you have any idea what might be going on here? I've been trying in > vain to setup a small reproduction here: > https://github.com/mscrivo/sequel-async-bug but have so far been unable > to reproduce the exact conditions that trigger the issue in our codebase. > I'm not sure if it's pg specific, or some other piece of code in our > codebase interacting badly with the fiber_concurrency and async. > The fiber_concurrency extension will result in a separate connection per-fiber. If your code is dealing with transactions (and models use transactions by default), it's fairly easy to run into situations where the connection which ran a query has not committed a transaction, and the fiber you are using implicitly via Async uses a different connection and does not see the changes. Note that I don't have any experience using Async, so I'm not exactly sure what failure conditions are possible with it. When debugging, I recommend making sure your Database has a logger attached and that log_connection_info is true so you can see which connection is used for all queries. If you still need help, please put together a minimal self contained example and post it here (not a link to a repository), so I can review. Thanks, Jeremy -- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/sequel-talk/CADGZSSd2Hy5QWYxUCi%3DaHKbP6nA5_Sp-MtpXJ5WMOXpmo7o0dA%40mail.gmail.com.
