Thank you, that connection logging tip is very helpful! On Friday, July 21, 2023 at 12:27:11 PM UTC-4 Jeremy Evans wrote:
> On Fri, Jul 21, 2023 at 8:58 AM 'Michael Scrivo' via sequel-talk < > seque...@googlegroups.com> wrote: > >> Hey Jeremy, >> >> We currently use Sequel with Sharded Connection Pooling (5-10 threads >> usually) in our codebase, connecting to postgres instances, along with some >> select usages of Async blocks using the Async gem. All is working great, >> but recently I wanted to try enabling the fiber_concurrency extension to >> see if that would squeeze out any additional performance gains. As soon as >> I enabled it, some of our tests broke, specifically in scenarios where an >> Async block was used in the context of a model. I started seeing errors >> like this: >> >> 0.0s warn: Async::Task [oid=0x27ee8] [ec=0x27efc] [pid=4388] >> [2023-07-21 08:48:12 -0700] >> | Task may have ended with unhandled exception. >> | NoMethodError: undefined method `foo' for nil:NilClass >> >> where foo was a method on some model called within an Async block. The >> really strange thing, in one case, is that it only threw the error if >> calling refresh on the model right before hand. ie given this test >> scernario: >> >> models = create models >> do some updates on the models >> models.each(&refresh) >> Model.some_function_with_async_block(models) >> >> dies with above exception where foo was called within the async block in >> some_function_with_async_block. >> >> However, if I remove the call to refresh, it all works fine! >> >> In another case, there was a a model that had a function with an async >> block, and in that block it called another model to get some records. With >> fiber_concurrency on, that dataset was returning nothing when called inside >> the block. In this case, it didn't need to be in the async block, so moving >> it outside is a fine solution, but these issues are a bit scary to say the >> least. >> >> Do you have any idea what might be going on here? I've been trying in >> vain to setup a small reproduction here: >> https://github.com/mscrivo/sequel-async-bug but have so far been unable >> to reproduce the exact conditions that trigger the issue in our codebase. >> I'm not sure if it's pg specific, or some other piece of code in our >> codebase interacting badly with the fiber_concurrency and async. >> > > The fiber_concurrency extension will result in a separate connection > per-fiber. If your code is dealing with transactions (and models use > transactions by default), it's fairly easy to run into situations where the > connection which ran a query has not committed a transaction, and the fiber > you are using implicitly via Async uses a different connection and does not > see the changes. Note that I don't have any experience using Async, so I'm > not exactly sure what failure conditions are possible with it. > > When debugging, I recommend making sure your Database has a logger > attached and that log_connection_info is true so you can see which > connection is used for all queries. > > If you still need help, please put together a minimal self contained > example and post it here (not a link to a repository), so I can review. > > Thanks, > Jeremy > -- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To unsubscribe from this group and stop receiving emails from it, send an email to sequel-talk+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/sequel-talk/93e9cd84-fb6a-4b75-b7b7-0d0ef150bed6n%40googlegroups.com.
