On Friday, July 20, 2012 1:25:48 AM UTC-7, Matt Hauck wrote: > > I'm working through some performance issues here and have found that the > largest amount of time in my requests is being spent inside of > Sequel::ThreadedConnectionPool.hold, and that the second most number of > time is being spent in what appears to be the YAML parser. This makes sense > because I have a lot of data being serialized. > > I am looking into faster ways to serialize than YAML, but in the meantime > thought I'd ask a question: does Sequel hold a connection while it is doing > this serialization? I ask because when I run load-tests at high > concurrency, a slow system will quickly start running to PoolTimeout > errors, which should not really happen in my mind since all the different > threads are really writing to different rows and should not really > experience any locking. My theory is that it appears that the connection is > locked down by the thread before the serialization happens, thus keeping it > longer than it needs to be, and blocks out others threads who won't even > start serializing until they get a connection from the pool. > > In the end, I'm not in a whole lot better of a situation without this, in > that it will still serialize YAML just as slow, but it will tax the cpu and > not the db connection pool and I should see fewer timeouts... > > Is my theory correct? If so, can the serialization plugin be reworked to > avoid this situation? >
The serialization plugin itself does not rely on having a connection open. However, the serialization is done in a before_save, and by default, model saves are run in transactions, transactions require checking out and keeping a connection open, and before_save is inside that transaction. If you turn off transactions when saving, your problem should go away. You can pass :transaction=>false to Model#save, or set use_transactions = false on the class or instance. The only time you need transactions when saving is if you are performing other queries in before_save/after_save hooks. They are only turned by default for safety, but unless you need that safety, you'll get better performance by turning them off. Jeremy -- You received this message because you are subscribed to the Google Groups "sequel-talk" group. To view this discussion on the web visit https://groups.google.com/d/msg/sequel-talk/-/uJLi29-YmUkJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/sequel-talk?hl=en.
