I've just pushed a commit to Sequel's threaded connection pools that should
make them more reliable in heavily constrained threaded environments (ones
with many more threads than allowed connections). If you are using Sequel
in a threaded environment, I'd like you to test the master branch and
provide feedback here on whether things are more or less reliable with it.
Here's one of the test benchmarks I wrote related to this change:
require 'benchmark'
require 'sequel'
db = Sequel.mock
timeouts = []
bm = Benchmark.measure do |x|
(0..25).map do
Thread.new do
50.times do
t = Time.now
db.synchronize{10.times{sleep 0.005}} rescue(timeouts << (Time.now -
t))
end
end
end.each(&:join)
end
puts "#{bm.to_s.chomp} (#{timeouts.length} timeouts)"
Basically, this uses Sequel's default connection pool settings (4
connections, 5 second timeout), then loads up 26 threads that each try to
grab a connection. With this benchmark, I got the following result for
Sequel 4.21.0:
$ for x in 1 2 3; do RUBYLIB= time ruby bench_pool.rb; done
> 0.460000 3.130000 3.590000 ( 61.355562) (143 timeouts)
> 61.73 real 0.78 user 3.17 sys
> 0.360000 1.390000 1.750000 ( 60.680346) (130 timeouts)
> 61.05 real 0.69 user 1.42 sys
> 0.610000 1.990000 2.600000 ( 61.035718) (133 timeouts)
> 61.40 real 0.91 user 2.05 sys
Note the number of timeouts, of the 1300 (26x50) attempts to checkout a
connection, over 10% were failing. With the master branch:
$ for x in 1 2 3; do RUBYLIB=lib time ruby bench_pool.rb; done
> 0.080000 0.470000 0.550000 ( 65.155083) (0 timeouts)
> 65.49 real 0.36 user 0.52 sys
> 0.120000 0.450000 0.570000 ( 65.155824) (0 timeouts)
> 65.48 real 0.36 user 0.52 sys
> 0.120000 0.430000 0.550000 ( 65.157323) (0 timeouts)
> 65.49 real 0.37 user 0.50 sys
Note how there are no timeouts. Also, note the significantly decreased
user and system time, though increased real time. So under heavily
resource constrained environments, this commit makes things slightly slower
(about 7%), but much more reliable, which I think is a good tradeoff.
I honestly don't think this will make much difference in most real world
apps, as in a real world app, you should try to avoid situations where
there are no connections available and you already reached the maximum
number of connections. However, it's possible there are some real world
apps where it will help. I encourage all users to try it out and let me
know their results.
Thanks,
Jeremy
--
You received this message because you are subscribed to the Google Groups
"sequel-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/sequel-talk.
For more options, visit https://groups.google.com/d/optout.
