On Feb 24, 1:30 pm, Myron Marston <[email protected]> wrote:
> > * It leaks Database objects. All database objects are stored in the
> > Sequel::DATABASES array, and if Sequel.connect isn't called with a
> > block, they need to be removed manually.
>
> That make sense. I didn't realize that a reference would be kept to
> the DB object outside of any variables I use.
It is mentioned in the documentation, but it doesn't really stand
out. Few people create arbitrary number of Database instances,
though.
> > 1) The ability to have a method that takes a block which sets the
> > default shard to use inside the block.
>
> If it's not too much trouble to add, I can also imagine needing/
> wanting an API to set the the default shard to use for all subsequent
> DB queries (or is that already present?). Within the sinatra
> application, I imagine we'll always use the block API but I can
> imagine other cases (such as one-off rake tasks) where we'd want to
> just set the shard at the start and not worry about wrapping
> everything in a block.
Currently, you would just make the main options for the connection the
same as the default shard. You can override the default shard
connection options by using add_servers(:default=>...), and then
calling Database#disconnect to clear the pool. Naturally, any block-
based method approach using begin/yield/ensure can be easily modified
to get the method you want simply by running the code between begin
and yield. I'll try to make that a single method you can call by
yourself to get the API you want (though it will most likely be
private and not public).
> > 1) is fairly easy to address, requiring just a little refactoring so
> > that instead of defaulting to :default, it can default to the shard
> > set by the method. The main complication is that sometimes Sequel
> > uses :default as the server if no server is given, and there needs to
> > be a way to detect the difference between Sequel's use of :default and
> > a possible manual use of :default. I'll probably switch Sequel to
> > use :_sequel_default for the cases, so that it is easy to tell the
> > difference.
>
> With this approach, how would Sequel handle it when a shard hasn't
> been set? I would like it to raise an error since there is no default
> shard that makes sense.
You can do the following when connecting:
Sequel.connect(..., :servers=>{:default=>proc{raise ...}})
That way, if you attempt to use the default server, an error is
raised.
> > 2) is not difficult to address in the simple case, but it causes other
> > questions, such as what to do with the connections for arbitrary
> > shards. I think the only way to handle arbitrary shards intelligently
> > is to remove the connection from the connection pool immediately after
> > it's usage is completed. I'll probably make it so that you can set
> > shard to an arbitrary hash that gets merged into the default database
> > options.
>
> I think disconnecting once the shard I'm done using the shard is the
> only thing that makes sense. That said: what are the performance
> implications of needing to connect and disconnect to a different DB
> shard for each request? I can (and will) benchmark if necessary, but
> I figured you might have a typical ballpark number for how long those
> operations take :).
The performance impact depends greatly on the environment. For an
SQLite database on a local drive, it's probably fairly small. For an
Oracle database over a satellite network connection, it's probably
large.
For a truly arbitrary number of connections, I see no way around the
issue. However, even for a largish number of possible shards
(1-10000?), you could probably stick all possible ones in a hash and
reference them by shard. While the documentation shows no examples of
non-symbol based shards, I'm pretty sure it works with them. You
should already be able to do this:
DB = Sequel.connect(..., :servers=>{['host1', 'db1']=>{}, ['host2',
'db2']=>{}})
DB[:table].with_server(['host1', 'db1']).all
If you never attempt to connect to a shard, you are only wasting a
small amount of memory (probably <1K) to store it in the servers hash.
> > I plan to address both of these issues, hopefully before 3.33.0 is
> > released next week. Here's my planned API for your use case:
>
> > DB.with_server(:host=>'foo', :database=>'bar') do
> > # Will use host foo and database bar for connections inside the
> > block.
> > end
>
> That API looks great. One question, though: how do I get the DB
> object? Presumably through `Sequel.connect`, but I'm not sure what
> connection string I would pass, given that there's not default DB that
> makes sense.
If you don't have a default DB that makes sense, then having
a :default entry that raises an error is fine. You would probably
want to set your default/shared options in the main connection string/
options, and have the shards only override the parts they need.
Putting it all together, this could possibly work for you currently:
DB = Sequel.connect('postgres://user:password@host/db',
:servers=>{:default=>proc{raise ...},
['host1', 'db1']=>{:host=>'host1', :database=>'db1'},
['host2', 'db2']=>{:host=>'host2', :database=>'db2'},
...})
Again, you have to know the shards up front, or be willing to add them
on the fly using add_servers. You only really need arbitrary
connections if you have no idea where you will be connecting, and
consider add_servers/remove_servers to be to arduous.
> I'm impressed at how quickly you responded and that you're willing to
> add features to meet our needs so quickly--thanks! That said, bear in
> mind that it's not definite that we'll use Sequel yet. We're
> evaluating it at the moment. I'm definitely leaning strongly towards
> it but the fact that we all have ActiveRecord experience (and, in my
> case, a good 4-5 years of it), and none of us have Sequel experience,
> counts for a lot.
Well, I'm not doing this just for you. :) Multiple people have asked
for the block based method before, so there is apparently general
demand for the feature. I've resisted the feature before as I wasn't
able to think of a good way to integrate it, but I've thought of
something that may work well.
The arbitrary connection thing should be fairly simple to do, and
makes Sequel much more flexible, so in both cases I think I'm just
trying to improve the library in general. I'll be implementing the
features regardless of whether you stay with ActiveRecord or choose to
go with Sequel. That said, I do hope you decide that Sequel meets
your needs best. :)
Thanks,
Jeremy
--
You received this message because you are subscribed to the Google Groups
"sequel-talk" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sequel-talk?hl=en.
