On Dec 10, 12:36 pm, Chuck Remes <[email protected]> wrote:
> On Dec 9, 2009, at 8:55 PM, Jeremy Evans wrote:
> > You should not be using the sharding support to modify schema on
> > specific shards. Anything with different schemas or modifying schemas
> > should be using different Database objects (mentioned near the top of
> >http://sequel.rubyforge.org/rdoc/files/doc/sharding_rdoc.html).
>
> Yes, I agree with a few caveats.
>
> 1. Create Table
> Since my patch allows for dynamic addition of shards, if the shard has
> not yet been populated with tables it would be nice to be able to
> handle that schema setup dynamically too. The burden is on the
> programmer to make sure all the schemas match.
>
> 2. Rolling migrations
> I can foresee the need to support rolling migrations across a set of
> shards. Let's say a new schema has been generated which alters a table
> to include another column. A code migration could be deployed that
> checks a shard for the proper layout when the connection is opened; if
> it is correct, no op. If it is incorrect, update the table schema and
> move on.
I agree that these are valid issues, but I don't think the sharding
support was designed for these use cases, which is why I recommend the
separate Database object approach.
> >> I can get to the correct database shard by using the
> >> Sequel:Database#synchronize call, but it yields (in my case) a
> >> Java::OrgH2Jdbc::JdbcConnection. I can't use any of the nice schema
> >> generator calls on that object (it's a Java object).
>
> >> Is it possible for me to go from a JdbcConnection object back to a
> >> Sequel:Database reference?
>
> > If you called DB.synchronize to get the connection object, then DB is
> > the Sequel::Database object you are looking for.
>
> No, DB is pointing at the :default server. I need it to point at a
> specific shard.
Technically, the DB does not point to any server. It defaults
to :default only if nothing else is specified. Your issue is that you
want to specify the server to use for schema modification (DDL)
queries, and Sequel currently does not provide a good way to do
that.
>
> >> Ideally, I would like to be able to do something like this:
>
> >> shards = #some hash
> >> DB = Sequel.connect "jdbc:h2:master_server", :servers => shards
> >> ....
> >> DB.server(some_shard_id).create_table :test do
> >> ..
> >> ..
> >> end
>
> > Nope, the design isn't set to work like that. This might work:
>
> > def DB.execute_ddl(sql, opts={}, &block)
> > super(sql, {:server=>some_shard_id}.merge(opts), &block)
> > end
>
> > All schema-modifying methods should go through execute_ddl.
>
> Good to know. I'll play around with this a bit to see if I can find an
> elegant way to patch in this behavior.
You can't do it in a thread safe manner unless you want to use your
own mutex. Assuming you are OK with using your own mutex:
USING_SHARD_MUTEX = Mutex.new
def DB.using_shard(server)
USING_SHARD_MUTEX.synchronize do
@schema_shard = server
yield
end
end
def DB.execute_ddl(sql, opts={}, &block)
super(sql, {:server=>@schema_shard}.merge(opts), &block)
end
DB.using_shard(some_shard) do
DB.create_table(:test){...}
DB.create_table(:test){...}
end
That should work fine in a thread safe manner, assuming you don't mind
the very coarse locking and always remember to have your create_table
calls inside a using_shard block.
> > Personally, I'd just make a connection to a single shard using a
> > separate database object:
>
> > Sequel.connect("jdbc:h2:shard1") do |db|
> > db.create_table(:test){...}
> > end
>
> This is what I will do in the short term until I figure out a more
> elegant long term solution. Thanks for the pointer.
The advantage of this approach is it is already thread safe and
doesn't require locking (you can update the schemas for multiple
databases simultaneously). It's also simpler and less likely to
break, which is why I recommend it over the execute_ddl hack.
Jeremy
--
You received this message because you are subscribed to the Google Groups
"sequel-talk" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/sequel-talk?hl=en.
