We use ZooKeeper + kazoo’s lock implementation. Kazoo is a Python client library for ZooKeeper.
- Max > Yes this is quite annoying. How did you implement that "external lock"? I > also thought of doing an external service that would be dedicated to that. > Cassandra client apps would send create instruction to that service, that > would receive them and do the creates 1 by 1, and the client app would wait > the response from it before starting to insert. > > Best, > > Sébastien. > > Le mar. 1 juin 2021 à 05:21, Max C. <mc_cassand...@core43.com > <mailto:mc_cassand...@core43.com>> a écrit : > In our case we have a shared dev cluster with (for example) a key space for > each developer, a key space for each CI runner, etc. As part of > initializing our test suite we setup the schema to match the code that is > about to be tested. This can mean multiple CI runners each adding/dropping > tables at the same time but for different key spaces. > > Our experience is even though the schema changes do not conflict, we still > run into schema mismatch problems. Our solution to this was to have a lock > (external to Cassandra) that ensures only a single schema change operation is > being issued at a time. > > People assume schema changes in Cassandra work the same way as MySQL or > multiple users editing files on disk — i.e. as long as you’re not editing the > same file (or same MySQL table), then there’s no problem. This is NOT the > case. Cassandra schema changes are more like “git push”ing a commit to the > same branch — i.e. at most one change can be outstanding at a time (across > all tables, all key spaces)…otherwise you will run into trouble. > > Hope that helps. Best of luck. > > - Max > > > Hello, > > I have a more general question about that, I cannot find clear answer. > > In my use case I have many tables (around 10k new tables created per months) > and they are created from many clients and only dynamically, with several > clients creating same tables simulteanously. > > What is the recommended way of creating tables dynamically? If I am doing "if > not exists" queries + wait for schema aggreement before and after each create > statement, will it work correctly for Cassandra? > > Sébastien. >
