Hi Joe, In our case we only do this in the test environment and it could be the case that there are several seconds or even minutes between when a schema change occurs vs when a test executes that depends on said schema change. Perhaps we have been lucky thus far. :-)
I wonder if there’s a way to query the driver to see if your schema change has fully propagated. I haven’t looked into this. - Max > On Jun 3, 2021, at 8:23 am, Joe Obernberger <joseph.obernber...@gmail.com> > wrote: > > How does this work? I have a program that runs a series of alter table > statements, and then does inserts. In some cases, the insert happens > immediately after the alter table statement and the insert fails because the > schema (apparently) has not had time to propagate. I get an Undefined column > name error. > > The alter statements run single threaded, but the inserts run in multiple > threads. The alter statement is run in a synchronized block (Java). Should > I put an artificial delay after the alter statement? > > -Joe > > On 6/1/2021 2:59 PM, Max C. wrote: >> We use ZooKeeper + kazoo’s lock implementation. Kazoo is a Python client >> library for ZooKeeper. >> >> - Max >> >>> Yes this is quite annoying. How did you implement that "external lock"? I >>> also thought of doing an external service that would be dedicated to that. >>> Cassandra client apps would send create instruction to that service, that >>> would receive them and do the creates 1 by 1, and the client app would wait >>> the response from it before starting to insert. >>> >>> Best, >>> >>> Sébastien. >>> >>> Le mar. 1 juin 2021 à 05:21, Max C. <mc_cassand...@core43.com >>> <mailto:mc_cassand...@core43.com>> a écrit : >>> In our case we have a shared dev cluster with (for example) a key space for >>> each developer, a key space for each CI runner, etc. As part of >>> initializing our test suite we setup the schema to match the code that is >>> about to be tested. This can mean multiple CI runners each adding/dropping >>> tables at the same time but for different key spaces. >>> >>> >>> Our experience is even though the schema changes do not conflict, we still >>> run into schema mismatch problems. Our solution to this was to have a >>> lock (external to Cassandra) that ensures only a single schema change >>> operation is being issued at a time. >>> >>> People assume schema changes in Cassandra work the same way as MySQL or >>> multiple users editing files on disk — i.e. as long as you’re not editing >>> the same file (or same MySQL table), then there’s no problem. This is NOT >>> the case. Cassandra schema changes are more like “git push”ing a commit to >>> the same branch — i.e. at most one change can be outstanding at a time >>> (across all tables, all key spaces)…otherwise you will run into trouble. >>> >>> Hope that helps. Best of luck. >>> >>> - Max >>> >>> >>> Hello, >>> >>> I have a more general question about that, I cannot find clear answer. >>> >>> In my use case I have many tables (around 10k new tables created per >>> months) and they are created from many clients and only dynamically, with >>> several clients creating same tables simulteanously. >>> >>> What is the recommended way of creating tables dynamically? If I am doing >>> "if not exists" queries + wait for schema aggreement before and after each >>> create statement, will it work correctly for Cassandra? >>> >>> Sébastien. >>> >> >> >> >> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> >> Virus-free. www.avg.com >> <http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient> >> <x-msg://2/#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
