How does this work? I have a program that runs a series of alter table
statements, and then does inserts. In some cases, the insert happens
immediately after the alter table statement and the insert fails because
the schema (apparently) has not had time to propagate. I get an
Undefined column name error.
The alter statements run single threaded, but the inserts run in
multiple threads. The alter statement is run in a synchronized block
(Java). Should I put an artificial delay after the alter statement?
-Joe
On 6/1/2021 2:59 PM, Max C. wrote:
We use ZooKeeper + kazoo’s lock implementation. Kazoo is a Python
client library for ZooKeeper.
- Max
Yes this is quite annoying. How did you implement that "external
lock"? I also thought of doing an external service that would be
dedicated to that. Cassandra client apps would send create
instruction to that service, that would receive them and do the
creates 1 by 1, and the client app would wait the response from it
before starting to insert.
Best,
Sébastien.
Le mar. 1 juin 2021 à 05:21, Max C. <mc_cassand...@core43.com> a
écrit :
In our case we have a shared dev cluster with (for example) a key
space for each developer, a key space for each CI runner, etc.
As part of initializing our test suite we setup the schema to
match the code that is about to be tested.� This can mean
multiple CI runners each adding/dropping tables at the same time
but for different key spaces.
Our experience is even though the schema changes do not conflict,
we still run into schema mismatch problems. Our solution to
this was to have a lock (external to Cassandra) that ensures only
a single schema change operation is being issued at a time.
People assume schema changes in Cassandra work the same way as
MySQL or multiple users editing files on disk — i.e. as long as
you’re not editing the same file (or same MySQL table), then
there’s no problem. � *_This is NOT the case._* Cassandra
schema changes are more like “git push”ing a commit to the
same branch — i.e. at most one change can be outstanding at a
time (across all tables, all key spaces)…otherwise you will run
into trouble.
Hope that helps. Best of luck.
- Max
Hello,
I have a more general question about that, I cannot find
clear answer.
In my use case I have many tables (around 10k new tables
created per months) and they are created from many clients
and only dynamically, with several clients creating same
tables simulteanously.
What is the recommended way of creating tables dynamically?
If I am doing "if not exists" queries + wait for schema
aggreement before and after each create statement, will it
work correctly for Cassandra?
Sébastien.
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
Virus-free. www.avg.com
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
