You can start here: https://issues.apache.org/jira/browse/CASSANDRA-10699
And here: http://stackoverflow.com/questions/20293897/cassandra-resolution-of-concurrent-schema-changes In a nutshell, schema changes works best when issued serially, when all nodes are up, and reachable. When these 3 conditions are not met a variety of behavior can be observed. On Tue, Nov 15, 2016 at 1:04 PM, Josh Smith <jsm...@ionicsecurity.com> wrote: > Would someone please explain how schema changes happen? > > Here are some of the ring details > > We have 5 nodes in 1 DC and 5 nodes in another DC across the country. > > Here is our problem, we have a tool which automates our schema creation. > Our schema consists of 7 keyspaces with 21 tables in each keyspace, so a > total of 147 tables are created at the initial provisioning. During this > schema creation we end up with system_schema keyspace corruption, we have > found that it is due to schema version disagreement. To combat this we > setup a wait until there is only one version in both system.local and > system.peers tables. > > The way I understand it schema changes are made on the local node only; > changes are then propagated through either Thrift or Gossip, I could not > find a definitive answer online if thrift or gossip was the carrier. So if > I make all of the schema changes to one node it should propagate the > changes to the other nodes one at a time. This is how I used to think that > schema changes are propagated but we still get schema disagreement when > changing the schema only on one node. Is the only option to introduce a > wait after every table creation? Should we be looking at another table > besides system.local and peers? Any help would be appreciated. > > > > Josh Smith >
