Hi,
We use a four-node Cassandra-Cluster in Version 2.1.2. Our
Client-Applications creates Tables dynamically. At one point two (or
more) of our Clients connected to two (or more) different
Cassandra-Nodes will create the same table simultaneously. We get the
"Column family ID mismatch"-Error-Messages on every node. Why is this
simultanous schema modification not possible? How can we handle this?
Every Help is appreciated.
The lengthy Error-Messages from two nodes follows:
On Node1 we got:
INFO [SharedPool-Worker-2] 2014-11-26 13:37:28,987
MigrationManager.java:248 - Create new ColumnFamily:
org.apache.cassandra.config.CFMetaData@7edad3a3[cfId=fbd24eb0-7568-11e4-bd04-b3ae3abaeff4,ksName=myplayground,cfName=test_table,
INFO [MigrationStage:1] 2014-11-26 13:37:29,607 DefsTables.java:373 -
Loading
org.apache.cassandra.config.CFMetaData@7adc8efd[cfId=fbd24eb0-7568-11e4-bd04-b3ae3abaeff4,ksName=myplayground,cfName=test_table,
INFO [MigrationStage:1] 2014-11-26 13:37:29,629
ColumnFamilyStore.java:284 - Initializing myplayground.test_table
ERROR [MigrationStage:1] 2014-11-26 13:37:30,282
CassandraDaemon.java:153 - Exception in thread
Thread[MigrationStage:1,5,main]
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ConfigurationException: Column family
ID mismatch (
found fbd275c0-7568-11e4-b9ea-3934eddce895; expected
fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1171)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:422)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:295)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeSchemaInternal(DefsTables.java:194)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:166)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:49)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_25]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
Caused by: org.apache.cassandra.exceptions.ConfigurationException:
Column family ID mismatch (found fbd275c0-7568-11e4-b9ea-3934eddce895;
expected fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.config.CFMetaData.validateCompatility(CFMetaData.java:1254)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.config.CFMetaData.apply(CFMetaData.java:1186)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1167)
~[apache-cassandra-2.1.1.jar:2.1.1]
... 11 common frames omitted
On Node2 we got:
INFO [SharedPool-Worker-1] 2014-11-26 13:37:28,989
MigrationManager.java:248 - Create new ColumnFamily:
org.apache.cassandra.config.CFMetaData@16d0bc0d[cfId=fbd275c0-7568-11e4-b9ea-3934eddce895,ksName=myplayground,cfName=test_table,
INFO [MigrationStage:1] 2014-11-26 13:37:29,539 DefsTables.java:373 -
Loading
org.apache.cassandra.config.CFMetaData@3777e24b[cfId=fbd24eb0-7568-11e4-bd04-b3ae3abaeff4,ksName=myplayground,cfName=test_table,
INFO [MigrationStage:1] 2014-11-26 13:37:29,541
ColumnFamilyStore.java:284 - Initializing myplayground.test_table
ERROR [SharedPool-Worker-1] 2014-11-26 13:37:29,984
QueryMessage.java:130 - Unexpected error during query
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ConfigurationException: Column family
ID mismatch (
found fbd275c0-7568-11e4-b9ea-3934eddce895; expected
fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:397)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:374)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.service.MigrationManager.announceNewColumnFamily(MigrationManager.java:249)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.cql3.statements.CreateTableStatement.announceMigration(CreateTableStatement.java:114)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:80)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:226)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:118)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.1.jar:2.1.1]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_25]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.1.jar:2.1.1]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
Caused by: java.util.concurrent.ExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ConfigurationException: Column family
ID mismatch (found fbd275c0-7568-11e4-b9ea-3934eddce895; expected
fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
java.util.concurrent.FutureTask.report(FutureTask.java:122)
~[na:1.8.0_25]
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
~[na:1.8.0_25]
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:393)
~[apache-cassandra-2.1.1.jar:2.1.1]
... 17 common frames omitted
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ConfigurationException: Column family
ID mismatch (found fbd275c0-7568-11e4-b9ea-3934eddce895; expected
fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1171)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:422)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:295)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeSchemaInternal(DefsTables.java:194)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:166)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.service.MigrationManager$2.runMayThrow(MigrationManager.java:393)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_25]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_25]
... 1 common frames omitted
Caused by: org.apache.cassandra.exceptions.ConfigurationException:
Column family ID mismatch (found fbd275c0-7568-11e4-b9ea-3934eddce895;
expected fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.config.CFMetaData.validateCompatility(CFMetaData.java:1254)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.config.CFMetaData.apply(CFMetaData.java:1186)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1167)
~[apache-cassandra-2.1.1.jar:2.1.1]
... 11 common frames omitted
ERROR [MigrationStage:1] 2014-11-26 13:37:29,984
CassandraDaemon.java:153 - Exception in thread
Thread[MigrationStage:1,5,main]
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ConfigurationException: Column family
ID mismatch (
found fbd275c0-7568-11e4-b9ea-3934eddce895; expected
fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1171)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:422)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:295)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeSchemaInternal(DefsTables.java:194)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:166)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.service.MigrationManager$2.runMayThrow(MigrationManager.java:393)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_25]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_25]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
Caused by: org.apache.cassandra.exceptions.ConfigurationException:
Column family ID mismatch (found fbd275c0-7568-11e4-b9ea-3934eddce895;
expected fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.config.CFMetaData.validateCompatility(CFMetaData.java:1254)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.config.CFMetaData.apply(CFMetaData.java:1186)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1167)
~[apache-cassandra-2.1.1.jar:2.1.1]
... 11 common frames omitted
ERROR [SharedPool-Worker-1] 2014-11-26 13:37:29,985
ErrorMessage.java:243 - Unexpected exception during request
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ConfigurationException: Column family
ID mismatch (
found fbd275c0-7568-11e4-b9ea-3934eddce895; expected
fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:397)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.service.MigrationManager.announce(MigrationManager.java:374)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.service.MigrationManager.announceNewColumnFamily(MigrationManager.java:249)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.cql3.statements.CreateTableStatement.announceMigration(CreateTableStatement.java:114)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.cql3.statements.SchemaAlteringStatement.execute(SchemaAlteringStatement.java:80)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:226)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:248)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:118)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:439)
[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:335)
[apache-cassandra-2.1.1.jar:2.1.1]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:333)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext.access$700(AbstractChannelHandlerContext.java:32)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
io.netty.channel.AbstractChannelHandlerContext$8.run(AbstractChannelHandlerContext.java:324)
[netty-all-4.0.23.Final.jar:4.0.23.Final]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_25]
at
org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-2.1.1.jar:2.1.1]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_25]
Caused by: java.util.concurrent.ExecutionException:
java.lang.RuntimeException:
org.apache.cassandra.exceptions.ConfigurationException: Column family
ID mismatch (found fbd275c0-7568-11e4-b9ea-3934eddce895; expected
fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
java.util.concurrent.FutureTask.report(FutureTask.java:122)
~[na:1.8.0_25]
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
~[na:1.8.0_25]
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:393)
~[apache-cassandra-2.1.1.jar:2.1.1]
... 17 common frames omitted
Caused by: java.lang.RuntimeException:
org.apache.cassandra.exceptions.ConfigurationException: Column family
ID mismatch (found fbd275c0-7568-11e4-b9ea-3934eddce895; expected
fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1171)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.updateColumnFamily(DefsTables.java:422)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeColumnFamilies(DefsTables.java:295)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeSchemaInternal(DefsTables.java:194)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.db.DefsTables.mergeSchema(DefsTables.java:166)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.service.MigrationManager$2.runMayThrow(MigrationManager.java:393)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[na:1.8.0_25]
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_25]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
~[na:1.8.0_25]
... 1 common frames omitted
Caused by: org.apache.cassandra.exceptions.ConfigurationException:
Column family ID mismatch (found fbd275c0-7568-11e4-b9ea-3934eddce895;
expected fbd24eb0-7568-11e4-bd04-b3ae3abaeff4)
at
org.apache.cassandra.config.CFMetaData.validateCompatility(CFMetaData.java:1254)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.config.CFMetaData.apply(CFMetaData.java:1186)
~[apache-cassandra-2.1.1.jar:2.1.1]
at
org.apache.cassandra.config.CFMetaData.reload(CFMetaData.java:1167)
~[apache-cassandra-2.1.1.jar:2.1.1]
... 11 common frames omitted
with best regards
Peter
