[
https://issues.apache.org/jira/browse/CASSANDRA-4356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397208#comment-13397208
]
David B commented on CASSANDRA-4356:
------------------------------------
Hi Pavel,
Here's the procedure I just used.
Wipe the nodes:
Remove /data, /saved_caches and /commit_log directories on both HOST_1 and
HOST_2
Restart both nodes
> nodetool -h HOST_1 move 0
> nodetool -h HOST_2 move 85070591730234615865843651857942052864
both nodes now own 50% of the ring
> cassandra-cli -h HOST_1
> create keyspace MyKeyspace with placement_strategy =
> 'org.apache.cassandra.locator.SimpleStrategy' and strategy_options =
> {replication_factor:2};
> use MyKeyspace;
> create column family MyColumnFamily with caching = 'keys_only' and
> key_validation_class = 'LongType' and compression_options = {
> sstable_compression: SnappyCompressor, chunk_length_kb: 64 };
[default@MyKeyspace] create column family MyColumnFamily with caching =
'keys_only' and key_validation_class = 'LongType' and compression_options = {
sstable_compression: SnappyCompressor, chunk_length_kb: 64 };
8b63c283-5850-3d36-9172-287208e680ec
Waiting for schema agreement...
... schemas agree across the cluster
But now, just like what I originally reported when I was trying to drop an
existing column family, creating MyColumnFamily fails. On HOST_2, system.log
contains the following error:
ERROR [MigrationStage:1] 2012-06-20 01:23:59,802 AbstractCassandraDaemon.java
(line 134) Exception in thread Thread[MigrationStage:1,5,main]
org.apache.cassandra.db.marshal.MarshalException: invalid UTF8 bytes 4fe12632
at org.apache.cassandra.db.marshal.UTF8Type.getString(UTF8Type.java:56)
at
org.apache.cassandra.cql3.ColumnIdentifier.<init>(ColumnIdentifier.java:47)
at
org.apache.cassandra.cql3.CFDefinition.getKeyId(CFDefinition.java:125)
at org.apache.cassandra.cql3.CFDefinition.<init>(CFDefinition.java:59)
Here's what I get from querying schema_columnfamilies:
[default@unknown] use system;
Authenticated to keyspace: system
[default@system] list schema_columnfamilies;
Using default limit of 100
Using default column limit of 100
-------------------
RowKey: MyKeyspace
=> (column=MyColumnFamily:caching, value=4b4559535f4f4e4c59,
timestamp=1211997307706930)
=> (column=MyColumnFamily:column_aliases, value=5b5d,
timestamp=1211997307706930)
=> (column=MyColumnFamily:comment, value=, timestamp=1211997307706930)
=> (column=MyColumnFamily:compaction_strategy_class,
value=6f72672e6170616368652e63617373616e6472612e64622e636f6d70616374696f6e2e53697a65546965726564436f6d70616374696f6e5374726174656779,
timestamp=1211997307706930)
=> (column=MyColumnFamily:compaction_strategy_options, value=7b7d,
timestamp=1211997307706930)
=> (column=MyColumnFamily:comparator,
value=6f72672e6170616368652e63617373616e6472612e64622e6d61727368616c2e427974657354797065,
timestamp=1211997307706930)
=> (column=MyColumnFamily:compression_parameters,
value=7b2273737461626c655f636f6d7072657373696f6e223a226f72672e6170616368652e63617373616e6472612e696f2e636f6d70726573732e536e61707079436f6d70726573736f72222c226368756e6b5f6c656e6774685f6b62223a223634227d,
timestamp=1211997307706930)
=> (column=MyColumnFamily:default_validator,
value=6f72672e6170616368652e63617373616e6472612e64622e6d61727368616c2e427974657354797065,
timestamp=1211997307706930)
=> (column=MyColumnFamily:gc_grace_seconds, value=000d2f00,
timestamp=1211997307706930)
=> (column=MyColumnFamily:id, value=000003e8, timestamp=1211997307706930)
=> (column=MyColumnFamily:key_validator,
value=6f72672e6170616368652e63617373616e6472612e64622e6d61727368616c2e4c6f6e6754797065,
timestamp=1211997307706930)
=> (column=MyColumnFamily:local_read_repair_chance, value=0000000000000000,
timestamp=1211997307706930)
=> (column=MyColumnFamily:max_compaction_threshold, value=00000020,
timestamp=1211997307706930)
=> (column=MyColumnFamily:min_compaction_threshold, value=00000004,
timestamp=1211997307706930)
=> (column=MyColumnFamily:read_repair_chance, value=3fb999999999999a,
timestamp=1211997307706930)
=> (column=MyColumnFamily:replicate_on_write, value=01,
timestamp=1211997307706930)
=> (column=MyColumnFamily:type, value=5374616e64617264,
timestamp=1211997307706930)
I don't see anything with '4fe12632'.
If I restart HOST_2, (/etc/init.d/cassandra restart), the column family is now
available...
> nodetool -h HOST_2
...
[default@unknown] use MyKeyspace;
Authenticated to keyspace: MyKeyspace
[default@MyKeyspace] describe;
Keyspace: MyKeyspace:
Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
Durable Writes: true
Options: [replication_factor:2]
Column Families:
ColumnFamily: MyColumnFamily
Key Validation Class: org.apache.cassandra.db.marshal.LongType
Default column value validator: org.apache.cassandra.db.marshal.BytesType
Columns sorted by: org.apache.cassandra.db.marshal.BytesType
GC grace seconds: 864000
Compaction min/max thresholds: 4/32
Read repair chance: 0.1
DC Local Read repair chance: 0.0
Replicate on write: true
Caching: KEYS_ONLY
Bloom Filter FP chance: default
Built indexes: []
Compaction Strategy:
org.apache.cassandra.db.compaction.SizeTieredCompactionStrategy
Compression Options:
chunk_length_kb: 64
sstable_compression: org.apache.cassandra.io.compress.SnappyCompressor
Now let's drop the column family
> nodetool -h HOST_1
[default@MyKeyspace] drop column family MyColumnFamily;
e1ffd8cd-5032-38c7-807f-35beb63cac43
Waiting for schema agreement...
... schemas agree across the cluster
Now a slightly different error message in HOST_2's system.log:
ERROR [MigrationStage:1] 2012-06-20 01:29:54,954 AbstractCassandraDaemon.java
(line 134) Exception in thread Thread[MigrationStage:1,5,main]
org.apache.cassandra.cql.jdbc.MarshalException: invalid UTF8 bytes 4fe12795
at org.apache.cassandra.cql.jdbc.JdbcUTF8.getString(JdbcUTF8.java:81)
at org.apache.cassandra.cql.jdbc.JdbcUTF8.compose(JdbcUTF8.java:97)
at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:35)
And here is the schema_columnfamilies information from HOST_2:
> nodetool -h HOST_2
[default@unknown] use system;
Authenticated to keyspace: system
[default@system] list schema_columnfamilies;
Using default limit of 100
Using default column limit of 100
-------------------
RowKey: MyKeyspace
1 Row Returned.
Elapsed time: 50 msec(s).
Hope this helps.
> Drop column family fail in Cassandra 1.1.1
> ------------------------------------------
>
> Key: CASSANDRA-4356
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4356
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 1.1.1
> Environment: 2-node cluster running on Ubuntu 10.10
> Reporter: David B
> Assignee: Sylvain Lebresne
>
> 2-node v1.1.1 cluster. Attempts to drop column families appears to succeed,
> but fails behind the scenes. More specifically, the column family is deleted
> as follows:
> cassandra-cli -h HOST_1
> > use MyKeyspace
> > drop column family MyColumnFamily
> 79f7ab8c-da08-355c-87a0-8e9630eb4945
> Waiting for schema agreement...
> ... schemas agree across the cluster
> Data files in fact have been deleted on HOST_1. However, the files on HOST_2
> still exist in the /data directory. Also, the following stack trace appears
> in /var/log/system.log on HOST_2:
> ERROR [MigrationStage:1] 2012-06-19 22:05:56,172 AbstractCassandraDaemon.java
> (line 134) Exception in thread Thread[MigrationStage:1,5,main]
> org.apache.cassandra.cql.jdbc.MarshalException: invalid UTF8 bytes 4fe0f7c7
> at org.apache.cassandra.cql.jdbc.JdbcUTF8.getString(JdbcUTF8.java:81)
> at org.apache.cassandra.cql.jdbc.JdbcUTF8.compose(JdbcUTF8.java:97)
> at org.apache.cassandra.db.marshal.UTF8Type.compose(UTF8Type.java:35)
> at
> org.apache.cassandra.cql3.UntypedResultSet$Row.getString(UntypedResultSet.java:87)
> at
> org.apache.cassandra.config.CFMetaData.fromSchemaNoColumns(CFMetaData.java:1170)
> at
> org.apache.cassandra.config.CFMetaData.fromSchema(CFMetaData.java:1215)
> at
> org.apache.cassandra.config.KSMetaData.deserializeColumnFamilies(KSMetaData.java:291)
> at
> org.apache.cassandra.db.DefsTable.mergeColumnFamilies(DefsTable.java:396)
> at org.apache.cassandra.db.DefsTable.mergeSchema(DefsTable.java:271)
> at
> org.apache.cassandra.db.DefsTable.mergeRemoteSchema(DefsTable.java:249)
> at
> org.apache.cassandra.db.DefinitionsUpdateVerbHandler$1.runMayThrow(DefinitionsUpdateVerbHandler.java:48)
> at
> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> From this point on, all other Migration commands (e.g. creating new column
> families) similarly report no errors in cassandra-cli, but fail behind the
> scenes. Recovering from the erroneous state requires a cluster restart.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
