[
https://issues.apache.org/jira/browse/CASSANDRA-21133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18053910#comment-18053910
]
Stefan Miklosovic commented on CASSANDRA-21133:
-----------------------------------------------
Unfortunately we can not do this so simply. The reason why is that in Cassandra
we have a machinery for assigning new SSTable IDs when we go to flush. This
"counter" is internal to Cassandra process. Logically, it works like this:
1) cqlsh and insert data
2) flush
3) upon flush, ID generator figures out what the next ID is, e.g. 1.
4) SSTable written on disk.
5) I go to execute bin/sstableupgrade -a ks test (-a being new flag, including
"all sstable").
6) So it sees that, okay, we are going to rewrite SSTable with id 1 as well.
7) Internally, it will generate id 2, SSTable is written with such id
8) cqlsh, insert data, flush
9) now this flushing will fail to create a new SSTable
{code}
java.lang.AssertionError
at
org.apache.cassandra.db.ColumnFamilyStore.newSSTableDescriptor(ColumnFamilyStore.java:1025)
at
org.apache.cassandra.db.ColumnFamilyStore.newSSTableDescriptor(ColumnFamilyStore.java:1015)
at
org.apache.cassandra.db.memtable.Flushing.flushRunnable(Flushing.java:115)
at
org.apache.cassandra.db.memtable.Flushing.flushRunnables(Flushing.java:87)
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.flushMemtable(ColumnFamilyStore.java:1334)
at
org.apache.cassandra.db.ColumnFamilyStore$Flush.run(ColumnFamilyStore.java:1289)
at
org.apache.cassandra.concurrent.ExecutionFailure$1.run(ExecutionFailure.java:138)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}
The reason this is happening is that, from the perspective of Cassandra
process, that "counter" of IDs was never increased. Cassandra's counter is
still on 1. So it goes to 2, but 2 already exists from what bin/sstableupgrade
generated.
bin/sstableupgrade is a standalone tool, running outside of Cassandra process
context, it just reads data on disk. On the other hand nodetool version talks
to Cassandra process so we can do "-a" there because it will increase internal
ID counter.
I do not think this has easy solution. This is something I hit when checking
the logic in CASSANDRA-21111, that we have actually two generators, one when
constructing CFS and another when bulkloading / stress testing. Here it shows
too.
> bin/sstableupgrade is not functionally on par with nodetool upgradesstables
> ---------------------------------------------------------------------------
>
> Key: CASSANDRA-21133
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21133
> Project: Apache Cassandra
> Issue Type: Improvement
> Reporter: Stefan Miklosovic
> Priority: Normal
>
> I am not sure if this is done on purpose or not but it seems to me that what
> I can achieve with nodetool version can not be achieved with bin version of
> SSTable upgrading.
> Nodetool version enables a user to specify "-a" flag which will include
> SSTables even they are on the same version / format.
> {code}
> @Option(paramLabel = "include_all",
> names = { "-a", "--include-all-sstables" },
> description = "Use -a to include all sstables, even those already
> on the current version")
> private boolean includeAll = false;
> /// and later
> return performSSTableRewrite(cfs, (sstable) -> {
> // Skip if descriptor version matches current version
> if (skipIfCurrentVersion &&
> sstable.descriptor.version.equals(sstable.descriptor.getFormat().getLatestVersion()))
> return false;
> {code}
> On the other hand, bin/sstableupgrade does not make this possible:
> {code}
> SSTableReader sstable =
> SSTableReader.openNoValidation(entry.getKey(), components, cfs);
> if
> (sstable.descriptor.version.equals(DatabaseDescriptor.getSelectedSSTableFormat().getLatestVersion()))
> {
> sstable.selfRef().release();
> continue;
> }
> readers.add(sstable);
> {code}
> This might be viewed as a functional gap / discrepancy between these two.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
