[
https://issues.apache.org/jira/browse/CASSANDRA-21620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-21620:
-------------------------------------------
Test and Documentation Plan:
MetadataSerializerTest passes: 4 tests, 0 failures, JDK 17. SSTableMetadataTest
passes: 3 tests, 0 failures.
The patch adds
MetadataSerializerTest.testStatsSerializedSizeMatchesTheBytesWritten. It
serializes a StatsMetadata whose covered clustering holds a null component,
once per compatible sstable version, and asserts that the written length equals
the size the serializer computes. Reverting the two production lines fails that
test and only that test, 1 of 4.
Status: Patch Available (was: Open)
> Stats metadata over-sizes the legacy minimum and maximum clustering values
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-21620
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21620
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Local/SSTable
> Reporter: Michael Semb Wever
> Priority: Normal
> Fix For: 7.x
>
>
> For an sstable version that still carries the legacy minimum and maximum
> clustering values, {{StatsMetadata.StatsMetadataSerializer.serialize}} writes
> the components of each bound up to the first null one, and writes that count:
> {code:java}
> out.writeInt(countUntilNull(minClusteringValues.getBufferArray()));
> for (ByteBuffer value : minClusteringValues.getBufferArray())
> {
> if (value == null)
> break;
> ByteBufferUtil.writeWithShortLength(value, out);
> }
> {code}
> {{serializedSize}} charges two length bytes for every component the bound
> holds, null ones included:
> {code:java}
> size += minClusteringValues.size() * 2 /* short length */ +
> minClusteringValues.dataSize();
> {code}
> A bound that {{MetadataCollector}} builds holds one entry per clustering
> column, so a covered clustering that stops short of the last column makes the
> two disagree. {{dataSize()}} already counts a null component as zero bytes,
> so the difference is the two length bytes alone.
> {{MetadataSerializer.serialize}} derives the table-of-contents offset of each
> component from the returned size, and writes the whole backing array of a
> buffer allocated to that size. The STATS component therefore ends with two
> unused bytes per absent component. The file stays readable: the reader takes
> the length of each component from the offsets and stops once it has read the
> recorded count.
> No production write reaches the disagreement today, which is why the ticket
> is Low. A writer uses the latest version of its format, where
> {{hasLegacyMinMax()}} is false, and a metadata mutation on an older sstable
> rebuilds the bounds from disk, where they carry exactly the components that
> were written and no nulls. The legacy branch is deprecated and due for
> removal after {{oa}}; until then the two halves of the serializer must agree,
> because the offset arithmetic depends on it.
> The patch sizes both bounds with {{countUntilNull}}, the same count the
> writer uses.
> Patch:
> [mck/upstream/stats-metadata-legacy-min-max-size/trunk|https://github.com/thelastpickle/cassandra/tree/mck/upstream/stats-metadata-legacy-min-max-size/trunk]
> Provenance:
> [f57af096a0|https://github.com/datastax/cassandra/commit/f57af096a0a9b5225d46dc860ab683d26156a013]
> by Jacek Lewandowski. That commit repairs the size computation only; the
> writer already counts to the first null upstream.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
