[
https://issues.apache.org/jira/browse/CASSANDRA-21620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-21620:
-------------------------------------------
Description:
For an sstable version that still carries the legacy minimum and maximum
clustering values, {{StatsMetadata.StatsMetadataSerializer.serialize}} writes
the components of each bound up to the first null one, and writes that count:
{code:java}
out.writeInt(countUntilNull(minClusteringValues.getBufferArray()));
for (ByteBuffer value : minClusteringValues.getBufferArray())
{
if (value == null)
break;
ByteBufferUtil.writeWithShortLength(value, out);
}
{code}
{{serializedSize}} charges two length bytes for every component the bound
holds, null ones included:
{code:java}
size += minClusteringValues.size() * 2 /* short length */ +
minClusteringValues.dataSize();
{code}
A bound that {{MetadataCollector}} builds holds one entry per clustering
column, so a covered clustering that stops short of the last column makes the
two disagree. {{dataSize()}} already counts a null component as zero bytes, so
the difference is the two length bytes alone.
{{MetadataSerializer.serialize}} derives the table-of-contents offset of each
component from the returned size, and writes the whole backing array of a
buffer allocated to that size. The STATS component therefore ends with two
unused bytes per absent component. The file stays readable: the reader takes
the length of each component from the offsets and stops once it has read the
recorded count.
No production write reaches the disagreement today, which is why the ticket is
Low. A writer uses the latest version of its format, where
{{hasLegacyMinMax()}} is false, and a metadata mutation on an older sstable
rebuilds the bounds from disk, where they carry exactly the components that
were written and no nulls. The legacy branch is deprecated and due for removal
after {{oa}}; until then the two halves of the serializer must agree, because
the offset arithmetic depends on it.
The patch sizes both bounds with {{countUntilNull}}, the same count the writer
uses.
Patch:
[mck/CASSANDRA-21620/trunk|https://github.com/thelastpickle/cassandra/tree/mck/CASSANDRA-21620/trunk]
Provenance:
[f57af096a0|https://github.com/datastax/cassandra/commit/f57af096a0a9b5225d46dc860ab683d26156a013]
by Jacek Lewandowski. That commit repairs the size computation only; the
writer already counts to the first null upstream.
was:
For an sstable version that still carries the legacy minimum and maximum
clustering values, {{StatsMetadata.StatsMetadataSerializer.serialize}} writes
the components of each bound up to the first null one, and writes that count:
{code:java}
out.writeInt(countUntilNull(minClusteringValues.getBufferArray()));
for (ByteBuffer value : minClusteringValues.getBufferArray())
{
if (value == null)
break;
ByteBufferUtil.writeWithShortLength(value, out);
}
{code}
{{serializedSize}} charges two length bytes for every component the bound
holds, null ones included:
{code:java}
size += minClusteringValues.size() * 2 /* short length */ +
minClusteringValues.dataSize();
{code}
A bound that {{MetadataCollector}} builds holds one entry per clustering
column, so a covered clustering that stops short of the last column makes the
two disagree. {{dataSize()}} already counts a null component as zero bytes, so
the difference is the two length bytes alone.
{{MetadataSerializer.serialize}} derives the table-of-contents offset of each
component from the returned size, and writes the whole backing array of a
buffer allocated to that size. The STATS component therefore ends with two
unused bytes per absent component. The file stays readable: the reader takes
the length of each component from the offsets and stops once it has read the
recorded count.
No production write reaches the disagreement today, which is why the ticket is
Low. A writer uses the latest version of its format, where
{{hasLegacyMinMax()}} is false, and a metadata mutation on an older sstable
rebuilds the bounds from disk, where they carry exactly the components that
were written and no nulls. The legacy branch is deprecated and due for removal
after {{oa}}; until then the two halves of the serializer must agree, because
the offset arithmetic depends on it.
The patch sizes both bounds with {{countUntilNull}}, the same count the writer
uses.
Patch:
[mck/upstream/stats-metadata-legacy-min-max-size/trunk|https://github.com/thelastpickle/cassandra/tree/mck/upstream/stats-metadata-legacy-min-max-size/trunk]
Provenance:
[f57af096a0|https://github.com/datastax/cassandra/commit/f57af096a0a9b5225d46dc860ab683d26156a013]
by Jacek Lewandowski. That commit repairs the size computation only; the
writer already counts to the first null upstream.
> Stats metadata over-sizes the legacy minimum and maximum clustering values
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-21620
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21620
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Local/SSTable
> Reporter: Michael Semb Wever
> Priority: Normal
> Fix For: 7.x
>
>
> For an sstable version that still carries the legacy minimum and maximum
> clustering values, {{StatsMetadata.StatsMetadataSerializer.serialize}} writes
> the components of each bound up to the first null one, and writes that count:
> {code:java}
> out.writeInt(countUntilNull(minClusteringValues.getBufferArray()));
> for (ByteBuffer value : minClusteringValues.getBufferArray())
> {
> if (value == null)
> break;
> ByteBufferUtil.writeWithShortLength(value, out);
> }
> {code}
> {{serializedSize}} charges two length bytes for every component the bound
> holds, null ones included:
> {code:java}
> size += minClusteringValues.size() * 2 /* short length */ +
> minClusteringValues.dataSize();
> {code}
> A bound that {{MetadataCollector}} builds holds one entry per clustering
> column, so a covered clustering that stops short of the last column makes the
> two disagree. {{dataSize()}} already counts a null component as zero bytes,
> so the difference is the two length bytes alone.
> {{MetadataSerializer.serialize}} derives the table-of-contents offset of each
> component from the returned size, and writes the whole backing array of a
> buffer allocated to that size. The STATS component therefore ends with two
> unused bytes per absent component. The file stays readable: the reader takes
> the length of each component from the offsets and stops once it has read the
> recorded count.
> No production write reaches the disagreement today, which is why the ticket
> is Low. A writer uses the latest version of its format, where
> {{hasLegacyMinMax()}} is false, and a metadata mutation on an older sstable
> rebuilds the bounds from disk, where they carry exactly the components that
> were written and no nulls. The legacy branch is deprecated and due for
> removal after {{oa}}; until then the two halves of the serializer must agree,
> because the offset arithmetic depends on it.
> The patch sizes both bounds with {{countUntilNull}}, the same count the
> writer uses.
> Patch:
> [mck/CASSANDRA-21620/trunk|https://github.com/thelastpickle/cassandra/tree/mck/CASSANDRA-21620/trunk]
> Provenance:
> [f57af096a0|https://github.com/datastax/cassandra/commit/f57af096a0a9b5225d46dc860ab683d26156a013]
> by Jacek Lewandowski. That commit repairs the size computation only; the
> writer already counts to the first null upstream.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
