[
https://issues.apache.org/jira/browse/CASSANDRA-21701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-21701:
-------------------------------------------
Description:
{{CompactionIterator}} refreshes its {{bytesRead}} field once every
{{UNFILTERED_TO_UPDATE_PROGRESS}} unfiltereds:
{code:java}
if ((++compactedUnfiltered) % UNFILTERED_TO_UPDATE_PROGRESS == 0)
updateBytesRead();
{code}
and {{getBytesRead()}} returned that field. Two things follow. The field lags
behind the scanners by up to a hundred unfiltereds, and it is still short of
the total once the iteration is over, because the last refresh lands on the
last multiple of a hundred.
{{ValidationManager}} assigns the value to {{ValidationState.bytesRead}} once
per partition:
{code:java}
state.bytesRead = vi.getBytesRead();
{code}
and {{LocalRepairTables}} reports it as the {{bytes_read}} column of
{{system_views.repair_validations}}. An operator watching a repair therefore
sees a figure that trails the work done, and a validation of fewer than a
hundred unfiltereds reports zero throughout.
{{getBytesRead()}} now sums the scanners on every call. The field remains for
{{getCompactionInfo}}, which reports from a background thread, so the refresh
interval still keeps that path off the scanners. The validation path asks once
per partition rather than once per unfiltered, so the summation is not on a hot
loop.
Patch:
[mck/CASSANDRA-21701/5.0|https://github.com/thelastpickle/cassandra/tree/mck/CASSANDRA-21701/5.0]
Provenance:
[fa9e504c11|https://github.com/datastax/cassandra/commit/fa9e504c1178927d12d40b340048818714f969c7]
by [~blambov]. That commit reaches the same result, on a class that has since
diverged: it folds a second accessor, {{getTotalBytesScanned}}, into
{{bytesRead()}}, and that second accessor does not exist here. That commit
carries tests for classes that do not exist here; this patch adds one that fits
the upstream class.
was:
{{CompactionIterator}} refreshes its {{bytesRead}} field once every
{{UNFILTERED_TO_UPDATE_PROGRESS}} unfiltereds:
{code:java}
if ((++compactedUnfiltered) % UNFILTERED_TO_UPDATE_PROGRESS == 0)
updateBytesRead();
{code}
and {{getBytesRead()}} returned that field. Two things follow. The field lags
behind the scanners by up to a hundred unfiltereds, and it is still short of
the total once the iteration is over, because the last refresh lands on the
last multiple of a hundred.
{{ValidationManager}} assigns the value to {{ValidationState.bytesRead}} once
per partition:
{code:java}
state.bytesRead = vi.getBytesRead();
{code}
and {{LocalRepairTables}} reports it as the {{bytes_read}} column of
{{system_views.repair_validations}}. An operator watching a repair therefore
sees a figure that trails the work done, and a validation of fewer than a
hundred unfiltereds reports zero throughout.
{{getBytesRead()}} now sums the scanners on every call. The field remains for
{{getCompactionInfo}}, which reports from a background thread, so the refresh
interval still keeps that path off the scanners. The validation path asks once
per partition rather than once per unfiltered, so the summation is not on a hot
loop.
Patch:
[mck/upstream/stale-bytes-read/5.0|https://github.com/thelastpickle/cassandra/tree/mck/upstream/stale-bytes-read/5.0]
Provenance:
[fa9e504c11|https://github.com/datastax/cassandra/commit/fa9e504c1178927d12d40b340048818714f969c7]
by [~blambov]. That commit reaches the same result, on a class that has since
diverged: it folds a second accessor, {{getTotalBytesScanned}}, into
{{bytesRead()}}, and that second accessor does not exist here. That commit
carries tests for classes that do not exist here; this patch adds one that fits
the upstream class.
> repair_validations reports a bytes_read figure that trails the work done
> ------------------------------------------------------------------------
>
> Key: CASSANDRA-21701
> URL: https://issues.apache.org/jira/browse/CASSANDRA-21701
> Project: Apache Cassandra
> Issue Type: Bug
> Components: Observability/Logging
> Reporter: Michael Semb Wever
> Priority: Normal
> Fix For: 5.0.x, 6.0.x, 7.x
>
>
> {{CompactionIterator}} refreshes its {{bytesRead}} field once every
> {{UNFILTERED_TO_UPDATE_PROGRESS}} unfiltereds:
> {code:java}
> if ((++compactedUnfiltered) % UNFILTERED_TO_UPDATE_PROGRESS == 0)
> updateBytesRead();
> {code}
> and {{getBytesRead()}} returned that field. Two things follow. The field lags
> behind the scanners by up to a hundred unfiltereds, and it is still short of
> the total once the iteration is over, because the last refresh lands on the
> last multiple of a hundred.
> {{ValidationManager}} assigns the value to {{ValidationState.bytesRead}} once
> per partition:
> {code:java}
> state.bytesRead = vi.getBytesRead();
> {code}
> and {{LocalRepairTables}} reports it as the {{bytes_read}} column of
> {{system_views.repair_validations}}. An operator watching a repair therefore
> sees a figure that trails the work done, and a validation of fewer than a
> hundred unfiltereds reports zero throughout.
> {{getBytesRead()}} now sums the scanners on every call. The field remains for
> {{getCompactionInfo}}, which reports from a background thread, so the refresh
> interval still keeps that path off the scanners. The validation path asks
> once per partition rather than once per unfiltered, so the summation is not
> on a hot loop.
> Patch:
> [mck/CASSANDRA-21701/5.0|https://github.com/thelastpickle/cassandra/tree/mck/CASSANDRA-21701/5.0]
> Provenance:
> [fa9e504c11|https://github.com/datastax/cassandra/commit/fa9e504c1178927d12d40b340048818714f969c7]
> by [~blambov]. That commit reaches the same result, on a class that has
> since diverged: it folds a second accessor, {{getTotalBytesScanned}}, into
> {{bytesRead()}}, and that second accessor does not exist here. That commit
> carries tests for classes that do not exist here; this patch adds one that
> fits the upstream class.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
