[
https://issues.apache.org/jira/browse/CASSANDRA-16325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677968#comment-17677968
]
Isaac Reath commented on CASSANDRA-16325:
-----------------------------------------
Thank you [~dcapwell]. I've now updated my PR to update metrics through a
{{StreamEventListener}}, overall seems like a much better outcome.
In testing I found that the {{CassandraCompressedStreamReader}} seemed to be
reporting way more bytes than were written. I did some digging and found that
{{cis.bytesRead()}} does not reset after each section is finished causing the
delta calculation to be incorrect after we've completed writing a section of an
SSTable. Moving that to outside of the {{for
(SSTableReader.PartitionPositionBounds section : sections)}} loop made the
metrics collection accurate.
[~jmckenzie], I'm not sure if moving this logic to a StreamEventListener
addresses your concerns about hammering the counters. Moving to a
{{StreamEventListener}} makes a threshold counter a bit more complicated (since
you will have 1:many relationship between {{StreamEventListener}} and peer,
forcing us to track the deltas per peer). Per [~dcapwell] we get a progress
event for every 64kb which may still be too often. I kept the patch simple for
now but we could move to either a batch-based approach or having a thread to
update the counters if you still feel this is too frequent.
> Update streaming metrics incrementally
> --------------------------------------
>
> Key: CASSANDRA-16325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16325
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Metrics
> Reporter: Paulo Motta
> Assignee: Isaac Reath
> Priority: Normal
> Labels: lhf
> Fix For: 4.2
>
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> Currently the inbound and outbound streamed bytes metrics are incremented
> after each file is streamed, what doesn't represent the current number of
> bytes streamed since it can take a long time for a large file to be streamed.
> We should update the metric incrementally as data is streamed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
