[
https://issues.apache.org/jira/browse/CASSANDRA-16325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656998#comment-17656998
]
David Capwell commented on CASSANDRA-16325:
-------------------------------------------
bq. I think if we updated the ProgressInfo object to store the delta from the
last time progress was called
I just looked into this and it would require light caching to the callers, such
as
{code}
diff --git
src/java/org/apache/cassandra/db/streaming/CassandraCompressedStreamReader.java
src/java/org/apache/cassandra/db/streaming/CassandraCompressedStreamReader.java
index dda874ba40..b9f4566aac 100644
---
src/java/org/apache/cassandra/db/streaming/CassandraCompressedStreamReader.java
+++
src/java/org/apache/cassandra/db/streaming/CassandraCompressedStreamReader.java
@@ -79,6 +79,7 @@ public class CassandraCompressedStreamReader extends
CassandraStreamReader
writer = createWriter(cfs, totalSize, repairedAt, pendingRepair,
format);
String filename = writer.getFilename();
int sectionIdx = 0;
+ long read = 0;
for (SSTableReader.PartitionPositionBounds section : sections)
{
assert cis.chunkBytesRead() <= totalSize;
@@ -93,7 +94,10 @@ public class CassandraCompressedStreamReader extends
CassandraStreamReader
{
writePartition(deserializer, writer);
// when compressed, report total bytes of compressed
chunks read since remoteFile.size is the sum of chunks transferred
- session.progress(filename + '-' + fileSeqNum,
ProgressInfo.Direction.IN, cis.chunkBytesRead(), totalSize);
+ long bytes = cis.chunkBytesRead();
+ long delta = bytes - read;
+ read = bytes;
+ session.progress(filename + '-' + fileSeqNum,
ProgressInfo.Direction.IN, bytes, delta, totalSize);
}
assert in.getBytesRead() == sectionLength;
}
{code}
So, if the reason to avoid using a listener was tracking progress info, we can
compute the delta at the event creation side (non look hard), I personally find
that better than embedding your custom metrics listener as it gives benefits to
other listeners (such as removing the cache in StreamingState)
> Update streaming metrics incrementally
> --------------------------------------
>
> Key: CASSANDRA-16325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16325
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Metrics
> Reporter: Paulo Motta
> Assignee: Isaac Reath
> Priority: Normal
> Labels: lhf
> Fix For: 4.2
>
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> Currently the inbound and outbound streamed bytes metrics are incremented
> after each file is streamed, what doesn't represent the current number of
> bytes streamed since it can take a long time for a large file to be streamed.
> We should update the metric incrementally as data is streamed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
