[
https://issues.apache.org/jira/browse/CASSANDRA-16325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17656750#comment-17656750
]
Josh McKenzie commented on CASSANDRA-16325:
-------------------------------------------
I'm a little concerned about us hammering those Counters every time we transfer
any date:
{code:java}
// stream each of the required sections of the file
for (SSTableReader.PartitionPositionBounds section : sections)
{
long start = validator == null ? section.lowerPosition :
validator.chunkStart(section.lowerPosition);
// if the transfer does not start on the valididator's chunk boundary, this
is the number of bytes to offset by
int transferOffset = (int) (section.lowerPosition - start);
if (validator != null)
validator.seek(start);
// length of the section to read
long length = section.upperPosition - start;
// tracks write progress
long bytesRead = 0;
while (bytesRead < length)
{
int toTransfer = (int) Math.min(bufferSize, length - bytesRead);
long lastBytesRead = write(proxy, validator, out, start,
transferOffset, toTransfer, bufferSize);
start += lastBytesRead;
bytesRead += lastBytesRead;
progress += (lastBytesRead - transferOffset);
session.progress(sstable.descriptor.filenameFor(Component.DATA),
ProgressInfo.Direction.OUT, progress, totalSize);
fileStreamMetricsListener.onStreamingBytesTransferred(progress); //
REVIEWER NOTE: This is where we're going to update the counters on each call
transferOffset = 0;
}{code}
ISTM we might benefit from a simple threshold counter inside
{{CassandraStreamWriter}} where it'll update the metrics every N multiples of
{{CassandraStreamWriter.DEFAULT_CHUNK_SIZE}} so we don't spam the metrics
updates.
> Update streaming metrics incrementally
> --------------------------------------
>
> Key: CASSANDRA-16325
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16325
> Project: Cassandra
> Issue Type: Improvement
> Components: Observability/Metrics
> Reporter: Paulo Motta
> Assignee: Isaac Reath
> Priority: Normal
> Labels: lhf
> Fix For: 4.2
>
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> Currently the inbound and outbound streamed bytes metrics are incremented
> after each file is streamed, what doesn't represent the current number of
> bytes streamed since it can take a long time for a large file to be streamed.
> We should update the metric incrementally as data is streamed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
