Steve Loughran created HADOOP-15348:
---------------------------------------
Summary: S3A Input Stream bytes read counter isn't getting through
to StorageStatistics/insturmentation properly
Key: HADOOP-15348
URL: https://issues.apache.org/jira/browse/HADOOP-15348
Project: Hadoop Common
Issue Type: Sub-task
Components: fs/s3
Affects Versions: 3.0.0, 3.1.0
Reporter: Steve Loughran
TL;DR: we should have common storage statistics for bytes read and bytes
written, and S3A should use them in its instrumentation and have enum names to
match.
# in the S3AInputStream we call
{{S3AInstrumentation.StreamStatistics.bytesRead(long)}}, which adds the amount
to {{bytesRead}}, in a read(), readFully, or forward seek() reading in data
# and in {{S3AInstrumentation.mergeInputStreamStatistics}}, that is pulled into
streamBytesRead.
# which has a Statistics name of ""stream_bytes_read"
# but that is served up in the Storage statistics as "STREAM_SEEK_BYTES_READ",
which is the wrong name.
# and there isn't a common name for the counter across other filesystems.
For now: people can use the wrong name in the enum; we may want to think about
retaining it when adding the correct name. And maybe add a
@Evolving/@LimitedPrivate scope pair to the enum
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
