steveloughran commented on pull request #1982:
URL: https://github.com/apache/hadoop/pull/1982#issuecomment-630284411
This patch adds LocalFS/RawLocalFS/ChecksummedFileSystem statistics
passthrough and collection.
Getting everything passed through is actually the harder part of the
process...
I ended up having to debug things just work out what is going on there.
The fact that the raw local streams are buffered complicates testing.
The write tests expect the counter to not update until the stream
has closed; I need to expand the read tests this way too.
Although it makes for a bigger patch, it means that we get unit tests in
hadoop-common and that passthrough is all correct. It will also permit
applications to collect IO statistics on local storage operations.
new interface/impl to make it easy to instrument a class;
a map of key -> atomic long is built up as well as the stats mapping,
all the stream needs to add is a varags list of counters
```
private final CounterIOStatistics ioStatistics = counterIOStatistics(
STREAM_READ_BYTES,
STREAM_READ_EXCEPTIONS,
STREAM_READ_SEEK_OPERATIONS,
STREAM_READ_SKIP_OPERATIONS,
STREAM_READ_SKIP_BYTES);
```
which can then be set or incremented
```
ioStatistics.increment(STREAM_READ_BYTES, 1);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
