Steve Loughran created HADOOP-17469:
---------------------------------------
Summary: IOStatistics Phase II
Key: HADOOP-17469
URL: https://issues.apache.org/jira/browse/HADOOP-17469
Project: Hadoop Common
Issue Type: New Feature
Components: fs, fs/azure, fs/s3
Affects Versions: 3.3.1
Reporter: Steve Loughran
Continue IOStatistics development with goals of
* Easy adoption in applications
* better instrumentation in hadoop codebase (distcp?)
* more stats in abfs and s3a connectors
A key has to be a thread level context for statistics so that app code doesn't
have to explicitly ask for the stats for each worker thread. Instead
filesystem components update the context stats as well as thread stats (when?)
and then apps can pick up.
* need to manage performance by minimising inefficient lookups, lock
acquisition etc on what should be memory-only ops (read()), (write()),
* and for duration tracking, cut down on calls to System.currentTime() so that
only 1 should be made per operation,
* need to propagate the context into worker threads
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
