[
https://issues.apache.org/jira/browse/HADOOP-13028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15254358#comment-15254358
]
Colin Patrick McCabe commented on HADOOP-13028:
-----------------------------------------------
Hi [~steve_l],
This is a really interesting idea. I think this ties in with some of the
discussions we've been having on HDFS-10175 with adding a way to fetch
arbitrary statistics from FileSystem (and FileContext) instances.
Basically, HDFS-10175 provides a way for MR to enumerate all the statistics and
their values. It also provides interfaces for finding just one statistic, of
course. This would also enable the use of those statistics in unit tests,
since the stats could be per-FS rather than global per type.
> add counter and timer metrics for S3A HTTP & low-level operations
> -----------------------------------------------------------------
>
> Key: HADOOP-13028
> URL: https://issues.apache.org/jira/browse/HADOOP-13028
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, metrics
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
>
> against S3 (and other object stores), opening connections can be expensive,
> closing connections may be expensive (a sign of a regression).
> S3A FS and individual input streams should have counters of the # of
> open/close/failure+reconnect operations, timers of how long things take. This
> can be used downstream to measure efficiency of the code (how often
> connections are being made), connection reliability, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
