[
https://issues.apache.org/jira/browse/HADOOP-13028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-13028:
------------------------------------
Attachment: HADOOP-13028-001.patch
Patch 001
# lifted and tuned {{AzureFileSystemInstrumentation}}
# removed the gauges of current bandwidth and things that s3 doesn't offer
# added counters for the stream-level operations
# added counters for copy operations and size of files
# went through S3A FS and input stream invoking the instrumentation operations
as required
# cleaned up the the debug statements to be fully "SLF4J" in the process.
> add counter and timer metrics for S3A HTTP & low-level operations
> -----------------------------------------------------------------
>
> Key: HADOOP-13028
> URL: https://issues.apache.org/jira/browse/HADOOP-13028
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, metrics
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Minor
> Attachments: HADOOP-13028-001.patch
>
>
> against S3 (and other object stores), opening connections can be expensive,
> closing connections may be expensive (a sign of a regression).
> S3A FS and individual input streams should have counters of the # of
> open/close/failure+reconnect operations, timers of how long things take. This
> can be used downstream to measure efficiency of the code (how often
> connections are being made), connection reliability, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
