[
https://issues.apache.org/jira/browse/HADOOP-13028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278800#comment-15278800
]
Colin Patrick McCabe commented on HADOOP-13028:
-----------------------------------------------
bq. Patrick: regarding fs.s3a.readahead.range versus calling it
fs.s3a.readahead.default, I think "default" could be a bit confusing too. How
about I make it clear that the if setReadahead() is set, then it supercedes any
previous value?
Sure.
bq. I absolutely need that printing in there, otherwise the value of this patch
is significantly reduced. If you want me to add a line like "WARNING: UNSTABLE"
or something to that string value, I'm happy to do so. Or the output is
published in a way that is deliberately hard to parse by machine but which we
humans can read. But without that information, we can't so easily tell which
Perhaps I'm missing something, but why not just do this in
{{S3AInstrumentation#InputStreamStatistics#toString}}? I don't see why this is
"absolutely needed" in {{S3AInputStream#toString}}.
> add low level counter metrics for S3A; use in read performance tests
> --------------------------------------------------------------------
>
> Key: HADOOP-13028
> URL: https://issues.apache.org/jira/browse/HADOOP-13028
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, metrics
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-13028-001.patch, HADOOP-13028-002.patch,
> HADOOP-13028-004.patch, HADOOP-13028-005.patch, HADOOP-13028-006.patch,
> HADOOP-13028-007.patch, HADOOP-13028-008.patch, HADOOP-13028-009.patch,
> HADOOP-13028-branch-2-008.patch, HADOOP-13028-branch-2-009.patch,
> HADOOP-13028-branch-2-010.patch, HADOOP-13028-branch-2-011.patch,
> org.apache.hadoop.fs.s3a.scale.TestS3AInputStreamPerformance-output.txt,
> org.apache.hadoop.fs.s3a.scale.TestS3AInputStreamPerformance-output.txt
>
>
> against S3 (and other object stores), opening connections can be expensive,
> closing connections may be expensive (a sign of a regression).
> S3A FS and individual input streams should have counters of the # of
> open/close/failure+reconnect operations, timers of how long things take. This
> can be used downstream to measure efficiency of the code (how often
> connections are being made), connection reliability, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
