[
https://issues.apache.org/jira/browse/HADOOP-13028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276778#comment-15276778
]
Colin Patrick McCabe commented on HADOOP-13028:
-----------------------------------------------
bq. I think this is OK. The whole close method is synchronized, so we won't
have two threads concurrently doing the actual close. Almost all other accesses
of closed are within synchronized methods too. It's marked volatile to help
with one unsynchronized access from readFully, calling into checkNotClosed.
That's only a read, not an update, so volatile is sufficient.
Thanks for the explanation. I missed the interaction between {{synchronized}}
and the assignment. Suggest adding a comment to the assignment in {{close()}}
explaining why this is atomic, or simply using AtomicBoolean to future-proof
this against later code changes.
bq. I'd like to keep \[the toString changes\]. It's very convenient for
logging. TestS3AInputStreamPerformance uses it for both logging output and
detailed assertion messages. It's poor practice to rely on a Java object's
toString output as a stable, parseable format. This is something that I'd like
to see clarified in our compatibility documentation.
The problem is, this is not consistent with how {{toString}} operates in other
FS streams. We also don't have anything in our compatibility documentation
stating that the output of {{toString}} is not a stable, parseable format.
We've had many, many JIRAs to "make toString act like some previous behavior"
for various Hadoop classes. I think we need to accept that currently the
stream's {{toString}} method is viewed as a public, stable API whether we like
it or not.
How about just adding this information to the {{toString}} method of the stream
statistics object? That makes more sense anyway.
> add low level counter metrics for S3A; use in read performance tests
> --------------------------------------------------------------------
>
> Key: HADOOP-13028
> URL: https://issues.apache.org/jira/browse/HADOOP-13028
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3, metrics
> Affects Versions: 2.8.0
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Attachments: HADOOP-13028-001.patch, HADOOP-13028-002.patch,
> HADOOP-13028-004.patch, HADOOP-13028-005.patch, HADOOP-13028-006.patch,
> HADOOP-13028-007.patch, HADOOP-13028-008.patch, HADOOP-13028-009.patch,
> HADOOP-13028-branch-2-008.patch, HADOOP-13028-branch-2-009.patch,
> HADOOP-13028-branch-2-010.patch,
> org.apache.hadoop.fs.s3a.scale.TestS3AInputStreamPerformance-output.txt,
> org.apache.hadoop.fs.s3a.scale.TestS3AInputStreamPerformance-output.txt
>
>
> against S3 (and other object stores), opening connections can be expensive,
> closing connections may be expensive (a sign of a regression).
> S3A FS and individual input streams should have counters of the # of
> open/close/failure+reconnect operations, timers of how long things take. This
> can be used downstream to measure efficiency of the code (how often
> connections are being made), connection reliability, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
