[
https://issues.apache.org/jira/browse/HADOOP-13453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15878886#comment-15878886
]
Steve Loughran commented on HADOOP-13453:
-----------------------------------------
thanks, I've hit the submit patch button, but jenkins will fail as the patch
isn't going to apply
the way Hadoop builds work is you need to include the branch name if its not
trunk, here
{code}
HADOOP-13453-HADOOP-13345-001.patch
{code}
Try that with the existing code and you should have the machines review the
patch (which we rely on to do the basic checks).
For object store tests we also require the submitter to declare which s3
infrastructure they tested against —because Jenkins doesn't run those tests.
Here's the details:
https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/testing.md
Patch-wise, base design looks good: you've added a new statistic and passing
the instrumentation down to add it.
We just need to think of all the statistics to collect.
Thanks for doing this.
> S3Guard: Instrument new functionality with Hadoop metrics.
> ----------------------------------------------------------
>
> Key: HADOOP-13453
> URL: https://issues.apache.org/jira/browse/HADOOP-13453
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Ai Deng
> Attachments: HADOOP-13453.wip-01.patch
>
>
> Provide Hadoop metrics showing operational details of the S3Guard
> implementation.
> The metrics will be implemented in this ticket:
> ● S3GuardRechecksNthPercentileLatency (MutableQuantiles) Percentile time
> spent
> in rechecks attempting to achieve consistency. Repeated for multiple
> percentile values
> of N. This metric is an indicator of the additional latency cost of running
> S3A with
> S3Guard.
> ● S3GuardRechecksNumOps (MutableQuantiles) Number of times a consistency
> recheck was required while attempting to achieve consistency.
> ● S3GuardStoreNthPercentileLatency (MutableQuantiles) Percentile time
> spent in
> operations against the consistent store, including both write operations
> during file system
> mutations and read operations during file system consistency checks. Repeated
> for
> multiple percentile values of N. This metric is an indicator of latency to
> the consistent
> store implementation.
> ● S3GuardConsistencyStoreNumOps (MutableQuantiles) Number of operations
> against the consistent store, including both write operations during file
> system mutations
> and read operations during file system consistency checks.
> ● S3GuardConsistencyStoreFailures (MutableCounterLong) Number of failures
> during operations against the consistent store implementation.
> ● S3GuardConsistencyStoreTimeouts (MutableCounterLong) Number of timeouts
> during operations against the consistent store implementation.
> ● S3GuardInconsistencies (MutableCounterLong) C ount of times S3Guard
> failed to
> achieve consistency, even after exhausting all rechecks. A high count may
> indicate
> unexpected outofband modification of the S3 bucket contents, such as by an
> external
> tool that does not make corresponding updates to the consistent store.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
