[
https://issues.apache.org/jira/browse/HADOOP-13453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967777#comment-15967777
]
Steve Loughran commented on HADOOP-13453:
-----------------------------------------
I see your point about quantile config: don't know what to do there.
* be careful with those imports, S3AInstrumentation had all its imports
expanded and moved around. Import sections are one of the main merge-conflict
areas, so its critical to keep changes there to a minimum. We normally turn off
any IDE automatic features to avoid this.
* moved off the split of separate interface and impl for the instrumentation;
everything is closely couple enough we don't need to abstract things away.
I'm doing a patch with my changes; if everything is happy and I can run a full
integration test suite (HADOOP-14216 has broken this), then I'll +1 it; once it
is in we can expand the metrics
> S3Guard: Instrument new functionality with Hadoop metrics.
> ----------------------------------------------------------
>
> Key: HADOOP-13453
> URL: https://issues.apache.org/jira/browse/HADOOP-13453
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Chris Nauroth
> Assignee: Ai Deng
> Attachments: HADOOP-13453-HADOOP-13345-001.patch,
> HADOOP-13453-HADOOP-13345-002.patch, HADOOP-13453-HADOOP-13345-003.patch
>
>
> Provide Hadoop metrics showing operational details of the S3Guard
> implementation.
> The metrics will be implemented in this ticket:
> ● S3GuardRechecksNthPercentileLatency (MutableQuantiles) Percentile time
> spent
> in rechecks attempting to achieve consistency. Repeated for multiple
> percentile values
> of N. This metric is an indicator of the additional latency cost of running
> S3A with
> S3Guard.
> ● S3GuardRechecksNumOps (MutableQuantiles) Number of times a consistency
> recheck was required while attempting to achieve consistency.
> ● S3GuardStoreNthPercentileLatency (MutableQuantiles) Percentile time
> spent in
> operations against the consistent store, including both write operations
> during file system
> mutations and read operations during file system consistency checks. Repeated
> for
> multiple percentile values of N. This metric is an indicator of latency to
> the consistent
> store implementation.
> ● S3GuardConsistencyStoreNumOps (MutableQuantiles) Number of operations
> against the consistent store, including both write operations during file
> system mutations
> and read operations during file system consistency checks.
> ● S3GuardConsistencyStoreFailures (MutableCounterLong) Number of failures
> during operations against the consistent store implementation.
> ● S3GuardConsistencyStoreTimeouts (MutableCounterLong) Number of timeouts
> during operations against the consistent store implementation.
> ● S3GuardInconsistencies (MutableCounterLong) C ount of times S3Guard
> failed to
> achieve consistency, even after exhausting all rechecks. A high count may
> indicate
> unexpected outofband modification of the S3 bucket contents, such as by an
> external
> tool that does not make corresponding updates to the consistent store.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
