[
https://issues.apache.org/jira/browse/HADOOP-19365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18028359#comment-18028359
]
Steve Loughran commented on HADOOP-19365:
-----------------------------------------
I can verify this works as we got a counter failure when I was changing VPN
settings during a test run
{code}
[ERROR]
org.apache.hadoop.fs.s3a.ITestS3AAnalyticsAcceleratorStreamReading.testMultiRowGroupParquet
-- Time elapsed: 4.123 s <<< FAILURE!
org.opentest4j.AssertionFailedError:
[Counter named audit_request_execution with expected value 4]
Expecting:
<12L>
to be equal to:
<4L>
but was not.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
org.apache.hadoop.fs.statistics.IOStatisticAssertions.verifyStatisticValue(IOStatisticAssertions.java:274)
at
org.apache.hadoop.fs.statistics.IOStatisticAssertions.verifyStatisticCounterValue(IOStatisticAssertions.java:175)
at
org.apache.hadoop.fs.s3a.ITestS3AAnalyticsAcceleratorStreamReading.testMultiRowGroupParquet(ITestS3AAnalyticsAcceleratorStreamReading.java:186)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at java.util.ArrayList.forEach(ArrayList.java:1259)
{code}
the SDK must have retried and so the counter incremented
> S3A Analytics-Accelerator: Add audit header support
> ----------------------------------------------------
>
> Key: HADOOP-19365
> URL: https://issues.apache.org/jira/browse/HADOOP-19365
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Reporter: Ahmar Suhail
> Priority: Major
> Labels: pull-request-available
>
> S3A adds audit information as referrer headers, see
> [https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing.md]
> for documentation on this.
> These are attached using execution interceptors, see ActiveAuditManagerS3A.
> createExecutionInterceptors().
>
> analyitcs-accelerator-s3 makes the GET requests now but does not update the
> referrer with GETs.
>
> See LoggingAuditor.attachRangeFromRequest() for how this is done.
>
> When using analytics-accelerator-s3, audit headers from S3A should be sent
> through.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
