[
https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713416&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713416
]
ASF GitHub Bot logged work on HADOOP-18091:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 23/Jan/22 20:55
Start Date: 23/Jan/22 20:55
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #3916:
URL: https://github.com/apache/hadoop/pull/3916#issuecomment-1019565240
two ARN failures from SDK update (noted on
https://issues.apache.org/jira/browse/HADOOP-18085 ); other test failures are
due to audit events not being collected. will address those by enabling
auditing for those tests
```
[ERROR] Failures:
[ERROR]
ITestS3AContractUnbuffer>AbstractContractUnbufferTest.testUnbufferMultipleReads:116->AbstractContractUnbufferTest.validateFileContents:139->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
failed to read expected number of bytes from stream. This may be transient
expected:<512> but was:<391>
[ERROR]
ITestS3AMiscOperationCost.testGetContentMissingPath:129->AbstractS3ACostTest.verifyMetricsIntercepting:317->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
operation returning java.io.FileNotFoundException: No such file or directory:
s3a://stevel-london/fork-0005/test/testGetContentMissingPath[delete-markers]:
audit_span_creation expected:<1> but was:<0>
[ERROR]
ITestS3AMiscOperationCost.testGetContentMissingPath:129->AbstractS3ACostTest.verifyMetricsIntercepting:317->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
operation returning java.io.FileNotFoundException: No such file or directory:
s3a://stevel-london/fork-0005/test/testGetContentMissingPath[keep-markers]:
audit_span_creation expected:<1> but was:<0>
[ERROR]
ITestS3AMiscOperationCost.testGetContentSummaryDir:110->AbstractS3ACostTest.verifyMetrics:294->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
operation returning none inf none
inf 2 1 0 : audit_span_creation
expected:<1> but was:<0>
[ERROR]
ITestS3AMiscOperationCost.testGetContentSummaryDir:110->AbstractS3ACostTest.verifyMetrics:294->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
operation returning none inf none
inf 2 1 0 : audit_span_creation
expected:<1> but was:<0>
[ERROR]
ITestS3AMiscOperationCost.testMkdirOverDir:83->AbstractS3ACostTest.verifyMetrics:294->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
operation returning true: audit_span_creation expected:<1> but was:<0>
[ERROR]
ITestS3AMiscOperationCost.testMkdirOverDir:83->AbstractS3ACostTest.verifyMetrics:294->Assert.assertEquals:647->Assert.failNotEquals:835->Assert.fail:89
operation returning true: audit_span_creation expected:<1> but was:<0>
[ERROR] Errors:
[ERROR]
ITestS3ABucketExistence.testAccessPointProbingV2:171->expectUnknownStore:103->lambda$testAccessPointProbingV2$12:172
» IllegalArgument
[ERROR]
ITestS3ABucketExistence.testAccessPointRequired:188->expectUnknownStore:103->lambda$testAccessPointRequired$14:189
» IllegalArgument
[INFO]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 713416)
Time Spent: 40m (was: 0.5h)
> S3A auditing leaks memory through ThreadLocal references
> --------------------------------------------------------
>
> Key: HADOOP-18091
> URL: https://issues.apache.org/jira/browse/HADOOP-18091
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.2
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Blocker
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {{ActiveAuditManagerS3A}} uses thread locals to map to active audit spans,
> which (because they are wrapped) include back reference to the audit manager
> instance and the config it was created with.
> these *do not* get cleaned up when the FS instance is closed.
> if you have a long lived process creating and destroying many FS instances,
> then memory gets used up. l
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
