[
https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713681&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713681
]
ASF GitHub Bot logged work on HADOOP-18091:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jan/22 11:43
Start Date: 24/Jan/22 11:43
Worklog Time Spent: 10m
Work Description: mukund-thakur commented on a change in pull request
#3916:
URL: https://github.com/apache/hadoop/pull/3916#discussion_r790658237
##########
File path:
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing_architecture.md
##########
@@ -141,8 +152,19 @@ thread.
### class `NoopAuditor`
-This auditor creates spans which perform no auditing.
-It is very efficient and reliable.
+This auditor creates spans which doesn't do anything with the events.
+
+```xml
+<property>
+ <name>fs.s3a.audit.service.classname</name>
+ <value>org.apache.hadoop.fs.s3a.audit.impl.NoopAuditor</value>
+</property>
+```
+
+This is *not* the same as disabling auditing, as it still uses the
`ActiveAuditManagerS3A` class
Review comment:
Getting confused here a bit. I see you have removed the threadlocal
field from NoopAuditManagerS3A. So ideally now it shouldn't cause the memory
leak. But again you are saying it still uses the ActiveAuditManagerS3A which
is a source of memory leaks.
And at the same point, when auditing is disabled by default, we are creating
stubAuditManager which in turn creates NoopAuditManagerS3A in new code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 713681)
Time Spent: 2.5h (was: 2h 20m)
> S3A auditing leaks memory through ThreadLocal references
> --------------------------------------------------------
>
> Key: HADOOP-18091
> URL: https://issues.apache.org/jira/browse/HADOOP-18091
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.2
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Blocker
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> {{ActiveAuditManagerS3A}} uses thread locals to map to active audit spans,
> which (because they are wrapped) include back reference to the audit manager
> instance and the config it was created with.
> these *do not* get cleaned up when the FS instance is closed.
> if you have a long lived process creating and destroying many FS instances,
> then memory gets used up. l
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
