[
https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713718&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713718
]
ASF GitHub Bot logged work on HADOOP-18091:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jan/22 13:14
Start Date: 24/Jan/22 13:14
Worklog Time Spent: 10m
Work Description: steveloughran commented on a change in pull request
#3916:
URL: https://github.com/apache/hadoop/pull/3916#discussion_r790729182
##########
File path:
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing_architecture.md
##########
@@ -141,8 +152,19 @@ thread.
### class `NoopAuditor`
-This auditor creates spans which perform no auditing.
-It is very efficient and reliable.
+This auditor creates spans which doesn't do anything with the events.
+
+```xml
+<property>
+ <name>fs.s3a.audit.service.classname</name>
+ <value>org.apache.hadoop.fs.s3a.audit.impl.NoopAuditor</value>
+</property>
+```
+
+This is *not* the same as disabling auditing, as it still uses the
`ActiveAuditManagerS3A` class
Review comment:
there is a NoopAuditor, this is the one you could already enable, which
generated `NoopAuditSpans`
but there was also a stub audit manager, which is the class s3afs created to
handle all the audit integration for that instance. it was used in two places
* mock s3a FS instances in unit tests
* in s3afs initialize so even when the aws sdk was probing s3, the request
processing would find something.
the change now then is to make that no op manager the default everywhere,
with tests to verify this. the parameterisation where will cause a bit of
backport conflict with branches which still have s3guard on, i will address that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 713718)
Time Spent: 3h (was: 2h 50m)
> S3A auditing leaks memory through ThreadLocal references
> --------------------------------------------------------
>
> Key: HADOOP-18091
> URL: https://issues.apache.org/jira/browse/HADOOP-18091
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.2
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Blocker
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {{ActiveAuditManagerS3A}} uses thread locals to map to active audit spans,
> which (because they are wrapped) include back reference to the audit manager
> instance and the config it was created with.
> these *do not* get cleaned up when the FS instance is closed.
> if you have a long lived process creating and destroying many FS instances,
> then memory gets used up. l
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
