[
https://issues.apache.org/jira/browse/HADOOP-18091?focusedWorklogId=713639&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-713639
]
ASF GitHub Bot logged work on HADOOP-18091:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 24/Jan/22 10:27
Start Date: 24/Jan/22 10:27
Worklog Time Spent: 10m
Work Description: mehakmeet commented on a change in pull request #3916:
URL: https://github.com/apache/hadoop/pull/3916#discussion_r790601482
##########
File path:
hadoop-tools/hadoop-aws/src/site/markdown/tools/hadoop-aws/auditing.md
##########
@@ -63,27 +75,43 @@ ideally even identifying the process/job generating load.
## Using Auditing
-The Logging Auditor is enabled by default; it annotates the S3 logs.
+Auditing is disabled by default.
+When auditing enabled, a Logging Auditor will annotate the S3 logs through a
custom
+HTTP Referrer header in requests made to S3.
+Other auditor classes may be used instead.
### Auditor Options
| Option | Meaning | Default Value |
|--------|---------|---------------|
+| `fs.s3a.audit.enabled` | Is auditing enabled | `false` |
| `fs.s3a.audit.service.classname` | Auditor classname |
`org.apache.hadoop.fs.s3a.audit.impl.LoggingAuditor` |
| `fs.s3a.audit.request.handlers` | List of extra subclasses of AWS SDK
RequestHandler2 to include in handler chain | `""` |
| `fs.s3a.audit.referrer.enabled` | Logging auditor to publish the audit
information in the HTTP Referrer header | `true` |
| `fs.s3a.audit.referrer.filter` | List of audit fields to filter | `""` |
| `fs.s3a.audit.reject.out.of.span.operations` | Auditor to reject operations
"outside of a span" | `false` |
-### Disabling Auditing with the No-op Auditor
+### Disabling Auditing.
Review comment:
ah yes, you're right. Just realized, ActiveAuditManager with NoopAuditor
would still leak, but NoopAuditManager is the one without the ThreadLocal
variable now. Thanks for clearing it 👍
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 713639)
Time Spent: 2h 10m (was: 2h)
> S3A auditing leaks memory through ThreadLocal references
> --------------------------------------------------------
>
> Key: HADOOP-18091
> URL: https://issues.apache.org/jira/browse/HADOOP-18091
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/s3
> Affects Versions: 3.3.2
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Blocker
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> {{ActiveAuditManagerS3A}} uses thread locals to map to active audit spans,
> which (because they are wrapped) include back reference to the audit manager
> instance and the config it was created with.
> these *do not* get cleaned up when the FS instance is closed.
> if you have a long lived process creating and destroying many FS instances,
> then memory gets used up. l
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
