steveloughran opened a new pull request #2807: URL: https://github.com/apache/hadoop/pull/2807
Notion of AuditSpan which is created for a given operation; goal is to pass it along everywhere. It's thread local per FS-instance; store operations pick this up in their constructor from the StoreContext. The entryPoint() method in S3A FS has been enhanced to initiate the spans. For this to work, internal code SHALL NOT call those entry points (Done) and all public API points MUST be declared as entry points. This is done, with a marker attribute `@AuditEntryPoint` to indicate this. The audit span create/deactivate sequence is ~the same as the duration tracking so the operation is generally merged: most of the metrics S3AFS collects are now durations Part of the isolation into spans means that there's explicit operations for mkdirs() and getContentSummary() The auditing is intended to be a plugin point; currently there is the LoggingAuditor which * logs at debug * adds an HTTP "referer" header with audit tracing * can be set to raise an exception if the SDK is handed an AWS Request and there's no active span (skipped for the multipart upload part and complete calls as TransferManager in the SDK does that out of span). NoopAuditor which: *does nothing A recent change is that we want every span to have a spanID (string, unique across all spans of that FS instance); even the no-op span has unique IDs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
