[
https://issues.apache.org/jira/browse/HADOOP-17511?focusedWorklogId=556501&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-556501
]
ASF GitHub Bot logged work on HADOOP-17511:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 23/Feb/21 18:20
Start Date: 23/Feb/21 18:20
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #2675:
URL: https://github.com/apache/hadoop/pull/2675#issuecomment-784410906
This hits the big milestone of *this has got complicated*.
For best-effort tracking of what is the FS operation this thread initiated,
we are functionally complete.
For anything more rigorous where the lifespan of operations is tracked: no.
I also think the attempts to preserve and restore the previous span get too
complex/unreliable once you start considering spans with more complex
lifecycles (what if you the previous span has already been closed?)
I'm going to take a couple of days off from this and do other stuff for the
3.3.1 release (which this is too big/complex to make).
Then think about how better to do this.
What do we want?
* Maybe differentiate bounded and long-lived spans? with different API to
create?
* Or declare that close() must be callable > once and do nothing but log in
an audit service
* and that its not an error to call it on a closed span.
The S3A FS code will be slightly more sophisticated in that bounded
try-with-resources spans
will, when close()d, switch the current thread to the unbonded span (i.e. no
attempt to restore
to previous one). And when we handoff() a span, we get a new span and the
active one is made inactive
immediately.
Like I said: I need to think of what we are trying to achieve here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 556501)
Time Spent: 8h (was: 7h 50m)
> Add an Audit plugin point for S3A auditing/context
> --------------------------------------------------
>
> Key: HADOOP-17511
> URL: https://issues.apache.org/jira/browse/HADOOP-17511
> Project: Hadoop Common
> Issue Type: Sub-task
> Affects Versions: 3.3.1
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Add a way for auditing tools to correlate S3 object calls with Hadoop FS API
> calls.
> Initially just to log/forward to an auditing service.
> Later: let us attach them as parameters in S3 requests, such as opentrace
> headeers or (my initial idea: http referrer header -where it will get into
> the log)
> Challenges
> * ensuring the audit span is created for every public entry point. That will
> have to include those used in s3guard tools, some defacto public APIs
> * and not re-entered for active spans. s3A code must not call back into the
> FS API points
> * Propagation across worker threads
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
