[
https://issues.apache.org/jira/browse/HADOOP-16906?focusedWorklogId=549508&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-549508
]
ASF GitHub Bot logged work on HADOOP-16906:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 08/Feb/21 10:56
Start Date: 08/Feb/21 10:56
Worklog Time Spent: 10m
Work Description: steveloughran commented on pull request #2684:
URL: https://github.com/apache/hadoop/pull/2684#issuecomment-775058042
Thanks @mukund-thakur
Had some thoughts over the w/e
1. The call should return something, `AbortOutcome` which can include a list
of non-critical exceptions caught during the call, just for diags. And to give
us the option of adding IOStatistics to an impl
2. change the spec to differentiate "ops which MUST succeed for abort"
(which for s3a is a no-op) and ops which MAY succeed and are which needed for
cleanup. The MAY succeed operations SHOULD be best effort, but MUST NOT block
for retries etc on failures.
The s3a stream abort() does do retries right now, and I think that's
probably a mistake for something which is only cleanup and is usually invoked
when things are starting to go wrong (e.g. data post/complete). Saying "no
retries here", means that we can have a fast abort() rather than a 60-120
second spin before the failure is caught and swallowed. I'll change the
relevant bits of s3a abort to `once()` in this situation
Makes sense?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 549508)
Time Spent: 4h 20m (was: 4h 10m)
> Add some Abortable.abort() interface for streams etc which can be terminated
> ----------------------------------------------------------------------------
>
> Key: HADOOP-16906
> URL: https://issues.apache.org/jira/browse/HADOOP-16906
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/azure, fs/s3
> Affects Versions: 3.3.0
> Reporter: Steve Loughran
> Assignee: Jungtaek Lim
> Priority: Blocker
> Labels: pull-request-available
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Some IO we want to be able to abort rather than close cleanly, especially if
> the inner stream is an HTTP connection which itself supports some abort()
> method. For example: uploads to an object where we want to cancel the upload
> without close() making an incomplete write visible.
> Proposed: Add a generic interface which things like streams can implement
> {code}
> AbortableIO {
> public void abortIO() throws IOE;
> }
> {code}
> +do for s3a output stream. I wouldn't do this a passthrough on
> FSDataOutputStream because we need to consider what expectations callers have
> of an operation being "aborted"
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
