steveloughran opened a new pull request #3611: URL: https://github.com/apache/hadoop/pull/3611
Compared to #3597 this moves the recovery handling into the application code. However, the nominally simpler "fs api" strategy had actually got more complicated with interfaces in RawLocalFS for testing, etc etc, and the hive team were showing a dangerous amount of interest and using the same API, judging by the comments. This solution at one new interface in hadoop fs, which is straight forward it to declare stable as s3 and abfs and others can all serve this field up immediately. With the interface and part capabilities probes, anything is free to use this for: Tracking changes across files, verifying that objects are unchanged and this special case of resilience recovery. What else do we need? We would like, but don't need, an option for abfs to throw meaningful exceptions on rename failures and for it not to attempt to use modtimes to recover from source file not found events. ### For code changes: - [X] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
