steveloughran opened a new pull request, #8623:
URL: https://github.com/apache/hadoop/pull/8623
All upload operations are in MultipartIO service, which has been renamed and
move to package org.apache.hadoop.fs.s3a.impl.write to match.
For completeness deletion should also go into this class or an adjacent one
on deletion.
Pulled out multipart IO such that
there are no back references from it to
S3AStore -the final change is to define a store
statistics class which it and other things can
use to update stats.
Executors in hadoop-common to
- pick up shutdown of inner executor and shut themselves down.
- semaphore executor to decrement counters in this process so that queue
state is updated
- semaphored delegating executor unit test in common This stops callers
being able to submit work when the inner executor has shut down.
WriteOperationHelper
* make all calls through its callback interface, rather than given a ref to
S3AFS.
* Move WriteOperationHelper callbacks to S3Store layer,
Multipart IO operations
* move nearly all Multipart IO operationss out of s3afs and into a new
mulitpart service interface and impl
* Multipart service retrieved and invoked as appropriate
* StoreImpl stores a map of ServiceName -> service. with a lookupService()
method in S3AStore interface, it's possible to retrieve services through the
API just by knowing their name and type
* registering all current services this way
StoreImpl to IllegalStateException on method invocation whene the service
isn't running.
Some methods are kept open as they do seem needed.
### How was this patch tested?
### For code changes:
- [X] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
### AI Tooling
If an AI tool was used:
- [ ] The PR includes the phrase "Contains content generated by <tool>"
where <tool> is the name of the AI tool used.
- [ ] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
