[
https://issues.apache.org/jira/browse/HADOOP-19303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928886#comment-17928886
]
ASF GitHub Bot commented on HADOOP-19303:
-----------------------------------------
steveloughran opened a new pull request, #7418:
URL: https://github.com/apache/hadoop/pull/7418
HADOOP-19303.
Adds a new method with a release method to
readVectored(List<? extends FileRange> ranges,
IntFunction<ByteBuffer> allocate,
Consumer<ByteBuffer> release)
This is return buffers to pools even in failures.
The default implementation hands back to readVectored/2, so that existing
custom implementations of that will get invoked.
S3A, ABFS and HDFS don't do this.
This is the hadoop-common side of #7105, which I'm going to retire to
focus on analytics vector read performance.
Getting the new API into hadoop releases before that means that it is
ready for applications to use, if built against it.
### How was this patch tested?
### For code changes:
- [ ] Does the title or this PR starts with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [=] Object storage: have the integration tests been executed and the
endpoint declared according to the connector-specific documentation?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files?
> VectorIO API to support releasing buffers on failure
> ----------------------------------------------------
>
> Key: HADOOP-19303
> URL: https://issues.apache.org/jira/browse/HADOOP-19303
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/s3
> Affects Versions: 3.4.1
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
>
> extend for vector IO API with a method that takes a ByteBufferPool
> implementation rather than just an allocator. This allows for buffers to be
> returned to the pool when problems occur, before throwing an exception.
> The Parquet API is already designed for this
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
