[
https://issues.apache.org/jira/browse/HADOOP-19303?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17929577#comment-17929577
]
ASF GitHub Bot commented on HADOOP-19303:
-----------------------------------------
anmolanmol1234 commented on PR #7418:
URL: https://github.com/apache/hadoop/pull/7418#issuecomment-2677501986
> @cnauroth @anujmodi2021 have either of you two implemented the vector read
API yet?
>
> I ask as this PR currently maps the readVectored/3 call to the
readVectored/2 call unless overridden, so the default implementation will leak
buffers on failure, even if a release function is passed in.
>
> If I change it to passing the release call down, then any input stream
which implemented readVectored/2 will not have the readVectored/3 call invoking
it, unless they override that explicitly too. In this PR, everything in hadoop
common does, and I will in S3AInputStream.
>
> I'm just trying to work out the best design for other streams. IF all the
implementation are in the hadoop source tree, I can do the overrides there and
have a default which does release buffers everywhere else.
>
> * @mukund-thakur @ahmarsuhail @saikatroy038 @shameersss1
Hi @steveloughran, I am working on the vectored read API feature from the
ABFS driver team. We are still working on the design part of the feature and
will pick up the implementation soon.
> VectorIO API to support releasing buffers on failure
> ----------------------------------------------------
>
> Key: HADOOP-19303
> URL: https://issues.apache.org/jira/browse/HADOOP-19303
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs, fs/s3
> Affects Versions: 3.4.1
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Major
> Labels: pull-request-available
>
> extend for vector IO API with a method that takes a ByteBufferPool
> implementation rather than just an allocator. This allows for buffers to be
> returned to the pool when problems occur, before throwing an exception.
> The Parquet API is already designed for this
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
