[
https://issues.apache.org/jira/browse/ARROW-18113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17634076#comment-17634076
]
Percy Camilo Triveño Aucahuasi commented on ARROW-18113:
--------------------------------------------------------
Thanks for all the feedback!
Quick question: I see we have {_}arrow::io::CacheOptions{_}, but it seems that
most of its contents are related to coalescing (except perhaps
{_}arrow::io::CacheOptions::lazy{_}); does it sounds a good idea to rename it
to _arrow::io::CoalesceOptions_ and move it into {_}interfaces.h{_}?
The reason why I think we may need to move it into _interfaces.h_ is that
_ReadManyAsync_ would be part of _RandomAccessFile_ and we may need some of the
attributes we have in _arrow::io::CacheOptions_ (like _hole_size_limit_ and
{_}range_size_limit{_}) ... mostly becouse for the the implementation of
_ReadManyAsync_ we may need to use _arrow::io::internal::CoalesceReadRanges_
and this function needs _hole_size_limit_ and {_}range_size_limit{_}.
> Implement a read range process without caching
> ----------------------------------------------
>
> Key: ARROW-18113
> URL: https://issues.apache.org/jira/browse/ARROW-18113
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Percy Camilo Triveño Aucahuasi
> Assignee: Percy Camilo Triveño Aucahuasi
> Priority: Major
>
> The current
> [ReadRangeCache|https://github.com/apache/arrow/blob/e06e98db356e602212019cfbae83fd3d5347292d/cpp/src/arrow/io/caching.h#L100]
> is mixing caching with coalescing and making difficult to implement readers
> capable to really perform concurrent reads on coalesced data (see this
> [github
> comment|https://github.com/apache/arrow/pull/14226#discussion_r999334979] for
> additional context); for instance, right now the prebuffering feature of
> those readers cannot handle concurrent invocations.
> The goal for this ticket is to implement a similar component to
> ReadRangeCache for performing non-cache reads (doing only the coalescing part
> instead). So, once we have that new capability, we can port the parquet and
> IPC readers to this new component and keep improving the reading process
> (that would be part of other set of follow-up tickets). Similar ideas were
> mentioned here https://issues.apache.org/jira/browse/ARROW-17599
> Maybe a good place to implement this new capability is inside the file system
> abstraction (as part of a dedicated method to read coalesced data) and where
> the abstract file system can provide a default implementation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
