[
https://issues.apache.org/jira/browse/ARROW-10524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17422342#comment-17422342
]
David Li commented on ARROW-10524:
----------------------------------
Relevant discussion from the Skyhook PR:
[https://github.com/apache/arrow/pull/10431#discussion_r683685887]
In summary, for backends like this where we may be able to push down some
computation, we need to consider how best to integrate that into the rest of
the pipeline. The latest PR in ARROW-13607 introduces a flag on Fragment that
toggles whether the scanner applies filtering/projection or not, which should
fit with the presumed approach here. (The story is more complicated for Skyhook
because they are halfway between a file and a fragment; for data from Flight it
is fairly clear that it should be a Fragment.)
> [C++][Dataset] Add FlightFragment
> ---------------------------------
>
> Key: ARROW-10524
> URL: https://issues.apache.org/jira/browse/ARROW-10524
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Affects Versions: 2.0.0
> Reporter: Ben Kietzman
> Assignee: Ben Kietzman
> Priority: Major
> Labels: dataset
> Fix For: 6.0.0
>
>
> Allow wrapping a flight service as a dataset/fragment
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
