[
https://issues.apache.org/jira/browse/ARROW-11782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17291419#comment-17291419
]
Weston Pace commented on ARROW-11782:
-------------------------------------
Thanks for pointing this out. I think we can't get rid of ScanTask. I will
create a ScanTaskInternal (or some similar name) for the work I'm doing and
leave ScanTask's interface as is.
In the future though it might be better to change Scanner::Scan to return a
RecordBatchIterator instead of ScanTaskIterator. Otherwise we're putting the
burden on the user to manage how many files are loaded concurrently. I think
that should either be hidden completely or controlled by some option
(MaxConcurrentFiles or, better yet, MaxAllocatedBytes)
> [GLib][Dataset] Remove bindings for internal classes
> ----------------------------------------------------
>
> Key: ARROW-11782
> URL: https://issues.apache.org/jira/browse/ARROW-11782
> Project: Apache Arrow
> Issue Type: Improvement
> Components: GLib
> Affects Versions: 3.0.0
> Reporter: Ben Kietzman
> Priority: Major
> Fix For: 4.0.0
>
>
> GLib and ruby include bindings for internal classes such as ScanOptions,
> ScanContext, InMemoryScanTask, ScanTask, ... These are probably unnecessary
> and should be removed to present a simpler interface less prone to breakage
> under refactoring of the wrapped classes
> https://github.com/apache/arrow/pull/9532/checks?check_run_id=1974229719#step:8:2071
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
