Ben Kietzman created ARROW-13263:
------------------------------------
Summary: [C++][Compute] Allow Fragments to attach guarantees to
scanned batches
Key: ARROW-13263
URL: https://issues.apache.org/jira/browse/ARROW-13263
Project: Apache Arrow
Issue Type: Improvement
Components: C++
Reporter: Ben Kietzman
A Fragment may be able to attach guarantee expressions to individual batches
beyond just its partition expression. For example, a parquet fragment can
attach row group statistics. These guarantees can be leveraged by subsequent
ExecNodes to optimize execution, for example to skip execution of unnecessary
filter expressions.
In its simplest form this is probably an overload of ScanBatchesAsync which
yields ExecBatches (which have a {{guarantee}} property) instead of
RecordBatches. When transforming to an ExecBatch, (see
{{expression.h::MakeExecBatch}} it'd be useful to also eagerly drop columns
which are not referenced by any nodes in the graph- just in case the Fragment
couldn't push the projection down any further.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
