[
https://issues.apache.org/jira/browse/ARROW-17648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Weston Pace updated ARROW-17648:
--------------------------------
Description:
When converting from an Acero plan to a Substrait plan not all scan options are
fully supported. The `projection` (partial column selection) and `filter`
(pushdown filtering) support needs to be added.
Ref:
{code:java}
Result<std::unique_ptr<substrait::ReadRel>> ScanRelationConverter(const
std::shared_ptr<Schema>& schema, const compute::Declaration& declaration,
ExtensionSet* ext_set, const ConversionOptions& conversion_options) {
auto read_rel = make_unique<substrait::ReadRel>();
const auto& scan_node_options = checked_cast<const
dataset::ScanNodeOptions&>(*declaration.options);
auto dataset =
dynamic_cast<dataset::FileSystemDataset*>(scan_node_options.dataset.get());
// TODO handle here{code}
was:
In the current ToProto the `ScanOptions` are not fully supported, the `project`
and `filter` support needs to be added.
Ref:
{code:java}
Result<std::unique_ptr<substrait::ReadRel>> ScanRelationConverter(const
std::shared_ptr<Schema>& schema, const compute::Declaration& declaration,
ExtensionSet* ext_set, const ConversionOptions& conversion_options) {
auto read_rel = make_unique<substrait::ReadRel>();
const auto& scan_node_options = checked_cast<const
dataset::ScanNodeOptions&>(*declaration.options);
auto dataset =
dynamic_cast<dataset::FileSystemDataset*>(scan_node_options.dataset.get());
// TODO handle here{code}
> [C++] Add ScanOptions to support projection and filter in ToProto Read
> ----------------------------------------------------------------------
>
> Key: ARROW-17648
> URL: https://issues.apache.org/jira/browse/ARROW-17648
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Vibhatha Lakmal Abeykoon
> Assignee: Vibhatha Lakmal Abeykoon
> Priority: Major
> Labels: query-engine, substrait, substrait-roundtrip
>
> When converting from an Acero plan to a Substrait plan not all scan options
> are fully supported. The `projection` (partial column selection) and
> `filter` (pushdown filtering) support needs to be added.
> Ref:
>
> {code:java}
> Result<std::unique_ptr<substrait::ReadRel>> ScanRelationConverter(const
> std::shared_ptr<Schema>& schema, const compute::Declaration& declaration,
> ExtensionSet* ext_set, const ConversionOptions& conversion_options) {
> auto read_rel = make_unique<substrait::ReadRel>();
> const auto& scan_node_options = checked_cast<const
> dataset::ScanNodeOptions&>(*declaration.options);
> auto dataset =
> dynamic_cast<dataset::FileSystemDataset*>(scan_node_options.dataset.get());
> // TODO handle here{code}
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
