[
https://issues.apache.org/jira/browse/BEAM-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16903192#comment-16903192
]
Rui Wang edited comment on BEAM-7929 at 8/8/19 6:01 PM:
--------------------------------------------------------
Calcite has well support on filter/projection pushdown so we don't need to
analyzing Calcite plan and Calcite will help us do that by optimization rules
and calling convention.
For example, BeamSQL has adopted some rules:
https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/planner/BeamRuleSets.java#L73
I can find many useful links on this topic, like:
http://mail-archives.apache.org/mod_mbox/calcite-dev/201610.mbox/%3c96745466-0bfb-49b4-8396-70a992cae...@apache.org%3E
https://stackoverflow.com/questions/40217160/how-to-push-down-project-filter-aggregation-to-tablescan-in-calcite
There is a also a good slides to cover many piece of calcite:
https://www.slideshare.net/JordanHalterman/introduction-to-apache-calcite
was (Author: amaliujia):
Calcite has well support on filter/pushdown so we don't need to analyzing
Calcite plan and Calcite will help us do that by optimization rules and calling
convention.
For example, BeamSQL has adopted some rules:
https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/planner/BeamRuleSets.java#L73
I can find many useful links on this topic, like:
http://mail-archives.apache.org/mod_mbox/calcite-dev/201610.mbox/%3c96745466-0bfb-49b4-8396-70a992cae...@apache.org%3E
https://stackoverflow.com/questions/40217160/how-to-push-down-project-filter-aggregation-to-tablescan-in-calcite
There is a also a good slides to cover many piece of calcite:
https://www.slideshare.net/JordanHalterman/introduction-to-apache-calcite
> ParquetTable.buildIOReader should support column projection and filter
> predicate
> --------------------------------------------------------------------------------
>
> Key: BEAM-7929
> URL: https://issues.apache.org/jira/browse/BEAM-7929
> Project: Beam
> Issue Type: New Feature
> Components: dsl-sql
> Affects Versions: 2.14.0
> Reporter: Neville Li
> Priority: Critical
>
> To leverage the performance improvements in Parquet.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
