[jira] [Commented] (ASTERIXDB-2933) Support more pushdowns when reading Parquet files
[ https://issues.apache.org/jira/browse/ASTERIXDB-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409718#comment-17409718 ] ASF subversion and git services commented on ASTERIXDB-2933: Commit c84b073159f0f1de23abb6543238ea0f794dbb6a in asterixdb's branch refs/heads/master from Wail Alkowaileet [ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=c84b073 ] [ASTERIXDB-2933][COMP][EXT] Pushdowns Part3: Enable pushdown - user model changes: no - storage format changes: no - interface changes: no Details: - Enable more pushdown when reading Parquet files - Issue warning on type-mismatch between the expected schema and the actual Parquet file's schema - Add testcases Change-Id: I411f0f5b98c1bc1e174473344cda666f96151e4a Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/13064 Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Wael Alkowaileet Reviewed-by: Dmitry Lychagin > Support more pushdowns when reading Parquet files > - > > Key: ASTERIXDB-2933 > URL: https://issues.apache.org/jira/browse/ASTERIXDB-2933 > Project: Apache AsterixDB > Issue Type: Improvement > Components: COMP - Compiler, EXT - External data >Affects Versions: 0.9.7 >Reporter: Wail Y. Alkowaileet >Assignee: Wail Y. Alkowaileet >Priority: Major > Fix For: 0.9.7 > > > Currently, AsterixDB only supports simple pushdowns for field access. We > should allow for more pushdowns including array functions (e.g., array_star). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ASTERIXDB-2933) Support more pushdowns when reading Parquet files
[ https://issues.apache.org/jira/browse/ASTERIXDB-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409119#comment-17409119 ] ASF subversion and git services commented on ASTERIXDB-2933: Commit ca2a2e034b2b9349f5c3664432cd6ced36c1903a in asterixdb's branch refs/heads/master from Wail Alkowaileet [ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=ca2a2e0 ] [ASTERIXDB-2933][COMP][EXT] Pushdowns Part2: Pushdown Rule - user model changes: no - storage format changes: no - interface changes: yes Details: Add a rule that computes the expected schema and sets it to the DataSourceScanOperator. The computed schema is then passed to the Parquet Reader to 'clip' the Parquet file's schema. The resulting clipped schema is then used to tell the reader what column should be read. Interface changes: - Change IProjectionInfo> to IProjectionInfo Change-Id: If0c0d05473be72df6f08dfcbab2d25c36c71368e Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12964 Tested-by: Jenkins Integration-Tests: Jenkins Reviewed-by: Wael Alkowaileet Reviewed-by: Dmitry Lychagin > Support more pushdowns when reading Parquet files > - > > Key: ASTERIXDB-2933 > URL: https://issues.apache.org/jira/browse/ASTERIXDB-2933 > Project: Apache AsterixDB > Issue Type: Improvement > Components: COMP - Compiler, EXT - External data >Affects Versions: 0.9.7 >Reporter: Wail Y. Alkowaileet >Assignee: Wail Y. Alkowaileet >Priority: Major > Fix For: 0.9.7 > > > Currently, AsterixDB only supports simple pushdowns for field access. We > should allow for more pushdowns including array functions (e.g., array_star). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ASTERIXDB-2933) Support more pushdowns when reading Parquet files
[ https://issues.apache.org/jira/browse/ASTERIXDB-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403470#comment-17403470 ] ASF subversion and git services commented on ASTERIXDB-2933: Commit 57577846a75c23d8d247e9f05f76a57d353d400d in asterixdb's branch refs/heads/master from Wail Alkowaileet [ https://gitbox.apache.org/repos/asf?p=asterixdb.git;h=5757784 ] [ASTERIXDB-2933][COMP][EXT] Pushdowns Part1: Expected Schema Node - user model changes: no - storage format changes: no - interface changes: no Details: We want to have a more aggressive pushdown for Parquet. To do so, we need to compute the expected schema from the expressions in the query plan. This change introduces the data structure that holds the expected schema information. Note: Moved from asterix-om to asterix-algebra as the expected schema should not be exposed outside the compiler. Change-Id: I4b5aef414050a518fe230326e92ed66734b7b1d2 Reviewed-on: https://asterix-gerrit.ics.uci.edu/c/asterixdb/+/12765 Integration-Tests: Jenkins Tested-by: Jenkins Reviewed-by: Wael Alkowaileet Reviewed-by: Dmitry Lychagin > Support more pushdowns when reading Parquet files > - > > Key: ASTERIXDB-2933 > URL: https://issues.apache.org/jira/browse/ASTERIXDB-2933 > Project: Apache AsterixDB > Issue Type: Improvement > Components: COMP - Compiler, EXT - External data >Affects Versions: 0.9.7 >Reporter: Wail Y. Alkowaileet >Assignee: Wail Y. Alkowaileet >Priority: Major > Fix For: 0.9.7 > > > Currently, AsterixDB only supports simple pushdowns for field access. We > should allow for more pushdowns including array functions (e.g., array_star). -- This message was sent by Atlassian Jira (v8.3.4#803005)
