[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Limitation: zipping unnest is not supported for mixed complex types, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Reviewed-on: http://gerrit.cloudera.org:8080/19322 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_complex_types.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 24 files changed, 1,192 insertions(+), 262 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 22 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 21: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 21 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 07 Mar 2023 13:22:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 21: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12565/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 21 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 07 Mar 2023 08:36:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 20: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12564/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 20 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 07 Mar 2023 08:35:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#20). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Limitation: zipping unnest is not supported for mixed complex types, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_complex_types.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 24 files changed, 1,192 insertions(+), 262 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/20 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 20 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#21). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Limitation: zipping unnest is not supported for mixed complex types, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_complex_types.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 24 files changed, 1,192 insertions(+), 262 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/21 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 21 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 19: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/19322/19/be/src/runtime/complex-value-writer.inline.h File be/src/runtime/complex-value-writer.inline.h: http://gerrit.cloudera.org:8080/#/c/19322/19/be/src/runtime/complex-value-writer.inline.h@108 PS19, Line 108: ; nit: unneeded ; -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 19 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 07 Mar 2023 08:00:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 19: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 19 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Mon, 06 Mar 2023 13:59:55 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 19: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9110/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 19 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Mon, 06 Mar 2023 08:44:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 19: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/9099/ -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 19 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 03 Mar 2023 16:04:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 19: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9099/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 19 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 03 Mar 2023 13:54:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 19: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12531/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 19 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 03 Mar 2023 11:32:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#19). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Limitation: zipping unnest is not supported for mixed complex types, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/ranger_column_masking_complex_types.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 24 files changed, 1,192 insertions(+), 262 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/19 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 19 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 18: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/9096/ -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 18 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 02 Mar 2023 19:33:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 18: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12517/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 18 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 02 Mar 2023 14:43:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 18: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9096/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 18 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 02 Mar 2023 14:22:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#18). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Limitation: zipping unnest is not supported for mixed complex types, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 1,188 insertions(+), 261 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/18 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 18 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 17: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12513/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 17 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 02 Mar 2023 12:01:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#17). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Limitation: zipping unnest is not supported for mixed complex types, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 1,188 insertions(+), 261 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/17 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 17 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 16: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 16 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Wed, 01 Mar 2023 22:38:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 16: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12508/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 16 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Wed, 01 Mar 2023 17:21:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#16). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Limitation: zipping unnest is not supported for mixed complex types, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 1,167 insertions(+), 261 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/16 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 16 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 16: (9 comments) I disabled zipping unnest with mixed complex types as getting it to work well with views and WHERE predicates is difficult. I'll try to do it in a later patch. http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h File be/src/runtime/complex-value-writer.inline.h: http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h@90 PS15, Line 90: struct_slot_desc.children_tuple_descriptor(); > nit: +2 indentation Done http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h@116 PS15, Line 116: reinterpret_cast(child); > +2 Done http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h@118 PS15, Line 118: child_slot_desc.children_tuple_descriptor(); > +2 Done http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h@236 PS15, Line 236: children_item_tuple_desc->slots(); > +2 Done http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/service/hs2-util.cc File be/src/service/hs2-util.cc: http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/service/hs2-util.cc@390 PS15, Line 390: static_cast(scalar_expr).GetSlotDescriptor(); > +2 indentation Done http://gerrit.cloudera.org:8080/#/c/19322/15/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java File fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java: http://gerrit.cloudera.org:8080/#/c/19322/15/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java@311 PS15, Line 311: > else if? Done http://gerrit.cloudera.org:8080/#/c/19322/15/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java@417 PS15, Line 417: // The path can be null in the case of the sorting tuple. > nit, here and next rows: indentation Done http://gerrit.cloudera.org:8080/#/c/19322/15/testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test File testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test: http://gerrit.cloudera.org:8080/#/c/19322/15/testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test@257 PS15, Line 257: sub.arr_contai > If we look at it strictly this is not zipping unnest, just simply unnest, a In this patch I disallowed zipping unnest with mixed complex types. http://gerrit.cloudera.org:8080/#/c/19322/15/testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test@387 PS15, Line 387: 2-12"],null]' > Can you also add one where different view levels have where conditions on t Not done yet, will do in a following patch. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 16 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Wed, 01 Mar 2023 17:00:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 15: Code-Review+1 (10 comments) looks good to me, just indentation + few more test ideas http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h File be/src/runtime/complex-value-writer.inline.h: http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h@90 PS15, Line 90: struct_slot_desc.children_tuple_descriptor(); nit: +2 indentation http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h@116 PS15, Line 116: reinterpret_cast(child); +2 http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h@118 PS15, Line 118: child_slot_desc.children_tuple_descriptor(); +2 http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/runtime/complex-value-writer.inline.h@236 PS15, Line 236: children_item_tuple_desc->slots(); +2 http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/service/hs2-util.cc File be/src/service/hs2-util.cc: http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/service/hs2-util.cc@390 PS15, Line 390: static_cast(scalar_expr).GetSlotDescriptor(); +2 indentation http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/service/query-result-set.cc File be/src/service/query-result-set.cc: http://gerrit.cloudera.org:8080/#/c/19322/15/be/src/service/query-result-set.cc@256 PS15, Line 256: static_cast(scalar_expr).GetSlotDescriptor(); +2 http://gerrit.cloudera.org:8080/#/c/19322/15/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java File fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java: http://gerrit.cloudera.org:8080/#/c/19322/15/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java@311 PS15, Line 311: if (colExpr.getType().isStructType()) { else if? http://gerrit.cloudera.org:8080/#/c/19322/15/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java@417 PS15, Line 417: childSlotDescRawPath.get(childSlotDescRawPath.size() - 1); nit, here and next rows: indentation http://gerrit.cloudera.org:8080/#/c/19322/15/testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test File testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test: http://gerrit.cloudera.org:8080/#/c/19322/15/testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test@257 PS15, Line 257: Zipping unnest If we look at it strictly this is not zipping unnest, just simply unnest, as the zipping part comes in when there are at least 2 columns, e.g arr_contains_nested_struct and arr_contains_struct Other zipping unnest related test idedas: - test zipping unnest in the from clause - test zipping unnest with where filter on an unnested col http://gerrit.cloudera.org:8080/#/c/19322/15/testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test@387 PS15, Line 387: WHERE clauses at different levels. Can you also add one where different view levels have where conditions on the same struct member? -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 15 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 31 Jan 2023 15:55:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 15: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 15 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Mon, 30 Jan 2023 18:24:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 15: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12264/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 15 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Mon, 30 Jan 2023 17:45:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#15). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. There is a limitation: zipping unnest for arrays that are within a struct is not supported, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 1,108 insertions(+), 259 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/15 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 15 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 15: (3 comments) http://gerrit.cloudera.org:8080/#/c/19322/14/be/src/runtime/complex-value-writer.inline.h File be/src/runtime/complex-value-writer.inline.h: http://gerrit.cloudera.org:8080/#/c/19322/14/be/src/runtime/complex-value-writer.inline.h@120 PS14, Line 120: CollectionValueToJSON(*nested_collection_val, child_type.type, > child_slot_desc.type() could be replaced with child_type Done http://gerrit.cloudera.org:8080/#/c/19322/14/be/src/runtime/descriptors.cc File be/src/runtime/descriptors.cc: http://gerrit.cloudera.org:8080/#/c/19322/14/be/src/runtime/descriptors.cc@364 PS14, Line 364: target_tuple->collection_slots_.push_back(slot); > This common part could be extracted from the conditions Done http://gerrit.cloudera.org:8080/#/c/19322/14/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java File fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java: http://gerrit.cloudera.org:8080/#/c/19322/14/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java@408 PS14, Line 408: private static void verifySameChild(Path childSlotDescPath, Path childColExprPath, > The access modifier should precede the static modifier Done -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 15 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Mon, 30 Jan 2023 17:24:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Peter Rozsa has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 14: (3 comments) http://gerrit.cloudera.org:8080/#/c/19322/14/be/src/runtime/complex-value-writer.inline.h File be/src/runtime/complex-value-writer.inline.h: http://gerrit.cloudera.org:8080/#/c/19322/14/be/src/runtime/complex-value-writer.inline.h@120 PS14, Line 120: CollectionValueToJSON(*nested_collection_val, child_slot_desc.type().type, child_slot_desc.type() could be replaced with child_type http://gerrit.cloudera.org:8080/#/c/19322/14/be/src/runtime/descriptors.cc File be/src/runtime/descriptors.cc: http://gerrit.cloudera.org:8080/#/c/19322/14/be/src/runtime/descriptors.cc@364 PS14, Line 364: TupleDescriptor* target_tuple = this; This common part could be extracted from the conditions http://gerrit.cloudera.org:8080/#/c/19322/14/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java File fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java: http://gerrit.cloudera.org:8080/#/c/19322/14/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java@408 PS14, Line 408: static private void verifySameChild(Path childSlotDescPath, Path childColExprPath, The access modifier should precede the static modifier -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 14 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Mon, 30 Jan 2023 12:18:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 14: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 14 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 26 Jan 2023 17:11:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 14: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12238/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 14 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 26 Jan 2023 13:01:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#14). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. There is a limitation: zipping unnest for arrays that are within a struct is not supported, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 1,108 insertions(+), 255 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/14 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 14 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 13: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12231/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 13 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Wed, 25 Jan 2023 17:13:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 13: Code-Review+1 (4 comments) http://gerrit.cloudera.org:8080/#/c/19322/12/be/src/runtime/complex-value-writer.h File be/src/runtime/complex-value-writer.h: http://gerrit.cloudera.org:8080/#/c/19322/12/be/src/runtime/complex-value-writer.h@48 PS12, Line 48: // out field names and types. This function can call itself recursively in case of > Done Done http://gerrit.cloudera.org:8080/#/c/19322/12/be/src/runtime/complex-value-writer.inline.h File be/src/runtime/complex-value-writer.inline.h: http://gerrit.cloudera.org:8080/#/c/19322/12/be/src/runtime/complex-value-writer.inline.h@226 PS12, Line 226: template > Removed this comment because this method is documented in the .h file. Done http://gerrit.cloudera.org:8080/#/c/19322/12/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java File fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java: http://gerrit.cloudera.org:8080/#/c/19322/12/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java@404 PS12, Line 404: } > Good point, moved to putOneStructMemberIntoSmaps(). Done http://gerrit.cloudera.org:8080/#/c/19322/12/tests/query_test/test_nested_types.py File tests/query_test/test_nested_types.py: http://gerrit.cloudera.org:8080/#/c/19322/12/tests/query_test/test_nested_types.py@185 PS12, Line 185: if vector.get_value('exec_option')['disable_codegen'] == 'False': > Having struct types in the select list is not yet supported with codegen - A code comment about why it's disabled (mention the ticket) would be welcome. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 13 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Wed, 25 Jan 2023 16:54:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#13). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. There is a limitation: zipping unnest for arrays that are within a struct is not supported, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 1,106 insertions(+), 255 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/13 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 13 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 12: (5 comments) http://gerrit.cloudera.org:8080/#/c/19322/12/be/src/runtime/complex-value-writer.h File be/src/runtime/complex-value-writer.h: http://gerrit.cloudera.org:8080/#/c/19322/12/be/src/runtime/complex-value-writer.h@48 PS12, Line 48: // 'column_type' to figure out field names and types. This function can call itself The 'column_type' argument was removed, so this comment no longer seems appropriate. http://gerrit.cloudera.org:8080/#/c/19322/12/be/src/runtime/complex-value-writer.inline.h File be/src/runtime/complex-value-writer.inline.h: http://gerrit.cloudera.org:8080/#/c/19322/12/be/src/runtime/complex-value-writer.inline.h@226 PS12, Line 226: // Gets a StructVal and writes it in JSON format using 'writer'. Uses 'column_type' to 'column_type' doesn't exist anymore. http://gerrit.cloudera.org:8080/#/c/19322/12/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java File fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java: http://gerrit.cloudera.org:8080/#/c/19322/12/fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java@404 PS12, Line 404: if (childColExprSlotRef.getType().isStructType()) { It seems strange to handle struct types here, but collection types down in putOneStructMemberIntoSmaps. http://gerrit.cloudera.org:8080/#/c/19322/12/tests/query_test/test_nested_types.py File tests/query_test/test_nested_types.py: http://gerrit.cloudera.org:8080/#/c/19322/12/tests/query_test/test_nested_types.py@185 PS12, Line 185: if vector.get_value('exec_option')['disable_codegen'] == 'False': Why are we skipping when codegen is disabled? http://gerrit.cloudera.org:8080/#/c/19322/12/tests/query_test/test_nested_types.py@213 PS12, Line 213: if vector.get_value('exec_option')['disable_codegen'] == 'False': Why are we skipping when codegen is disabled? -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 12 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 24 Jan 2023 23:04:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12171/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 12 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Mon, 16 Jan 2023 12:51:45 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. There is a limitation: zipping unnest for arrays that are within a struct is not supported, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 1,113 insertions(+), 257 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/12 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 12 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 11: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12146/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 11 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Wed, 11 Jan 2023 14:29:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. There is a limitation: zipping unnest for arrays that are within a struct is not supported, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 942 insertions(+), 257 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/11 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 11 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 10: (3 comments) only looked through the tests so far http://gerrit.cloudera.org:8080/#/c/19322/10/be/src/exec/unnest-node.cc File be/src/exec/unnest-node.cc: http://gerrit.cloudera.org:8080/#/c/19322/10/be/src/exec/unnest-node.cc@71 PS10, Line 71: parent_tuple : master_tuple; nit: +2 indentation http://gerrit.cloudera.org:8080/#/c/19322/10/testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test File testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test: http://gerrit.cloudera.org:8080/#/c/19322/10/testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test@90 PS10, Line 90: collection_struct_mix_view Can you add a test where all 3 kind of views are used? e.g with sub as (select id, struct_contains_nested_arr from collection_struct_mix_view) select id, struct_contains_nested_arr from (select id, struct_contains_nested_arr from sub) sub2; http://gerrit.cloudera.org:8080/#/c/19322/10/testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test@91 PS10, Line 91: RESULTS Can you also test star expansion? e.g. - test if mixed complex types are correctly passed through views when EXPAND_COMPLEX_TYPES=1 - test that a structs's members can be expanded with struct_col/member.* in mixed complex types, both with EXPAND_COMPLEX_TYPES=1 and 0 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 10 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Wed, 11 Jan 2023 12:36:58 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12135/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 10 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 10 Jan 2023 10:28:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. There is a limitation: zipping unnest for arrays that are within a struct is not supported, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 917 insertions(+), 247 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/10 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 10 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 9: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12126/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 9 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Mon, 09 Jan 2023 16:11:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. There is a limitation: zipping unnest for arrays that are within a struct is not supported, for example the following query: use functional_parquet; select unnest(struct_contains_nested_arr.arr) from collection_struct_mix; Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/CollectionTableRef.java M fe/src/main/java/org/apache/impala/analysis/InlineViewRef.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/analysis/UnnestExpr.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv M testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 23 files changed, 917 insertions(+), 247 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/9 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 9 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12119/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 8 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 06 Jan 2023 14:29:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. TODO: - zipping unnest - add tests with maps with NULL keys within structs in testdata/workloads/functional-query/queries/QueryTest/map_null_keys.test Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 19 files changed, 660 insertions(+), 132 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/8 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 8 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12107/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 7 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 05 Jan 2023 17:08:08 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exec/unnest-node.cc M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/SlotDescriptor.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 19 files changed, 590 insertions(+), 132 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/7 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 7 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12039/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 6 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Fri, 16 Dec 2022 15:56:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.h M be/src/runtime/complex-value-writer.inline.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M be/src/service/query-result-set.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 16 files changed, 466 insertions(+), 128 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/6 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 6 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 4: We should also test this with views before merging: if it works, we should add tests for it; if it doesn't we can disable it until we add support for them too. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 4 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 15 Dec 2022 14:20:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12009/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 4 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 08 Dec 2022 17:05:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12008/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 3 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 08 Dec 2022 16:59:09 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/19322/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java File fe/src/main/java/org/apache/impala/analysis/Analyzer.java: http://gerrit.cloudera.org:8080/#/c/19322/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@1529 PS2, Line 1529: SlotDescriptor result = createAndRegister > nit: this would be clearer as Done -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 4 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Thu, 08 Dec 2022 16:39:46 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.cc M be/src/runtime/complex-value-writer.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 15 files changed, 474 insertions(+), 129 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/4 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 4 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exprs/slot-ref.h M be/src/runtime/complex-value-writer.cc M be/src/runtime/complex-value-writer.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 15 files changed, 476 insertions(+), 127 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/3 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 3 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Daniel Becker Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/19322/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java File fe/src/main/java/org/apache/impala/analysis/Analyzer.java: http://gerrit.cloudera.org:8080/#/c/19322/2/fe/src/main/java/org/apache/impala/analysis/Analyzer.java@1529 PS2, Line 1529: if (slotPath.destType().isStructType()) { nit: this would be clearer as SlotDescriptor result = createAndRegisterRawSlotDesc(slotPath, true); if (slotPath.destType().isStructType()) createStructTuplesAndSlotDescs(result); return result; -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Smith Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 06 Dec 2022 18:42:03 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/11972/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 06 Dec 2022 10:32:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19322 ) Change subject: IMPALA-9551: Allow mixed complex types in select list .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/19322/2/be/src/service/hs2-util.cc File be/src/service/hs2-util.cc: http://gerrit.cloudera.org:8080/#/c/19322/2/be/src/service/hs2-util.cc@452 PS2, Line 452: static void StructInCollectionToJSON(Tuple* item_tuple, const SlotDescriptor& struct_slot_desc, line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/19322/2/be/src/service/hs2-util.cc@454 PS2, Line 454: const TupleDescriptor* children_tuple_desc = struct_slot_desc.children_tuple_descriptor(); line too long (92 > 90) http://gerrit.cloudera.org:8080/#/c/19322/2/be/src/service/hs2-util.cc@566 PS2, Line 566: const SlotDescriptor* slot_desc = static_cast(scalar_expr).GetSlotDescriptor(); line too long (101 > 90) -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Peter Rozsa Gerrit-Comment-Date: Tue, 06 Dec 2022 10:13:50 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9551: Allow mixed complex types in select list
Daniel Becker has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19322 Change subject: IMPALA-9551: Allow mixed complex types in select list .. IMPALA-9551: Allow mixed complex types in select list Currently collections and structs are supported in the select list, also when they are nested (structs in structs and collections in collections), but mixing different kinds of complex types, i.e. having structs in collections or vice versa, is not supported. This patch adds support for mixed complex types in the select list. Testing: - Created a new test table, 'collection_struct_mix', that contains mixed complex types. - Added tests in mixed-collections-and-structs.test that test having mixed complex types in the select list. These tests are called from test_nested_types.py::TestMixedCollectionsAndStructsInSelectList. - Ran existing tests that test collections and structs in the select list; test queries that expected a failure in case of mixed complex types have been moved to mixed-collections-and-structs.test and now expect success. Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 --- M be/src/exprs/slot-ref.h M be/src/runtime/descriptors.cc M be/src/runtime/raw-value.cc M be/src/service/hs2-util.cc M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/SlotRef.java M fe/src/main/java/org/apache/impala/analysis/TupleDescriptor.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv A testdata/workloads/functional-query/queries/QueryTest/mixed-collections-and-structs.test M testdata/workloads/functional-query/queries/QueryTest/struct-in-select-list.test M tests/query_test/test_nested_types.py 13 files changed, 459 insertions(+), 122 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/22/19322/2 -- To view, visit http://gerrit.cloudera.org:8080/19322 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I476d98884b5fd192dfcd4feeec7947526aebe993 Gerrit-Change-Number: 19322 Gerrit-PatchSet: 2 Gerrit-Owner: Daniel Becker Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Peter Rozsa