[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 5: @Yongzhi can this be closed now that IMPALA-8718 is marked as resolved. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Wed, 18 Mar 2020 20:23:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 5: (1 comment) Sorry that it takes time for me to examine all the code paths in the Planner side. This patch makes sense to me. But I need to go deeper to make sure whether this is the best place to add an If-statement for this case. http://gerrit.cloudera.org:8080/#/c/13765/5/fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java File fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java: http://gerrit.cloudera.org:8080/#/c/13765/5/fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java@163 PS5, Line 163: // If the expr is a complex type, return false directly because I think we need to explain why the return type of an expression can be in complex type. In this case the expression is a SlofRef for a complex type column. It's produced by the SubPlan node. In the case of "count(*)", there're no IllegalStateException even without this patch. The reason is that AggregateInfo will generate intermediate TupleDescriptor which doesn't contain the complex type SlotRef. Some relative JIRAs: IMPALA-2875, IMPALA-2138 -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Mon, 29 Jul 2019 21:06:32 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3887/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Tue, 16 Jul 2019 07:09:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Yongzhi Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 5: submit patch set 5 with better tests -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Tue, 16 Jul 2019 06:31:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Hello Bharath Vissapragada, Quanlong Huang, Sahil Takiar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13765 to look at the new patch set (#5). Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column For Inline view with left outer join, in order to make right side exprs nullable, the planner makes a wrapper for the expr that return non-null when all its contained slotref is null. The wrap is TupleIsNullPredicate. Complex type does not need the wrapper for its value cannot both be null and non-null. But during its checking whether the wrapper needed, because of code limitation(IsNullPredicate is not supported for complex types), AnalysisException is thrown. This fix overcomes the limitation by avoiding to use unsupported IsNullPredicate for complex type. It should be a safe change for: 1. Other data types such as int return false after a call to backend to check isNotNull(nullvalue). 2.Left outer join with complex type works fine for the queries without inline view. Join code can handle complex type correctly without the wrapper. Tests: Added ee tests to test left outer joins. Manual tests. Ran all core tests. Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 --- M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java A testdata/workloads/functional-query/queries/QueryTest/complex_joins.test M tests/query_test/test_nested_types.py 3 files changed, 63 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/13765/5 -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 5 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 4: (1 comment) > Patch Set 4: > > (1 comment) Sorry for the wrong comment. I have interests in this problem, but may need more time to verify more context. http://gerrit.cloudera.org:8080/#/c/13765/4/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test File testdata/workloads/functional-query/queries/QueryTest/complex_joins.test: http://gerrit.cloudera.org:8080/#/c/13765/4/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@38 PS4, Line 38: select count(*) from leftSide l left join rightSide r on l.col1 = r.col1 limit 1; > I think we don't need the "limit 1" here. BTW, if I change the last line to Sorry for my wrong comment... The error I mentioned above is fixed by this patch. Somehow I forgot to restart the cluster after building on this patch. However, this test case (the count(*) one) can run successfully without this patch. We may need another more accurate test case. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 4 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Mon, 15 Jul 2019 22:15:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/13765/4/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test File testdata/workloads/functional-query/queries/QueryTest/complex_joins.test: http://gerrit.cloudera.org:8080/#/c/13765/4/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@38 PS4, Line 38: select count(*) from leftSide l left join rightSide r on l.col1 = r.col1 limit 1; I think we don't need the "limit 1" here. BTW, if I change the last line to "select * from leftSide l left join rightSide r on l.col1 = r.col1", an AnalysisException will still occur: AnalysisException: IS NOT NULL predicate does not support complex types: col2 IS NOT NULL -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 4 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Mon, 15 Jul 2019 19:11:28 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3815/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 4 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Wed, 03 Jul 2019 09:47:16 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Yongzhi Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 3: (5 comments) Answer Sahil's question: IsNull Check may be necessary for other places, but use IsNullPredicate in requiresNullWrapping is not required. We can workaround it. Attach patch 4 with fixed tests. Run all core tests. http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@16 PS3, Line 16: cannot both be null and non-null. But during its > This line is a bit confusing. Can you clarify what you mean here? Although we do not have the predict to tell if a value is null, we know that a valid should be either null or nonnull and cannot be both. http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@17 PS3, Line 17: checking whether the wrapper needed, because of code > what do you mean by "because of code limitation"? We do not have IsNullPredicate for complex type, it is mentioned in the latter part of the message. I will make a change to add it here too. http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test File testdata/workloads/functional-query/queries/QueryTest/complex_joins.test: http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@5 PS3, Line 5: create table array_tbl (col1 string, col2 array ) stored as parquet; > databases such as "functional_parquet" already have tables such as "complex This is the test case that the customer provided. And our functional table does not have data. http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@35 PS3, Line 35: select t.col1, rank() over(order by t.col1) as rnk > is the analytical function necessary to re-produce the error? otherwise I t Yes, it is the customer's case, I must make sure the tests passed with/without data. Why create two: The testing framework parallels the tests which make test flaky if switching between impala and hive. http://gerrit.cloudera.org:8080/#/c/13765/3/tests/query_test/test_complex.py File tests/query_test/test_complex.py: http://gerrit.cloudera.org:8080/#/c/13765/3/tests/query_test/test_complex.py@23 PS3, Line 23: class TestComplex(ImpalaTestSuite): > You can just add this test to test_nested_types.py instead of creating a ne I will move -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Wed, 03 Jul 2019 09:11:42 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Hello Bharath Vissapragada, Sahil Takiar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13765 to look at the new patch set (#4). Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column For Inline view with left outer join, in order to make right side exprs nullable, the planner makes a wrapper for the expr that return non-null when all its contained slotref is null. The wrap is TupleIsNullPredicate. Complex type does not need the wrapper for its value cannot both be null and non-null. But during its checking whether the wrapper needed, because of code limitation(IsNullPredicate is not supported for complex types), AnalysisException is thrown. This fix overcomes the limitation by avoiding to use unsupported IsNullPredicate for complex type. It should be a safe change for: 1. Other data types such as int return false after a call to backend to check isNotNull(nullvalue). 2.Left outer join with complex type works fine for the queries without inline view. Join code can handle complex type correctly without the wrapper. Tests: Added ee tests to test left outer joins. Manual tests. Ran all core tests. Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 --- M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java A testdata/workloads/functional-query/queries/QueryTest/complex_joins.test M tests/query_test/test_nested_types.py 3 files changed, 51 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/13765/4 -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 4 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 3: (6 comments) Still trying to understand the overall approach here. The commit message seems to indicate that the NULL check is not necessary for complex types, but I don't really understand why; plus the JIRA description in IMPALA-8718 seems to indicate the NULL check is necessary. http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@11 PS3, Line 11: make right side exprs nullable, it makes a wrapper nit: what is "it", the Impala planner? http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@16 PS3, Line 16: cannot both be null and non-null. But during its This line is a bit confusing. Can you clarify what you mean here? http://gerrit.cloudera.org:8080/#/c/13765/3//COMMIT_MSG@17 PS3, Line 17: checking whether the wrapper needed, because of code what do you mean by "because of code limitation"? http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test File testdata/workloads/functional-query/queries/QueryTest/complex_joins.test: http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@5 PS3, Line 5: create table array_tbl (col1 string, col2 array ) stored as parquet; databases such as "functional_parquet" already have tables such as "complextypestbl" that you can use. See "test_nested_types.py" for examples. http://gerrit.cloudera.org:8080/#/c/13765/3/testdata/workloads/functional-query/queries/QueryTest/complex_joins.test@35 PS3, Line 35: select t.col1, rank() over(order by t.col1) as rnk is the analytical function necessary to re-produce the error? otherwise I think it is just adding unnecessary complexity to the test case. http://gerrit.cloudera.org:8080/#/c/13765/3/tests/query_test/test_complex.py File tests/query_test/test_complex.py: http://gerrit.cloudera.org:8080/#/c/13765/3/tests/query_test/test_complex.py@23 PS3, Line 23: class TestComplex(ImpalaTestSuite): You can just add this test to test_nested_types.py instead of creating a new file. I think the pattern in the Impala codebase is to refer to array, map, and struct types as "nested" types, rather than "complex" types. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Tue, 02 Jul 2019 15:55:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3789/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Sat, 29 Jun 2019 00:56:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Hello Bharath Vissapragada, Sahil Takiar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13765 to look at the new patch set (#3). Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column For Inline view with left outer join, in order to make right side exprs nullable, it makes a wrapper for the expr that return non-null when all its contained slotref is null. The wrap is TupleIsNullPredicate. Complex type does not need the wrapper for its value cannot both be null and non-null. But during its checking whether the wrapper needed, because of code limitation, AnalysisException is thrown. This fix overcomes the limitation by avoiding to use unsupported IsNullPredicate for complex type. It should be a safe change for: 1. Other data types such as int return false after a call to backend to check isNotNull(nullvalue). 2.Left outer join with complex type works fine for the queries without inline view. Join code can handle complex type correctly without the wrapper. Tests: Added ee tests to test left outer joins. Manual tests. Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 --- M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java A testdata/workloads/functional-query/queries/QueryTest/complex_joins.test A tests/query_test/test_complex.py 3 files changed, 84 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/13765/3 -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 3 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3785/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 2 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Fri, 28 Jun 2019 22:44:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/13765/2/tests/query_test/test_complex.py File tests/query_test/test_complex.py: http://gerrit.cloudera.org:8080/#/c/13765/2/tests/query_test/test_complex.py@22 PS2, Line 22: class TestComplex(ImpalaTestSuite): flake8: E302 expected 2 blank lines, found 1 -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 2 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Fri, 28 Jun 2019 22:06:41 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Hello Bharath Vissapragada, Sahil Takiar, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13765 to look at the new patch set (#2). Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column For Inline view with left outer join, in order to make right side exprs nullable, it makes a wrapper for the expr that return non-null when all its contained slotref is null. The wrap is TupleIsNullPredicate. Complex type does not need the wrapper for its value cannot both be null and non-null. But during its checking whether the wrapper needed, because of code limitation, AnalysisException is thrown. This fix overcomes the limitation by avoiding to use unsupported IsNullPredicate for complex type. It should be a safe change for: 1. Other data types such as int return false after a call to backend to check isNotNull(nullvalue). 2.Left outer join with complex type works fine for the queries without inline view. Join code can handle complex type correctly without the wrapper. Tests: Added ee tests to test left outer joins. Manual tests. Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 --- M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java A testdata/workloads/functional-query/queries/QueryTest/complex_joins.test A tests/query_test/test_complex.py 3 files changed, 83 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/13765/2 -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 2 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3783/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Fri, 28 Jun 2019 22:02:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13765 ) Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13765/1/tests/query_test/test_complex.py File tests/query_test/test_complex.py: http://gerrit.cloudera.org:8080/#/c/13765/1/tests/query_test/test_complex.py@19 PS1, Line 19: import pytest flake8: F401 'pytest' imported but unused -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 28 Jun 2019 21:25:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column
Yongzhi Chen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13765 Change subject: IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column .. IMPALA-8718: Fixed AnalysisException in inline view with outer join complex type column For Inline view with left outer join, in order to make right side exprs nullable, it makes a wrapper for the expr that return non-null when all its contained slotref is null. The wrap is TupleIsNullPredicate. Complex type does not need the wrapper for its value cannot both be null and non-null. But during its checking whether the wrapper needed, because of code limitation, AnalysisException is thrown. This fix overcomes the limitation by avoiding to use unsupported IsNullPredicate for complex type. It should be a safe change for: 1. Other data types such as int return false after a call to backend to check isNotNull(nullvalue). 2.Left outer join with complex type works fine for the queries without inline view. Join code can handle complex type correctly without the wrapper. Tests: Added ee tests to test left outer joins. Manual tests. Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 --- M fe/src/main/java/org/apache/impala/analysis/TupleIsNullPredicate.java A testdata/workloads/functional-query/queries/QueryTest/complex_joins.test A tests/query_test/test_complex.py 3 files changed, 85 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/65/13765/1 -- To view, visit http://gerrit.cloudera.org:8080/13765 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ida65503ca4b1342b8fe0049753bc664da227dca9 Gerrit-Change-Number: 13765 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen
