[jira] [Commented] (HIVE-15721) Allow IN/NOT IN correlated subquery with aggregates
[ https://issues.apache.org/jira/browse/HIVE-15721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839566#comment-15839566 ] Lefty Leverenz commented on HIVE-15721: --- Doc note: This removes the restriction that HIVE-15544 said would need to be documented (also in release 2.2.0), so actually this is an "undocumentation note" -- see these comments on HIVE-15544: * [restriction comment on HIVE-15544 | https://issues.apache.org/jira/browse/HIVE-15544?focusedCommentId=15829209=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15829209] * [doc comment on HIVE-15544 | https://issues.apache.org/jira/browse/HIVE-15544?focusedCommentId=1584=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-1584] > Allow IN/NOT IN correlated subquery with aggregates > > > Key: HIVE-15721 > URL: https://issues.apache.org/jira/browse/HIVE-15721 > Project: Hive > Issue Type: Sub-task > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Labels: TODOC2.2, sub-query > Fix For: 2.2.0 > > Attachments: HIVE-15721.1.patch, HIVE-15721.2.patch > > > With HIVE-15544 IN/NOT IN correlated subqueries with aggregates were disabled > since re-writting them into JOIN could have produced wrong result. > Wrong results would occur if subquery produces zero row, since aggregate > always produce result lower such query into LEFT JOIN or SEMI JOIN would not > take these case into consideration. > We propose to allow such queries with an added run time check which will > throw an error/exception if subquery produces zero row. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15721) Allow IN/NOT IN correlated subquery with aggregates
[ https://issues.apache.org/jira/browse/HIVE-15721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15839347#comment-15839347 ] Ashutosh Chauhan commented on HIVE-15721: - +1 > Allow IN/NOT IN correlated subquery with aggregates > > > Key: HIVE-15721 > URL: https://issues.apache.org/jira/browse/HIVE-15721 > Project: Hive > Issue Type: Sub-task > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Labels: sub-query > Attachments: HIVE-15721.1.patch, HIVE-15721.2.patch > > > With HIVE-15544 IN/NOT IN correlated subqueries with aggregates were disabled > since re-writting them into JOIN could have produced wrong result. > Wrong results would occur if subquery produces zero row, since aggregate > always produce result lower such query into LEFT JOIN or SEMI JOIN would not > take these case into consideration. > We propose to allow such queries with an added run time check which will > throw an error/exception if subquery produces zero row. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15721) Allow IN/NOT IN correlated subquery with aggregates
[
https://issues.apache.org/jira/browse/HIVE-15721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15838715#comment-15838715
]
Hive QA commented on HIVE-15721:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12849360/HIVE-15721.2.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 11000 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=93)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3182/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3182/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3182/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12849360 - PreCommit-HIVE-Build
> Allow IN/NOT IN correlated subquery with aggregates
>
>
> Key: HIVE-15721
> URL: https://issues.apache.org/jira/browse/HIVE-15721
> Project: Hive
> Issue Type: Sub-task
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Labels: sub-query
> Attachments: HIVE-15721.1.patch, HIVE-15721.2.patch
>
>
> With HIVE-15544 IN/NOT IN correlated subqueries with aggregates were disabled
> since re-writting them into JOIN could have produced wrong result.
> Wrong results would occur if subquery produces zero row, since aggregate
> always produce result lower such query into LEFT JOIN or SEMI JOIN would not
> take these case into consideration.
> We propose to allow such queries with an added run time check which will
> throw an error/exception if subquery produces zero row.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15721) Allow IN/NOT IN correlated subquery with aggregates
[
https://issues.apache.org/jira/browse/HIVE-15721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837637#comment-15837637
]
Hive QA commented on HIVE-15721:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12849221/HIVE-15721.1.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 10984 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=119)
[load_dyn_part2.q,smb_mapjoin_7.q,vectorization_5.q,smb_mapjoin_2.q,ppd_join_filter.q,column_access_stats.q,stats0.q,vector_between_in.q,vectorized_string_funcs.q,bucket_map_join_2.q,groupby4_map_skew.q,groupby_ppr_multi_distinct.q,temp_table_join1.q,vectorized_case.q,stats_noscan_1.q]
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=140)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
(batchId=93)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14]
(batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23]
(batchId=223)
org.apache.hadoop.hive.ql.parse.TestQBSubQuery.testCheckAggOrWindowing
(batchId=258)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3170/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3170/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3170/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12849221 - PreCommit-HIVE-Build
> Allow IN/NOT IN correlated subquery with aggregates
>
>
> Key: HIVE-15721
> URL: https://issues.apache.org/jira/browse/HIVE-15721
> Project: Hive
> Issue Type: Sub-task
> Components: Query Planning
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Labels: sub-query
> Attachments: HIVE-15721.1.patch
>
>
> With HIVE-15544 IN/NOT IN correlated subqueries with aggregates were disabled
> since re-writting them into JOIN could have produced wrong result.
> Wrong results would occur if subquery produces zero row, since aggregate
> always produce result lower such query into LEFT JOIN or SEMI JOIN would not
> take these case into consideration.
> We propose to allow such queries with an added run time check which will
> throw an error/exception if subquery produces zero row.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15721) Allow IN/NOT IN correlated subquery with aggregates
[ https://issues.apache.org/jira/browse/HIVE-15721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15837311#comment-15837311 ] Ashutosh Chauhan commented on HIVE-15721: - [~vgarg] Can you create a RB request for it? > Allow IN/NOT IN correlated subquery with aggregates > > > Key: HIVE-15721 > URL: https://issues.apache.org/jira/browse/HIVE-15721 > Project: Hive > Issue Type: Sub-task > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Labels: sub-query > Attachments: HIVE-15721.1.patch > > > With HIVE-15544 IN/NOT IN correlated subqueries with aggregates were disabled > since re-writting them into JOIN could have produced wrong result. > Wrong results would occur if subquery produces zero row, since aggregate > always produce result lower such query into LEFT JOIN or SEMI JOIN would not > take these case into consideration. > We propose to allow such queries with an added run time check which will > throw an error/exception if subquery produces zero row. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
