[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281227#comment-16281227 ] Ashutosh Chauhan commented on HIVE-18173: - +1 > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch, > HIVE-18173.3.patch, HIVE-18173.4.patch, HIVE-18173.5.patch, > HIVE-18173.6.patch, HIVE-18173.7.patch, HIVE-18173.8.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281159#comment-16281159 ] Vineet Garg commented on HIVE-18173: [~ashutoshc] Yes hadn't rebased my branch with master yet. Attached another patch which is rebased. > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch, > HIVE-18173.3.patch, HIVE-18173.4.patch, HIVE-18173.5.patch, > HIVE-18173.6.patch, HIVE-18173.7.patch, HIVE-18173.8.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281052#comment-16281052 ] Ashutosh Chauhan commented on HIVE-18173: - rebase gone wrong? > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch, > HIVE-18173.3.patch, HIVE-18173.4.patch, HIVE-18173.5.patch, > HIVE-18173.6.patch, HIVE-18173.7.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280665#comment-16280665 ] Hive QA commented on HIVE-18173: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12900807/HIVE-18173.6.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11512 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=48) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=160) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1] (batchId=178) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_in] (batchId=133) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] (batchId=113) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] (batchId=136) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] (batchId=122) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] (batchId=122) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] (batchId=110) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=227) org.apache.hive.jdbc.TestJdbcDriver2.testSelectExecAsync2 (batchId=235) org.apache.hive.jdbc.TestJdbcDriver2.testYarnATSGuid (batchId=235) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8133/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8133/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8133/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 14 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12900807 - PreCommit-HIVE-Build > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch, > HIVE-18173.3.patch, HIVE-18173.4.patch, HIVE-18173.5.patch, HIVE-18173.6.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16280592#comment-16280592 ] Hive QA commented on HIVE-18173: | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 50s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} ql: The patch generated 0 new + 401 unchanged - 105 fixed = 401 total (was 506) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 11s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 12m 50s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / 95dadac | | Default Java | 1.8.0_111 | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-8133/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch, > HIVE-18173.3.patch, HIVE-18173.4.patch, HIVE-18173.5.patch, HIVE-18173.6.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275922#comment-16275922 ] Hive QA commented on HIVE-18173: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12900362/HIVE-18173.4.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11507 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=48) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join] (batchId=44) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=160) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] (batchId=113) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=227) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8088/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8088/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8088/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12900362 - PreCommit-HIVE-Build > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch, > HIVE-18173.3.patch, HIVE-18173.4.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275900#comment-16275900 ] Hive QA commented on HIVE-18173: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 1s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 37s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 35s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 37s{color} | {color:red} ql: The patch generated 33 new + 457 unchanged - 49 fixed = 490 total (was 506) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 12m 36s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / 5f12cb8 | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-8088/yetus/diff-checkstyle-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-8088/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch, > HIVE-18173.3.patch, HIVE-18173.4.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275783#comment-16275783 ] Hive QA commented on HIVE-18173: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12900353/HIVE-18173.3.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11507 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=48) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join] (batchId=44) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=160) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] (batchId=113) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=227) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8085/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8085/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8085/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12900353 - PreCommit-HIVE-Build > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch, > HIVE-18173.3.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16275765#comment-16275765 ] Hive QA commented on HIVE-18173: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 13s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 37s{color} | {color:red} ql: The patch generated 26 new + 488 unchanged - 18 fixed = 514 total (was 506) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 13m 8s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / 5f12cb8 | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-8085/yetus/diff-checkstyle-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-8085/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch, > HIVE-18173.3.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273959#comment-16273959 ] Hive QA commented on HIVE-18173: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12900084/HIVE-18173.2.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 11493 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_exists] (batchId=41) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_complex_join] (batchId=44) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=160) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_exists] (batchId=161) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in] (batchId=165) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in_having] (batchId=164) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=160) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_exists] (batchId=123) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] (batchId=113) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=227) org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=233) org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=233) org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=233) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8072/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8072/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8072/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12900084 - PreCommit-HIVE-Build > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16273935#comment-16273935 ] Hive QA commented on HIVE-18173: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 44s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 36s{color} | {color:red} ql: The patch generated 26 new + 488 unchanged - 18 fixed = 514 total (was 506) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 12m 45s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / 508d7e6 | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-8072/yetus/diff-checkstyle-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-8072/yetus/patch-asflicense-problems.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-8072/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272550#comment-16272550 ] Hive QA commented on HIVE-18173: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12899966/HIVE-18173.2.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8055/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8055/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8055/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2017-11-30 11:32:32.466 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-8055/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2017-11-30 11:32:32.469 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 2efb7d3 HIVE-17972: Implement Parquet vectorization reader for Map type (Colin Ma, reviewed by Ferdinand Xu) + git clean -f -d Removing ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommandsForMmTable.java Removing ql/src/test/org/apache/hadoop/hive/ql/TestTxnCommandsForOrcMmTable.java + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 2efb7d3 HIVE-17972: Implement Parquet vectorization reader for Map type (Colin Ma, reviewed by Ferdinand Xu) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2017-11-30 11:32:38.751 + rm -rf ../yetus + mkdir ../yetus + cp -R . ../yetus + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8055/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch patch: Only garbage was found in the patch input. patch: Only garbage was found in the patch input. patch: Only garbage was found in the patch input. fatal: unrecognized input The patch does not appear to apply with p0, p1, or p2 + exit 1 ' {noformat} This message is automatically generated. ATTACHMENT ID: 12899966 - PreCommit-HIVE-Build > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch, HIVE-18173.2.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270908#comment-16270908 ] Hive QA commented on HIVE-18173: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12899762/HIVE-18173.1.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 24 failed/errored test(s), 11413 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notexists] (batchId=88) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_notin_having] (batchId=49) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[subquery_unqualcolumnrefs] (batchId=18) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1] (batchId=156) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=162) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan] (batchId=160) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_multi] (batchId=152) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin] (batchId=164) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_views] (batchId=151) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=157) org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1] (batchId=175) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_in] (batchId=131) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_multi] (batchId=111) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_notin] (batchId=134) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar] (batchId=120) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select] (batchId=120) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_views] (batchId=109) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=224) org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=230) org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=230) org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=230) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8044/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8044/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8044/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 24 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12899762 - PreCommit-HIVE-Build > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18173) Improve plans for correlated subqueries with non-equi predicate
[ https://issues.apache.org/jira/browse/HIVE-18173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270786#comment-16270786 ] Hive QA commented on HIVE-18173: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 42s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 33s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 34s{color} | {color:red} ql: The patch generated 26 new + 89 unchanged - 19 fixed = 115 total (was 108) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 1s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 12m 39s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / 07fe7e2 | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-8044/yetus/diff-checkstyle-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-8044/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Improve plans for correlated subqueries with non-equi predicate > --- > > Key: HIVE-18173 > URL: https://issues.apache.org/jira/browse/HIVE-18173 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-18173.1.patch > > > HIVE-17767 optimized plan to not generate value generator (i.e. an extra join > with outer query to fetch correlated columns) for EQUAL and NOT EQUAL > predicates e.g. > {code:sql} > select * from src b where b.key in (select key from src a where b.value <> > a.value) > {code} > This should be improved and implemented for rest of the predicates e.g. LESS > THAN etc -- This message was sent by Atlassian JIRA (v6.4.14#64029)