[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682751#comment-16682751 ] Hive QA commented on HIVE-20744: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947725/HIVE-20744.07.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 15537 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14872/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14872/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14872/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12947725 - PreCommit-HIVE-Build > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.05.patch, > HIVE-20744.06.patch, HIVE-20744.06.patch, HIVE-20744.07.patch, > HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682741#comment-16682741 ] Hive QA commented on HIVE-20744: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m 3s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 43s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 36s{color} | {color:red} ql: The patch generated 21 new + 51 unchanged - 13 fixed = 72 total (was 64) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 52s{color} | {color:red} ql generated 1 new + 2315 unchanged - 0 fixed = 2316 total (was 2315) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 22m 32s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Redundant nullcheck of leftInputResult, which is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:[line 366] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14872/dev-support/hive-personality.sh | | git revision | master / c049cac | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-14872/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-14872/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14872/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.05.patch, > HIVE-20744.06.patch, HIVE-20744.06.patch, HIVE-20744.07.patch, > HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682728#comment-16682728 ] Hive QA commented on HIVE-20744: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947723/HIVE-20744.06.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14871/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14871/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14871/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-11-11 03:59:35.375 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-14871/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-11-11 03:59:35.378 + cd apache-github-source-source + git fetch origin >From https://github.com/apache/hive 7ae4a2c..c049cac master -> origin/master + git reset --hard HEAD HEAD is now at 7ae4a2c HIVE-20833: package.jdo needs to be updated to conform with HIVE-20221 changes (Vineet Garg, reviewed by Jason Dere) + git clean -f -d Removing standalone-metastore/metastore-server/src/gen/ + git checkout master Already on 'master' Your branch is behind 'origin/master' by 2 commits, and can be fast-forwarded. (use "git pull" to update your local branch) + git reset --hard origin/master HEAD is now at c049cac HIVE-20826 : Enhance HiveSemiJoin rule to convert join + group by on left side to Left Semi Join (Vineet Garg via Ashutosh Chauhan) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-11-11 03:59:36.602 + rm -rf ../yetus_PreCommit-HIVE-Build-14871 + mkdir ../yetus_PreCommit-HIVE-Build-14871 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-14871 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14871/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelOptUtil.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinConstraintsRule.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdSelectivity.java: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query17.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query24.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query25.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query54.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query64.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query72.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query84.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query17.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query24.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query25.q.out: does not exist in index error: a/ql/src/test/results/clientpositive/perf/tez/constraints/query54.q.out: does not exist in index error:
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682710#comment-16682710 ] Hive QA commented on HIVE-20744: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947719/HIVE-20744.06.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15537 tests executed *Failed tests:* {noformat} org.apache.hive.jdbc.TestJdbcDriver2.testSelectExecAsync2 (batchId=259) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14869/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14869/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14869/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12947719 - PreCommit-HIVE-Build > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.05.patch, > HIVE-20744.06.patch, HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682700#comment-16682700 ] Hive QA commented on HIVE-20744: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 34s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 37s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 36s{color} | {color:red} ql: The patch generated 21 new + 51 unchanged - 13 fixed = 72 total (was 64) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 54s{color} | {color:red} ql generated 1 new + 2315 unchanged - 0 fixed = 2316 total (was 2315) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 22m 13s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Redundant nullcheck of leftInputResult, which is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:[line 366] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14869/dev-support/hive-personality.sh | | git revision | master / 7ae4a2c | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-14869/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-14869/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14869/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.05.patch, > HIVE-20744.06.patch, HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682010#comment-16682010 ] Ashutosh Chauhan commented on HIVE-20744: - +1 > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.05.patch, > HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678440#comment-16678440 ] Hive QA commented on HIVE-20744: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947157/HIVE-20744.05.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15526 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2] (batchId=85) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14791/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14791/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14791/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12947157 - PreCommit-HIVE-Build > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.05.patch, > HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678369#comment-16678369 ] Hive QA commented on HIVE-20744: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 35s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 38s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 39s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 36s{color} | {color:red} ql: The patch generated 21 new + 51 unchanged - 13 fixed = 72 total (was 64) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 55s{color} | {color:red} ql generated 1 new + 2315 unchanged - 0 fixed = 2316 total (was 2315) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 22m 14s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Redundant nullcheck of leftInputResult, which is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:[line 366] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14791/dev-support/hive-personality.sh | | git revision | master / 6d713b6 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-14791/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-14791/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14791/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.05.patch, > HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677134#comment-16677134 ] Hive QA commented on HIVE-20744: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947013/HIVE-20744.04.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14772/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14772/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14772/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12947013/HIVE-20744.04.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12947013 - PreCommit-HIVE-Build > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677131#comment-16677131 ] Hive QA commented on HIVE-20744: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12947013/HIVE-20744.04.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15526 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_rp_limit] (batchId=171) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query54] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query84] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query54] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query84] (batchId=272) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14771/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14771/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14771/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12947013 - PreCommit-HIVE-Build > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677068#comment-16677068 ] Hive QA commented on HIVE-20744: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 39s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 37s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 38s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 56s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 35s{color} | {color:red} ql: The patch generated 21 new + 51 unchanged - 13 fixed = 72 total (was 64) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 52s{color} | {color:red} ql generated 1 new + 2315 unchanged - 0 fixed = 2316 total (was 2315) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 22m 11s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Redundant nullcheck of leftInputResult, which is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:[line 366] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14771/dev-support/hive-personality.sh | | git revision | master / f4a48fd | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-14771/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-14771/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14771/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.04.patch, HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16676150#comment-16676150 ] Jesus Camacho Rodriguez commented on HIVE-20744: [~ashutoshc], uploaded the latest version to https://reviews.apache.org/r/69258/ . Thanks > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.03.patch, HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16675837#comment-16675837 ] Ashutosh Chauhan commented on HIVE-20744: - [~jcamachorodriguez] can you add RB for this? > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673768#comment-16673768 ] Hive QA commented on HIVE-20744: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12946288/HIVE-20744.02.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 15520 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query17] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query24] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query25] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query64] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query72] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query17] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query24] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query25] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query64] (batchId=272) org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query72] (batchId=272) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14711/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14711/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14711/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12946288 - PreCommit-HIVE-Build > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16673719#comment-16673719 ] Hive QA commented on HIVE-20744: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 42s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 39s{color} | {color:blue} ql in master has 2315 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s{color} | {color:red} ql: The patch generated 21 new + 52 unchanged - 12 fixed = 73 total (was 64) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m 54s{color} | {color:red} ql generated 1 new + 2315 unchanged - 0 fixed = 2316 total (was 2315) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 13s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 22m 25s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Redundant nullcheck of leftInputResult, which is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:is known to be non-null in org.apache.hadoop.hive.ql.optimizer.calcite.stats.HiveRelMdRowCount.constraintsBasedAnalyzeJoinForPKFK(Join, RelMetadataQuery) Redundant null check at HiveRelMdRowCount.java:[line 360] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-14711/dev-support/hive-personality.sh | | git revision | master / f3fab45 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-14711/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-14711/yetus/new-findbugs-ql.html | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-14711/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Use SQL constraints to improve join reordering algorithm > > > Key: HIVE-20744 > URL: https://issues.apache.org/jira/browse/HIVE-20744 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-20744.01.patch, HIVE-20744.02.patch, > HIVE-20744.patch > > > Till now, it was all based on stats stored for the base tables and their > columns. Now the optimizer can rely on constraints. Hence, this patch is for > the join reordering costing to use constraints, and if it does not find any, > rely on old code path. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20744) Use SQL constraints to improve join reordering algorithm
[ https://issues.apache.org/jira/browse/HIVE-20744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16662852#comment-16662852 ] Hive QA commented on HIVE-20744: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12945438/HIVE-20744.01.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/14627/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14627/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14627/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Tests exited with: NonZeroExitCodeException Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and output '+ date '+%Y-%m-%d %T.%3N' 2018-10-24 21:11:12.271 + [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]] + export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64 + export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games + export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m ' + ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m ' + export 'MAVEN_OPTS=-Xmx1g ' + MAVEN_OPTS='-Xmx1g ' + cd /data/hiveptest/working/ + tee /data/hiveptest/logs/PreCommit-HIVE-Build-14627/source-prep.txt + [[ false == \t\r\u\e ]] + mkdir -p maven ivy + [[ git = \s\v\n ]] + [[ git = \g\i\t ]] + [[ -z master ]] + [[ -d apache-github-source-source ]] + [[ ! -d apache-github-source-source/.git ]] + [[ ! -d apache-github-source-source ]] + date '+%Y-%m-%d %T.%3N' 2018-10-24 21:11:12.274 + cd apache-github-source-source + git fetch origin + git reset --hard HEAD HEAD is now at 94d4991 HIVE-18876: Remove Superfluous Logging in Driver (Alice Fan, reviewed by Naveen Gangam) + git clean -f -d Removing standalone-metastore/metastore-server/src/gen/ + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 94d4991 HIVE-18876: Remove Superfluous Logging in Driver (Alice Fan, reviewed by Naveen Gangam) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-10-24 21:11:12.842 + rm -rf ../yetus_PreCommit-HIVE-Build-14627 + mkdir ../yetus_PreCommit-HIVE-Build-14627 + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-14627 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-14627/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HiveRelOptUtil.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinConstraintsRule.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdRowCount.java: does not exist in index error: a/ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/stats/HiveRelMdSelectivity.java: does not exist in index Going to apply patch with: git apply -p1 + [[ maven == \m\a\v\e\n ]] + rm -rf /data/hiveptest/working/maven/org/apache/hive + mvn -B clean install -DskipTests -T 4 -q -Dmaven.repo.local=/data/hiveptest/working/maven protoc-jar: executing: [/tmp/protoc3067556178129132626.exe, --version] libprotoc 2.5.0 protoc-jar: executing: [/tmp/protoc3067556178129132626.exe, -I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore, --java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources, /data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto] ANTLR Parser Generator Version 3.5.2 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (process-resource-bundles) on project hive-shims-0.23: Execution process-resource-bundles of goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process failed. ConcurrentModificationException -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1]