[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526867#comment-16526867 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12929599/HIVE-18140.01-branch-3.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 14348 tests executed *Failed tests:* {noformat} TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=257) TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=257) TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=257) TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=257) TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95) [infer_bucket_sort_num_buckets.q,infer_bucket_sort_reducers_power_two.q,parallel_orderby.q,bucket_num_reducers_acid.q,scriptfile1.q,infer_bucket_sort_map_operators.q,infer_bucket_sort_merge.q,root_dir_external_table.q,infer_bucket_sort_dyn_part.q,udf_using.q] TestReplicationScenariosAcrossInstances - did not produce a TEST-*.xml file (likely timed out) (batchId=234) TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=257) org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths (batchId=234) org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation (batchId=243) org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=309) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12224/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12224/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12224/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 10 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12929599 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01-branch-3.patch, HIVE-18140.01.patch, > HIVE-18140.01wip01.patch, HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, > HIVE-18140.02.patch, HIVE-18140.02wip01.patch, HIVE-18140.03.patch, > HIVE-18140.04.patch, HIVE-18140.04.patch, HIVE-18140.05.patch, > HIVE-19140.02wip02.patch, HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526794#comment-16526794 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color} | {color:red} /data/hiveptest/logs/PreCommit-HIVE-Build-12224/patches/PreCommit-HIVE-Build-12224.patch does not apply to master. Rebase required? Wrong Branch? See http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12224/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01-branch-3.patch, HIVE-18140.01.patch, > HIVE-18140.01wip01.patch, HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, > HIVE-18140.02.patch, HIVE-18140.02wip01.patch, HIVE-18140.03.patch, > HIVE-18140.04.patch, HIVE-18140.04.patch, HIVE-18140.05.patch, > HIVE-19140.02wip02.patch, HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16526172#comment-16526172 ] Zoltan Haindrich commented on HIVE-18140: - committed to master. Thank you Ashutosh for reviewing the changes! will submit it for a qa run for branch-3 > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02.patch, > HIVE-18140.02wip01.patch, HIVE-18140.03.patch, HIVE-18140.04.patch, > HIVE-18140.04.patch, HIVE-18140.05.patch, HIVE-19140.02wip02.patch, > HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525355#comment-16525355 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12929438/HIVE-18140.05.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 14611 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12191/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12191/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12191/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12929438 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02.patch, > HIVE-18140.02wip01.patch, HIVE-18140.03.patch, HIVE-18140.04.patch, > HIVE-18140.04.patch, HIVE-18140.05.patch, HIVE-19140.02wip02.patch, > HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16525339#comment-16525339 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 57s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 56s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 4s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m 29s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 6m 25s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m 3s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 42s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 28s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 57s{color} | {color:red} ql: The patch generated 18 new + 80 unchanged - 5 fixed = 98 total (was 85) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 3s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 6m 35s{color} | {color:red} ql generated 1 new + 2280 unchanged - 0 fixed = 2281 total (was 2280) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 1m 36s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 19s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 40m 6s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Dead store to results in org.apache.hadoop.hive.ql.stats.StatsUtils.getNumRows(HiveConf, List, Table, PrunedPartitionList, AtomicInteger) At StatsUtils.java:org.apache.hadoop.hive.ql.stats.StatsUtils.getNumRows(HiveConf, List, Table, PrunedPartitionList, AtomicInteger) At StatsUtils.java:[line 189] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc xml compile findbugs checkstyle | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-12191/dev-support/hive-personality.sh | | git revision | master / c7fe4ef | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-12191/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-12191/yetus/new-findbugs-ql.html | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-12191/yetus/diff-javadoc-javadoc-ql.txt | | modules | C: ql itests/qtest U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-12191/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments:
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523837#comment-16523837 ] Ashutosh Chauhan commented on HIVE-18140: - +1 > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02.patch, > HIVE-18140.02wip01.patch, HIVE-18140.03.patch, HIVE-18140.04.patch, > HIVE-19140.02wip02.patch, HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16523536#comment-16523536 ] Zoltan Haindrich commented on HIVE-18140: - rebased patch [~ashutoshc] Could you please take a look? > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02.patch, > HIVE-18140.02wip01.patch, HIVE-18140.03.patch, HIVE-18140.04.patch, > HIVE-19140.02wip02.patch, HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16517140#comment-16517140 ] Zoltan Haindrich commented on HIVE-18140: - [~ashutoshc] Could you please take a look? > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02.patch, > HIVE-18140.02wip01.patch, HIVE-18140.03.patch, HIVE-19140.02wip02.patch, > HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516938#comment-16516938 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12928177/HIVE-18140.03.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 14535 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/11917/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11917/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11917/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12928177 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02.patch, > HIVE-18140.02wip01.patch, HIVE-18140.03.patch, HIVE-19140.02wip02.patch, > HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516915#comment-16516915 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 31s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 15s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 34s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 50s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m 8s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 16s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 9s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 56s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 33s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 37s{color} | {color:red} ql: The patch generated 18 new + 80 unchanged - 5 fixed = 98 total (was 85) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 13s{color} | {color:red} ql generated 1 new + 2280 unchanged - 0 fixed = 2281 total (was 2280) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 54s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 25m 41s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Dead store to results in org.apache.hadoop.hive.ql.stats.StatsUtils.getNumRows(HiveConf, List, Table, PrunedPartitionList, AtomicInteger) At StatsUtils.java:org.apache.hadoop.hive.ql.stats.StatsUtils.getNumRows(HiveConf, List, Table, PrunedPartitionList, AtomicInteger) At StatsUtils.java:[line 189] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc xml compile findbugs checkstyle | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-11917/dev-support/hive-personality.sh | | git revision | master / 2394e40 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-11917/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-11917/yetus/new-findbugs-ql.html | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-11917/yetus/diff-javadoc-javadoc-ql.txt | | modules | C: itests/qtest ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-11917/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments:
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515194#comment-16515194 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12927974/HIVE-18140.02.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14526 tests executed *Failed tests:* {noformat} TestHCatExternalDynamicPartitioned - did not produce a TEST-*.xml file (likely timed out) (batchId=206) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/11861/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11861/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11861/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 1 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12927974 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02.patch, > HIVE-18140.02wip01.patch, HIVE-19140.02wip02.patch, HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515184#comment-16515184 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 30s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 53s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 31s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 51s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 56s{color} | {color:blue} ql in master has 2281 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 35s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 38s{color} | {color:red} ql: The patch generated 18 new + 80 unchanged - 5 fixed = 98 total (was 85) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 2s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m 10s{color} | {color:red} ql generated 1 new + 2281 unchanged - 0 fixed = 2282 total (was 2281) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 58s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 26m 3s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:ql | | | Dead store to results in org.apache.hadoop.hive.ql.stats.StatsUtils.getNumRows(HiveConf, List, Table, PrunedPartitionList, AtomicInteger) At StatsUtils.java:org.apache.hadoop.hive.ql.stats.StatsUtils.getNumRows(HiveConf, List, Table, PrunedPartitionList, AtomicInteger) At StatsUtils.java:[line 189] | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc xml compile findbugs checkstyle | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-11861/dev-support/hive-personality.sh | | git revision | master / 24da460 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-11861/yetus/diff-checkstyle-ql.txt | | findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-11861/yetus/new-findbugs-ql.html | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-11861/yetus/diff-javadoc-javadoc-ql.txt | | modules | C: itests/qtest ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-11861/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments:
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502927#comment-16502927 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12926312/HIVE-19727.02wip03.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:green}SUCCESS:{color} +1 due to 14469 tests passed Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/11550/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11550/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11550/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase {noformat} This message is automatically generated. ATTACHMENT ID: 12926312 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02wip01.patch, > HIVE-19140.02wip02.patch, HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502894#comment-16502894 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 24s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 13s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 22s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 46s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 24s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 7s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 20s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 20s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 33s{color} | {color:red} ql: The patch generated 19 new + 80 unchanged - 5 fixed = 99 total (was 85) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 36s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 47s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 11s{color} | {color:red} The patch generated 2 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 22m 48s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc xml compile findbugs checkstyle | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-11550/dev-support/hive-personality.sh | | git revision | master / 0992d82 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-11550/yetus/diff-checkstyle-ql.txt | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-11550/yetus/diff-javadoc-javadoc-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-11550/yetus/patch-asflicense-problems.txt | | modules | C: itests/qtest ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-11550/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02wip01.patch, > HIVE-19140.02wip02.patch, HIVE-19727.02wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then >
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499235#comment-16499235 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12925871/HIVE-19140.02wip02.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/11453/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11453/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11453/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Tests exited with: Exception: Patch URL https://issues.apache.org/jira/secure/attachment/12925871/HIVE-19140.02wip02.patch was found in seen patch url's cache and a test was probably run already on it. Aborting... {noformat} This message is automatically generated. ATTACHMENT ID: 12925871 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02wip01.patch, > HIVE-19140.02wip02.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497068#comment-16497068 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12925871/HIVE-19140.02wip02.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14427 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert1_overwrite_partitions] (batchId=171) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[bucket_mapjoin_mismatch1] (batchId=97) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[bucketmapjoin7] (batchId=122) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/11391/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11391/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11391/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 3 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12925871 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02wip01.patch, > HIVE-19140.02wip02.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16497025#comment-16497025 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 27s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 36s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 24s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 47s{color} | {color:green} master passed {color} | | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m 50s{color} | {color:blue} ql in master has 2278 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 12s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 25s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 36s{color} | {color:red} ql: The patch generated 19 new + 79 unchanged - 4 fixed = 98 total (was 83) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s{color} | {color:green} The patch has no ill-formed XML file. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m 47s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 51s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 24m 28s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc xml compile findbugs checkstyle | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-11391/dev-support/hive-personality.sh | | git revision | master / 338a2d4 | | Default Java | 1.8.0_111 | | findbugs | v3.0.0 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-11391/yetus/diff-checkstyle-ql.txt | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-11391/yetus/diff-javadoc-javadoc-ql.txt | | modules | C: itests/qtest ql U: . | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-11391/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01.patch, HIVE-18140.01wip01.patch, > HIVE-18140.01wip03.patch, HIVE-18140.01wip04.patch, HIVE-18140.02wip01.patch, > HIVE-19140.02wip02.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16410623#comment-16410623 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12915527/HIVE-18140.01.patch {color:red}ERROR:{color} -1 due to build exiting with an error Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9776/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9776/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9776/ Messages: {noformat} This message was trimmed, see log for full details + git fetch origin Auto packing the repository in background for optimum performance. See "git help gc" for manual housekeeping. + git reset --hard HEAD HEAD is now at 353a9cf HIVE-19013. Fix various storage-api build issues. (Owen O'Malley, reviewed by Prasanth Jayachandran) + git clean -f -d + git checkout master Already on 'master' Your branch is up-to-date with 'origin/master'. + git reset --hard origin/master HEAD is now at 353a9cf HIVE-19013. Fix various storage-api build issues. (Owen O'Malley, reviewed by Prasanth Jayachandran) + git merge --ff-only origin/master Already up-to-date. + date '+%Y-%m-%d %T.%3N' 2018-03-23 00:29:23.410 + rm -rf ../yetus_PreCommit-HIVE-Build-9776 + mkdir ../yetus_PreCommit-HIVE-Build-9776 + git gc + sleep 1s + git gc + sleep 1s + git gc + sleep 1s + git gc + sleep 1s + git gc + cp -R . ../yetus_PreCommit-HIVE-Build-9776 + mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-9776/yetus + patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh + patchFilePath=/data/hiveptest/working/scratch/build.patch + [[ -f /data/hiveptest/working/scratch/build.patch ]] + chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh + /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch error: ql/src/test/results/clientpositive/insert1_overwrite_partitions.q.out: does not exist in index error: qtest/pom.xml: does not exist in index error: src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java: does not exist in index error: src/java/org/apache/hadoop/hive/ql/plan/Statistics.java: does not exist in index error: src/java/org/apache/hadoop/hive/ql/stats/Partish.java: does not exist in index error: src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java: does not exist in index error: src/test/queries/clientpositive/stats8.q: does not exist in index error: src/test/results/clientnegative/bucket_mapjoin_mismatch1.q.out: does not exist in index error: src/test/results/clientpositive/analyze_table_null_partition.q.out: does not exist in index error: src/test/results/clientpositive/annotate_stats_part.q.out: does not exist in index error: src/test/results/clientpositive/autoColumnStats_2.q.out: does not exist in index error: src/test/results/clientpositive/auto_sortmerge_join_1.q.out: does not exist in index error: src/test/results/clientpositive/auto_sortmerge_join_11.q.out: does not exist in index error: src/test/results/clientpositive/auto_sortmerge_join_12.q.out: does not exist in index error: src/test/results/clientpositive/auto_sortmerge_join_2.q.out: does not exist in index error: src/test/results/clientpositive/auto_sortmerge_join_3.q.out: does not exist in index error: src/test/results/clientpositive/auto_sortmerge_join_4.q.out: does not exist in index error: src/test/results/clientpositive/auto_sortmerge_join_7.q.out: does not exist in index error: src/test/results/clientpositive/beeline/smb_mapjoin_10.q.out: does not exist in index error: src/test/results/clientpositive/bucket_map_join_spark1.q.out: does not exist in index error: src/test/results/clientpositive/bucket_map_join_spark2.q.out: does not exist in index error: src/test/results/clientpositive/bucket_map_join_spark3.q.out: does not exist in index error: src/test/results/clientpositive/bucketcontext_1.q.out: does not exist in index error: src/test/results/clientpositive/bucketcontext_2.q.out: does not exist in index error: src/test/results/clientpositive/bucketcontext_3.q.out: does not exist in index error: src/test/results/clientpositive/bucketcontext_4.q.out: does not exist in index error: src/test/results/clientpositive/bucketcontext_6.q.out: does not exist in index error: src/test/results/clientpositive/bucketcontext_7.q.out: does not exist in index error: src/test/results/clientpositive/bucketcontext_8.q.out: does not exist in index error: src/test/results/clientpositive/bucketmapjoin10.q.out: does not exist in index error: src/test/results/clientpositive/bucketmapjoin11.q.out: does not exist in index error: src/test/results/clientpositive/bucketmapjoin12.q.out: does not exist in index error: src/test/results/clientpositive/bucketmapjoin5.q.out: does not exist in index error:
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404979#comment-16404979 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12915126/HIVE-18140.01wip04.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 112 failed/errored test(s), 13425 tests executed *Failed tests:* {noformat} TestNegativeCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=95)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404909#comment-16404909 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 39s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 0s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 39s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 2s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 43s{color} | {color:red} ql: The patch generated 21 new + 77 unchanged - 6 fixed = 98 total (was 83) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 55s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 14m 39s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9708/dev-support/hive-personality.sh | | git revision | master / 94152c9 | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9708/yetus/diff-checkstyle-ql.txt | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-9708/yetus/diff-javadoc-javadoc-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9708/yetus/patch-asflicense-problems.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9708/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01wip01.patch, HIVE-18140.01wip03.patch, > HIVE-18140.01wip04.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390752#comment-16390752 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913402/HIVE-18140.01wip03.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 204 failed/errored test(s), 13747 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_10] (batchId=250) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats] (batchId=53) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart] (batchId=87) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_table_null_partition] (batchId=82) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part] (batchId=16) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_2] (batchId=84) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_11] (batchId=87) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_12] (batchId=33) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_1] (batchId=46) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=49) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_3] (batchId=2) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_4] (batchId=63) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_7] (batchId=90) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark1] (batchId=68) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark2] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark3] (batchId=45) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_1] (batchId=33) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_2] (batchId=66) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_3] (batchId=68) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_4] (batchId=42) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_6] (batchId=84) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_7] (batchId=39) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_8] (batchId=37) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin10] (batchId=52) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin11] (batchId=72) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin12] (batchId=36) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin5] (batchId=85) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin8] (batchId=13) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin9] (batchId=17) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative2] (batchId=69) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative] (batchId=23) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl] (batchId=36) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp] (batchId=52) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_6] (batchId=9) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input25] (batchId=45) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert1_overwrite_partitions] (batchId=68) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert2_overwrite_partitions] (batchId=88) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_1] (batchId=60) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_2] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_3] (batchId=10) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition2] (batchId=16) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition3] (batchId=70) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition] (batchId=42) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullgroup3] (batchId=2) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullgroup5] (batchId=31) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=78) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_10] (batchId=88) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats12] (batchId=62) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats13] (batchId=81)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390702#comment-16390702 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 59s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 35s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 40s{color} | {color:red} ql: The patch generated 24 new + 77 unchanged - 6 fixed = 101 total (was 83) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 50s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 13s{color} | {color:red} The patch generated 49 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 13m 34s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9544/dev-support/hive-personality.sh | | git revision | master / 073dc88 | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9544/yetus/diff-checkstyle-ql.txt | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-9544/yetus/diff-javadoc-javadoc-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9544/yetus/patch-asflicense-problems.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9544/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01wip01.patch, HIVE-18140.01wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390687#comment-16390687 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12913402/HIVE-18140.01wip03.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 200 failed/errored test(s), 13747 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_10] (batchId=250) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats] (batchId=53) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alterColumnStatsPart] (batchId=87) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_table_null_partition] (batchId=82) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part] (batchId=16) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_2] (batchId=84) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_11] (batchId=87) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_12] (batchId=33) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_1] (batchId=46) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2] (batchId=49) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_3] (batchId=2) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_4] (batchId=63) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_7] (batchId=90) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark1] (batchId=68) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark2] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucket_map_join_spark3] (batchId=45) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_1] (batchId=33) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_2] (batchId=66) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_3] (batchId=68) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_4] (batchId=42) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_6] (batchId=84) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_7] (batchId=39) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketcontext_8] (batchId=37) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin10] (batchId=52) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin11] (batchId=72) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin12] (batchId=36) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin5] (batchId=85) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin8] (batchId=13) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin9] (batchId=17) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative2] (batchId=69) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketmapjoin_negative] (batchId=23) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl] (batchId=36) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp] (batchId=52) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_sort_6] (batchId=9) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input25] (batchId=45) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert1_overwrite_partitions] (batchId=68) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert2_overwrite_partitions] (batchId=88) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_1] (batchId=60) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_2] (batchId=3) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_query_oneskew_3] (batchId=10) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition2] (batchId=16) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition3] (batchId=70) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_dynamic_partition] (batchId=42) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullgroup3] (batchId=2) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullgroup5] (batchId=31) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=78) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_10] (batchId=88) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats12] (batchId=62) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats13] (batchId=81)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390624#comment-16390624 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 0s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 49s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 37s{color} | {color:red} ql: The patch generated 24 new + 77 unchanged - 6 fixed = 101 total (was 83) {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 55s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 12s{color} | {color:red} The patch generated 49 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 13m 43s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-9543/dev-support/hive-personality.sh | | git revision | master / 073dc88 | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-9543/yetus/diff-checkstyle-ql.txt | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-9543/yetus/diff-javadoc-javadoc-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-9543/yetus/patch-asflicense-problems.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-9543/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-18140.01wip01.patch, HIVE-18140.01wip03.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16282172#comment-16282172 ] Ashutosh Chauhan commented on HIVE-18140: - [~kgyrtkirk] Does this need more work? > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18140.01wip01.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269196#comment-16269196 ] Zoltan Haindrich commented on HIVE-18140: - I've merged HIVE-18108 locally into this... > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18140.01wip01.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268857#comment-16268857 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12899061/HIVE-18140.01wip01.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 11414 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_table_null_partition] (batchId=80) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part] (batchId=15) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp] (batchId=50) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_var_samp] (batchId=84) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid] (batchId=158) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=162) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_reordering_no_stats] (batchId=162) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan] (batchId=159) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats8] (batchId=134) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=224) org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=230) org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=230) org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=230) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8021/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8021/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8021/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 15 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12899061 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18140.01wip01.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268782#comment-16268782 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m 52s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 33s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 55s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 1s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 33s{color} | {color:red} ql: The patch generated 15 new + 125 unchanged - 5 fixed = 140 total (was 130) {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 57s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 15s{color} | {color:red} The patch generated 5 ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 12m 45s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / a21742e | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-8021/yetus/diff-checkstyle-ql.txt | | whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-8021/yetus/whitespace-eol.txt | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-8021/yetus/diff-javadoc-javadoc-ql.txt | | asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-8021/yetus/patch-asflicense-problems.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-8021/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18140.01wip01.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265273#comment-16265273 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12899061/HIVE-18140.01wip01.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 11412 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries] (batchId=236) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_table_null_partition] (batchId=80) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part] (batchId=15) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp] (batchId=50) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid] (batchId=158) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=162) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_reordering_no_stats] (batchId=162) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats8] (batchId=134) org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning (batchId=297) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=224) org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=230) org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=230) org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=230) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7994/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7994/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7994/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 14 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12899061 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18140.01wip01.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16265235#comment-16265235 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 1s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 13s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 2s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 33s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 32s{color} | {color:red} ql: The patch generated 15 new + 126 unchanged - 5 fixed = 141 total (was 131) {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 51s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 12m 40s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / d9924ab | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-7994/yetus/diff-checkstyle-ql.txt | | whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-7994/yetus/whitespace-eol.txt | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-7994/yetus/diff-javadoc-javadoc-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-7994/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18140.01wip01.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264771#comment-16264771 ] Hive QA commented on HIVE-18140: Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12899061/HIVE-18140.01wip01.patch {color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 11412 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_table_null_partition] (batchId=80) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part] (batchId=15) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[columnstats_partlvl_dp] (batchId=50) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization_acid] (batchId=158) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata] (batchId=162) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_reordering_no_stats] (batchId=162) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[stats8] (batchId=134) org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints (batchId=224) org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=230) org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=230) org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=230) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7990/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7990/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7990/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.YetusPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 12 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12899061 - PreCommit-HIVE-Build > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18140.01wip01.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264747#comment-16264747 ] Hive QA commented on HIVE-18140: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || || || || || {color:brown} Prechecks {color} || | {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m 0s{color} | {color:blue} Findbugs executables are not available. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | || || || || {color:brown} master Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 14s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 32s{color} | {color:green} master passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s{color} | {color:green} master passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 18s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 32s{color} | {color:red} ql: The patch generated 15 new + 126 unchanged - 5 fixed = 141 total (was 131) {color} | | {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m 0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 50s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 12s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 12m 42s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Optional Tests | asflicense javac javadoc findbugs checkstyle compile | | uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux | | Build tool | maven | | Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh | | git revision | master / d9924ab | | Default Java | 1.8.0_111 | | checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-7990/yetus/diff-checkstyle-ql.txt | | whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-7990/yetus/whitespace-eol.txt | | javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-7990/yetus/diff-javadoc-javadoc-ql.txt | | modules | C: ql U: ql | | Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-7990/yetus.txt | | Powered by | Apache Yetushttp://yetus.apache.org | This message was automatically generated. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich > Attachments: HIVE-18140.01wip01.patch > > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18140) Partitioned tables statistics can go wrong in basic stats mixed case
[ https://issues.apache.org/jira/browse/HIVE-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264276#comment-16264276 ] Zoltan Haindrich commented on HIVE-18140: - HIVE-18108 also collaborates to mess-up this thing...I'll attach the refactored fix here - which is designed to not break many case ; it started fixing this issue because its what naturally does ; I started it as HIVE-18015 ... but it looks like it will be safer to separate the refactor and fixing all the issues. > Partitioned tables statistics can go wrong in basic stats mixed case > > > Key: HIVE-18140 > URL: https://issues.apache.org/jira/browse/HIVE-18140 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich > > suppose the following scenario: > * part1 has basic stats {{RC=10,DS=1K}} > * all other partition has no basic stats (and a bunch of rows) > then > [this|https://github.com/apache/hive/blob/d9924ab3e285536f7e2cc15ecbea36a78c59c66d/ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java#L378] > condition would be false; which in turn produces estimations for the whole > partitioned table: {{RC=10,DS=1K}} -- This message was sent by Atlassian JIRA (v6.4.14#64029)