[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733657#comment-16733657
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12953681/HIVE-16957.10.patch
{color:green}SUCCESS:{color} +1 due to 31 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15760 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15494/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15494/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15494/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12953681 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.07.patch, HIVE-16957.07.patch,
> HIVE-16957.08.patch, HIVE-16957.08.patch, HIVE-16957.09.patch,
> HIVE-16957.09.patch, HIVE-16957.10.patch, HIVE-16957.10.patch,
> HIVE-16957.10.patch, HIVE-16957.10.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733632#comment-16733632
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 6 new + 563 unchanged - 7
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
55s{color} | {color:red} ql generated 1 new + 2310 unchanged - 2 fixed = 2311
total (was 2312) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
54s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 6s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15494/dev-support/hive-personality.sh
|
| git revision | master / 691c4cb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15494/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15494/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15494/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hcatalog-unit itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15494/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733478#comment-16733478
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12953667/HIVE-16957.10.patch
{color:green}SUCCESS:{color} +1 due to 31 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15745 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=155)
[intersect_all.q,unionDistinct_1.q,table_nonprintable.q,orc_llap_counters1.q,mm_cttas.q,whroot_external1.q,global_limit.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,results_cache_diff_fs.q,cttl.q,parallel_colstats.q,load_hdfs_file_with_space_in_the_name.q]
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15492/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15492/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15492/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12953667 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.07.patch, HIVE-16957.07.patch,
> HIVE-16957.08.patch, HIVE-16957.08.patch, HIVE-16957.09.patch,
> HIVE-16957.09.patch, HIVE-16957.10.patch, HIVE-16957.10.patch,
> HIVE-16957.10.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733425#comment-16733425
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 6 new + 563 unchanged - 7
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
58s{color} | {color:red} ql generated 1 new + 2310 unchanged - 2 fixed = 2311
total (was 2312) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
53s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 44s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15492/dev-support/hive-personality.sh
|
| git revision | master / 691c4cb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15492/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15492/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15492/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hcatalog-unit itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15492/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733127#comment-16733127
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12953619/HIVE-16957.09.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15480/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15480/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15480/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12953619/HIVE-16957.09.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12953619 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.07.patch, HIVE-16957.07.patch,
> HIVE-16957.08.patch, HIVE-16957.08.patch, HIVE-16957.09.patch,
> HIVE-16957.09.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733126#comment-16733126
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12953619/HIVE-16957.09.patch
{color:green}SUCCESS:{color} +1 due to 31 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15760 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid2]
(batchId=161)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15479/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15479/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15479/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12953619 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.07.patch, HIVE-16957.07.patch,
> HIVE-16957.08.patch, HIVE-16957.08.patch, HIVE-16957.09.patch,
> HIVE-16957.09.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733088#comment-16733088
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 6 new + 563 unchanged - 7
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
50s{color} | {color:red} ql generated 1 new + 2310 unchanged - 2 fixed = 2311
total (was 2312) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
53s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 44s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15479/dev-support/hive-personality.sh
|
| git revision | master / 691c4cb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15479/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15479/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15479/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hcatalog-unit itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15479/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733039#comment-16733039
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12953616/HIVE-16957.09.patch
{color:green}SUCCESS:{color} +1 due to 31 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15760 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid2]
(batchId=161)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=262)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=262)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes
(batchId=262)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15478/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15478/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15478/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12953616 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.07.patch, HIVE-16957.07.patch,
> HIVE-16957.08.patch, HIVE-16957.08.patch, HIVE-16957.09.patch,
> HIVE-16957.09.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733005#comment-16733005
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 6 new + 563 unchanged - 7
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
46s{color} | {color:red} ql generated 1 new + 2310 unchanged - 2 fixed = 2311
total (was 2312) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
54s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 2s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15478/dev-support/hive-personality.sh
|
| git revision | master / 691c4cb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15478/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15478/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15478/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hcatalog-unit itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15478/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732554#comment-16732554
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12953542/HIVE-16957.07.patch
{color:green}SUCCESS:{color} +1 due to 30 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15759 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid2]
(batchId=161)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15460/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15460/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15460/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12953542 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.07.patch, HIVE-16957.07.patch,
> HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732537#comment-16732537
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
42s{color} | {color:blue} ql in master has 2312 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 6 new + 563 unchanged - 7
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
6s{color} | {color:red} ql generated 1 new + 2310 unchanged - 2 fixed = 2311
total (was 2312) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
56s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15460/dev-support/hive-personality.sh
|
| git revision | master / dc215b1 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15460/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15460/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15460/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hcatalog-unit itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15460/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732195#comment-16732195
]
Ashutosh Chauhan commented on HIVE-16957:
-
+1
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.07.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731850#comment-16731850
]
Jesus Camacho Rodriguez commented on HIVE-16957:
[~ashutoshc], I have rebased the latest patch that addressed your comments in
RB.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.07.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727126#comment-16727126
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12952735/HIVE-16957.06.patch
{color:green}SUCCESS:{color} +1 due to 30 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15759 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=334)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15422/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15422/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15422/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12952735 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727115#comment-16727115
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 6 new + 563 unchanged - 7
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
54s{color} | {color:red} ql generated 1 new + 2307 unchanged - 2 fixed = 2308
total (was 2309) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
53s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15422/dev-support/hive-personality.sh
|
| git revision | master / 49f8f84 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15422/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15422/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15422/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hcatalog-unit itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15422/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727088#comment-16727088
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12952733/HIVE-16957.05.patch
{color:green}SUCCESS:{color} +1 due to 29 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15759 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid2]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=190)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.sqlCTAS
(batchId=267)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15421/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15421/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15421/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12952733 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.05.patch,
> HIVE-16957.06.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16727062#comment-16727062
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 6 new + 563 unchanged - 7
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
7s{color} | {color:red} ql generated 1 new + 2307 unchanged - 2 fixed = 2308
total (was 2309) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
54s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 49s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15421/dev-support/hive-personality.sh
|
| git revision | master / 49f8f84 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15421/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15421/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15421/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15421/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726696#comment-16726696
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12952641/HIVE-16957.04.patch
{color:green}SUCCESS:{color} +1 due to 29 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15759 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.hcatalog.listener.TestDbNotificationListener.sqlCTAS
(batchId=267)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15417/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15417/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15417/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12952641 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.04.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726681#comment-16726681
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
4s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 4 new + 565 unchanged - 5
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
48s{color} | {color:red} ql generated 1 new + 2308 unchanged - 2 fixed = 2309
total (was 2310) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
52s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 54s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15417/dev-support/hive-personality.sh
|
| git revision | master / 04cc681 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15417/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15417/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15417/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15417/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726181#comment-16726181
]
Ashutosh Chauhan commented on HIVE-16957:
-
Left some comments for golden file updates. Still going through code changes.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721904#comment-16721904
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12951854/HIVE-16957.03.patch
{color:green}SUCCESS:{color} +1 due to 29 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15724 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15333/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15333/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15333/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12951854 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.03.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721887#comment-16721887
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 4 new + 565 unchanged - 5
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
59s{color} | {color:red} ql generated 1 new + 2308 unchanged - 2 fixed = 2309
total (was 2310) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
54s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15333/dev-support/hive-personality.sh
|
| git revision | master / 4e41560 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15333/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15333/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15333/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15333/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720671#comment-16720671
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12951706/HIVE-16957.02.patch
{color:green}SUCCESS:{color} +1 due to 29 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15660 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_opt_vectorization]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid2]
(batchId=160)
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=255)
org.apache.hive.spark.client.rpc.TestRpc.testClientTimeout (batchId=325)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15304/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15304/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15304/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12951706 - PreCommit-HIVE-Build
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720645#comment-16720645
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
17s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 4 new + 565 unchanged - 5
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
51s{color} | {color:red} ql generated 1 new + 2308 unchanged - 2 fixed = 2309
total (was 2310) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
54s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15304/dev-support/hive-personality.sh
|
| git revision | master / b5b6371 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15304/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15304/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15304/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15304/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720394#comment-16720394
]
Jesus Camacho Rodriguez commented on HIVE-16957:
HIVE-16957.02.patch contains further q file updates and no code changes, ptest
should come clean this time.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.02.patch,
> HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719846#comment-16719846
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12951582/HIVE-16957.01.patch
{color:green}SUCCESS:{color} +1 due to 29 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 42 failed/errored test(s), 15570 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input1_limit]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input3_limit]
(batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input_part10] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert2_overwrite_partitions]
(batchId=95)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into1]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into2]
(batchId=96)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into3]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into4]
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into5]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_into6]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[limit_pushdown_negative]
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[load_dyn_part14]
(batchId=97)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge4] (batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonreserved_keywords_insert_into1]
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udtf_explode]
(batchId=55)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[insert_into1]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[insert_into2]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_3]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_opt_vectorization]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynpart_sort_optimization]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[enforce_constraint_notnull]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert1_overwrite_partitions]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_into_default_keyword]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid2]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[runtime_stats_merge]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[semijoin_hint]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sqlmerge_stats]
(batchId=179)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select]
(batchId=169)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_nway_join]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr_2]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_like_2]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_udf2]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_mapjoin3]
(batchId=160)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_scalar]
(batchId=128)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[subquery_select]
(batchId=128)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query70]
(batchId=272)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query70]
(batchId=270)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query70]
(batchId=270)
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
(batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15293/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15293/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15293/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719824#comment-16719824
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2310 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
36s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
41s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 4 new + 565 unchanged - 5
fixed = 569 total (was 570) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
50s{color} | {color:red} ql generated 1 new + 2308 unchanged - 2 fixed = 2309
total (was 2310) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
59s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15293/dev-support/hive-personality.sh
|
| git revision | master / a43581b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15293/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15293/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15293/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15293/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.patch
>
>
>
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719597#comment-16719597
]
Jesus Camacho Rodriguez commented on HIVE-16957:
[~ashutoshc], could you take a look? https://reviews.apache.org/r/69562/
Thanks
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.01.patch, HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718441#comment-16718441
]
Hive QA commented on HIVE-16957:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12951441/HIVE-16957.patch
{color:green}SUCCESS:{color} +1 due to 27 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 136 failed/errored test(s), 15660 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=270)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_update_status_disable_bitvector]
(batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cbo_rp_cross_product_check_2]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_colname]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas_uses_database_location]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_3] (batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_4] (batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_3] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_4] (batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[cte_mat_5] (batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[decimal_serde]
(batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_ddl] (batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby_duplicate_key]
(batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join42] (batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join_filters_overlap]
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_mv] (batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge3] (batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_default] (batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[named_column_join]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nonmr_fetch] (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[nullformatCTAS]
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_createas1]
(batchId=95)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parallel_orderby]
(batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rcfile_default_format]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample_islocalmode_hook_use_metadata]
(batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_materialized_views]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_noskew]
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[skewjoin_onesideskew]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin9]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats5] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union24] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union25] (batchId=75)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_6]
(batchId=15)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv]
(batchId=194)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[cte_4]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_nullscan]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_udf]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llapdecider]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[rcfile_createas1]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[convert_decimal64_to_decimal]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cross_prod_1]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cross_prod_3]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cross_prod_4]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cross_product_check_1]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cross_product_check_2]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ctas]
(batchId=159)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718424#comment-16718424
]
Hive QA commented on HIVE-16957:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2311 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 3 new + 562 unchanged - 5
fixed = 565 total (was 567) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
57s{color} | {color:red} ql generated 1 new + 2309 unchanged - 2 fixed = 2310
total (was 2311) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
56s{color} | {color:red} ql generated 2 new + 98 unchanged - 2 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 10s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| |
org.apache.hadoop.hive.ql.parse.ColumnStatsSemanticAnalyzer.genPartitionClause(Table,
Map) makes inefficient use of keySet iterator instead of entrySet iterator At
ColumnStatsSemanticAnalyzer.java:of keySet iterator instead of entrySet
iterator At ColumnStatsSemanticAnalyzer.java:[line 160] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15271/dev-support/hive-personality.sh
|
| git revision | master / b650083 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15271/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15271/yetus/new-findbugs-ql.html
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15271/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15271/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718351#comment-16718351
]
Jesus Camacho Rodriguez commented on HIVE-16957:
ALTER MV... REBUILD is working correctly. When incremental rebuild translates
into a MERGE operation, i.e. MV contains a Group By statement, column stats are
not present because the MERGE contains in turn an UPDATE operation, which
currently invalidates column stats. When incremental rebuild translates into
INSERT operation, i.e., MV does not contain a Group By statement, column stats
for the MV are updated correctly.
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16957) Support CTAS for auto gather column stats
[
https://issues.apache.org/jira/browse/HIVE-16957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16718332#comment-16718332
]
Jesus Camacho Rodriguez commented on HIVE-16957:
Uploaded initial patch to trigger ptests. I realized that ALTER MV... REBUILD
is still not collecting the column stats automatically yet, I will work on that.
Cc [~ashutoshc]
> Support CTAS for auto gather column stats
> -
>
> Key: HIVE-16957
> URL: https://issues.apache.org/jira/browse/HIVE-16957
> Project: Hive
> Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-16957.patch
>
>
> The idea is to rely as much as possible on the logic in
> ColumnStatsSemanticAnalyzer as other operations do. In particular, they
> create a 'analyze table t compute statistics for columns', use
> ColumnStatsSemanticAnalyzer to parse it, and connect resulting plan to
> existing INSERT/INSERT OVERWRITE statement. The challenge for CTAS or CREATE
> MATERIALIZED VIEW is that the table object does not exist yet, hence we
> cannot rely fully on ColumnStatsSemanticAnalyzer.
> Thus, we use same process, but ColumnStatsSemanticAnalyzer produces a
> statement for column stats collection that uses a table values clause instead
> of the original table reference:
> {code}
> select compute_stats(col1), compute_stats(col2), compute_stats(col3)
> from table(values(cast(null as int), cast(null as int), cast(null as
> string))) as t(col1, col2, col3);
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
