[jira] [Commented] (HIVE-22510) Support decimal64 operations for column operands with different scales
[
https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010367#comment-17010367
]
Hive QA commented on HIVE-22510:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990149/HIVE-22510.18.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17859 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20105/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20105/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20105/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990149 - PreCommit-HIVE-Build
> Support decimal64 operations for column operands with different scales
> --
>
> Key: HIVE-22510
> URL: https://issues.apache.org/jira/browse/HIVE-22510
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22510.11.patch, HIVE-22510.13.patch,
> HIVE-22510.14.patch, HIVE-22510.15.patch, HIVE-22510.16.patch,
> HIVE-22510.17.patch, HIVE-22510.18.patch, HIVE-22510.2.patch,
> HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch,
> HIVE-22510.7.patch, HIVE-22510.9.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Right now, if the operands on the decimal64 operations are columns with
> different scales, then we do not use the decimal64 vectorized version and
> fall back to HiveDecimal vectorized version of the operator. In this Jira, we
> will check if we can use decimal64 vectorized version, even if the scales are
> different.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22510) Support decimal64 operations for column operands with different scales
[
https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010345#comment-17010345
]
Hive QA commented on HIVE-22510:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 2 new + 780 unchanged - 0
fixed = 782 total (was 780) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20105/dev-support/hive-personality.sh
|
| git revision | master / f71a243 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20105/yetus/diff-checkstyle-ql.txt
|
| modules | C: vector-code-gen ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20105/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support decimal64 operations for column operands with different scales
> --
>
> Key: HIVE-22510
> URL: https://issues.apache.org/jira/browse/HIVE-22510
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22510.11.patch, HIVE-22510.13.patch,
> HIVE-22510.14.patch, HIVE-22510.15.patch, HIVE-22510.16.patch,
> HIVE-22510.17.patch, HIVE-22510.18.patch, HIVE-22510.2.patch,
> HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch,
> HIVE-22510.7.patch, HIVE-22510.9.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Right now, if the operands on the decimal64 operations are columns with
> different scales, then we do not use the decimal64 vectorized version and
> fall back to HiveDecimal vectorized version of the operator. In this Jira, we
> will check if we can use decimal64 vectorized version, even if the scales are
> different.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22261) Materialized view rewriting does not support window functions
[
https://issues.apache.org/jira/browse/HIVE-22261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010321#comment-17010321
]
Hive QA commented on HIVE-22261:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990145/HIVE-22261.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17858 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20104/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20104/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20104/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990145 - PreCommit-HIVE-Build
> Materialized view rewriting does not support window functions
> -
>
> Key: HIVE-22261
> URL: https://issues.apache.org/jira/browse/HIVE-22261
> Project: Hive
> Issue Type: Sub-task
> Components: CBO, Materialized views
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22261.patch, af2.sql
>
>
> Materialized views don't support window functions. At a minimum, we should
> print a friendlier message when the rewrite fails (it can still be created
> with a "disable rewrite")
> Script is attached
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22261) Materialized view rewriting does not support window functions
[
https://issues.apache.org/jira/browse/HIVE-22261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010265#comment-17010265
]
Hive QA commented on HIVE-22261:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 2m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20104/dev-support/hive-personality.sh
|
| git revision | master / f71a243 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20104/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Materialized view rewriting does not support window functions
> -
>
> Key: HIVE-22261
> URL: https://issues.apache.org/jira/browse/HIVE-22261
> Project: Hive
> Issue Type: Sub-task
> Components: CBO, Materialized views
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22261.patch, af2.sql
>
>
> Materialized views don't support window functions. At a minimum, we should
> print a friendlier message when the rewrite fails (it can still be created
> with a "disable rewrite")
> Script is attached
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010260#comment-17010260
]
Hive QA commented on HIVE-22699:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990142/HIVE-22699.2.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17857 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20103/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20103/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20103/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990142 - PreCommit-HIVE-Build
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch, HIVE-22699.2.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22510) Support decimal64 operations for column operands with different scales
[ https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22510: Attachment: HIVE-22510.18.patch Status: Patch Available (was: Open) > Support decimal64 operations for column operands with different scales > -- > > Key: HIVE-22510 > URL: https://issues.apache.org/jira/browse/HIVE-22510 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22510.11.patch, HIVE-22510.13.patch, > HIVE-22510.14.patch, HIVE-22510.15.patch, HIVE-22510.16.patch, > HIVE-22510.17.patch, HIVE-22510.18.patch, HIVE-22510.2.patch, > HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch, > HIVE-22510.7.patch, HIVE-22510.9.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Right now, if the operands on the decimal64 operations are columns with > different scales, then we do not use the decimal64 vectorized version and > fall back to HiveDecimal vectorized version of the operator. In this Jira, we > will check if we can use decimal64 vectorized version, even if the scales are > different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22510) Support decimal64 operations for column operands with different scales
[ https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22510: Attachment: (was: HIVE-22510.18.patch) > Support decimal64 operations for column operands with different scales > -- > > Key: HIVE-22510 > URL: https://issues.apache.org/jira/browse/HIVE-22510 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22510.11.patch, HIVE-22510.13.patch, > HIVE-22510.14.patch, HIVE-22510.15.patch, HIVE-22510.16.patch, > HIVE-22510.17.patch, HIVE-22510.18.patch, HIVE-22510.2.patch, > HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch, > HIVE-22510.7.patch, HIVE-22510.9.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Right now, if the operands on the decimal64 operations are columns with > different scales, then we do not use the decimal64 vectorized version and > fall back to HiveDecimal vectorized version of the operator. In this Jira, we > will check if we can use decimal64 vectorized version, even if the scales are > different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22510) Support decimal64 operations for column operands with different scales
[ https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22510: Status: Open (was: Patch Available) > Support decimal64 operations for column operands with different scales > -- > > Key: HIVE-22510 > URL: https://issues.apache.org/jira/browse/HIVE-22510 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22510.11.patch, HIVE-22510.13.patch, > HIVE-22510.14.patch, HIVE-22510.15.patch, HIVE-22510.16.patch, > HIVE-22510.17.patch, HIVE-22510.18.patch, HIVE-22510.2.patch, > HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch, > HIVE-22510.7.patch, HIVE-22510.9.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Right now, if the operands on the decimal64 operations are columns with > different scales, then we do not use the decimal64 vectorized version and > fall back to HiveDecimal vectorized version of the operator. In this Jira, we > will check if we can use decimal64 vectorized version, even if the scales are > different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010226#comment-17010226
]
Hive QA commented on HIVE-22699:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20103/dev-support/hive-personality.sh
|
| git revision | master / f71a243 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20103/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch, HIVE-22699.2.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22261) Materialized view rewriting does not support window functions
[ https://issues.apache.org/jira/browse/HIVE-22261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22261: --- Attachment: HIVE-22261.patch > Materialized view rewriting does not support window functions > - > > Key: HIVE-22261 > URL: https://issues.apache.org/jira/browse/HIVE-22261 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22261.patch, af2.sql > > > Materialized views don't support window functions. At a minimum, we should > print a friendlier message when the rewrite fails (it can still be created > with a "disable rewrite") > Script is attached > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22261) Materialized view rewriting does not support window functions
[ https://issues.apache.org/jira/browse/HIVE-22261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22261: --- Status: Patch Available (was: In Progress) > Materialized view rewriting does not support window functions > - > > Key: HIVE-22261 > URL: https://issues.apache.org/jira/browse/HIVE-22261 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: af2.sql > > > Materialized views don't support window functions. At a minimum, we should > print a friendlier message when the rewrite fails (it can still be created > with a "disable rewrite") > Script is attached > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-22261) Materialized view rewriting does not support window functions
[ https://issues.apache.org/jira/browse/HIVE-22261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22261 started by Jesus Camacho Rodriguez. -- > Materialized view rewriting does not support window functions > - > > Key: HIVE-22261 > URL: https://issues.apache.org/jira/browse/HIVE-22261 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: af2.sql > > > Materialized views don't support window functions. At a minimum, we should > print a friendlier message when the rewrite fails (it can still be created > with a "disable rewrite") > Script is attached > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated HIVE-22699:
--
Status: Patch Available (was: In Progress)
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch, HIVE-22699.2.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated HIVE-22699:
--
Status: In Progress (was: Patch Available)
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch, HIVE-22699.2.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated HIVE-22699:
--
Attachment: HIVE-22699.2.patch
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch, HIVE-22699.2.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22261) Materialized view rewriting does not support window functions
[ https://issues.apache.org/jira/browse/HIVE-22261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-22261: -- Assignee: Jesus Camacho Rodriguez > Materialized view rewriting does not support window functions > - > > Key: HIVE-22261 > URL: https://issues.apache.org/jira/browse/HIVE-22261 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: af2.sql > > > Materialized views don't support window functions. At a minimum, we should > print a friendlier message when the rewrite fails (it can still be created > with a "disable rewrite") > Script is attached > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22609) Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
[
https://issues.apache.org/jira/browse/HIVE-22609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-22609:
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Thanks [~ashutoshc]. Committed to master.
> Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
> -
>
> Key: HIVE-22609
> URL: https://issues.apache.org/jira/browse/HIVE-22609
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22609.1.patch, HIVE-22609.2.patch,
> HIVE-22609.3.patch, HIVE-22609.4.patch, HIVE-22609.5.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L1380]
> ACID delta folder contains {{_orc_acid_version}} and {{bucket_0}} files.
> For both these files, parent dir is the same. Number of getFileStatus in such
> cases should be reduced by 1/2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22510) Support decimal64 operations for column operands with different scales
[
https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010122#comment-17010122
]
Hive QA commented on HIVE-22510:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990122/HIVE-22510.18.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17859 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestRestrictedList.org.apache.hive.jdbc.TestRestrictedList
(batchId=288)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20102/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20102/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20102/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990122 - PreCommit-HIVE-Build
> Support decimal64 operations for column operands with different scales
> --
>
> Key: HIVE-22510
> URL: https://issues.apache.org/jira/browse/HIVE-22510
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22510.11.patch, HIVE-22510.13.patch,
> HIVE-22510.14.patch, HIVE-22510.15.patch, HIVE-22510.16.patch,
> HIVE-22510.17.patch, HIVE-22510.18.patch, HIVE-22510.2.patch,
> HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch,
> HIVE-22510.7.patch, HIVE-22510.9.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Right now, if the operands on the decimal64 operations are columns with
> different scales, then we do not use the decimal64 vectorized version and
> fall back to HiveDecimal vectorized version of the operator. In this Jira, we
> will check if we can use decimal64 vectorized version, even if the scales are
> different.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22510) Support decimal64 operations for column operands with different scales
[
https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010090#comment-17010090
]
Hive QA commented on HIVE-22510:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
25s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 2 new + 780 unchanged - 0
fixed = 782 total (was 780) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20102/dev-support/hive-personality.sh
|
| git revision | master / 8a4392f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20102/yetus/diff-checkstyle-ql.txt
|
| modules | C: vector-code-gen ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20102/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support decimal64 operations for column operands with different scales
> --
>
> Key: HIVE-22510
> URL: https://issues.apache.org/jira/browse/HIVE-22510
> Project: Hive
> Issue Type: Bug
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22510.11.patch, HIVE-22510.13.patch,
> HIVE-22510.14.patch, HIVE-22510.15.patch, HIVE-22510.16.patch,
> HIVE-22510.17.patch, HIVE-22510.18.patch, HIVE-22510.2.patch,
> HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch,
> HIVE-22510.7.patch, HIVE-22510.9.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Right now, if the operands on the decimal64 operations are columns with
> different scales, then we do not use the decimal64 vectorized version and
> fall back to HiveDecimal vectorized version of the operator. In this Jira, we
> will check if we can use decimal64 vectorized version, even if the scales are
> different.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010073#comment-17010073
]
Hive QA commented on HIVE-22489:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990119/HIVE-22489.10.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20101/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20101/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20101/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-01-07 20:32:43.033
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20101/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-01-07 20:32:43.036
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 8a4392f HIVE-22652: TopNKey push through Group by with Grouping
sets (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 8a4392f HIVE-22652: TopNKey push through Group by with Grouping
sets (Krisztian Kasa, reviewed by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-01-07 20:32:43.759
+ rm -rf ../yetus_PreCommit-HIVE-Build-20101
+ mkdir ../yetus_PreCommit-HIVE-Build-20101
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20101
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20101/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: patch failed:
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out:99
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out'
with conflicts.
error: patch failed: ql/src/test/results/clientpositive/spark/cbo_limit.q.out:8
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/spark/cbo_limit.q.out'
cleanly.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:48738: trailing whitespace.
z
/data/hiveptest/working/scratch/build.patch:48742: trailing whitespace.
z
/data/hiveptest/working/scratch/build.patch:48840: trailing whitespace.
z
/data/hiveptest/working/scratch/build.patch:48844: trailing whitespace.
z
/data/hiveptest/working/scratch/build.patch:48937: trailing whitespace.
z
error: patch failed:
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out:99
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out'
with conflicts.
error: patch failed: ql/src/test/results/clientpositive/spark/cbo_limit.q.out:8
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/spark/cbo_limit.q.out'
cleanly.
U
ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets_limit.q.out
warning: squelched 20 whitespace errors
warning: 25 lines add whitespace errors.
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-20101
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990119 - PreCommit-HIVE-Build
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
>
[jira] [Commented] (HIVE-22698) Support Statement#closeOnCompletion()
[
https://issues.apache.org/jira/browse/HIVE-22698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010071#comment-17010071
]
Hive QA commented on HIVE-22698:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990100/HIVE-22698.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17860 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20100/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20100/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20100/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990100 - PreCommit-HIVE-Build
> Support Statement#closeOnCompletion()
> -
>
> Key: HIVE-22698
> URL: https://issues.apache.org/jira/browse/HIVE-22698
> Project: Hive
> Issue Type: Task
> Components: JDBC
>Affects Versions: 3.1.2, 2.3.6
>Reporter: Iwao AVE!
>Priority: Major
> Labels: patch
> Attachments: HIVE-22698.2.patch, HIVE-22698.3.patch, HIVE-22698.patch
>
>
> I am a member of [MyBatis|https://github.com/mybatis] team and a user
> [reported|https://github.com/mybatis/mybatis-3/issues/1351#issuecomment-523771341]
> that Hive does not support java.sql.Statement#closeOnCompletion() yet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22698) Support Statement#closeOnCompletion()
[
https://issues.apache.org/jira/browse/HIVE-22698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010017#comment-17010017
]
Hive QA commented on HIVE-22698:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
12s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} jdbc in master has 16 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20100/dev-support/hive-personality.sh
|
| git revision | master / 8a4392f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: jdbc itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20100/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support Statement#closeOnCompletion()
> -
>
> Key: HIVE-22698
> URL: https://issues.apache.org/jira/browse/HIVE-22698
> Project: Hive
> Issue Type: Task
> Components: JDBC
>Affects Versions: 3.1.2, 2.3.6
>Reporter: Iwao AVE!
>Priority: Major
> Labels: patch
> Attachments: HIVE-22698.2.patch, HIVE-22698.3.patch, HIVE-22698.patch
>
>
> I am a member of [MyBatis|https://github.com/mybatis] team and a user
> [reported|https://github.com/mybatis/mybatis-3/issues/1351#issuecomment-523771341]
> that Hive does not support java.sql.Statement#closeOnCompletion() yet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20934) ACID: Query based compactor for minor compaction
[
https://issues.apache.org/jira/browse/HIVE-20934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010001#comment-17010001
]
Hive QA commented on HIVE-20934:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990104/HIVE-20934.05.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17865 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20099/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20099/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20099/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990104 - PreCommit-HIVE-Build
> ACID: Query based compactor for minor compaction
>
>
> Key: HIVE-20934
> URL: https://issues.apache.org/jira/browse/HIVE-20934
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20934.01.patch, HIVE-20934.02.patch,
> HIVE-20934.03.patch, HIVE-20934.04.patch, HIVE-20934.05.patch
>
>
> Follow up of HIVE-20699. This is to enable running minor compactions as a
> HiveQL query
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22510) Support decimal64 operations for column operands with different scales
[ https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22510: Status: Open (was: Patch Available) > Support decimal64 operations for column operands with different scales > -- > > Key: HIVE-22510 > URL: https://issues.apache.org/jira/browse/HIVE-22510 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22510.11.patch, HIVE-22510.13.patch, > HIVE-22510.14.patch, HIVE-22510.15.patch, HIVE-22510.16.patch, > HIVE-22510.17.patch, HIVE-22510.18.patch, HIVE-22510.2.patch, > HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch, > HIVE-22510.7.patch, HIVE-22510.9.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Right now, if the operands on the decimal64 operations are columns with > different scales, then we do not use the decimal64 vectorized version and > fall back to HiveDecimal vectorized version of the operator. In this Jira, we > will check if we can use decimal64 vectorized version, even if the scales are > different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22510) Support decimal64 operations for column operands with different scales
[ https://issues.apache.org/jira/browse/HIVE-22510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22510: Attachment: HIVE-22510.18.patch Status: Patch Available (was: Open) > Support decimal64 operations for column operands with different scales > -- > > Key: HIVE-22510 > URL: https://issues.apache.org/jira/browse/HIVE-22510 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22510.11.patch, HIVE-22510.13.patch, > HIVE-22510.14.patch, HIVE-22510.15.patch, HIVE-22510.16.patch, > HIVE-22510.17.patch, HIVE-22510.18.patch, HIVE-22510.2.patch, > HIVE-22510.3.patch, HIVE-22510.4.patch, HIVE-22510.5.patch, > HIVE-22510.7.patch, HIVE-22510.9.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Right now, if the operands on the decimal64 operations are columns with > different scales, then we do not use the decimal64 vectorized version and > fall back to HiveDecimal vectorized version of the operator. In this Jira, we > will check if we can use decimal64 vectorized version, even if the scales are > different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22609) Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
[
https://issues.apache.org/jira/browse/HIVE-22609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009993#comment-17009993
]
Ashutosh Chauhan commented on HIVE-22609:
-
+1
> Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
> -
>
> Key: HIVE-22609
> URL: https://issues.apache.org/jira/browse/HIVE-22609
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-22609.1.patch, HIVE-22609.2.patch,
> HIVE-22609.3.patch, HIVE-22609.4.patch, HIVE-22609.5.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L1380]
> ACID delta folder contains {{_orc_acid_version}} and {{bucket_0}} files.
> For both these files, parent dir is the same. Number of getFileStatus in such
> cases should be reduced by 1/2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22652) TopNKey push through Group by with Grouping sets
[ https://issues.apache.org/jira/browse/HIVE-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22652: --- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master, thanks [~kkasa]! > TopNKey push through Group by with Grouping sets > > > Key: HIVE-22652 > URL: https://issues.apache.org/jira/browse/HIVE-22652 > Project: Hive > Issue Type: Improvement >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22652.1.patch, HIVE-22652.2.patch, > HIVE-22652.3.patch, HIVE-22652.4.patch, HIVE-22652.5.patch, HIVE-22652.6.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22549) RS deduplication should not merge final aggregation without keys
[
https://issues.apache.org/jira/browse/HIVE-22549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22549:
---
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
> RS deduplication should not merge final aggregation without keys
>
>
> Key: HIVE-22549
> URL: https://issues.apache.org/jira/browse/HIVE-22549
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22549.patch
>
>
> This may lead to performance degradation. For instance, this can happen for
> the following query:
> {code}
> set hive.support.concurrency=true;
> set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
> EXPLAIN
> CREATE TABLE x STORED AS ORC TBLPROPERTIES('transactional'='true') AS
> SELECT * FROM SRC x CLUSTER BY x.key;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20934) ACID: Query based compactor for minor compaction
[
https://issues.apache.org/jira/browse/HIVE-20934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009947#comment-17009947
]
Hive QA commented on HIVE-20934:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
20s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
55s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} ql: The patch generated 0 new + 408 unchanged - 4
fixed = 408 total (was 412) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 138
unchanged - 25 fixed = 139 total (was 163) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20099/dev-support/hive-personality.sh
|
| git revision | master / 57fe5d6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20099/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20099/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Query based compactor for minor compaction
>
>
> Key: HIVE-20934
> URL: https://issues.apache.org/jira/browse/HIVE-20934
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20934.01.patch, HIVE-20934.02.patch,
> HIVE-20934.03.patch, HIVE-20934.04.patch, HIVE-20934.05.patch
>
>
> Follow up of HIVE-20699. This is to enable running minor compactions as a
> HiveQL query
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009925#comment-17009925
]
Hive QA commented on HIVE-22699:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990095/HIVE-22699.1.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17852 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_mask_first_n]
(batchId=88)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20098/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20098/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20098/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990095 - PreCommit-HIVE-Build
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Patch Available (was: Open)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.2.patch, HIVE-22489.3.patch,
> HIVE-22489.3.patch, HIVE-22489.4.patch, HIVE-22489.5.patch,
> HIVE-22489.6.patch, HIVE-22489.7.patch, HIVE-22489.8.patch,
> HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
>
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Open (was: Patch Available)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.2.patch, HIVE-22489.3.patch,
> HIVE-22489.3.patch, HIVE-22489.4.patch, HIVE-22489.5.patch,
> HIVE-22489.6.patch, HIVE-22489.7.patch, HIVE-22489.8.patch,
> HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
>
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Attachment: HIVE-22489.10.patch
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.10.patch, HIVE-22489.2.patch, HIVE-22489.3.patch,
> HIVE-22489.3.patch, HIVE-22489.4.patch, HIVE-22489.5.patch,
> HIVE-22489.6.patch, HIVE-22489.7.patch, HIVE-22489.8.patch,
> HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
>
[jira] [Updated] (HIVE-20972) Enable TestMiniLlapLocalCliDriver cbo_limit and cbo_rp_limit
[ https://issues.apache.org/jira/browse/HIVE-20972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-20972: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you Peter for reviewing! > Enable TestMiniLlapLocalCliDriver cbo_limit and cbo_rp_limit > > > Key: HIVE-20972 > URL: https://issues.apache.org/jira/browse/HIVE-20972 > Project: Hive > Issue Type: Test >Reporter: Vihang Karajgaonkar >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-20972.01.patch, HIVE-20972.02.patch, > HIVE-20972.02.patch, HIVE-20972.03.patch, HIVE-20972.03.patch, > HIVE-20972.03.patch, HIVE-20972.03.patch, HIVE-20972.03.patch, > HIVE-20972.03.patch, HIVE-20972.03.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22635) Disable scheduled query executor for unittests
[ https://issues.apache.org/jira/browse/HIVE-22635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22635: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you Peter for reviewing! > Disable scheduled query executor for unittests > -- > > Key: HIVE-22635 > URL: https://issues.apache.org/jira/browse/HIVE-22635 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22635.01.patch, HIVE-22635.01.patch, > HIVE-22635.02.patch > > > HIVE-21884 missed to set the default to off; so it may sometime interfere > with unit tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009891#comment-17009891
]
Hive QA commented on HIVE-22699:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
2s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20098/dev-support/hive-personality.sh
|
| git revision | master / 47b22ce |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20098/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009876#comment-17009876
]
Hive QA commented on HIVE-22489:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990089/HIVE-22489.10.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17853 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2]
(batchId=90)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20097/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20097/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20097/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990089 - PreCommit-HIVE-Build
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.2.patch, HIVE-22489.3.patch, HIVE-22489.3.patch,
> HIVE-22489.4.patch, HIVE-22489.5.patch, HIVE-22489.6.patch,
> HIVE-22489.7.patch, HIVE-22489.8.patch, HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was
[jira] [Commented] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009850#comment-17009850
]
Hive QA commented on HIVE-22489:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
7s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} accumulo-handler in master has 20 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} contrib in master has 11 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} kudu-handler in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} serde: The patch generated 0 new + 564 unchanged - 2
fixed = 564 total (was 566) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} ql: The patch generated 0 new + 794 unchanged - 1
fixed = 794 total (was 795) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch accumulo-handler passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch contrib passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hbase-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch kudu-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} The patch hive-blobstore passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} serde generated 0 new + 197 unchanged - 1 fixed =
197 total (was 198) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
11s{color} | {color:red} ql generated 1 new + 1531 unchanged - 0 fixed = 1532
total (was 1531) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} accumulo-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} contrib in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} hbase-handler in the patch passed.
[jira] [Commented] (HIVE-22700) Compactions may leak memory when unauthorized
[ https://issues.apache.org/jira/browse/HIVE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009814#comment-17009814 ] Peter Vary commented on HIVE-22700: --- +1 > Compactions may leak memory when unauthorized > - > > Key: HIVE-22700 > URL: https://issues.apache.org/jira/browse/HIVE-22700 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-22700.01.patch > > > Initiator class determines compaction type periodically. Initiator either > runs as hive user or impersonates the owner of the table. When impersonation > is used, Initiator#checkForCompaction may leak memory. If impersonation > (ugi.doAs()) call fails, FileSystem.closeAllForUGI does not run, therefore > does not clean the file system cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22654) ACID: Allow TxnHandler::checkLock to chunk partitions by 1000
[
https://issues.apache.org/jira/browse/HIVE-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009801#comment-17009801
]
Peter Vary commented on HIVE-22654:
---
Thanks [~rajesh.balamohan]! Fair point about the complicated query.
Not sure if you already have a reviewer or not, here are my questions:
* We have a "global" locksForPartitions for every batch. Would it make sense to
calculate this for every batch instead? If I did not miss something we still
might end up with too much element in the in clause if the result of
getLocksBeingHeldForPartitions(locksBeingChecked) is too high. The specific
place where the in clause could grow is this:
{code}
if (!locksForPartitions.isEmpty()) {
query.append(" and (hl_partition is null or hl_partition in(");
first = true;
for (String s : locksForPartitions) {
if (first) {
first = false;
} else {
query.append(", ");
}
query.append('\'');
query.append(s);
query.append('\'');
}
query.append("))");
}
{code}
* Is the checkLock(Connection dbConn, long extLockId, Set
locksForPartitions, Statement stmt, List locksBeingChecked) really
checks the locks? My understanding is that the main goal of this method is to
collect the related locks, and by the way really checks the concurrent writes
too... My feeling is that the name is a little misleading.
* As part of the suggestion above, I would move out the writeset check from the
current private checkLock method. It does not have to be batched, and we might
be better issuing it in one select instead of issuing it in every batch. (also
if we do it before collecting the lock data we might save some execution time
by failing fast)
* Handling/closing the Statement and the ResultSet objects seems a little dodgy
for me. {stmt} is input for the private checkLock method, but closed by us if
there is a write, but not recreated without closure in other cases. We might
want to review this part of the code as well.
* The calculation of {{sawNull}} was "global" for every batch. Now with
batching this could be problematic. We might have to calculate the value before
starting the batching, and use this as an input parameter.
These are the potential issues I have been able to identify, maybe having a
pull request or review board would help us to have better communication
channels.
Thanks for taking this up!
Peter
> ACID: Allow TxnHandler::checkLock to chunk partitions by 1000
> --
>
> Key: HIVE-22654
> URL: https://issues.apache.org/jira/browse/HIVE-22654
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal Vijayaraghavan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-22654.1.patch, HIVE-22654.2.patch
>
>
> The following loop can end up with too many entries within the IN clause
> throwing.
> https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L4428
> {code:java}
> // If any of the partition requests are null, then I need to pull all
> // partition locks for this table.
> sawNull = false;
> strings.clear();
> for (LockInfo info : locksBeingChecked) {
> if (info.partition == null) {
> sawNull = true;
> break;
> } else {
> strings.add(info.partition);
> }
> }
> {code}
> {code}
> 2019-12-17T04:28:57,991 ERROR [pool-8-thread-143]:
> metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(201)) -
> MetaException(message:Unable to update transaction database
> java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in
> a list is 1000
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22700) Compactions may leak memory when unauthorized
[
https://issues.apache.org/jira/browse/HIVE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009787#comment-17009787
]
Hive QA commented on HIVE-22700:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990086/HIVE-22700.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17852 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20096/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20096/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20096/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990086 - PreCommit-HIVE-Build
> Compactions may leak memory when unauthorized
> -
>
> Key: HIVE-22700
> URL: https://issues.apache.org/jira/browse/HIVE-22700
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22700.01.patch
>
>
> Initiator class determines compaction type periodically. Initiator either
> runs as hive user or impersonates the owner of the table. When impersonation
> is used, Initiator#checkForCompaction may leak memory. If impersonation
> (ugi.doAs()) call fails, FileSystem.closeAllForUGI does not run, therefore
> does not clean the file system cache.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-20934) ACID: Query based compactor for minor compaction
[ https://issues.apache.org/jira/browse/HIVE-20934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter updated HIVE-20934: - Attachment: HIVE-20934.05.patch > ACID: Query based compactor for minor compaction > > > Key: HIVE-20934 > URL: https://issues.apache.org/jira/browse/HIVE-20934 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-20934.01.patch, HIVE-20934.02.patch, > HIVE-20934.03.patch, HIVE-20934.04.patch, HIVE-20934.05.patch > > > Follow up of HIVE-20699. This is to enable running minor compactions as a > HiveQL query -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22696) Break up DDLSemanticAnalyzer - extract Table partition analyzers
[ https://issues.apache.org/jira/browse/HIVE-22696?focusedWorklogId=367484=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-367484 ] ASF GitHub Bot logged work on HIVE-22696: - Author: ASF GitHub Bot Created on: 07/Jan/20 14:22 Start Date: 07/Jan/20 14:22 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #874: HIVE-22696 Break up DDLSemanticAnalyzer - extract Table partition analyzers URL: https://github.com/apache/hive/pull/874 DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is to refactor it in order to have everything cut into more handleable classes under the package org.apache.hadoop.hive.ql.exec.ddl: - have a separate class for each analyzers - have a package for each operation, containing an analyzer, a description, and an operation, so the amount of classes under a package is more manageable Step #12: extract the table partition related analyzers from DDLSemanticAnalyzer, and move them under the new package. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 367484) Remaining Estimate: 0h Time Spent: 10m > Break up DDLSemanticAnalyzer - extract Table partition analyzers > > > Key: HIVE-22696 > URL: https://issues.apache.org/jira/browse/HIVE-22696 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22696.01.patch, HIVE-22696.02.patch, > HIVE-22696.03.patch > > Time Spent: 10m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #12: extract the table partition related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22696) Break up DDLSemanticAnalyzer - extract Table partition analyzers
[ https://issues.apache.org/jira/browse/HIVE-22696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22696: -- Labels: pull-request-available refactor-ddl (was: refactor-ddl) > Break up DDLSemanticAnalyzer - extract Table partition analyzers > > > Key: HIVE-22696 > URL: https://issues.apache.org/jira/browse/HIVE-22696 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22696.01.patch, HIVE-22696.02.patch, > HIVE-22696.03.patch > > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #12: extract the table partition related analyzers from > DDLSemanticAnalyzer, and move them under the new package. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22662) java.lang.NegativeArraySizeException when trying to get HashPartition
[
https://issues.apache.org/jira/browse/HIVE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009745#comment-17009745
]
Diego Patricio Moncayo edited comment on HIVE-22662 at 1/7/20 2:04 PM:
---
Sorry for the late answer. I confirm that with the change in the config, the
error does not happen. Although some OOM appeared in other queries.
Thank you [~gopalv]
was (Author: dmonmu):
Sorry for the late answer. I confirm that with the change in the parameter, the
error does not happen. Although some OOM appeared in other queries.
Thank you [~gopalv]
> java.lang.NegativeArraySizeException when trying to get HashPartition
> -
>
> Key: HIVE-22662
> URL: https://issues.apache.org/jira/browse/HIVE-22662
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
> Environment: Using hdp 3.1.4.0-315
>Reporter: Diego Patricio Moncayo
>Priority: Minor
>
> Running a query between a big table and a small one, this error appeared:
> {code:java}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:361)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:266)
> ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Async
> Initialization failed. abortRequested=false
> at
> org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:465)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:399)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:572)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:524)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:342)
> ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NegativeArraySizeException
> at
> org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:103)
> at
> org.apache.hadoop.hive.ql.exec.tez.ObjectCache$1.call(ObjectCache.java:113)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NegativeArraySizeException
> at
> org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:271)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:344)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:413)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:215)
> at
> org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:96)
> ... 5 more
> Caused by: java.lang.NegativeArraySizeException
> at
> org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer.(HybridHashTableContainer.java:362)
> at
> org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer.(HybridHashTableContainer.java:277)
> at
> org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:231)
> ... 9 more
> {code}
> When looking into the stacktrace, this line appeared.
> {code:java}
> hashPartitions = new HashPartition[numPartitions];
> {code}
> Apparently the int variable numPartition overflows
> When this parameter is set hive.auto.convert.join=false the issue disappears
> Similar to this Issue https://issues.apache.org/jira/browse/HIVE-12547 but
> not the same
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22662) java.lang.NegativeArraySizeException when trying to get HashPartition
[
https://issues.apache.org/jira/browse/HIVE-22662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009745#comment-17009745
]
Diego Patricio Moncayo commented on HIVE-22662:
---
Sorry for the late answer. I confirm that with the change in the parameter, the
error does not happen. Although some OOM appeared in other queries.
Thank you [~gopalv]
> java.lang.NegativeArraySizeException when trying to get HashPartition
> -
>
> Key: HIVE-22662
> URL: https://issues.apache.org/jira/browse/HIVE-22662
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0
> Environment: Using hdp 3.1.4.0-315
>Reporter: Diego Patricio Moncayo
>Priority: Minor
>
> Running a query between a big table and a small one, this error appeared:
> {code:java}
> Caused by: java.lang.RuntimeException: Map operator initialization failed
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:361)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:266)
> ... 16 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Async
> Initialization failed. abortRequested=false
> at
> org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:465)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:399)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:572)
> at
> org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:524)
> at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:385)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:342)
> ... 17 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NegativeArraySizeException
> at
> org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:103)
> at
> org.apache.hadoop.hive.ql.exec.tez.ObjectCache$1.call(ObjectCache.java:113)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.NegativeArraySizeException
> at
> org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:271)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTableInternal(MapJoinOperator.java:344)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:413)
> at
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.lambda$initializeOp$0(MapJoinOperator.java:215)
> at
> org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:96)
> ... 5 more
> Caused by: java.lang.NegativeArraySizeException
> at
> org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer.(HybridHashTableContainer.java:362)
> at
> org.apache.hadoop.hive.ql.exec.persistence.HybridHashTableContainer.(HybridHashTableContainer.java:277)
> at
> org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:231)
> ... 9 more
> {code}
> When looking into the stacktrace, this line appeared.
> {code:java}
> hashPartitions = new HashPartition[numPartitions];
> {code}
> Apparently the int variable numPartition overflows
> When this parameter is set hive.auto.convert.join=false the issue disappears
> Similar to this Issue https://issues.apache.org/jira/browse/HIVE-12547 but
> not the same
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22698) Support Statement#closeOnCompletion()
[ https://issues.apache.org/jira/browse/HIVE-22698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Iwao AVE! updated HIVE-22698: - Attachment: HIVE-22698.3.patch > Support Statement#closeOnCompletion() > - > > Key: HIVE-22698 > URL: https://issues.apache.org/jira/browse/HIVE-22698 > Project: Hive > Issue Type: Task > Components: JDBC >Affects Versions: 3.1.2, 2.3.6 >Reporter: Iwao AVE! >Priority: Major > Labels: patch > Attachments: HIVE-22698.2.patch, HIVE-22698.3.patch, HIVE-22698.patch > > > I am a member of [MyBatis|https://github.com/mybatis] team and a user > [reported|https://github.com/mybatis/mybatis-3/issues/1351#issuecomment-523771341] > that Hive does not support java.sql.Statement#closeOnCompletion() yet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009738#comment-17009738
]
Quanlong Huang commented on HIVE-22699:
---
[~Rajkumar Singh] I uploaded the patch and added you as the reviewer. Hope you
can have a look.
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22700) Compactions may leak memory when unauthorized
[
https://issues.apache.org/jira/browse/HIVE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009737#comment-17009737
]
Hive QA commented on HIVE-22700:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 27s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20096/dev-support/hive-personality.sh
|
| git revision | master / 47b22ce |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20096/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Compactions may leak memory when unauthorized
> -
>
> Key: HIVE-22700
> URL: https://issues.apache.org/jira/browse/HIVE-22700
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-22700.01.patch
>
>
> Initiator class determines compaction type periodically. Initiator either
> runs as hive user or impersonates the owner of the table. When impersonation
> is used, Initiator#checkForCompaction may leak memory. If impersonation
> (ugi.doAs()) call fails, FileSystem.closeAllForUGI does not run, therefore
> does not clean the file system cache.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22701) New Compaction for subsequent read's optimisations.
[ https://issues.apache.org/jira/browse/HIVE-22701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009724#comment-17009724 ] Peter Vary edited comment on HIVE-22701 at 1/7/20 1:44 PM: --- CC: [~lpinter], as you might be interested Work is ongoing on HIVE-20934 - ACID: Query based compactor for minor compaction was (Author: pvary): CC: [~lpinter], as you might be interested > New Compaction for subsequent read's optimisations. > --- > > Key: HIVE-22701 > URL: https://issues.apache.org/jira/browse/HIVE-22701 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Aditya Shah >Priority: Major > > Introducing a new Compaction Type say "OPTIMIZE" to have the following > optimizations for better reads: > 1. Sort data > 2. Re-bucket data > 3. z-ordering > 4. removing ROW_IDs > I've attached a [design doc| > https://docs.google.com/document/d/10zWk7FR6I0CMy57Uykbkcox4HZTMQv2sgLoZrHVeLYU/edit?usp=sharing] > with the JIRA. Feel free to comment on the same. > cc: [~t3rmin4t0r] [~pvary] [~lpinter] [~asomani] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22701) New Compaction for subsequent read's optimisations.
[ https://issues.apache.org/jira/browse/HIVE-22701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009724#comment-17009724 ] Peter Vary commented on HIVE-22701: --- CC: [~lpinter], as you might be interested > New Compaction for subsequent read's optimisations. > --- > > Key: HIVE-22701 > URL: https://issues.apache.org/jira/browse/HIVE-22701 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Aditya Shah >Priority: Major > > Introducing a new Compaction Type say "OPTIMIZE" to have the following > optimizations for better reads: > 1. Sort data > 2. Re-bucket data > 3. z-ordering > 4. removing ROW_IDs > I've attached a [design doc| > https://docs.google.com/document/d/10zWk7FR6I0CMy57Uykbkcox4HZTMQv2sgLoZrHVeLYU/edit?usp=sharing] > with the JIRA. Feel free to comment on the same. > cc: [~t3rmin4t0r] [~pvary] [~lpinter] [~asomani] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated HIVE-22699:
--
Attachment: HIVE-22699.1.patch
Status: Patch Available (was: In Progress)
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22699 started by Quanlong Huang.
-
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009721#comment-17009721
]
Quanlong Huang commented on HIVE-22699:
---
Hi [~Rajkumar Singh], the doc I mean is the result of "describe function
extended mask":
{code:java}
masks the given value
Examples:
mask(ccn)
mask(ccn, 'X', 'x', '0')
mask(ccn, 'x', 'x', 'x')
Arguments:
mask(value, upperChar, lowerChar, digitChar, otherChar, numberChar,
dayValue, monthValue, yearValue)
value - value to mask. Supported types: TINYINT, SMALLINT, INT,
BIGINT, STRING, VARCHAR, CHAR, DATE
upperChar - character to replace upper-case characters with. Specify -1
to retain original character. Default value: 'X'
lowerChar - character to replace lower-case characters with. Specify -1
to retain original character. Default value: 'x'
digitChar - character to replace digit characters with. Specify -1 to
retain original character. Default value: 'n'
otherChar - character to replace all other characters with. Specify -1 to
retain original character. Default value: -1
numberChar - character to replace digits in a number with. Valid values:
0-9. Default value: '1'
dayValue - value to replace day field in a date with. Specify -1 to
retain original value. Valid values: 1-31. Default value: 1
monthValue - value to replace month field in a date with. Specify -1 to
retain original value. Valid values: 0-11. Default value: 0
yearValue - value to replace year field in a date with. Specify -1 to
retain original value. Default value: 0 {code}
You are trying the argument in STRING type. If you try any numeric types
(TINYINT, SMALLINT, INT, BIGINT) with value 0, you can find the result is
always 0. It should be masked to 'numberChar' which defaults to 1.
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Priority: Major
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang reassigned HIVE-22699:
-
Assignee: Quanlong Huang
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22609) Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
[
https://issues.apache.org/jira/browse/HIVE-22609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009717#comment-17009717
]
Hive QA commented on HIVE-22609:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990073/HIVE-22609.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17852 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query30]
(batchId=303)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20095/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20095/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20095/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990073 - PreCommit-HIVE-Build
> Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
> -
>
> Key: HIVE-22609
> URL: https://issues.apache.org/jira/browse/HIVE-22609
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-22609.1.patch, HIVE-22609.2.patch,
> HIVE-22609.3.patch, HIVE-22609.4.patch, HIVE-22609.5.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L1380]
> ACID delta folder contains {{_orc_acid_version}} and {{bucket_0}} files.
> For both these files, parent dir is the same. Number of getFileStatus in such
> cases should be reduced by 1/2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22609) Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
[
https://issues.apache.org/jira/browse/HIVE-22609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009671#comment-17009671
]
Hive QA commented on HIVE-22609:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 14s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20095/dev-support/hive-personality.sh
|
| git revision | master / 47b22ce |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20095/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
> -
>
> Key: HIVE-22609
> URL: https://issues.apache.org/jira/browse/HIVE-22609
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-22609.1.patch, HIVE-22609.2.patch,
> HIVE-22609.3.patch, HIVE-22609.4.patch, HIVE-22609.5.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L1380]
> ACID delta folder contains {{_orc_acid_version}} and {{bucket_0}} files.
> For both these files, parent dir is the same. Number of getFileStatus in such
> cases should be reduced by 1/2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22652) TopNKey push through Group by with Grouping sets
[
https://issues.apache.org/jira/browse/HIVE-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009658#comment-17009658
]
Hive QA commented on HIVE-22652:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990071/HIVE-22652.6.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17855 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20094/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20094/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20094/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990071 - PreCommit-HIVE-Build
> TopNKey push through Group by with Grouping sets
>
>
> Key: HIVE-22652
> URL: https://issues.apache.org/jira/browse/HIVE-22652
> Project: Hive
> Issue Type: Improvement
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22652.1.patch, HIVE-22652.2.patch,
> HIVE-22652.3.patch, HIVE-22652.4.patch, HIVE-22652.5.patch, HIVE-22652.6.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Attachment: HIVE-22489.10.patch
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.2.patch, HIVE-22489.3.patch, HIVE-22489.3.patch,
> HIVE-22489.4.patch, HIVE-22489.5.patch, HIVE-22489.6.patch,
> HIVE-22489.7.patch, HIVE-22489.8.patch, HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
> serialization.lib
>
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Patch Available (was: Open)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.2.patch, HIVE-22489.3.patch, HIVE-22489.3.patch,
> HIVE-22489.4.patch, HIVE-22489.5.patch, HIVE-22489.6.patch,
> HIVE-22489.7.patch, HIVE-22489.8.patch, HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
>
[jira] [Updated] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22489:
--
Status: Open (was: Patch Available)
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.10.patch,
> HIVE-22489.2.patch, HIVE-22489.3.patch, HIVE-22489.3.patch,
> HIVE-22489.4.patch, HIVE-22489.5.patch, HIVE-22489.6.patch,
> HIVE-22489.7.patch, HIVE-22489.8.patch, HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 500 Data size: 43500 Basic
> stats: COMPLETE Column stats: COMPLETE
> tag: 0
> auto parallelism: true
> Execution mode: vectorized, llap
> LLAP IO: no inputs
> Path -> Alias:
> A masked pattern was here
> Path -> Partition:
> A masked pattern was here
> Partition
> base file name: src
> input format: org.apache.hadoop.mapred.TextInputFormat
> output format:
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
> properties:
> COLUMN_STATS_ACCURATE
> {"BASIC_STATS":"true","COLUMN_STATS":{"key":"true","value":"true"}}
> bucket_count -1
> bucketing_version 2
> column.name.delimiter ,
> columns key,value
> columns.comments 'default','default'
> columns.types string:string
> A masked pattern was here
> name default.src
> numFiles 1
> numRows 500
> rawDataSize 5312
> serialization.ddl struct src { string key, string value}
> serialization.format 1
>
[jira] [Commented] (HIVE-22652) TopNKey push through Group by with Grouping sets
[
https://issues.apache.org/jira/browse/HIVE-22652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009619#comment-17009619
]
Hive QA commented on HIVE-22652:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20094/dev-support/hive-personality.sh
|
| git revision | master / 47b22ce |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20094/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TopNKey push through Group by with Grouping sets
>
>
> Key: HIVE-22652
> URL: https://issues.apache.org/jira/browse/HIVE-22652
> Project: Hive
> Issue Type: Improvement
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22652.1.patch, HIVE-22652.2.patch,
> HIVE-22652.3.patch, HIVE-22652.4.patch, HIVE-22652.5.patch, HIVE-22652.6.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22661) Compaction fails on non bucketed table with data loaded inpath
[
https://issues.apache.org/jira/browse/HIVE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009613#comment-17009613
]
Peter Vary commented on HIVE-22661:
---
+1
> Compaction fails on non bucketed table with data loaded inpath
> --
>
> Key: HIVE-22661
> URL: https://issues.apache.org/jira/browse/HIVE-22661
> Project: Hive
> Issue Type: Bug
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22661.0.patch, HIVE-22661.1.patch,
> HIVE-22661.2.patch
>
>
> Compaction cannot handle situations where:
> * data was ingested with {{LOAD DATA INPATH}}
> * this ingest method is run multiple times, and
> ** with different number of files getting created in the delta directories
> Therefore, for file/dir structures such as:
> {code:java}
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/02_0
> {code}
> Although the table is not bucketed, bucket is calculated from the (raw)
> files' names. Compaction in the above case will fail on delta1-1 not having
> data for 'bucket' 2.
> Steps to repro using small dataset:
> {code:java}
> set tez.grouping.min-size=8;
> set tez.grouping.max-size=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> create external table comp0 (a string);
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp1 stored as orc as select * from comp0;
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp2 stored as orc as select * from comp0;
> create table comp3 (a string);
> load data inpath '/warehouse/tablespace/external/hive/comp1' into table comp3;
> load data inpath '/warehouse/tablespace/external/hive/comp2' into table
> comp3;{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22700) Compactions may leak memory when unauthorized
[ https://issues.apache.org/jira/browse/HIVE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter updated HIVE-22700: - Attachment: HIVE-22700.01.patch > Compactions may leak memory when unauthorized > - > > Key: HIVE-22700 > URL: https://issues.apache.org/jira/browse/HIVE-22700 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-22700.01.patch > > > Initiator class determines compaction type periodically. Initiator either > runs as hive user or impersonates the owner of the table. When impersonation > is used, Initiator#checkForCompaction may leak memory. If impersonation > (ugi.doAs()) call fails, FileSystem.closeAllForUGI does not run, therefore > does not clean the file system cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22700) Compactions may leak memory when unauthorized
[ https://issues.apache.org/jira/browse/HIVE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter updated HIVE-22700: - Status: Patch Available (was: Open) > Compactions may leak memory when unauthorized > - > > Key: HIVE-22700 > URL: https://issues.apache.org/jira/browse/HIVE-22700 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-22700.01.patch > > > Initiator class determines compaction type periodically. Initiator either > runs as hive user or impersonates the owner of the table. When impersonation > is used, Initiator#checkForCompaction may leak memory. If impersonation > (ugi.doAs()) call fails, FileSystem.closeAllForUGI does not run, therefore > does not clean the file system cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009598#comment-17009598
]
Hive QA commented on HIVE-22489:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990070/HIVE-22489.9.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 17853 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert1_overwrite_partitions]
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_insert2_overwrite_partitions]
(batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_merge_dynamic_partition]
(batchId=66)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[temp_table_partition_multilevels]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[temp_table_llap_partitioned]
(batchId=167)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20093/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20093/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20093/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990070 - PreCommit-HIVE-Build
> Reduce Sink operator should order nulls by parameter
> -
>
> Key: HIVE-22489
> URL: https://issues.apache.org/jira/browse/HIVE-22489
> Project: Hive
> Issue Type: Bug
> Components: Query Planning
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22489.1.patch, HIVE-22489.2.patch,
> HIVE-22489.3.patch, HIVE-22489.3.patch, HIVE-22489.4.patch,
> HIVE-22489.5.patch, HIVE-22489.6.patch, HIVE-22489.7.patch,
> HIVE-22489.8.patch, HIVE-22489.9.patch, HIVE-22489.9.patch
>
>
> When the property hive.default.nulls.last is set to true and no null order is
> explicitly specified in the ORDER BY clause of the query null ordering should
> be NULLS LAST.
> But some of the Reduce Sink operators still orders null first.
> {code}
> SET hive.default.nulls.last=true;
> EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key LIMIT 5;
> {code}
> {code}
> PREHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> PREHOOK: type: QUERY
> PREHOOK: Input: default@src
> A masked pattern was here
> POSTHOOK: query: EXPLAIN EXTENDED
> SELECT src1.key, src2.value FROM src src1 JOIN src src2 ON (src1.key =
> src2.key) ORDER BY src1.key
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@src
> A masked pattern was here
> OPTIMIZED SQL: SELECT `t0`.`key`, `t2`.`value`
> FROM (SELECT `key`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t0`
> INNER JOIN (SELECT `key`, `value`
> FROM `default`.`src`
> WHERE `key` IS NOT NULL) AS `t2` ON `t0`.`key` = `t2`.`key`
> ORDER BY `t0`.`key`
> STAGE DEPENDENCIES:
> Stage-1 is a root stage
> Stage-0 depends on stages: Stage-1
> STAGE PLANS:
> Stage: Stage-1
> Tez
> A masked pattern was here
> Edges:
> Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 4 (SIMPLE_EDGE)
> Reducer 3 <- Reducer 2 (SIMPLE_EDGE)
> A masked pattern was here
> Vertices:
> Map 1
> Map Operator Tree:
> TableScan
> alias: src1
> filterExpr: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> GatherStats: false
> Filter Operator
> isSamplingPred: false
> predicate: key is not null (type: boolean)
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Select Operator
> expressions: key (type: string)
> outputColumnNames: _col0
> Statistics: Num rows: 500 Data size: 43500 Basic stats:
> COMPLETE Column stats: COMPLETE
> Reduce Output Operator
> key expressions: _col0 (type: string)
> null sort order: a
>
[jira] [Assigned] (HIVE-22700) Compactions may leak memory when unauthorized
[ https://issues.apache.org/jira/browse/HIVE-22700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter reassigned HIVE-22700: > Compactions may leak memory when unauthorized > - > > Key: HIVE-22700 > URL: https://issues.apache.org/jira/browse/HIVE-22700 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Major > > Initiator class determines compaction type periodically. Initiator either > runs as hive user or impersonates the owner of the table. When impersonation > is used, Initiator#checkForCompaction may leak memory. If impersonation > (ugi.doAs()) call fails, FileSystem.closeAllForUGI does not run, therefore > does not clean the file system cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22489) Reduce Sink operator should order nulls by parameter
[
https://issues.apache.org/jira/browse/HIVE-22489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009573#comment-17009573
]
Hive QA commented on HIVE-22489:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
57s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
57s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
29s{color} | {color:blue} accumulo-handler in master has 20 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} contrib in master has 11 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} kudu-handler in master has 1 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} serde: The patch generated 0 new + 564 unchanged - 2
fixed = 564 total (was 566) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
50s{color} | {color:red} ql: The patch generated 1 new + 794 unchanged - 1
fixed = 795 total (was 795) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch accumulo-handler passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch contrib passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch hbase-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch kudu-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} The patch hive-blobstore passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} serde generated 0 new + 197 unchanged - 1 fixed =
197 total (was 198) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
55s{color} | {color:red} ql generated 1 new + 1531 unchanged - 0 fixed = 1532
total (was 1531) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s{color} | {color:green} accumulo-handler in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} contrib in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} hbase-handler in the patch passed. {color} |
[jira] [Commented] (HIVE-22661) Compaction fails on non bucketed table with data loaded inpath
[
https://issues.apache.org/jira/browse/HIVE-22661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009540#comment-17009540
]
Laszlo Pinter commented on HIVE-22661:
--
[~szita] Thanks for the patch. +1 (non-binding)
> Compaction fails on non bucketed table with data loaded inpath
> --
>
> Key: HIVE-22661
> URL: https://issues.apache.org/jira/browse/HIVE-22661
> Project: Hive
> Issue Type: Bug
>Reporter: Ádám Szita
>Assignee: Ádám Szita
>Priority: Major
> Attachments: HIVE-22661.0.patch, HIVE-22661.1.patch,
> HIVE-22661.2.patch
>
>
> Compaction cannot handle situations where:
> * data was ingested with {{LOAD DATA INPATH}}
> * this ingest method is run multiple times, and
> ** with different number of files getting created in the delta directories
> Therefore, for file/dir structures such as:
> {code:java}
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_001_001_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/00_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/01_0
> /warehouse/tablespace/managed/hive/comp3/delta_002_002_/02_0
> {code}
> Although the table is not bucketed, bucket is calculated from the (raw)
> files' names. Compaction in the above case will fail on delta1-1 not having
> data for 'bucket' 2.
> Steps to repro using small dataset:
> {code:java}
> set tez.grouping.min-size=8;
> set tez.grouping.max-size=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> set mapreduce.input.fileinputformat.split.minsize=8;
> create external table comp0 (a string);
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp1 stored as orc as select * from comp0;
> insert into comp0 values ("qwertyuiopasdfghjklzxcvbnm");
> create external table comp2 stored as orc as select * from comp0;
> create table comp3 (a string);
> load data inpath '/warehouse/tablespace/external/hive/comp1' into table comp3;
> load data inpath '/warehouse/tablespace/external/hive/comp2' into table
> comp3;{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22696) Break up DDLSemanticAnalyzer - extract Table partition analyzers
[
https://issues.apache.org/jira/browse/HIVE-22696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009524#comment-17009524
]
Hive QA commented on HIVE-22696:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12990069/HIVE-22696.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17852 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20092/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20092/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20092/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12990069 - PreCommit-HIVE-Build
> Break up DDLSemanticAnalyzer - extract Table partition analyzers
>
>
> Key: HIVE-22696
> URL: https://issues.apache.org/jira/browse/HIVE-22696
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
> Attachments: HIVE-22696.01.patch, HIVE-22696.02.patch,
> HIVE-22696.03.patch
>
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #12: extract the table partition related analyzers from
> DDLSemanticAnalyzer, and move them under the new package.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22696) Break up DDLSemanticAnalyzer - extract Table partition analyzers
[
https://issues.apache.org/jira/browse/HIVE-22696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17009460#comment-17009460
]
Hive QA commented on HIVE-22696:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 1531 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} hcatalog/core in master has 36 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
36s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 1 new + 213 unchanged - 16
fixed = 214 total (was 229) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
2s{color} | {color:red} ql generated 1 new + 1530 unchanged - 1 fixed = 1531
total (was 1531) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 26s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Class
org.apache.hadoop.hive.ql.ddl.table.partition.add.AlterTableAddPartitionDesc
defines non-transient non-serializable instance field partitions In
AlterTableAddPartitionDesc.java:instance field partitions In
AlterTableAddPartitionDesc.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20092/dev-support/hive-personality.sh
|
| git revision | master / 47b22ce |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20092/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20092/yetus/new-findbugs-ql.html
|
| modules | C: ql hcatalog/core itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20092/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Break up DDLSemanticAnalyzer - extract Table partition analyzers
>
>
> Key: HIVE-22696
> URL: https://issues.apache.org/jira/browse/HIVE-22696
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-ddl
>
[jira] [Updated] (HIVE-22609) Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
[
https://issues.apache.org/jira/browse/HIVE-22609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated HIVE-22609:
Attachment: HIVE-22609.5.patch
> Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
> -
>
> Key: HIVE-22609
> URL: https://issues.apache.org/jira/browse/HIVE-22609
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-22609.1.patch, HIVE-22609.2.patch,
> HIVE-22609.3.patch, HIVE-22609.4.patch, HIVE-22609.5.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L1380]
> ACID delta folder contains {{_orc_acid_version}} and {{bucket_0}} files.
> For both these files, parent dir is the same. Number of getFileStatus in such
> cases should be reduced by 1/2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-22609) Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
[
https://issues.apache.org/jira/browse/HIVE-22609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan reassigned HIVE-22609:
---
Assignee: Rajesh Balamohan
> Reduce number of FS getFileStatus calls in AcidUtils::getHdfsDirSnapshots
> -
>
> Key: HIVE-22609
> URL: https://issues.apache.org/jira/browse/HIVE-22609
> Project: Hive
> Issue Type: Bug
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Major
> Attachments: HIVE-22609.1.patch, HIVE-22609.2.patch,
> HIVE-22609.3.patch, HIVE-22609.4.patch, HIVE-22609.5.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L1380]
> ACID delta folder contains {{_orc_acid_version}} and {{bucket_0}} files.
> For both these files, parent dir is the same. Number of getFileStatus in such
> cases should be reduced by 1/2.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
