[jira] [Updated] (HIVE-22576) tez.queue.name setting is invalid when no default queue is set on yarn
[
https://issues.apache.org/jira/browse/HIVE-22576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hezhang updated HIVE-22576:
---
Attachment: HIVE-22576.1.patch
> tez.queue.name setting is invalid when no default queue is set on yarn
> ---
>
> Key: HIVE-22576
> URL: https://issues.apache.org/jira/browse/HIVE-22576
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.1.1, 2.3.2
>Reporter: hezhang
>Assignee: hezhang
>Priority: Major
> Fix For: 2.3.2
>
> Attachments: HIVE-22576.1.patch, image-2019-12-04-15-49-03-475.png
>
>
> h2. hive cli started:
> hive> set tez.queue.name=prod;
> hive>
> > select count(*) from test;
> Query ID = hadoop_20191204152859_d9f7fff6-c0a1-4984-9f56-dd958419992b
> Total jobs = 1
> Launching Job 1 out of 1
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask
> err stack
>
> {code:java}
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> org.apache.tez.dag.api.TezException:
> org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit
> application_1575443228569_0002 to YARN : Application
> application_1575443228569_0002 submitted by user hadoop to unknown queue:
> default
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.isOpen(TezSessionState.java:163)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:144)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2183)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1839)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1526)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_191]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_191]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_191]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> ~[hadoop-common-2.7.3.jar:?]
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> ~[hadoop-common-2.7.3.jar:?]
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.tez.dag.api.TezException:
> org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit
> application_1575443228569_0002 to YARN : Application
> application_1575443228569_0002 submitted by user hadoop to unknown queue:
> default
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> ~[?:1.8.0_191]
> at java.util.concurrent.FutureTask.get(FutureTask.java:206)
> ~[?:1.8.0_191]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.isOpen(TezSessionState.java:158)
> ~[hive-exec-2.3.2.jar:2.3.2]
> ... 20 more
> Caused by: org.apache.tez.dag.api.TezException:
> org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit
> application_1575443228569_0002 to YARN : Application
> application_1575443228569_0002 submitted by user hadoop to unknown queue:
> default
> at org.apache.tez.client.TezClient.start(TezClient.java:492)
> ~[tez-api-0.8.5.jar:0.8.5]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:376)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.access$000(TezSessionState.java:87)
>
[jira] [Assigned] (HIVE-22576) tez.queue.name setting is invalid when no default queue is set on yarn
[
https://issues.apache.org/jira/browse/HIVE-22576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hezhang reassigned HIVE-22576:
--
> tez.queue.name setting is invalid when no default queue is set on yarn
> ---
>
> Key: HIVE-22576
> URL: https://issues.apache.org/jira/browse/HIVE-22576
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 2.3.2, 2.1.1
>Reporter: hezhang
>Assignee: hezhang
>Priority: Major
> Fix For: 2.3.2
>
> Attachments: image-2019-12-04-15-49-03-475.png
>
>
> h2. hive cli started:
> hive> set tez.queue.name=prod;
> hive>
> > select count(*) from test;
> Query ID = hadoop_20191204152859_d9f7fff6-c0a1-4984-9f56-dd958419992b
> Total jobs = 1
> Launching Job 1 out of 1
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask
> err stack
>
> {code:java}
> java.lang.RuntimeException: java.util.concurrent.ExecutionException:
> org.apache.tez.dag.api.TezException:
> org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit
> application_1575443228569_0002 to YARN : Application
> application_1575443228569_0002 submitted by user hadoop to unknown queue:
> default
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.isOpen(TezSessionState.java:163)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:144)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2183)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1839)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1526)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1237)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1227)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
> ~[hive-cli-2.3.2.jar:2.3.2]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:1.8.0_191]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_191]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_191]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_191]
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> ~[hadoop-common-2.7.3.jar:?]
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> ~[hadoop-common-2.7.3.jar:?]
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.tez.dag.api.TezException:
> org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit
> application_1575443228569_0002 to YARN : Application
> application_1575443228569_0002 submitted by user hadoop to unknown queue:
> default
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> ~[?:1.8.0_191]
> at java.util.concurrent.FutureTask.get(FutureTask.java:206)
> ~[?:1.8.0_191]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.isOpen(TezSessionState.java:158)
> ~[hive-exec-2.3.2.jar:2.3.2]
> ... 20 more
> Caused by: org.apache.tez.dag.api.TezException:
> org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit
> application_1575443228569_0002 to YARN : Application
> application_1575443228569_0002 submitted by user hadoop to unknown queue:
> default
> at org.apache.tez.client.TezClient.start(TezClient.java:492)
> ~[tez-api-0.8.5.jar:0.8.5]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:376)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
> org.apache.hadoop.hive.ql.exec.tez.TezSessionState.access$000(TezSessionState.java:87)
> ~[hive-exec-2.3.2.jar:2.3.2]
> at
>
[jira] [Updated] (HIVE-16690) Configure Tez cartesian product edge based on LLAP cluster size
[ https://issues.apache.org/jira/browse/HIVE-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] László Bodor updated HIVE-16690: Attachment: HIVE-16690.2.patch > Configure Tez cartesian product edge based on LLAP cluster size > --- > > Key: HIVE-16690 > URL: https://issues.apache.org/jira/browse/HIVE-16690 > Project: Hive > Issue Type: Bug >Reporter: Zhiyuan Yang >Assignee: László Bodor >Priority: Major > Attachments: HIVE-16690.1.patch, HIVE-16690.2.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.2.patch, > HIVE-16690.2.patch, HIVE-16690.2.patch, HIVE-16690.2.patch, > HIVE-16690.addendum.patch > > > In HIVE-14731 we are using default value for target parallelism of fair > cartesian product edge. Ideally this should be set according to cluster size. > In case of LLAP it's pretty easy to get cluster size, i.e., number of > executors. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22553) Expose locks and transactions in sys db
[
https://issues.apache.org/jira/browse/HIVE-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987597#comment-16987597
]
Hive QA commented on HIVE-22553:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
57s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 3m 6s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19732/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| modules | C: metastore ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19732/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Expose locks and transactions in sys db
> ---
>
> Key: HIVE-22553
> URL: https://issues.apache.org/jira/browse/HIVE-22553
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22553.2.patch, HIVE-22553.patch
>
>
> Create new sysdb tables/views to access lock and transaction data.
> This allows to provide admins with live data about ongoing locks and
> transacions. Due to this being in the sys db access to this information can
> be restricted to select privileged users.
> Information about locks and compactions can be joined and accessed at the
> same time.
> Compaction related transactions would also be visible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22355) Beeline should not prompt for hive user and password when authentication is NONE
[
https://issues.apache.org/jira/browse/HIVE-22355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987590#comment-16987590
]
Hive QA commented on HIVE-22355:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987376/HIVE-22355.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17757 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19731/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19731/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19731/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987376 - PreCommit-HIVE-Build
> Beeline should not prompt for hive user and password when authentication is
> NONE
>
>
> Key: HIVE-22355
> URL: https://issues.apache.org/jira/browse/HIVE-22355
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: HIVE-22355.1.patch, HIVE-22355.2.patch,
> HIVE-22355.3.patch
>
>
> Beeline - without adding the jdbc url - prompts for username and password in
> case hive.server2.authentication=NONE, which is possibly pointless and can be
> misleading for users as any input is accepted.
> In addition, Sqoop has dropped hive cli recently in favor of beeline and if
> there is no authentication set in Hive, Sqoop fails to connect as the process
> stops waiting for the user/password input.
> I think it would be nice to check the auth type "NONE" before reading unused
> inputs from the console before this point:
> https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/Commands.java#L1641
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22355) Beeline should not prompt for hive user and password when authentication is NONE
[
https://issues.apache.org/jira/browse/HIVE-22355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987564#comment-16987564
]
Hive QA commented on HIVE-22355:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
3s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
32s{color} | {color:blue} beeline in master has 48 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
18s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 21
unchanged - 0 fixed = 22 total (was 21) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19731/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19731/yetus/diff-checkstyle-itests_hive-unit.txt
|
| modules | C: beeline itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19731/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Beeline should not prompt for hive user and password when authentication is
> NONE
>
>
> Key: HIVE-22355
> URL: https://issues.apache.org/jira/browse/HIVE-22355
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: HIVE-22355.1.patch, HIVE-22355.2.patch,
> HIVE-22355.3.patch
>
>
> Beeline - without adding the jdbc url - prompts for username and password in
> case hive.server2.authentication=NONE, which is possibly pointless and can be
> misleading for users as any input is accepted.
> In addition, Sqoop has dropped hive cli recently in favor of beeline and if
> there is no authentication set in Hive, Sqoop fails to connect as the process
> stops waiting for the user/password input.
> I think it would be nice to check the auth type "NONE" before reading unused
> inputs
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987546#comment-16987546
]
Hive QA commented on HIVE-22066:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987375/HIVE-22066.999.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19730/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19730/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19730/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12987375/HIVE-22066.999.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987375 - PreCommit-HIVE-Build
> Upgrade Apache parent POM to version 21
> ---
>
> Key: HIVE-22066
> URL: https://issues.apache.org/jira/browse/HIVE-22066
> Project: Hive
> Issue Type: Improvement
>Reporter: Jesus Camacho Rodriguez
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch,
> HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch,
> HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch,
> HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch,
> HIVE-22066.patch, HIVE-22066.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987545#comment-16987545
]
Hive QA commented on HIVE-22066:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987375/HIVE-22066.999.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19729/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19729/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19729/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-12-04 05:25:26.671
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-19729/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-12-04 05:25:26.675
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 23db35e HIVE-22526 Extract Compiler from Driver (Miklos Gergely,
reviewed by Zoltan Haindrich)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 23db35e HIVE-22526 Extract Compiler from Driver (Miklos Gergely,
reviewed by Zoltan Haindrich)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-12-04 05:25:27.867
+ rm -rf ../yetus_PreCommit-HIVE-Build-19729
+ mkdir ../yetus_PreCommit-HIVE-Build-19729
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-19729
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-19729/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
fatal: git diff header lacks filename information when removing 0 leading
pathname components (line 1647)
error: patch failed:
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java:130
Falling back to three-way merge...
Applied patch to
'storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java'
cleanly.
Going to apply patch with: git apply -p1
/data/hiveptest/working/scratch/build.patch:2162: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:1636: new blank line at EOF.
+
error: patch failed:
storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java:130
Falling back to three-way merge...
Applied patch to
'storage-api/src/java/org/apache/hadoop/hive/ql/exec/vector/BytesColumnVector.java'
cleanly.
warning: 2 lines add whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
[ERROR] The build could not read 31 projects -> [Help 1]
[ERROR]
[ERROR] The project org.apache.hive:hive-accumulo-handler:4.0.0-SNAPSHOT
(/data/hiveptest/working/apache-github-source-source/accumulo-handler/pom.xml)
has 1 error
[ERROR] Non-resolvable parent POM: Could not find artifact
org.apache.hive:hive:pom:4.0.0-SNAPSHOT and 'parent.relativePath' points at
wrong local POM @ line 19, column 11 -> [Help 2]
[ERROR]
[ERROR] The project org.apache.hive:hive-vector-code-gen:4.0.0-SNAPSHOT
(/data/hiveptest/working/apache-github-source-source/vector-code-gen/pom.xml)
has 1 error
[ERROR] Non-resolvable parent POM: Could not find artifact
org.apache.hive:hive:pom:4.0.0-SNAPSHOT and 'parent.relativePath' points at
wrong local POM @ line 19, column 11 -> [Help 2]
[ERROR]
[ERROR] The project org.apache.hive:hive-beeline:4.0.0-SNAPSHOT
(/data/hiveptest/working/apache-github-source-source/beeline/pom.xml) has 1
[jira] [Commented] (HIVE-22534) ACID: Improve Compactor thread logging
[
https://issues.apache.org/jira/browse/HIVE-22534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987542#comment-16987542
]
Hive QA commented on HIVE-22534:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987373/HIVE-22534.06.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17752 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=257)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19728/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19728/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19728/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987373 - PreCommit-HIVE-Build
> ACID: Improve Compactor thread logging
> --
>
> Key: HIVE-22534
> URL: https://issues.apache.org/jira/browse/HIVE-22534
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22534.01.patch, HIVE-22534.02.patch,
> HIVE-22534.03.patch, HIVE-22534.04.patch, HIVE-22534.05.patch,
> HIVE-22534.06.patch
>
>
> Make sure that it is easy to find issues when one of the compactor thread
> fails.
> Maybe:
> * MDC - with iteration / threadname - so we can easily grep the logs for a
> given run
> * MDC with table/partition data on which the worker is working
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22575) "set hive.default.fileformat = TextFile" ，modifying the configuration parameters does not work
[ https://issues.apache.org/jira/browse/HIVE-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987531#comment-16987531 ] Rajkumar Singh commented on HIVE-22575: --- starting hive-3, by default managed tables are acid and stored as ORC, I believe the property to control default managed storage is hive.default.fileformat.managed. > "set hive.default.fileformat = TextFile" ，modifying the configuration > parameters does not work > -- > > Key: HIVE-22575 > URL: https://issues.apache.org/jira/browse/HIVE-22575 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.1 >Reporter: lv haiyang >Priority: Major > > 0: jdbc:hive2://master:2181,sdw1:2181,sdw2:21> set hive.default.fileformat; > +---+ > |set| > +---+ > | hive.default.fileformat=TextFile | > +---+ > 1 row selected (0.024 seconds) > 0: jdbc:hive2://master:2181,sdw1:2181,sdw2:21> create textfile_table(id > int,desc string); > 0: jdbc:hive2://master:2181,sdw1:2181,sdw2:21> desc formatted textfile_table; > INFO : OK > +---+++ > | col_name| data_type >| comment | > +---+++ > | id| int >|| > | desc | string >|| > | # Storage Information | NULL >| NULL | > | SerDe Library:| org.apache.hadoop.hive.ql.io.orc.OrcSerde >| NULL | > | InputFormat: | > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat| NULL > | > | OutputFormat: | > org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat | NULL > | > | Compressed: | No >| NULL | > | Num Buckets: | -1 >| NULL | > | Bucket Columns: | [] >| NULL | > | Sort Columns: | [] >| NULL | > | Storage Desc Params: | NULL >| NULL | > | | serialization.format >| 1 | > +---+++ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22548) Optimise Utilities.removeTempOrDuplicateFiles when moving files to final location
[
https://issues.apache.org/jira/browse/HIVE-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-22548:
---
Attachment: HIVE-22548.01.patch
> Optimise Utilities.removeTempOrDuplicateFiles when moving files to final
> location
> -
>
> Key: HIVE-22548
> URL: https://issues.apache.org/jira/browse/HIVE-22548
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
> Attachments: HIVE-22548.01.patch
>
>
> {{Utilities.removeTempOrDuplicateFiles}}
> is very slow with cloud storage, as it executes {{listStatus}} twice and also
> runs in single threaded mode.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1629
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22548) Optimise Utilities.removeTempOrDuplicateFiles when moving files to final location
[
https://issues.apache.org/jira/browse/HIVE-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-22548:
---
Status: Patch Available (was: Open)
> Optimise Utilities.removeTempOrDuplicateFiles when moving files to final
> location
> -
>
> Key: HIVE-22548
> URL: https://issues.apache.org/jira/browse/HIVE-22548
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
> Attachments: HIVE-22548.01.patch
>
>
> {{Utilities.removeTempOrDuplicateFiles}}
> is very slow with cloud storage, as it executes {{listStatus}} twice and also
> runs in single threaded mode.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1629
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22548) Optimise Utilities.removeTempOrDuplicateFiles when moving files to final location
[
https://issues.apache.org/jira/browse/HIVE-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-22548:
---
Status: Open (was: Patch Available)
> Optimise Utilities.removeTempOrDuplicateFiles when moving files to final
> location
> -
>
> Key: HIVE-22548
> URL: https://issues.apache.org/jira/browse/HIVE-22548
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
>
> {{Utilities.removeTempOrDuplicateFiles}}
> is very slow with cloud storage, as it executes {{listStatus}} twice and also
> runs in single threaded mode.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1629
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22548) Optimise Utilities.removeTempOrDuplicateFiles when moving files to final location
[
https://issues.apache.org/jira/browse/HIVE-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-22548:
---
Attachment: (was: HIVE-22548.01.patch)
> Optimise Utilities.removeTempOrDuplicateFiles when moving files to final
> location
> -
>
> Key: HIVE-22548
> URL: https://issues.apache.org/jira/browse/HIVE-22548
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
>
> {{Utilities.removeTempOrDuplicateFiles}}
> is very slow with cloud storage, as it executes {{listStatus}} twice and also
> runs in single threaded mode.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1629
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22534) ACID: Improve Compactor thread logging
[
https://issues.apache.org/jira/browse/HIVE-22534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987515#comment-16987515
]
Hive QA commented on HIVE-22534:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
16s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 1 new + 48 unchanged - 4 fixed
= 49 total (was 52) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19728/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19728/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19728/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Improve Compactor thread logging
> --
>
> Key: HIVE-22534
> URL: https://issues.apache.org/jira/browse/HIVE-22534
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Attachments: HIVE-22534.01.patch, HIVE-22534.02.patch,
> HIVE-22534.03.patch, HIVE-22534.04.patch, HIVE-22534.05.patch,
> HIVE-22534.06.patch
>
>
> Make sure that it is easy to find issues when one of the compactor thread
> fails.
> Maybe:
> * MDC - with iteration / threadname - so we can easily grep the logs for a
> given run
> * MDC with table/partition data on which the worker is working
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22575) "set hive.default.fileformat = TextFile" ，modifying the configuration parameters does not work
[ https://issues.apache.org/jira/browse/HIVE-22575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987506#comment-16987506 ] lv haiyang commented on HIVE-22575: --- why the table's storage way still was orc fileformat ? > "set hive.default.fileformat = TextFile" ，modifying the configuration > parameters does not work > -- > > Key: HIVE-22575 > URL: https://issues.apache.org/jira/browse/HIVE-22575 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.1 >Reporter: lv haiyang >Priority: Major > > 0: jdbc:hive2://master:2181,sdw1:2181,sdw2:21> set hive.default.fileformat; > +---+ > |set| > +---+ > | hive.default.fileformat=TextFile | > +---+ > 1 row selected (0.024 seconds) > 0: jdbc:hive2://master:2181,sdw1:2181,sdw2:21> create textfile_table(id > int,desc string); > 0: jdbc:hive2://master:2181,sdw1:2181,sdw2:21> desc formatted textfile_table; > INFO : OK > +---+++ > | col_name| data_type >| comment | > +---+++ > | id| int >|| > | desc | string >|| > | # Storage Information | NULL >| NULL | > | SerDe Library:| org.apache.hadoop.hive.ql.io.orc.OrcSerde >| NULL | > | InputFormat: | > org.apache.hadoop.hive.ql.io.orc.OrcInputFormat| NULL > | > | OutputFormat: | > org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat | NULL > | > | Compressed: | No >| NULL | > | Num Buckets: | -1 >| NULL | > | Bucket Columns: | [] >| NULL | > | Sort Columns: | [] >| NULL | > | Storage Desc Params: | NULL >| NULL | > | | serialization.format >| 1 | > +---+++ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22547) Review txn compactor Package
[
https://issues.apache.org/jira/browse/HIVE-22547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987502#comment-16987502
]
Hive QA commented on HIVE-22547:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987372/HIVE-22547.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17756 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_reducers_power_two]
(batchId=195)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19727/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19727/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19727/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987372 - PreCommit-HIVE-Build
> Review txn compactor Package
>
>
> Key: HIVE-22547
> URL: https://issues.apache.org/jira/browse/HIVE-22547
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22547.1.patch, HIVE-22547.2.patch,
> HIVE-22547.2.patch
>
>
> * Remove log-and-throw anti-pattern
> * Use parameterized logging
> * Add a CompactionException class to improve debug-ability
> * Introduce Java Optional utility
> * Other clean up
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22547) Review txn compactor Package
[
https://issues.apache.org/jira/browse/HIVE-22547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987476#comment-16987476
]
Hive QA commented on HIVE-22547:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
15s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} service in master has 49 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 3 new + 70 unchanged - 10
fixed = 73 total (was 80) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
51s{color} | {color:red} ql generated 3 new + 1531 unchanged - 1 fixed = 1534
total (was 1532) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Unread public/protected field:At CompactorThread.java:[line 66] |
| | Unused public or protected
field:org.apache.hadoop.hive.ql.txn.compactor.CompactorThread.rs In
CompactorThread.java |
| | Null passed for non-null parameter of
RemoteCompactorThread.resolveTable(CompactionInfo) in
org.apache.hadoop.hive.ql.txn.compactor.Worker.run() Method invoked at
Worker.java:of RemoteCompactorThread.resolveTable(CompactionInfo) in
org.apache.hadoop.hive.ql.txn.compactor.Worker.run() Method invoked at
Worker.java:[line 115] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19727/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19727/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19727/yetus/new-findbugs-ql.html
|
| modules | C: ql service U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19727/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Review txn compactor Package
>
>
> Key: HIVE-22547
> URL: https://issues.apache.org/jira/browse/HIVE-22547
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments:
[jira] [Commented] (HIVE-20934) ACID: Query based compactor for minor compaction
[
https://issues.apache.org/jira/browse/HIVE-20934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987443#comment-16987443
]
Hive QA commented on HIVE-20934:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987364/HIVE-20934.01.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 17758 tests
executed
*Failed tests:*
{noformat}
TestCompactorUtil - did not produce a TEST-*.xml file (likely timed out)
(batchId=277)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mm_all] (batchId=75)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testOperationsOnCompletedTxnComponentsForMmTable
(batchId=312)
org.apache.hadoop.hive.ql.TestTxnCommandsForOrcMmTable.testOperationsOnCompletedTxnComponentsForMmTable
(batchId=334)
org.apache.hadoop.hive.ql.TestTxnConcatenate.testConcatenateMM (batchId=322)
org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx.org.apache.hadoop.hive.ql.parse.TestReplicationWithTableMigrationEx
(batchId=268)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.majorCompactAfterAbort
(batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.majorCompactAfterAbortNew
(batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.majorCompactWhileStreaming
(batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.majorCompactWhileStreamingForSplitUpdate
(batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.majorCompactWhileStreamingForSplitUpdateNew
(batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTable (batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTableBucketed
(batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTableOpenWriteId
(batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTableOriginalsOrc
(batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTableOriginalsText
(batchId=250)
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.mmTablePartitioned
(batchId=250)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19726/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19726/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19726/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987364 - PreCommit-HIVE-Build
> ACID: Query based compactor for minor compaction
>
>
> Key: HIVE-20934
> URL: https://issues.apache.org/jira/browse/HIVE-20934
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Vaibhav Gumashta
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-20934.01.patch
>
>
> Follow up of HIVE-20699. This is to enable running minor compactions as a
> HiveQL query
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22548) Optimise Utilities.removeTempOrDuplicateFiles when moving files to final location
[
https://issues.apache.org/jira/browse/HIVE-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987441#comment-16987441
]
mahesh kumar behera commented on HIVE-22548:
[~ste...@apache.org]
The directory listing is required by the caller. Earlier there were two calls
to list status. Now it's merged to one list status. The directory listing done
in removeEmptyDpDirectory is used by removeTempOrDuplicateFiles. The directory
listing is kept in removeEmptyDpDirectory and is called in parallel for
multiple partitions to reduce execution time.
> Optimise Utilities.removeTempOrDuplicateFiles when moving files to final
> location
> -
>
> Key: HIVE-22548
> URL: https://issues.apache.org/jira/browse/HIVE-22548
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
> Attachments: HIVE-22548.01.patch
>
>
> {{Utilities.removeTempOrDuplicateFiles}}
> is very slow with cloud storage, as it executes {{listStatus}} twice and also
> runs in single threaded mode.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1629
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Draese updated HIVE-22572: - Attachment: HIVE-22572.patch Status: Patch Available (was: In Progress) > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22572.patch > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Draese updated HIVE-22572: - Attachment: (was: HIVE-22572.patch) > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22572.patch > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-21283) Create Synonym mid for substr, position for locate
[ https://issues.apache.org/jira/browse/HIVE-21283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Andrunas reassigned HIVE-21283: Assignee: (was: Simon poortman) > Create Synonym mid for substr, position for locate > > > Key: HIVE-21283 > URL: https://issues.apache.org/jira/browse/HIVE-21283 > Project: Hive > Issue Type: New Feature > Components: UDF >Reporter: Mani M >Priority: Minor > Labels: UDF, pull-request-available, todoc4.0 > Fix For: 4.0.0 > > Attachments: HIVE.21283.03.PATCH, HIVE.21283.04.PATCH, > HIVE.21283.05.PATCH, HIVE.21283.06.PATCH, HIVE.21283.07.PATCH, > HIVE.21283.08.PATCH, HIVE.21283.09.PATCH, HIVE.21283.10.PATCH, > HIVE.21283.2.PATCH, HIVE.21283.PATCH, image-2019-03-16-21-31-15-541.png, > image-2019-03-16-21-33-18-898.png > > Time Spent: 3h 10m > Remaining Estimate: 0h > > Create new synonym for the existing function > > Mid for substr > postiion for locate -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-21367) Hive returns an incorrect result when using a simple select query
[
https://issues.apache.org/jira/browse/HIVE-21367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Andrunas reassigned HIVE-21367:
Assignee: (was: Simon poortman)
> Hive returns an incorrect result when using a simple select query
> -
>
> Key: HIVE-21367
> URL: https://issues.apache.org/jira/browse/HIVE-21367
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2, JDBC, SQL
>Affects Versions: 3.1.0
> Environment: - HDP 3.1
> - Hive 3.1.0
> - Spark 2.3.2
> - Sqoop 1.4.7
>Reporter: LEMBARKI Mohamed Amine
>Priority: Blocker
> Attachments: mapred_input_dir_recursive.png
>
>
> Hive returns an incorrect result when using a simple select query with a
> where clause
> While with an aggregation it returns a correct result
> The problem arises for tables created by Spark or Sqoop
> Also when we use spark-shell with HiveWarehouseConnector it returns a correct
> result
>
> Workflow:
> - Loading data with sqoop to hive
> - Data processing with spark using HiveWarehouseConnector and Storage to
> Hive
>
> below the error log :
>
> */-*
> *1 - Executing Query : select code from db1.tbl1 where code = '123'*
> */-*
> {code:java}
> [data@data1 ~]$ hive -e "select code from db1.tbl1 where code = '123'"
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Connecting to
> jdbc:hive2://data2:2181,data1:2181/default;password=data;serviceDiscoveryMode=zooKeeper;user=data;zooKeeperNamespace=hiveserver2
> 19/03/01 10:31:36 [main]: INFO jdbc.HiveConnection: Connected to data2:1
> Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
> Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> INFO : Compiling
> command(queryId=hive_20190301103129_d48e71f6-a8dd-490e-a574-04d8d4f893e2):
> select code from db1.tbl1 where code = '123'
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:code,
> type:string, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20190301103129_d48e71f6-a8dd-490e-a574-04d8d4f893e2);
> Time taken: 0.142 seconds
> INFO : Executing
> command(queryId=hive_20190301103129_d48e71f6-a8dd-490e-a574-04d8d4f893e2):
> select code from db1.tbl1 where code = '123'
> INFO : Completed executing
> command(queryId=hive_20190301103129_d48e71f6-a8dd-490e-a574-04d8d4f893e2);
> Time taken: 0.003 seconds
> INFO : OK
> +--+
> | code |
> +--+
> +--+
> No rows selected (4,307 seconds)
> Beeline version 3.1.0.3.1.0.0-78 by Apache Hive
> Closing: 0:
> jdbc:hive2://data2:2181,data1:2181/default;password=data;serviceDiscoveryMode=zooKeeper;user=data;zooKeeperNamespace=hiveserver2
> {code}
> */-*
> *2 - Executing Query using count :*
> *select count(code) from db1.tbl1 where code = '123'*
> */-*
> {code:java}
> [data@data1 ~]$ hive -e "select count(code) from db1.tbl1 where code = '123'"
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> Connecting to
> jdbc:hive2://data2:2181,data1:2181/default;password=data;serviceDiscoveryMode=zooKeeper;user=data;zooKeeperNamespace=hiveserver2
> 19/03/01 10:31:56 [main]: INFO jdbc.HiveConnection: Connected to data2:1
> Connected to: Apache Hive (version 3.1.0.3.1.0.0-78)
> Driver: Hive JDBC (version 3.1.0.3.1.0.0-78)
> Transaction isolation: TRANSACTION_REPEATABLE_READ
> INFO : Compiling
> command(queryId=hive_20190301103149_90aa338b-b99b-4f1c-b7e5-6b285f64cb3e):
> select count(code) from db1.tbl1 where code = '123'
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
[jira] [Updated] (HIVE-22573) HMS filter metadata using Ranger plugin
[ https://issues.apache.org/jira/browse/HIVE-22573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam An updated HIVE-22573: -- Attachment: HIVE-22573.patch Status: Patch Available (was: In Progress) ramesh's HMS ranger plugin code > HMS filter metadata using Ranger plugin > --- > > Key: HIVE-22573 > URL: https://issues.apache.org/jira/browse/HIVE-22573 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Sam An >Assignee: Sam An >Priority: Major > Attachments: HIVE-22573.patch > > > HMS needs to filter out rows based on Ranger plugin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22508) KeyWrapperComparator throws exception
[
https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987369#comment-16987369
]
Hive QA commented on HIVE-22508:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987363/HIVE-22508.1.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17773 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.service.server.TestHS2HttpServerPamConfiguration.testPamCorrectConfiguration
(batchId=241)
org.apache.hive.service.server.TestHS2HttpServerPamConfiguration.testPamServicesAreNotConfigured
(batchId=241)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19725/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19725/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19725/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987363 - PreCommit-HIVE-Build
> KeyWrapperComparator throws exception
> -
>
> Key: HIVE-22508
> URL: https://issues.apache.org/jira/browse/HIVE-22508
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch
>
>
> TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a
> copied key should be compared.
> The current implementation uses the standard object inspectors for all
> KeyWrapper instances. However when comparing untouched KeyWrappers the key
> object inspector should be used which can be extracted form
> Operator.inputObjectInspectors during initialization of the key's
> ExprNodeEvaluator.
> This can cause a ClassCastException when the comparator is used collections
> like TreeSet.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Draese updated HIVE-22572: - Status: In Progress (was: Patch Available) > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22572.patch > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22508) KeyWrapperComparator throws exception
[
https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987348#comment-16987348
]
Hive QA commented on HIVE-22508:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
59s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
23s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
20s{color} | {color:red} serde: The patch generated 18 new + 261 unchanged - 0
fixed = 279 total (was 261) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
46s{color} | {color:red} ql: The patch generated 1 new + 11 unchanged - 1 fixed
= 12 total (was 12) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
53s{color} | {color:green} serde in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
49s{color} | {color:green} ql generated 0 new + 1531 unchanged - 1 fixed = 1531
total (was 1532) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 32m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19725/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19725/yetus/diff-checkstyle-serde.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19725/yetus/diff-checkstyle-ql.txt
|
| modules | C: serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19725/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> KeyWrapperComparator throws exception
> -
>
> Key: HIVE-22508
> URL: https://issues.apache.org/jira/browse/HIVE-22508
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch
>
>
> TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a
> copied key should be compared.
> The current
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987340#comment-16987340 ] Jason Dere commented on HIVE-20801: --- Is anything going on with this Jira? I am +1 on this one, especially considering that on Apache Hive there is still the ability to have non-transactional managed tables. > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal Vijayaraghavan >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[ https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987326#comment-16987326 ] Slim Bouguerra commented on HIVE-22499: --- +1 > LLAP: Add an EncodedReaderOptions to extend ORC impl for options > > > Key: HIVE-22499 > URL: https://issues.apache.org/jira/browse/HIVE-22499 > Project: Hive > Issue Type: Bug > Components: llap, ORC >Reporter: Gopal Vijayaraghavan >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch, > HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch, > HIVE-22499.6.patch, HIVE-22499.7.patch, HIVE-22499.WIP.patch, HIVE-22499.patch > > Time Spent: 10m > Remaining Estimate: 0h > > ORC-570 is an ABI change to the way getFileSystem() by adding an another > exception to the implementation. > To accept and use that change requires waiting for an ORC release, while this > patch serves the same purpose though falls back for a retry for > FileSystem.get() in case the supplier fails at runtime. > Also as a side-note, the FS.get() call is always used in the cases where the > file is not being read from a cache such as EncodedOrcFile (so the upstream > API change might be overkill). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (HIVE-22573) HMS filter metadata using Ranger plugin
[ https://issues.apache.org/jira/browse/HIVE-22573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22573 started by Sam An. - > HMS filter metadata using Ranger plugin > --- > > Key: HIVE-22573 > URL: https://issues.apache.org/jira/browse/HIVE-22573 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Sam An >Assignee: Sam An >Priority: Major > > HMS needs to filter out rows based on Ranger plugin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22573) HMS filter metadata using Ranger plugin
[ https://issues.apache.org/jira/browse/HIVE-22573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam An reassigned HIVE-22573: - > HMS filter metadata using Ranger plugin > --- > > Key: HIVE-22573 > URL: https://issues.apache.org/jira/browse/HIVE-22573 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Affects Versions: 4.0.0 >Reporter: Sam An >Assignee: Sam An >Priority: Major > > HMS needs to filter out rows based on Ranger plugin. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20972) Enable TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
[
https://issues.apache.org/jira/browse/HIVE-20972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987320#comment-16987320
]
Hive QA commented on HIVE-20972:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987362/HIVE-20972.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17757 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
(batchId=188)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[cbo_limit]
(batchId=155)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19724/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19724/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19724/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987362 - PreCommit-HIVE-Build
> Enable TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
> --
>
> Key: HIVE-20972
> URL: https://issues.apache.org/jira/browse/HIVE-20972
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20972.01.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Draese updated HIVE-22572: - Attachment: HIVE-22572.patch Status: Patch Available (was: Open) > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22572.patch > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22572) NullPointerException when using dynamic semijoin reduction
[ https://issues.apache.org/jira/browse/HIVE-22572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oliver Draese reassigned HIVE-22572: > NullPointerException when using dynamic semijoin reduction > -- > > Key: HIVE-22572 > URL: https://issues.apache.org/jira/browse/HIVE-22572 > Project: Hive > Issue Type: Bug > Components: Query Planning >Reporter: Oliver Draese >Assignee: Oliver Draese >Priority: Major > Fix For: 4.0.0 > > > From HS2 logs > Caused by: java.lang.NullPointerException > at > org.apache.hadoop.hive.ql.parse.TezCompiler.removeSemijoinOptimizationByBenefit(TezCompiler.java:1541) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.semijoinRemovalBasedTransformations(TezCompiler.java:471) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TezCompiler.optimizeOperatorPlan(TezCompiler.java:182) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.TaskCompiler.compile(TaskCompiler.java:148) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12487) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at > org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869) > ~[hive-exec-3.1.0.3.1.0.142-1.jar:3.1.0.3.1.0.142-1] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20972) Enable TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
[
https://issues.apache.org/jira/browse/HIVE-20972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987287#comment-16987287
]
Hive QA commented on HIVE-20972:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
25s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
56s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
28s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 34m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19724/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql itests/util U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19724/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Enable TestMiniLlapLocalCliDriver.testCliDriver[cbo_limit]
> --
>
> Key: HIVE-20972
> URL: https://issues.apache.org/jira/browse/HIVE-20972
> Project: Hive
> Issue Type: Test
>Reporter: Vihang Karajgaonkar
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-20972.01.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22408) The fix for CALCITE-2991 creates wrong results on edge case
[
https://issues.apache.org/jira/browse/HIVE-22408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987257#comment-16987257
]
Hive QA commented on HIVE-22408:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987360/HIVE-22408.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17756 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query44]
(batchId=303)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19723/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19723/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19723/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987360 - PreCommit-HIVE-Build
> The fix for CALCITE-2991 creates wrong results on edge case
> ---
>
> Key: HIVE-22408
> URL: https://issues.apache.org/jira/browse/HIVE-22408
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22408.1.patch, HIVE-22408.2.patch,
> HIVE-22408.3.patch, HIVE-22408.4.patch, HIVE-22408.5.patch
>
>
> The Calcite upgrade from 1.19 to 1.21 includes a fix for CALCITE-2991. This
> patch causes a failure in subquery_ALL.q
> In order to proceed with the upgrade, the patch was bypassed by creating the
> HiveRelMdMaxRowCount class which overrides the behavior in Calcite.
> In order to recreate the regression, remove this file and the references to
> the class in HiveDefaultRelMetadataProvider.
> This Jira is being created before the HIVE-22274 PR has been pushed, so
> please wait until that is pushed before working on this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22568) Process compaction candidates in parallel by the Initiator
[ https://issues.apache.org/jira/browse/HIVE-22568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987241#comment-16987241 ] Peter Vary commented on HIVE-22568: --- CC: [~Rajkumar Singh] - this is back on the plate again. We decided to move forward with this parallelization with a parameter low enough not to kill the NameNode and the HMS > Process compaction candidates in parallel by the Initiator > -- > > Key: HIVE-22568 > URL: https://issues.apache.org/jira/browse/HIVE-22568 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22568.1.patch > > > `checkForCompaction` includes many file metadata checks and may be expensive. > Therefore, make sense using a thread pool here and running > `checkForCompactions` in parallel. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22408) The fix for CALCITE-2991 creates wrong results on edge case
[
https://issues.apache.org/jira/browse/HIVE-22408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987232#comment-16987232
]
Hive QA commented on HIVE-22408:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
25s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} ql: The patch generated 0 new + 2 unchanged - 22
fixed = 2 total (was 24) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19723/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19723/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> The fix for CALCITE-2991 creates wrong results on edge case
> ---
>
> Key: HIVE-22408
> URL: https://issues.apache.org/jira/browse/HIVE-22408
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22408.1.patch, HIVE-22408.2.patch,
> HIVE-22408.3.patch, HIVE-22408.4.patch, HIVE-22408.5.patch
>
>
> The Calcite upgrade from 1.19 to 1.21 includes a fix for CALCITE-2991. This
> patch causes a failure in subquery_ALL.q
> In order to proceed with the upgrade, the patch was bypassed by creating the
> HiveRelMdMaxRowCount class which overrides the behavior in Calcite.
> In order to recreate the regression, remove this file and the references to
> the class in HiveDefaultRelMetadataProvider.
> This Jira is being created before the HIVE-22274 PR has been pushed, so
> please wait until that is pushed before working on this.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22555) Upgrade ORC version to 1.5.8
[ https://issues.apache.org/jira/browse/HIVE-22555?focusedWorklogId=352920=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-352920 ] ASF GitHub Bot logged work on HIVE-22555: - Author: ASF GitHub Bot Created on: 03/Dec/19 19:35 Start Date: 03/Dec/19 19:35 Worklog Time Spent: 10m Work Description: mustafaiman commented on pull request #858: HIVE-22555: Upgrade ORC dependency to 1.5.8 URL: https://github.com/apache/hive/pull/858 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 352920) Time Spent: 20m (was: 10m) > Upgrade ORC version to 1.5.8 > > > Key: HIVE-22555 > URL: https://issues.apache.org/jira/browse/HIVE-22555 > Project: Hive > Issue Type: Bug > Components: ORC >Affects Versions: 4.0.0 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22555.2.patch, HIVE-22555.2.patch, > HIVE-22555.3.patch, HIVE-22555.4.patch, HIVE-22555.patch, HIVE-22555.patch > > Time Spent: 20m > Remaining Estimate: 0h > > Hive currently depends on ORC 1.5.6. We need 1.5.8 upgrade for > https://issues.apache.org/jira/browse/HIVE-22499 > ORC-1.5.7 includes https://issues.apache.org/jira/browse/ORC-361 . It causes > some tests overriding MemoryManager to fail. These need to be addressed while > upgrading. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22537) getAcidState() not saving directory snapshot causes multiple calls to S3 api
[ https://issues.apache.org/jira/browse/HIVE-22537?focusedWorklogId=352921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-352921 ] ASF GitHub Bot logged work on HIVE-22537: - Author: ASF GitHub Bot Created on: 03/Dec/19 19:35 Start Date: 03/Dec/19 19:35 Worklog Time Spent: 10m Work Description: mustafaiman commented on pull request #854: HIVE-22537: getAcidState() not saving directory causes multiple files… URL: https://github.com/apache/hive/pull/854 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 352921) Time Spent: 50m (was: 40m) > getAcidState() not saving directory snapshot causes multiple calls to S3 api > > > Key: HIVE-22537 > URL: https://issues.apache.org/jira/browse/HIVE-22537 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Mustafa Iman >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22537.2.patch, HIVE-22537.3.patch, > HIVE-22537.4.patch, HIVE-22537.4.patch, HIVE-22537.patch > > Time Spent: 50m > Remaining Estimate: 0h > > Fix for HIVE-21225 is not enabled in query coordinator codepath. The last > argument (generateDirSnapshots) for getAcidState() is set to false when > invoked by callInternal(). Also, snapshot is not used for file exists calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21266) Don't run cleaner if compaction is skipped (issue with single delta file)
[
https://issues.apache.org/jira/browse/HIVE-21266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987209#comment-16987209
]
Hive QA commented on HIVE-21266:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987359/HIVE-21266.04.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17758 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19722/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19722/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19722/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987359 - PreCommit-HIVE-Build
> Don't run cleaner if compaction is skipped (issue with single delta file)
> -
>
> Key: HIVE-21266
> URL: https://issues.apache.org/jira/browse/HIVE-21266
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21266.01.patch, HIVE-21266.02.patch,
> HIVE-21266.02.patch, HIVE-21266.03.patch, HIVE-21266.04.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java#L353-L357]
>
> {noformat}
> if ((deltaCount + (dir.getBaseDirectory() == null ? 0 : 1)) + origCount <= 1)
> {
> LOG.debug("Not compacting {}; current base is {} and there are {}
> deltas and {} originals", sd.getLocation(), dir
> .getBaseDirectory(), deltaCount, origCount);
> return;
> }
> {noformat}
> Is problematic.
> Suppose you have 1 delta file from streaming ingest: {{delta_11_20}} where
> {{txnid:13}} was aborted. The code above will not rewrite the delta (which
> drops anything that belongs to the aborted txn) and transition the compaction
> to "ready_for_cleaning" state which will drop the metadata about the aborted
> txn in {{markCleaned()}}. Now aborted data will come back as committed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[ https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HIVE-22539: Attachment: HIVE-22539.2.patch > HiveServer2 SPNEGO authentication should skip if authorization header is empty > -- > > Key: HIVE-22539 > URL: https://issues.apache.org/jira/browse/HIVE-22539 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Minor > Attachments: HIVE-22539.1.patch, HIVE-22539.2.patch > > > Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos > before checking header. This can be checked up front to avoid doing any > Kerberos related work if the header is empty. This is helpful in a lot of > cases since typically the first request is empty with the client waiting for > a 401 before returning the Authorization header. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[ https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HIVE-22539: Status: Patch Available (was: Open) > HiveServer2 SPNEGO authentication should skip if authorization header is empty > -- > > Key: HIVE-22539 > URL: https://issues.apache.org/jira/browse/HIVE-22539 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Minor > Attachments: HIVE-22539.1.patch, HIVE-22539.2.patch > > > Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos > before checking header. This can be checked up front to avoid doing any > Kerberos related work if the header is empty. This is helpful in a lot of > cases since typically the first request is empty with the client waiting for > a 401 before returning the Authorization header. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22539) HiveServer2 SPNEGO authentication should skip if authorization header is empty
[ https://issues.apache.org/jira/browse/HIVE-22539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Risden updated HIVE-22539: Status: Open (was: Patch Available) > HiveServer2 SPNEGO authentication should skip if authorization header is empty > -- > > Key: HIVE-22539 > URL: https://issues.apache.org/jira/browse/HIVE-22539 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Reporter: Kevin Risden >Assignee: Kevin Risden >Priority: Minor > Attachments: HIVE-22539.1.patch, HIVE-22539.2.patch > > > Currently HiveServer2 SPNEGO authentication waits until setting up Kerberos > before checking header. This can be checked up front to avoid doing any > Kerberos related work if the header is empty. This is helpful in a lot of > cases since typically the first request is empty with the client waiting for > a 401 before returning the Authorization header. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22571) Review of ExprNodeFieldDesc Class
[ https://issues.apache.org/jira/browse/HIVE-22571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22571: -- Attachment: HIVE-22571.1.patch > Review of ExprNodeFieldDesc Class > - > > Key: HIVE-22571 > URL: https://issues.apache.org/jira/browse/HIVE-22571 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22571.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22571) Review of ExprNodeFieldDesc Class
[ https://issues.apache.org/jira/browse/HIVE-22571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22571: -- Status: Patch Available (was: Open) > Review of ExprNodeFieldDesc Class > - > > Key: HIVE-22571 > URL: https://issues.apache.org/jira/browse/HIVE-22571 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22571.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22571) Review of ExprNodeFieldDesc Class
[ https://issues.apache.org/jira/browse/HIVE-22571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-22571: - > Review of ExprNodeFieldDesc Class > - > > Key: HIVE-22571 > URL: https://issues.apache.org/jira/browse/HIVE-22571 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22570) Review of ExprNodeDesc.java
[ https://issues.apache.org/jira/browse/HIVE-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22570: -- Status: Patch Available (was: Open) > Review of ExprNodeDesc.java > --- > > Key: HIVE-22570 > URL: https://issues.apache.org/jira/browse/HIVE-22570 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22570.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22570) Review of ExprNodeDesc.java
[ https://issues.apache.org/jira/browse/HIVE-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22570: -- Attachment: HIVE-22570.1.patch > Review of ExprNodeDesc.java > --- > > Key: HIVE-22570 > URL: https://issues.apache.org/jira/browse/HIVE-22570 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22570.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22570) Review of ExprNodeDesc.java
[ https://issues.apache.org/jira/browse/HIVE-22570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-22570: - > Review of ExprNodeDesc.java > --- > > Key: HIVE-22570 > URL: https://issues.apache.org/jira/browse/HIVE-22570 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE-22570.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21266) Don't run cleaner if compaction is skipped (issue with single delta file)
[
https://issues.apache.org/jira/browse/HIVE-21266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987159#comment-16987159
]
Hive QA commented on HIVE-21266:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
20s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
49s{color} | {color:red} ql: The patch generated 1 new + 758 unchanged - 2
fixed = 759 total (was 760) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19722/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19722/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19722/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Don't run cleaner if compaction is skipped (issue with single delta file)
> -
>
> Key: HIVE-21266
> URL: https://issues.apache.org/jira/browse/HIVE-21266
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21266.01.patch, HIVE-21266.02.patch,
> HIVE-21266.02.patch, HIVE-21266.03.patch, HIVE-21266.04.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java#L353-L357]
>
> {noformat}
> if ((deltaCount + (dir.getBaseDirectory() == null ? 0 : 1)) + origCount <= 1)
> {
> LOG.debug("Not compacting {}; current base is {} and there are {}
> deltas and {} originals", sd.getLocation(), dir
> .getBaseDirectory(), deltaCount, origCount);
> return;
> }
> {noformat}
> Is problematic.
> Suppose you have 1 delta file from streaming ingest: {{delta_11_20}} where
> {{txnid:13}} was aborted. The code above will not rewrite the delta (which
> drops anything that belongs to the aborted txn) and transition the compaction
> to "ready_for_cleaning" state which will drop the metadata about the aborted
> txn in {{markCleaned()}}. Now aborted data will come back as committed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22524) CommandProcessorException should utilize standard Exception fields
[ https://issues.apache.org/jira/browse/HIVE-22524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22524: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you Miklos for reviewing the changes! > CommandProcessorException should utilize standard Exception fields > -- > > Key: HIVE-22524 > URL: https://issues.apache.org/jira/browse/HIVE-22524 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22524.01.patch > > > CommandProcessorException right now has: > * getCause() inherited from Exception > * getException() local implementation > * getMessage() inherited from Exception > * getErrorMessage() local implementation -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22521) Both Driver and SessionState has a userName
[ https://issues.apache.org/jira/browse/HIVE-22521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22521: Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) pushed to master. Thank you Laszlo for reviewing the changes! > Both Driver and SessionState has a userName > --- > > Key: HIVE-22521 > URL: https://issues.apache.org/jira/browse/HIVE-22521 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22521.01.patch, HIVE-22521.01.patch, > HIVE-22521.01.patch, HIVE-22521.01.patch, HIVE-22521.01.patch, > HIVE-22521.01.patch > > > This caused some confusing behaviour to me...especially when the 2 values > were different. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22569) PartitionPruner use Collections Class
[
https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22569:
--
Description:
Make class {{PrunedPartitionList.java}} more immutable. It kinda is already.
Immutable classes are easier to reason about.
Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
instantiation at runtime.
was:
Make class {{PrunedPartitionList.java}} more immutable. It kinda of is
already. Immutable classes are easier to reason about.
Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
instantiation at runtime.
> PartitionPruner use Collections Class
> -
>
> Key: HIVE-22569
> URL: https://issues.apache.org/jira/browse/HIVE-22569
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE.22569.1.patch
>
>
> Make class {{PrunedPartitionList.java}} more immutable. It kinda is already.
> Immutable classes are easier to reason about.
> Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
> instantiation at runtime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22569) PartitionPruner use Collections Class
[
https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-22569:
--
Description:
Make class {{PrunedPartitionList.java}} more immutable. It kinda of is
already. Immutable classes are easier to reason about.
Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
instantiation at runtime.
> PartitionPruner use Collections Class
> -
>
> Key: HIVE-22569
> URL: https://issues.apache.org/jira/browse/HIVE-22569
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE.22569.1.patch
>
>
> Make class {{PrunedPartitionList.java}} more immutable. It kinda of is
> already. Immutable classes are easier to reason about.
> Use {{Collections.emptyXXX}} in {{PartitionPruner}} to save an object
> instantiation at runtime.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22280) Q tests for partitioned temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987124#comment-16987124
]
Hive QA commented on HIVE-22280:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987166/HIVE-22280.08.patch
{color:green}SUCCESS:{color} +1 due to 66 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17766 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.org.apache.hadoop.hive.ql.parse.TestReplicationScenarios
(batchId=266)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19721/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19721/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19721/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987166 - PreCommit-HIVE-Build
> Q tests for partitioned temporary tables
>
>
> Key: HIVE-22280
> URL: https://issues.apache.org/jira/browse/HIVE-22280
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22280.01.patch, HIVE-22280.02.patch,
> HIVE-22280.03.patch, HIVE-22280.04.patch, HIVE-22280.05.patch,
> HIVE-22280.06.patch, HIVE-22280.07.patch, HIVE-22280.08.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22569) PartitionPruner use Collections Class
[ https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22569: -- Status: Patch Available (was: Open) > PartitionPruner use Collections Class > - > > Key: HIVE-22569 > URL: https://issues.apache.org/jira/browse/HIVE-22569 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE.22569.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22569) PartitionPruner use Collections Class
[ https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22569: -- Attachment: HIVE.22569.1.patch > PartitionPruner use Collections Class > - > > Key: HIVE-22569 > URL: https://issues.apache.org/jira/browse/HIVE-22569 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Attachments: HIVE.22569.1.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-22569) PartitionPruner use Collections Class
[ https://issues.apache.org/jira/browse/HIVE-22569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor reassigned HIVE-22569: - > PartitionPruner use Collections Class > - > > Key: HIVE-22569 > URL: https://issues.apache.org/jira/browse/HIVE-22569 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[ https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22499: Status: Open (was: Patch Available) > LLAP: Add an EncodedReaderOptions to extend ORC impl for options > > > Key: HIVE-22499 > URL: https://issues.apache.org/jira/browse/HIVE-22499 > Project: Hive > Issue Type: Bug > Components: llap, ORC >Reporter: Gopal Vijayaraghavan >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch, > HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch, > HIVE-22499.6.patch, HIVE-22499.7.patch, HIVE-22499.WIP.patch, HIVE-22499.patch > > Time Spent: 10m > Remaining Estimate: 0h > > ORC-570 is an ABI change to the way getFileSystem() by adding an another > exception to the implementation. > To accept and use that change requires waiting for an ORC release, while this > patch serves the same purpose though falls back for a retry for > FileSystem.get() in case the supplier fails at runtime. > Also as a side-note, the FS.get() call is always used in the cases where the > file is not being read from a cache such as EncodedOrcFile (so the upstream > API change might be overkill). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[ https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-22499: Attachment: HIVE-22499.7.patch Status: Patch Available (was: Open) > LLAP: Add an EncodedReaderOptions to extend ORC impl for options > > > Key: HIVE-22499 > URL: https://issues.apache.org/jira/browse/HIVE-22499 > Project: Hive > Issue Type: Bug > Components: llap, ORC >Reporter: Gopal Vijayaraghavan >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch, > HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch, > HIVE-22499.6.patch, HIVE-22499.7.patch, HIVE-22499.WIP.patch, HIVE-22499.patch > > Time Spent: 10m > Remaining Estimate: 0h > > ORC-570 is an ABI change to the way getFileSystem() by adding an another > exception to the implementation. > To accept and use that change requires waiting for an ORC release, while this > patch serves the same purpose though falls back for a retry for > FileSystem.get() in case the supplier fails at runtime. > Also as a side-note, the FS.get() call is always used in the cases where the > file is not being read from a cache such as EncodedOrcFile (so the upstream > API change might be overkill). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22548) Optimise Utilities.removeTempOrDuplicateFiles when moving files to final location
[
https://issues.apache.org/jira/browse/HIVE-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987093#comment-16987093
]
Steve Loughran commented on HIVE-22548:
---
do you need that return code from removeEmptyDpDirectory()? As you are still
doing listStatus calls which you can avoid...if you just replaced the entire
function with delete(path, false) then only an empty dir will be deleted, so
you save the cost of a listing
> Optimise Utilities.removeTempOrDuplicateFiles when moving files to final
> location
> -
>
> Key: HIVE-22548
> URL: https://issues.apache.org/jira/browse/HIVE-22548
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
> Attachments: HIVE-22548.01.patch
>
>
> {{Utilities.removeTempOrDuplicateFiles}}
> is very slow with cloud storage, as it executes {{listStatus}} twice and also
> runs in single threaded mode.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1629
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22529) Make Debugging Stacktrace More Explicit
[
https://issues.apache.org/jira/browse/HIVE-22529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987086#comment-16987086
]
Vineet Garg commented on HIVE-22529:
+1
> Make Debugging Stacktrace More Explicit
> ---
>
> Key: HIVE-22529
> URL: https://issues.apache.org/jira/browse/HIVE-22529
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22529.1.patch, HIVE-22529.1.patch
>
>
> In some places, the following DEBUG logging was introduced:
> {code:java}
> LOG.debug("Message", new Exception());
> {code}
> The purpose of this is to log the stack trace of the Thread calling this
> debug logging method. However, the resulting log message includes the
> following:
> {code:none}
> 2019-11-19T08:13:31,392 DEBUG [Thread] Logger: Message
> java.lang.Exception: null
> at
> {code}
> To the observer, it appears that there was perhaps some sort of NPE. Add a
> message to the Exception being generated to make it more clear that this
> "Exception" is for debugging purposes and not an actual error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22529) Make Debugging Stacktrace More Explicit
[
https://issues.apache.org/jira/browse/HIVE-22529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987084#comment-16987084
]
David Mollitor commented on HIVE-22529:
---
[~ashutosh.bapat] Please review.
> Make Debugging Stacktrace More Explicit
> ---
>
> Key: HIVE-22529
> URL: https://issues.apache.org/jira/browse/HIVE-22529
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-22529.1.patch, HIVE-22529.1.patch
>
>
> In some places, the following DEBUG logging was introduced:
> {code:java}
> LOG.debug("Message", new Exception());
> {code}
> The purpose of this is to log the stack trace of the Thread calling this
> debug logging method. However, the resulting log message includes the
> following:
> {code:none}
> 2019-11-19T08:13:31,392 DEBUG [Thread] Logger: Message
> java.lang.Exception: null
> at
> {code}
> To the observer, it appears that there was perhaps some sort of NPE. Add a
> message to the Exception being generated to make it more clear that this
> "Exception" is for debugging purposes and not an actual error.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22280) Q tests for partitioned temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987071#comment-16987071
]
Hive QA commented on HIVE-22280:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
30s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
10s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 3m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19721/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19721/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Q tests for partitioned temporary tables
>
>
> Key: HIVE-22280
> URL: https://issues.apache.org/jira/browse/HIVE-22280
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22280.01.patch, HIVE-22280.02.patch,
> HIVE-22280.03.patch, HIVE-22280.04.patch, HIVE-22280.05.patch,
> HIVE-22280.06.patch, HIVE-22280.07.patch, HIVE-22280.08.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21050) Upgrade Parquet to 1.11.0 and use LogicalTypes
[
https://issues.apache.org/jira/browse/HIVE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987061#comment-16987061
]
Hive QA commented on HIVE-21050:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987358/HIVE-21050.7.patch
{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 41 failed/errored test(s), 17806 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_stats]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_parquet_projection]
(batchId=48)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parquet_complex_types_vectorization]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parquet_map_type_vectorization]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parquet_struct_type_vectorization]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[parquet_types_vectorization]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_partitioned_date_time]
(batchId=185)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet_types]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
(batchId=195)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_join]
(batchId=123)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_0]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_10]
(batchId=124)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_11]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_12]
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_13]
(batchId=138)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_14]
(batchId=131)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_15]
(batchId=154)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_16]
(batchId=152)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_17]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_1]
(batchId=118)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_2]
(batchId=115)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_3]
(batchId=150)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_4]
(batchId=134)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_5]
(batchId=146)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_6]
(batchId=133)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_7]
(batchId=153)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_8]
(batchId=120)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_9]
(batchId=128)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_decimal_date]
(batchId=128)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_div0]
(batchId=150)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_limit]
(batchId=125)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_offset_limit]
(batchId=129)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_part_project]
(batchId=130)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[parquet_vectorization_pushdown]
(batchId=129)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_input_format_excludes]
(batchId=135)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorization_parquet_projection]
(batchId=134)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[cbo_query21]
(batchId=301)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=276)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19720/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19720/console
Test logs:
[jira] [Commented] (HIVE-21050) Upgrade Parquet to 1.11.0 and use LogicalTypes
[
https://issues.apache.org/jira/browse/HIVE-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987053#comment-16987053
]
Hive QA commented on HIVE-21050:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
21s{color} | {color:blue} ql in master has 1532 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} ql: The patch generated 0 new + 145 unchanged - 4
fixed = 145 total (was 149) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
12s{color} | {color:green} root: The patch generated 0 new + 145 unchanged - 4
fixed = 145 total (was 149) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19720/dev-support/hive-personality.sh
|
| git revision | master / 23db35e |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19720/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade Parquet to 1.11.0 and use LogicalTypes
> --
>
> Key: HIVE-21050
> URL: https://issues.apache.org/jira/browse/HIVE-21050
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Labels: Parquet, parquet
> Attachments: HIVE-21050.1.patch, HIVE-21050.1.patch,
> HIVE-21050.1.patch, HIVE-21050.2.patch, HIVE-21050.3.patch,
> HIVE-21050.4.patch, HIVE-21050.4.patch, HIVE-21050.4.patch,
> HIVE-21050.5.patch, HIVE-21050.5.patch, HIVE-21050.5.patch,
> HIVE-21050.6.patch, HIVE-21050.6.patch, HIVE-21050.6.patch,
> HIVE-21050.6.patch.txt, HIVE-21050.7.patch, HIVE-21050.7.patch
>
>
> [WIP until Parquet community releases version 1.11.0]
> The new Parquet version (1.11.0) uses
> [LogicalTypes|https://github.com/apache/parquet-format/blob/master/LogicalTypes.md]
[jira] [Updated] (HIVE-22546) Postgres schema not using quoted identifiers for certain tables
[
https://issues.apache.org/jira/browse/HIVE-22546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Chovan updated HIVE-22546:
-
Attachment: HIVE-22546.4.patch
> Postgres schema not using quoted identifiers for certain tables
> ---
>
> Key: HIVE-22546
> URL: https://issues.apache.org/jira/browse/HIVE-22546
> Project: Hive
> Issue Type: Bug
> Components: Database/Schema
>Affects Versions: 4.0.0
>Reporter: Zoltan Chovan
>Assignee: Zoltan Chovan
>Priority: Major
> Attachments: HIVE-22546.2.patch, HIVE-22546.3.patch,
> HIVE-22546.4.patch, HIVE-22546.patch
>
>
> In the latest postgresql schema
> (standalone-metastore/metastore-server/src/main/sql/postgres/hive-schema-4.0.0.postgres.sql)
> the following tables have lowercase table and column names:
> {code:java}
> aux_table
> compaction_queue
> completed_compactions
> completed_txn_components
> hive_locks
> materialization_rebuild_locks
> min_history_level
> next_compaction_queue_id
> next_lock_id
> next_txn_id
> next_write_id
> repl_txn_map
> runtime_stats
> txn_components
> txn_to_write_id
> txns
> write_set{code}
> As these tables are referenced from the Hive sys database, the queries to
> these tables will fail with a "Table not found" error.
> The problem is that the table and column names are not enclosed in quotes, so
> postgres will turn these identifiers into lowercase.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22553) Expose locks and transactions in sys db
[ https://issues.apache.org/jira/browse/HIVE-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Chovan updated HIVE-22553: - Attachment: HIVE-22553.2.patch > Expose locks and transactions in sys db > --- > > Key: HIVE-22553 > URL: https://issues.apache.org/jira/browse/HIVE-22553 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0 >Reporter: Zoltan Chovan >Assignee: Zoltan Chovan >Priority: Major > Attachments: HIVE-22553.2.patch, HIVE-22553.patch > > > Create new sysdb tables/views to access lock and transaction data. > This allows to provide admins with live data about ongoing locks and > transacions. Due to this being in the sys db access to this information can > be restricted to select privileged users. > Information about locks and compactions can be joined and accessed at the > same time. > Compaction related transactions would also be visible. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20150) TopNKey pushdown
[
https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986986#comment-16986986
]
Hive QA commented on HIVE-20150:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987356/HIVE-20150.21.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17762 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19719/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19719/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19719/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987356 - PreCommit-HIVE-Build
> TopNKey pushdown
>
>
> Key: HIVE-20150
> URL: https://issues.apache.org/jira/browse/HIVE-20150
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Teddy Choi
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch,
> HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch,
> HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch,
> HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch,
> HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch,
> HIVE-20150.21.patch, HIVE-20150.4.patch, HIVE-20150.5.patch,
> HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch
>
>
> TopNKey operator is implemented in HIVE-17896, but it needs more work in
> pushdown implementation. So this issue covers TopNKey pushdown implementation
> with proper tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22355) Beeline should not prompt for hive user and password when authentication is NONE
[ https://issues.apache.org/jira/browse/HIVE-22355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986970#comment-16986970 ] Mate Juhasz commented on HIVE-22355: Thanks [~pvary] for your inputs! Uploading a new patch from a sightly different approach according to what we talked offline. Content: * The "auth" parameter could be provided as part of the JDBC url to ensure backward compatibility, also this way getting HiveConf is not necessary * Replaced the constant with "NONE" so its more readable * The "new Character()" is safe to be removed due to autoboxing in Java * Reverted the test changes > Beeline should not prompt for hive user and password when authentication is > NONE > > > Key: HIVE-22355 > URL: https://issues.apache.org/jira/browse/HIVE-22355 > Project: Hive > Issue Type: Bug > Components: Beeline >Reporter: Mate Juhasz >Assignee: Mate Juhasz >Priority: Major > Attachments: HIVE-22355.1.patch, HIVE-22355.2.patch, > HIVE-22355.3.patch > > > Beeline - without adding the jdbc url - prompts for username and password in > case hive.server2.authentication=NONE, which is possibly pointless and can be > misleading for users as any input is accepted. > In addition, Sqoop has dropped hive cli recently in favor of beeline and if > there is no authentication set in Hive, Sqoop fails to connect as the process > stops waiting for the user/password input. > I think it would be nice to check the auth type "NONE" before reading unused > inputs from the console before this point: > https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/Commands.java#L1641 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22355) Beeline should not prompt for hive user and password when authentication is NONE
[ https://issues.apache.org/jira/browse/HIVE-22355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated HIVE-22355: --- Attachment: HIVE-22355.3.patch > Beeline should not prompt for hive user and password when authentication is > NONE > > > Key: HIVE-22355 > URL: https://issues.apache.org/jira/browse/HIVE-22355 > Project: Hive > Issue Type: Bug > Components: Beeline >Reporter: Mate Juhasz >Assignee: Mate Juhasz >Priority: Major > Attachments: HIVE-22355.1.patch, HIVE-22355.2.patch, > HIVE-22355.3.patch > > > Beeline - without adding the jdbc url - prompts for username and password in > case hive.server2.authentication=NONE, which is possibly pointless and can be > misleading for users as any input is accepted. > In addition, Sqoop has dropped hive cli recently in favor of beeline and if > there is no authentication set in Hive, Sqoop fails to connect as the process > stops waiting for the user/password input. > I think it would be nice to check the auth type "NONE" before reading unused > inputs from the console before this point: > https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/Commands.java#L1641 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Status: Patch Available (was: Open) > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch, > HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986957#comment-16986957
]
David Mollitor edited comment on HIVE-22066 at 12/3/19 2:43 PM:
[~jcamachorodriguez] OK, so I think I have a better handle on what's going
wrong here...
{code:none}
maven install: master
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/branch-mvninstall-root.txt
2>&1
Elapsed: 8m 21s
master compilation: pre-patch
...
Pre-patch javadoc verification on master
...
Applying patch to master
...
Determining Maven Dependency Order (downloading dependencies in the process)
...
maven install: patch
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/storage-api
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-storage-api.txt
2>&1
Elapsed: 0m 13s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/shims/common
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-shims_common.txt
2>&1
Elapsed: 0m 11s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/shims/0.23
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-shims_0.23.txt
2>&1
Elapsed: 0m 12s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/shims/scheduler
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-shims_scheduler.txt
2>&1
Elapsed: 0m 9s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/standalone-metastore
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-standalone-metastore.txt
2>&1
Elapsed: 1m 15s
cd
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/standalone-metastore/metastore-common
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-standalone-metastore_metastore-common.txt
2>&1
Elapsed: 0m 37s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/common
mvn -DskipTests -Pitests -fae clean
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Attachment: HIVE-22066.999.patch > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch, > HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Attachment: (was: HIVE-22066.999.patch) > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch, > HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Status: Open (was: Patch Available) > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch, > HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22066) Upgrade Apache parent POM to version 21
[
https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986957#comment-16986957
]
David Mollitor commented on HIVE-22066:
---
[~jcamachorodriguez] OK, so I think I have a better handle on what's going
wrong here...
{code:none}
maven install: master
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/branch-mvninstall-root.txt
2>&1
Elapsed: 8m 21s
master compilation: pre-patch
...
Pre-patch javadoc verification on master
...
Applying patch to master
...
Determining Maven Dependency Order (downloading dependencies in the process)
...
maven install: patch
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/storage-api
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-storage-api.txt
2>&1
Elapsed: 0m 13s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/shims/common
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-shims_common.txt
2>&1
Elapsed: 0m 11s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/shims/0.23
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-shims_0.23.txt
2>&1
Elapsed: 0m 12s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/shims/scheduler
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-shims_scheduler.txt
2>&1
Elapsed: 0m 9s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/standalone-metastore
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-standalone-metastore.txt
2>&1
Elapsed: 1m 15s
cd
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/standalone-metastore/metastore-common
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true -Dcheckstyle.skip=true -Dfindbugs.skip=true >
/data/hiveptest/logs/PreCommit-HIVE-Build-19708/yetus/patch-mvninstall-standalone-metastore_metastore-common.txt
2>&1
Elapsed: 0m 37s
cd /data/hiveptest/working/yetus_PreCommit-HIVE-Build-19708/common
mvn -DskipTests -Pitests -fae clean install -DskipTests=true
-Dmaven.javadoc.skip=true
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Attachment: HIVE-22066.999.patch > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch, > HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Status: Open (was: Patch Available) > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch, > HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22066) Upgrade Apache parent POM to version 21
[ https://issues.apache.org/jira/browse/HIVE-22066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22066: -- Status: Patch Available (was: Open) > Upgrade Apache parent POM to version 21 > --- > > Key: HIVE-22066 > URL: https://issues.apache.org/jira/browse/HIVE-22066 > Project: Hive > Issue Type: Improvement >Reporter: Jesus Camacho Rodriguez >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22066.2.patch, HIVE-22066.3.patch, > HIVE-22066.4.patch, HIVE-22066.5.patch, HIVE-22066.6.patch, > HIVE-22066.6.patch, HIVE-22066.7.patch, HIVE-22066.8.patch, > HIVE-22066.8.patch, HIVE-22066.9.patch, HIVE-22066.999.patch, > HIVE-22066.patch, HIVE-22066.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20150) TopNKey pushdown
[
https://issues.apache.org/jira/browse/HIVE-20150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986944#comment-16986944
]
Hive QA commented on HIVE-20150:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
59s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
37s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} ql: The patch generated 0 new + 37 unchanged - 1
fixed = 37 total (was 38) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19719/dev-support/hive-personality.sh
|
| git revision | master / 601937d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19719/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TopNKey pushdown
>
>
> Key: HIVE-20150
> URL: https://issues.apache.org/jira/browse/HIVE-20150
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Teddy Choi
>Assignee: Krisztian Kasa
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20150.1.patch, HIVE-20150.10.patch,
> HIVE-20150.11.patch, HIVE-20150.11.patch, HIVE-20150.14.patch,
> HIVE-20150.15.patch, HIVE-20150.16.patch, HIVE-20150.17.patch,
> HIVE-20150.17.patch, HIVE-20150.18.patch, HIVE-20150.18.patch,
> HIVE-20150.19.patch, HIVE-20150.2.patch, HIVE-20150.20.patch,
> HIVE-20150.21.patch, HIVE-20150.4.patch, HIVE-20150.5.patch,
> HIVE-20150.6.patch, HIVE-20150.7.patch, HIVE-20150.8.patch, HIVE-20150.9.patch
>
>
> TopNKey operator is implemented in HIVE-17896, but it needs more work in
> pushdown implementation. So this issue covers TopNKey pushdown
[jira] [Updated] (HIVE-22534) ACID: Improve Compactor thread logging
[ https://issues.apache.org/jira/browse/HIVE-22534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter updated HIVE-22534: - Attachment: HIVE-22534.06.patch > ACID: Improve Compactor thread logging > -- > > Key: HIVE-22534 > URL: https://issues.apache.org/jira/browse/HIVE-22534 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Pinter >Assignee: Laszlo Pinter >Priority: Minor > Attachments: HIVE-22534.01.patch, HIVE-22534.02.patch, > HIVE-22534.03.patch, HIVE-22534.04.patch, HIVE-22534.05.patch, > HIVE-22534.06.patch > > > Make sure that it is easy to find issues when one of the compactor thread > fails. > Maybe: > * MDC - with iteration / threadname - so we can easily grep the logs for a > given run > * MDC with table/partition data on which the worker is working -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22526) Extract Compiler from Driver
[ https://issues.apache.org/jira/browse/HIVE-22526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22526: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Extract Compiler from Driver > > > Key: HIVE-22526 > URL: https://issues.apache.org/jira/browse/HIVE-22526 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22526.01.patch, HIVE-22526.02.patch, > HIVE-22526.03.patch, HIVE-22526.04.patch, HIVE-22526.05.patch, > HIVE-22526.06.patch, HIVE-22526.07.patch, HIVE-22526.08.patch, > HIVE-22526.09.patch > > > The Driver class contains ~600 lines of code responsible for compiling the > command. That means that from the command String a Plan needs to be created, > and also a transaction needs to be started (in most of the cases). This is a > thing done by the compile function, which has a lot of sub functions to help > this task, while itself is also really big. All these codes should be put > into a separate class, where it can do it's job without getting mixed with > the other codes in the Driver. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22547) Review txn compactor Package
[ https://issues.apache.org/jira/browse/HIVE-22547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22547: -- Status: Patch Available (was: Open) > Review txn compactor Package > > > Key: HIVE-22547 > URL: https://issues.apache.org/jira/browse/HIVE-22547 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22547.1.patch, HIVE-22547.2.patch, > HIVE-22547.2.patch > > > * Remove log-and-throw anti-pattern > * Use parameterized logging > * Add a CompactionException class to improve debug-ability > * Introduce Java Optional utility > * Other clean up -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22547) Review txn compactor Package
[ https://issues.apache.org/jira/browse/HIVE-22547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22547: -- Attachment: HIVE-22547.2.patch > Review txn compactor Package > > > Key: HIVE-22547 > URL: https://issues.apache.org/jira/browse/HIVE-22547 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22547.1.patch, HIVE-22547.2.patch, > HIVE-22547.2.patch > > > * Remove log-and-throw anti-pattern > * Use parameterized logging > * Add a CompactionException class to improve debug-ability > * Introduce Java Optional utility > * Other clean up -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22547) Review txn compactor Package
[ https://issues.apache.org/jira/browse/HIVE-22547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22547: -- Status: Open (was: Patch Available) > Review txn compactor Package > > > Key: HIVE-22547 > URL: https://issues.apache.org/jira/browse/HIVE-22547 > Project: Hive > Issue Type: Improvement >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Attachments: HIVE-22547.1.patch, HIVE-22547.2.patch > > > * Remove log-and-throw anti-pattern > * Use parameterized logging > * Add a CompactionException class to improve debug-ability > * Introduce Java Optional utility > * Other clean up -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22548) Optimise Utilities.removeTempOrDuplicateFiles when moving files to final location
[
https://issues.apache.org/jira/browse/HIVE-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986902#comment-16986902
]
Hive QA commented on HIVE-22548:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987353/HIVE-22548.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17756 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestHiveMetaStoreAlterColumnPar.org.apache.hadoop.hive.metastore.TestHiveMetaStoreAlterColumnPar
(batchId=249)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19718/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19718/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19718/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987353 - PreCommit-HIVE-Build
> Optimise Utilities.removeTempOrDuplicateFiles when moving files to final
> location
> -
>
> Key: HIVE-22548
> URL: https://issues.apache.org/jira/browse/HIVE-22548
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
> Attachments: HIVE-22548.01.patch
>
>
> {{Utilities.removeTempOrDuplicateFiles}}
> is very slow with cloud storage, as it executes {{listStatus}} twice and also
> runs in single threaded mode.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1629
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22548) Optimise Utilities.removeTempOrDuplicateFiles when moving files to final location
[
https://issues.apache.org/jira/browse/HIVE-22548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986865#comment-16986865
]
Hive QA commented on HIVE-22548:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
39s{color} | {color:blue} ql in master has 1534 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 1 new + 106 unchanged - 0
fixed = 107 total (was 106) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-19718/dev-support/hive-personality.sh
|
| git revision | master / 0a95b3a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19718/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-19718/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Optimise Utilities.removeTempOrDuplicateFiles when moving files to final
> location
> -
>
> Key: HIVE-22548
> URL: https://issues.apache.org/jira/browse/HIVE-22548
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.2
>Reporter: Rajesh Balamohan
>Assignee: mahesh kumar behera
>Priority: Major
> Attachments: HIVE-22548.01.patch
>
>
> {{Utilities.removeTempOrDuplicateFiles}}
> is very slow with cloud storage, as it executes {{listStatus}} twice and also
> runs in single threaded mode.
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1629
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22554) ACID: Wait timeout for blocking compaction should be configurable
[
https://issues.apache.org/jira/browse/HIVE-22554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986845#comment-16986845
]
Laszlo Pinter commented on HIVE-22554:
--
Thanks [~pvary], for the review & commit!
> ACID: Wait timeout for blocking compaction should be configurable
> -
>
> Key: HIVE-22554
> URL: https://issues.apache.org/jira/browse/HIVE-22554
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22554.01.patch, HIVE-22554.02.patch
>
>
> The wait timeout for blocking compaction is hardcoded to 5 minutes.
> {code:java}
> public class AlterTableCompactOperation extends
> DDLOperation {
> private static final int FIVE_MINUTES_IN_MILLIES = 5*60*1000;
> ...
> }{code}
> This should be configurable via a Hive Configuration parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22554) ACID: Wait timeout for blocking compaction should be configurable
[
https://issues.apache.org/jira/browse/HIVE-22554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-22554:
--
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master.
Thanks for the patch [~lpinter]!
> ACID: Wait timeout for blocking compaction should be configurable
> -
>
> Key: HIVE-22554
> URL: https://issues.apache.org/jira/browse/HIVE-22554
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22554.01.patch, HIVE-22554.02.patch
>
>
> The wait timeout for blocking compaction is hardcoded to 5 minutes.
> {code:java}
> public class AlterTableCompactOperation extends
> DDLOperation {
> private static final int FIVE_MINUTES_IN_MILLIES = 5*60*1000;
> ...
> }{code}
> This should be configurable via a Hive Configuration parameter.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22499) LLAP: Add an EncodedReaderOptions to extend ORC impl for options
[
https://issues.apache.org/jira/browse/HIVE-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986835#comment-16986835
]
Hive QA commented on HIVE-22499:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12987348/HIVE-22499.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17756 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/19717/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19717/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19717/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12987348 - PreCommit-HIVE-Build
> LLAP: Add an EncodedReaderOptions to extend ORC impl for options
>
>
> Key: HIVE-22499
> URL: https://issues.apache.org/jira/browse/HIVE-22499
> Project: Hive
> Issue Type: Bug
> Components: llap, ORC
>Reporter: Gopal Vijayaraghavan
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-22499.2.patch, HIVE-22499.2.patch,
> HIVE-22499.3.patch, HIVE-22499.4.patch, HIVE-22499.5.patch,
> HIVE-22499.6.patch, HIVE-22499.WIP.patch, HIVE-22499.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> ORC-570 is an ABI change to the way getFileSystem() by adding an another
> exception to the implementation.
> To accept and use that change requires waiting for an ORC release, while this
> patch serves the same purpose though falls back for a retry for
> FileSystem.get() in case the supplier fails at runtime.
> Also as a side-note, the FS.get() call is always used in the cases where the
> file is not being read from a cache such as EncodedOrcFile (so the upstream
> API change might be overkill).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-20934) ACID: Query based compactor for minor compaction
[ https://issues.apache.org/jira/browse/HIVE-20934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter updated HIVE-20934: - Status: Patch Available (was: Open) > ACID: Query based compactor for minor compaction > > > Key: HIVE-20934 > URL: https://issues.apache.org/jira/browse/HIVE-20934 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-20934.01.patch > > > Follow up of HIVE-20699. This is to enable running minor compactions as a > HiveQL query -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-20934) ACID: Query based compactor for minor compaction
[ https://issues.apache.org/jira/browse/HIVE-20934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Pinter updated HIVE-20934: - Attachment: HIVE-20934.01.patch > ACID: Query based compactor for minor compaction > > > Key: HIVE-20934 > URL: https://issues.apache.org/jira/browse/HIVE-20934 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Vaibhav Gumashta >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-20934.01.patch > > > Follow up of HIVE-20699. This is to enable running minor compactions as a > HiveQL query -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22355) Beeline should not prompt for hive user and password when authentication is NONE
[
https://issues.apache.org/jira/browse/HIVE-22355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986823#comment-16986823
]
Peter Vary commented on HIVE-22355:
---
Hi [~matijhs],
Thanks for the patch. Could you please open a pull request, or a review board
request?
Few questions:
* Why is this change necessary:
{code:java}
auth =
getHiveConf(false).get(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.varname);
{code}
getHiveConf does with false tries to read hive-conf from the classpath - which
usually should not be there. Why is this change necessary?
* I would prefer "NONE", or even better a constant in HiveConf instead of
.getDefaultValue() - this is not descriptive, and might change without anyone
noticing it:
{code:java}
if
(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.getDefaultValue().equals(auth))
{ {code}
* Are you sure of this change:
{code:java}
password = beeLine.getConsoleReader().readLine("Enter password for " +
urlForPrompt + ": ", '*'); {code}
Seems reasonable, also unnecessary, so I would want to be absolutely sure that
nothing changes with removing "new Character()" before doing :)
* The test changes suggest to me, that this is a backward incompatible change:
{code:java}
argList.add("-a");
argList.add("NOT_NONE"); {code}
Do I miss something, or this is intended? I would prefer a change that is
backward compatible (proven by tests)
* Please fix the *new* checkstyle errors
Thanks,
Peter
> Beeline should not prompt for hive user and password when authentication is
> NONE
>
>
> Key: HIVE-22355
> URL: https://issues.apache.org/jira/browse/HIVE-22355
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: HIVE-22355.1.patch, HIVE-22355.2.patch
>
>
> Beeline - without adding the jdbc url - prompts for username and password in
> case hive.server2.authentication=NONE, which is possibly pointless and can be
> misleading for users as any input is accepted.
> In addition, Sqoop has dropped hive cli recently in favor of beeline and if
> there is no authentication set in Hive, Sqoop fails to connect as the process
> stops waiting for the user/password input.
> I think it would be nice to check the auth type "NONE" before reading unused
> inputs from the console before this point:
> https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/Commands.java#L1641
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22355) Beeline should not prompt for hive user and password when authentication is NONE
[
https://issues.apache.org/jira/browse/HIVE-22355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986823#comment-16986823
]
Peter Vary edited comment on HIVE-22355 at 12/3/19 11:47 AM:
-
Hi [~matijhs],
Thanks for the patch. Could you please open a pull request, or a review board
request?
Few questions:
* Why is this change necessary:
{code:java}
auth =
getHiveConf(false).get(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.varname);
{code}
getHiveConf does with false tries to read hive-conf from the classpath - which
usually should not be there. Why is this change necessary?
* I would prefer "NONE", or even better a constant in HiveConf instead of
.getDefaultValue() - this is not descriptive, and might change without anyone
noticing it:
{code:java}
if
(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.getDefaultValue().equals(auth))
{ {code}
* Are you sure of this change:
{code:java}
password = beeLine.getConsoleReader().readLine("Enter password for " +
urlForPrompt + ": ", '*'); {code}
Seems reasonable, also unnecessary, so I would want to be absolutely sure that
nothing changes with removing "new Character()" before doing so :)
* The test changes suggest to me, that this is a backward incompatible change:
{code:java}
argList.add("-a");
argList.add("NOT_NONE"); {code}
Do I miss something, or this is intended? I would prefer a change that is
backward compatible (proven by tests)
* Please fix the *new* checkstyle errors
Thanks,
Peter
was (Author: pvary):
Hi [~matijhs],
Thanks for the patch. Could you please open a pull request, or a review board
request?
Few questions:
* Why is this change necessary:
{code:java}
auth =
getHiveConf(false).get(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.varname);
{code}
getHiveConf does with false tries to read hive-conf from the classpath - which
usually should not be there. Why is this change necessary?
* I would prefer "NONE", or even better a constant in HiveConf instead of
.getDefaultValue() - this is not descriptive, and might change without anyone
noticing it:
{code:java}
if
(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.getDefaultValue().equals(auth))
{ {code}
* Are you sure of this change:
{code:java}
password = beeLine.getConsoleReader().readLine("Enter password for " +
urlForPrompt + ": ", '*'); {code}
Seems reasonable, also unnecessary, so I would want to be absolutely sure that
nothing changes with removing "new Character()" before doing :)
* The test changes suggest to me, that this is a backward incompatible change:
{code:java}
argList.add("-a");
argList.add("NOT_NONE"); {code}
Do I miss something, or this is intended? I would prefer a change that is
backward compatible (proven by tests)
* Please fix the *new* checkstyle errors
Thanks,
Peter
> Beeline should not prompt for hive user and password when authentication is
> NONE
>
>
> Key: HIVE-22355
> URL: https://issues.apache.org/jira/browse/HIVE-22355
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: HIVE-22355.1.patch, HIVE-22355.2.patch
>
>
> Beeline - without adding the jdbc url - prompts for username and password in
> case hive.server2.authentication=NONE, which is possibly pointless and can be
> misleading for users as any input is accepted.
> In addition, Sqoop has dropped hive cli recently in favor of beeline and if
> there is no authentication set in Hive, Sqoop fails to connect as the process
> stops waiting for the user/password input.
> I think it would be nice to check the auth type "NONE" before reading unused
> inputs from the console before this point:
> https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/Commands.java#L1641
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (HIVE-22355) Beeline should not prompt for hive user and password when authentication is NONE
[
https://issues.apache.org/jira/browse/HIVE-22355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16986823#comment-16986823
]
Peter Vary edited comment on HIVE-22355 at 12/3/19 11:47 AM:
-
Hi [~matijhs],
Thanks for the patch. Could you please open a pull request, or a review board
request?
Few questions:
* Why is this change necessary:
{code:java}
auth =
getHiveConf(false).get(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.varname);
{code}
getHiveConf with false tries to read hive-conf from the classpath - which
usually should not be there. Why is this change necessary?
* I would prefer "NONE", or even better a constant in HiveConf instead of
.getDefaultValue() - this is not descriptive, and might change without anyone
noticing it:
{code:java}
if
(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.getDefaultValue().equals(auth))
{ {code}
* Are you sure of this change:
{code:java}
password = beeLine.getConsoleReader().readLine("Enter password for " +
urlForPrompt + ": ", '*'); {code}
Seems reasonable, also unnecessary, so I would want to be absolutely sure that
nothing changes with removing "new Character()" before doing so :)
* The test changes suggest to me, that this is a backward incompatible change:
{code:java}
argList.add("-a");
argList.add("NOT_NONE"); {code}
Do I miss something, or this is intended? I would prefer a change that is
backward compatible (proven by tests)
* Please fix the *new* checkstyle errors
Thanks,
Peter
was (Author: pvary):
Hi [~matijhs],
Thanks for the patch. Could you please open a pull request, or a review board
request?
Few questions:
* Why is this change necessary:
{code:java}
auth =
getHiveConf(false).get(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.varname);
{code}
getHiveConf does with false tries to read hive-conf from the classpath - which
usually should not be there. Why is this change necessary?
* I would prefer "NONE", or even better a constant in HiveConf instead of
.getDefaultValue() - this is not descriptive, and might change without anyone
noticing it:
{code:java}
if
(HiveConf.ConfVars.HIVE_SERVER2_AUTHENTICATION.getDefaultValue().equals(auth))
{ {code}
* Are you sure of this change:
{code:java}
password = beeLine.getConsoleReader().readLine("Enter password for " +
urlForPrompt + ": ", '*'); {code}
Seems reasonable, also unnecessary, so I would want to be absolutely sure that
nothing changes with removing "new Character()" before doing so :)
* The test changes suggest to me, that this is a backward incompatible change:
{code:java}
argList.add("-a");
argList.add("NOT_NONE"); {code}
Do I miss something, or this is intended? I would prefer a change that is
backward compatible (proven by tests)
* Please fix the *new* checkstyle errors
Thanks,
Peter
> Beeline should not prompt for hive user and password when authentication is
> NONE
>
>
> Key: HIVE-22355
> URL: https://issues.apache.org/jira/browse/HIVE-22355
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Attachments: HIVE-22355.1.patch, HIVE-22355.2.patch
>
>
> Beeline - without adding the jdbc url - prompts for username and password in
> case hive.server2.authentication=NONE, which is possibly pointless and can be
> misleading for users as any input is accepted.
> In addition, Sqoop has dropped hive cli recently in favor of beeline and if
> there is no authentication set in Hive, Sqoop fails to connect as the process
> stops waiting for the user/password input.
> I think it would be nice to check the auth type "NONE" before reading unused
> inputs from the console before this point:
> https://github.com/apache/hive/blob/master/beeline/src/java/org/apache/hive/beeline/Commands.java#L1641
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Attachment: HIVE-22508.1.patch > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22508) KeyWrapperComparator throws exception
[ https://issues.apache.org/jira/browse/HIVE-22508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Krisztian Kasa updated HIVE-22508: -- Status: Open (was: Patch Available) > KeyWrapperComparator throws exception > - > > Key: HIVE-22508 > URL: https://issues.apache.org/jira/browse/HIVE-22508 > Project: Hive > Issue Type: Bug > Components: Physical Optimizer >Reporter: Krisztian Kasa >Assignee: Krisztian Kasa >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22508.1.patch, HIVE-22508.1.patch > > > TopNKeyOperator.KeyWrapperComparator throws exception when a new key and a > copied key should be compared. > The current implementation uses the standard object inspectors for all > KeyWrapper instances. However when comparing untouched KeyWrappers the key > object inspector should be used which can be extracted form > Operator.inputObjectInspectors during initialization of the key's > ExprNodeEvaluator. > This can cause a ClassCastException when the comparator is used collections > like TreeSet. -- This message was sent by Atlassian Jira (v8.3.4#803005)
