[jira] [Commented] (HIVE-22825) Reduce directory lookup cost for acid tables
[
https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043197#comment-17043197
]
Hive QA commented on HIVE-22825:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994271/HIVE-22825.10.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 18056 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20796/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20796/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20796/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12994271 - PreCommit-HIVE-Build
> Reduce directory lookup cost for acid tables
>
>
> Key: HIVE-22825
> URL: https://issues.apache.org/jira/browse/HIVE-22825
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22825.1.patch, HIVE-22825.10.patch,
> HIVE-22825.2.patch, HIVE-22825.3.patch, HIVE-22825.4.patch,
> HIVE-22825.5.patch, HIVE-22825.6.patch, HIVE-22825.7.patch,
> HIVE-22825.8.patch, HIVE-22825.9.patch
>
>
> With objectstores, directory lookup costs are expensive. For acid tables, it
> would be good to have a directory cache to reduce number of lookup calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22825) Reduce directory lookup cost for acid tables
[
https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043177#comment-17043177
]
Hive QA commented on HIVE-22825:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
56s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 4 new + 541 unchanged - 4
fixed = 545 total (was 545) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20796/dev-support/hive-personality.sh
|
| git revision | master / fa213b2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20796/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20796/yetus/patch-asflicense-problems.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20796/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Reduce directory lookup cost for acid tables
>
>
> Key: HIVE-22825
> URL: https://issues.apache.org/jira/browse/HIVE-22825
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22825.1.patch, HIVE-22825.10.patch,
> HIVE-22825.2.patch, HIVE-22825.3.patch, HIVE-22825.4.patch,
> HIVE-22825.5.patch, HIVE-22825.6.patch, HIVE-22825.7.patch,
> HIVE-22825.8.patch, HIVE-22825.9.patch
>
>
> With objectstores, directory lookup costs are expensive. For acid tables, it
> would be good to have a directory cache to reduce number of lookup calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22771) Partition location incorrectly formed in FileOutputCommitterContainer
[
https://issues.apache.org/jira/browse/HIVE-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043173#comment-17043173
]
Shivam commented on HIVE-22771:
---
Any comments, does this patch looks fine?
> Partition location incorrectly formed in FileOutputCommitterContainer
> -
>
> Key: HIVE-22771
> URL: https://issues.apache.org/jira/browse/HIVE-22771
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 1.2.1
>Reporter: Shivam
>Assignee: Shivam
>Priority: Critical
> Labels: pull-request-available
> Attachments: HIVE-22771.2.patch, HIVE-22771.3.patch,
> HIVE-22771.4.patch, HIVE-22771.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Class _HCatOutputFormat_ in package _org.apache.hive.hcatalog.mapreduce_ uses
> function _setOutput_ to generate _idHash_ using below statement:
> *+In file org/apache/hive/hcatalog/mapreduce/HCatOutputFormat.java+*
> *line 116: idHash = String.valueOf(Math.random());*
> The output of idHash can be similar to values like this : 7.145347157239135E-4
>
> And, in class _FileOutputCommitterContainer_ in package
> _org.apache.hive.hcatalog.mapreduce;_
> Uses below statement to compute final partition path:
> +*In org/apache/hive/hcatalog/mapreduce/FileOutputCommitterContainer.java*+
> *line 366: String finalLocn = jobLocation.replaceAll(Path.SEPARATOR +
> SCRATCH_DIR_NAME + "{color:#ff}\\d\\.?
> d+"{color},"");*
> *line 367: partPath = new Path(finalLocn);*
>
> Regex used here is incorrect, since it will only remove integers after the
> *SCRATCH_DIR_NAME,* and hence will append 'E-4' (for the above example) in
> the final partition location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22825) Reduce directory lookup cost for acid tables
[ https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22825: Attachment: HIVE-22825.10.patch > Reduce directory lookup cost for acid tables > > > Key: HIVE-22825 > URL: https://issues.apache.org/jira/browse/HIVE-22825 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22825.1.patch, HIVE-22825.10.patch, > HIVE-22825.2.patch, HIVE-22825.3.patch, HIVE-22825.4.patch, > HIVE-22825.5.patch, HIVE-22825.6.patch, HIVE-22825.7.patch, > HIVE-22825.8.patch, HIVE-22825.9.patch > > > With objectstores, directory lookup costs are expensive. For acid tables, it > would be good to have a directory cache to reduce number of lookup calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22892) Unable to compile query if CTE joined
[
https://issues.apache.org/jira/browse/HIVE-22892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22892:
--
Component/s: (was: Logical Optimizer)
Query Processor
> Unable to compile query if CTE joined
> -
>
> Key: HIVE-22892
> URL: https://issues.apache.org/jira/browse/HIVE-22892
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22892.1.patch, HIVE-22892.2.patch,
> HIVE-22892.3.patch, HIVE-22892.4.patch
>
>
> Repro:
> {code}
> CREATE TABLE t1 (a int, b varchar(100));
> SELECT S.a, t1.a, t1.b FROM (
> WITH
> sub1 AS (SELECT a, b FROM t1 WHERE b = 'c')
> SELECT sub1.a, sub1.b FROM sub1
> ) S
> JOIN t1 ON S.a = t1.a;
> {code}
> {code}
> java.lang.AssertionError
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getUnescapedUnqualifiedTableName(BaseSemanticAnalyzer.java:463)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genJoinLogicalPlan(CalcitePlanner.java:2870)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:5047)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1787)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1734)
> at
> org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:130)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:915)
> at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:179)
> at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:125)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:471)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12550)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:361)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:286)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:219)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:103)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:197)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:756)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:750)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:125)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:229)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:249)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:193)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:415)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:346)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:709)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:679)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:169)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver(TestCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:135)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
>
[jira] [Updated] (HIVE-22892) Unable to compile query if CTE joined
[
https://issues.apache.org/jira/browse/HIVE-22892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22892:
--
Component/s: Logical Optimizer
> Unable to compile query if CTE joined
> -
>
> Key: HIVE-22892
> URL: https://issues.apache.org/jira/browse/HIVE-22892
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer
>Reporter: Krisztian Kasa
>Assignee: Krisztian Kasa
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22892.1.patch, HIVE-22892.2.patch,
> HIVE-22892.3.patch, HIVE-22892.4.patch
>
>
> Repro:
> {code}
> CREATE TABLE t1 (a int, b varchar(100));
> SELECT S.a, t1.a, t1.b FROM (
> WITH
> sub1 AS (SELECT a, b FROM t1 WHERE b = 'c')
> SELECT sub1.a, sub1.b FROM sub1
> ) S
> JOIN t1 ON S.a = t1.a;
> {code}
> {code}
> java.lang.AssertionError
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.getUnescapedUnqualifiedTableName(BaseSemanticAnalyzer.java:463)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genJoinLogicalPlan(CalcitePlanner.java:2870)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:5047)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1787)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1734)
> at
> org.apache.calcite.tools.Frameworks.lambda$withPlanner$0(Frameworks.java:130)
> at
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:915)
> at org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:179)
> at org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:125)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1495)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:471)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12550)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:361)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:286)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:219)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:103)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:197)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:810)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:756)
> at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:750)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:125)
> at
> org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:229)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:249)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:193)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:415)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:346)
> at
> org.apache.hadoop.hive.ql.QTestUtil.executeClientInternal(QTestUtil.java:709)
> at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:679)
> at
> org.apache.hadoop.hive.cli.control.CoreCliDriver.runTest(CoreCliDriver.java:169)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter.runTest(CliAdapter.java:157)
> at
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver(TestCliDriver.java:59)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.hadoop.hive.cli.control.CliAdapter$2$1.evaluate(CliAdapter.java:135)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.r
[jira] [Commented] (HIVE-22840) Race condition in formatters of TimestampColumnVector and DateColumnVector
[
https://issues.apache.org/jira/browse/HIVE-22840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043154#comment-17043154
]
Shubham Chaurasia commented on HIVE-22840:
--
[~jcamachorodriguez]
Oh sorry, created a PR - https://github.com/apache/hive/pull/922
Latest patch was -
https://issues.apache.org/jira/secure/attachment/12993999/HIVE-22840.patch
> Race condition in formatters of TimestampColumnVector and DateColumnVector
> ---
>
> Key: HIVE-22840
> URL: https://issues.apache.org/jira/browse/HIVE-22840
> Project: Hive
> Issue Type: Bug
> Components: storage-api
>Reporter: László Bodor
>Assignee: Shubham Chaurasia
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22840.1.patch, HIVE-22840.2.patch, HIVE-22840.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-22405 added support for proleptic calendar. It uses java's
> SimpleDateFormat/Calendar APIs which are not thread-safe and cause race in
> some scenarios.
> As a result of those race conditions, we see some exceptions like
> {code:java}
> 1) java.lang.NumberFormatException: For input string: ""
> OR
> java.lang.NumberFormatException: For input string: ".821582E.821582E44"
> OR
> 2) Caused by: java.lang.ArrayIndexOutOfBoundsException: -5325980
> at
> sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:453)
> at
> java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2397)
> {code}
> This issue is to address those thread-safety issues/race conditions.
> cc [~jcamachorodriguez] [~abstractdog] [~omalley]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22825) Reduce directory lookup cost for acid tables
[
https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043153#comment-17043153
]
Hive QA commented on HIVE-22825:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994265/HIVE-22825.9.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20795/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20795/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20795/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2020-02-24 04:16:54.977
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-20795/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2020-02-24 04:16:54.980
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
6c3ee53..fa213b2 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 6c3ee53 HIVE-21216: Write Parquet INT64 timestamp (Karen Coppage
via Marta Kuczora)
+ git clean -f -d
Removing ${project.basedir}/
Removing itests/${project.basedir}/
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at fa213b2 HIVE-22914: Make Hive Connection ZK Interactions Easier
to Troubleshoot (David Mollitor, reviewed by Peter Vary)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2020-02-24 04:16:56.443
+ rm -rf ../yetus_PreCommit-HIVE-Build-20795
+ mkdir ../yetus_PreCommit-HIVE-Build-20795
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-20795
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-20795/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Trying to apply the patch with -p0
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:35
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java' with
conflicts.
Going to apply patch with: git apply -p0
error: patch failed: ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java:35
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java' with
conflicts.
U ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-20795
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12994265 - PreCommit-HIVE-Build
> Reduce directory lookup cost for acid tables
>
>
> Key: HIVE-22825
> URL: https://issues.apache.org/jira/browse/HIVE-22825
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Minor
> Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch,
> HIVE-22825.3.patch, HIVE-22825.4.patch, HIVE-22825.5.patch,
> HIVE-22825.6.patch, HIVE-22825.7.patch, HIVE-22825.8.patch, HIVE-22825.9.patch
>
>
> With objectstores, directory lookup costs are expensive. For acid tables, it
> would be good to have a directory cache to reduce number of lookup calls.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22840) Race condition in formatters of TimestampColumnVector and DateColumnVector
[
https://issues.apache.org/jira/browse/HIVE-22840?focusedWorklogId=391500&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391500
]
ASF GitHub Bot logged work on HIVE-22840:
-
Author: ASF GitHub Bot
Created on: 24/Feb/20 04:15
Start Date: 24/Feb/20 04:15
Worklog Time Spent: 10m
Work Description: ShubhamChaurasia commented on pull request #922:
HIVE-22840: Race condition in formatters of TimestampColumnVector and…
URL: https://github.com/apache/hive/pull/922
… DateColumnVector
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 391500)
Remaining Estimate: 0h
Time Spent: 10m
> Race condition in formatters of TimestampColumnVector and DateColumnVector
> ---
>
> Key: HIVE-22840
> URL: https://issues.apache.org/jira/browse/HIVE-22840
> Project: Hive
> Issue Type: Bug
> Components: storage-api
>Reporter: László Bodor
>Assignee: Shubham Chaurasia
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22840.1.patch, HIVE-22840.2.patch, HIVE-22840.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HIVE-22405 added support for proleptic calendar. It uses java's
> SimpleDateFormat/Calendar APIs which are not thread-safe and cause race in
> some scenarios.
> As a result of those race conditions, we see some exceptions like
> {code:java}
> 1) java.lang.NumberFormatException: For input string: ""
> OR
> java.lang.NumberFormatException: For input string: ".821582E.821582E44"
> OR
> 2) Caused by: java.lang.ArrayIndexOutOfBoundsException: -5325980
> at
> sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:453)
> at
> java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2397)
> {code}
> This issue is to address those thread-safety issues/race conditions.
> cc [~jcamachorodriguez] [~abstractdog] [~omalley]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22840) Race condition in formatters of TimestampColumnVector and DateColumnVector
[
https://issues.apache.org/jira/browse/HIVE-22840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-22840:
--
Labels: pull-request-available (was: )
> Race condition in formatters of TimestampColumnVector and DateColumnVector
> ---
>
> Key: HIVE-22840
> URL: https://issues.apache.org/jira/browse/HIVE-22840
> Project: Hive
> Issue Type: Bug
> Components: storage-api
>Reporter: László Bodor
>Assignee: Shubham Chaurasia
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22840.1.patch, HIVE-22840.2.patch, HIVE-22840.patch
>
>
> HIVE-22405 added support for proleptic calendar. It uses java's
> SimpleDateFormat/Calendar APIs which are not thread-safe and cause race in
> some scenarios.
> As a result of those race conditions, we see some exceptions like
> {code:java}
> 1) java.lang.NumberFormatException: For input string: ""
> OR
> java.lang.NumberFormatException: For input string: ".821582E.821582E44"
> OR
> 2) Caused by: java.lang.ArrayIndexOutOfBoundsException: -5325980
> at
> sun.util.calendar.BaseCalendar.getCalendarDateFromFixedDate(BaseCalendar.java:453)
> at
> java.util.GregorianCalendar.computeFields(GregorianCalendar.java:2397)
> {code}
> This issue is to address those thread-safety issues/race conditions.
> cc [~jcamachorodriguez] [~abstractdog] [~omalley]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22825) Reduce directory lookup cost for acid tables
[ https://issues.apache.org/jira/browse/HIVE-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated HIVE-22825: Attachment: HIVE-22825.9.patch > Reduce directory lookup cost for acid tables > > > Key: HIVE-22825 > URL: https://issues.apache.org/jira/browse/HIVE-22825 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Rajesh Balamohan >Assignee: Rajesh Balamohan >Priority: Minor > Attachments: HIVE-22825.1.patch, HIVE-22825.2.patch, > HIVE-22825.3.patch, HIVE-22825.4.patch, HIVE-22825.5.patch, > HIVE-22825.6.patch, HIVE-22825.7.patch, HIVE-22825.8.patch, HIVE-22825.9.patch > > > With objectstores, directory lookup costs are expensive. For acid tables, it > would be good to have a directory cache to reduce number of lookup calls. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22763) 0 is accepted in 12-hour format during timestamp cast
[
https://issues.apache.org/jira/browse/HIVE-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043136#comment-17043136
]
David Mollitor commented on HIVE-22763:
---
... also, please add a proper JavaDoc to this method to include the parameters,
and the fact that it may throw an {{IllegalArgumentException}}
> 0 is accepted in 12-hour format during timestamp cast
> -
>
> Key: HIVE-22763
> URL: https://issues.apache.org/jira/browse/HIVE-22763
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.02.patch
>
>
> Having a timestamp string in 12-hour format can be parsed if the hour is 0,
> however, based on the [design
> document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
> it should be rejected.
> h3. How to reproduce
> Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd
> hh12 p.m. ss"){code}
> It shouldn' t be parsed, as the hour component is 0.
> h3. Spec
> ||Pattern||Meaning||Additional details||
> |HH12|Hour of day (1-12)|Same as HH|
> |HH|Hour of day (1-12)|{panel:borderStyle=none}
> - One digit inputs are possible in a string to datetime conversion but needs
> to be surrounded by separators.
> - In a datetime to string conversion one digit hours are prefixed with a zero.
> - Error if provided hour is not between 1 and 12.
> - Displaying an unformatted timestamp in Impala uses the HH24 format
> regardless if it was created using HH12.
> - If no AM/PM provided then defaults to AM.
> - In string to datetime conversion, conflicts with S and
> HH24.{panel:borderStyle=none}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22763) 0 is accepted in 12-hour format during timestamp cast
[
https://issues.apache.org/jira/browse/HIVE-22763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043135#comment-17043135
]
David Mollitor commented on HIVE-22763:
---
[~klcopp] OK. Please add some comments in the code about this and I'll grant a
+1 :)
Also note that since the "exceptions to the rule" perform an early return (or
throw an exception), there does not need to be if/else statement.
{code:java}
// exceptions to the rule
// Needs a comment (but can be left for a different day since it's outside
the scope of this path)
if (token.temporalField == ChronoField.AMPM_OF_DAY) {
return substring.toLowerCase().startsWith("a") ? AM : PM;
}
// Needs a comment explaining why "12" returns 0 and "0" throws an exception
if (token.temporalField == ChronoField.HOUR_OF_AMPM) {
if ("12".equals(substring)) {
return 0;
}
if ("0".equals(substring)) {
throw new IllegalArgumentException("Value of hour of day (hh/hh12) in
input is 0. "
+ "The value should be between 1 and 12.");
}
}
{code}
> 0 is accepted in 12-hour format during timestamp cast
> -
>
> Key: HIVE-22763
> URL: https://issues.apache.org/jira/browse/HIVE-22763
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.01.patch, HIVE-22763.01.patch,
> HIVE-22763.01.patch, HIVE-22763.02.patch
>
>
> Having a timestamp string in 12-hour format can be parsed if the hour is 0,
> however, based on the [design
> document|https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit],
> it should be rejected.
> h3. How to reproduce
> Run {code}select cast("2020-01-01 0 am 00" as timestamp format "-mm-dd
> hh12 p.m. ss"){code}
> It shouldn' t be parsed, as the hour component is 0.
> h3. Spec
> ||Pattern||Meaning||Additional details||
> |HH12|Hour of day (1-12)|Same as HH|
> |HH|Hour of day (1-12)|{panel:borderStyle=none}
> - One digit inputs are possible in a string to datetime conversion but needs
> to be surrounded by separators.
> - In a datetime to string conversion one digit hours are prefixed with a zero.
> - Error if provided hour is not between 1 and 12.
> - Displaying an unformatted timestamp in Impala uses the HH24 format
> regardless if it was created using HH12.
> - If no AM/PM provided then defaults to AM.
> - In string to datetime conversion, conflicts with S and
> HH24.{panel:borderStyle=none}|
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22914) Make Hive Connection ZK Interactions Easier to Troubleshoot
[ https://issues.apache.org/jira/browse/HIVE-22914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Mollitor updated HIVE-22914: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) [~pvary] Thanks for the review! Pushed to master. > Make Hive Connection ZK Interactions Easier to Troubleshoot > --- > > Key: HIVE-22914 > URL: https://issues.apache.org/jira/browse/HIVE-22914 > Project: Hive > Issue Type: Improvement >Affects Versions: 4.0.0, 3.1.2 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Minor > Fix For: 4.0.0 > > Attachments: HIVE-22914.1.patch, HIVE-22914.1.patch > > > Add better logging and make errors more consistent and meaningful. > Recently was trying to troubleshoot an issue where the ZK namespace of the > client and the HS2 were different and it was way too difficult to diagnose. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-21240) JSON SerDe Re-Write
[ https://issues.apache.org/jira/browse/HIVE-21240?focusedWorklogId=391469&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391469 ] ASF GitHub Bot logged work on HIVE-21240: - Author: ASF GitHub Bot Created on: 24/Feb/20 02:12 Start Date: 24/Feb/20 02:12 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #530: HIVE-21240: JSON SerDe Deserialize Re-Write URL: https://github.com/apache/hive/pull/530 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 391469) Time Spent: 2h 20m (was: 2h 10m) > JSON SerDe Re-Write > --- > > Key: HIVE-21240 > URL: https://issues.apache.org/jira/browse/HIVE-21240 > Project: Hive > Issue Type: Improvement > Components: Serializers/Deserializers >Affects Versions: 4.0.0, 3.1.1 >Reporter: David Mollitor >Assignee: David Mollitor >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Attachments: HIVE-21240.1.patch, HIVE-21240.1.patch, > HIVE-21240.10.patch, HIVE-21240.11.patch, HIVE-21240.12.patch, > HIVE-21240.12.patch, HIVE-21240.12.patch, HIVE-21240.12.patch, > HIVE-21240.12.patch, HIVE-21240.2.patch, HIVE-21240.3.patch, > HIVE-21240.4.patch, HIVE-21240.5.patch, HIVE-21240.6.patch, > HIVE-21240.7.patch, HIVE-21240.9.patch, HIVE-24240.8.patch, > kafka_storage_handler.diff > > Time Spent: 2h 20m > Remaining Estimate: 0h > > The JSON SerDe has a few issues, I will link them to this JIRA. > * Use Jackson Tree parser instead of manually parsing > * Added support for base-64 encoded data (the expected format when using JSON) > * Added support to skip blank lines (returns all columns as null values) > * Current JSON parser accepts, but does not apply, custom timestamp formats > in most cases > * Added some unit tests > * Added cache for column-name to column-index searches, currently O\(n\) for > each row processed, for each column in the row -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22920) Add row format OpenCSVSerde to the metastore column managed list
[ https://issues.apache.org/jira/browse/HIVE-22920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043127#comment-17043127 ] Ashutosh Chauhan commented on HIVE-22920: - +1 > Add row format OpenCSVSerde to the metastore column managed list > > > Key: HIVE-22920 > URL: https://issues.apache.org/jira/browse/HIVE-22920 > Project: Hive > Issue Type: Bug >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22920.1.patch, HIVE-22920.2.patch > > > Add row format OpenCSVSerde to the metastore column managed list -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22912) Support native submission of Hive queries to a Kubernetes Cluster
[ https://issues.apache.org/jira/browse/HIVE-22912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043126#comment-17043126 ] Surbhi Aggarwal commented on HIVE-22912: And what about the Kubernetes support with Hive released as an open-source project > Support native submission of Hive queries to a Kubernetes Cluster > - > > Key: HIVE-22912 > URL: https://issues.apache.org/jira/browse/HIVE-22912 > Project: Hive > Issue Type: New Feature >Reporter: Surbhi Aggarwal >Priority: Major > > So many big data applications are already integrated or trying to natively > integrate with Kubernetes engine. Should we not work together to support hive > with this engine? > If efforts are already being spent on this, please point me to it. Thanks ! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22888) Rewrite checkLock inner select with JOIN operator
[
https://issues.apache.org/jira/browse/HIVE-22888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043122#comment-17043122
]
Hive QA commented on HIVE-22888:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994255/HIVE-22888.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 17907 tests
executed
*Failed tests:*
{noformat}
TestCommandProcessorFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=361)
TestDbTxnManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestDummyTxnManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestEmbeddedLockManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestGetInputSummary - did not produce a TEST-*.xml file (likely timed out)
(batchId=342)
TestHiveCredentialProviders - did not produce a TEST-*.xml file (likely timed
out) (batchId=342)
TestHiveInputSplitComparator - did not produce a TEST-*.xml file (likely timed
out) (batchId=361)
TestHiveLockObject - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestKeyWrapperFactory - did not produce a TEST-*.xml file (likely timed out)
(batchId=342)
TestLog4j2Appenders - did not produce a TEST-*.xml file (likely timed out)
(batchId=342)
TestResetProcessor - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestSetProcessor - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestSlidingFilenameRolloverStrategy - did not produce a TEST-*.xml file (likely
timed out) (batchId=342)
TestSyslogInputFormat - did not produce a TEST-*.xml file (likely timed out)
(batchId=342)
TestTxnCommands - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
TestTxnCommandsWithSplitUpdateAndVectorization - did not produce a TEST-*.xml
file (likely timed out) (batchId=342)
TestZookeeperLockManager - did not produce a TEST-*.xml file (likely timed out)
(batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20794/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20794/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20794/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12994255 - PreCommit-HIVE-Build
> Rewrite checkLock inner select with JOIN operator
> -
>
> Key: HIVE-22888
> URL: https://issues.apache.org/jira/browse/HIVE-22888
> Project: Hive
> Issue Type: Improvement
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22888.1.patch, HIVE-22888.2.patch,
> HIVE-22888.3.patch, HIVE-22888.4.patch, HIVE-22888.5.patch, HIVE-22888.6.patch
>
>
> - Created extra (db, tbl, part) index on HIVE_LOCKS table;
> - Replaced inner select under checkLocks using multiple IN statements with
> JOIN operator;
> generated query looks like :
> {code}
> SELECT LS.* FROM (
> SELECT HL_LOCK_EXT_ID, HL_DB, HL_TABLE, HL_PARTITION, HL_LOCK_STATE,
> HL_LOCK_TYPE FROM HIVE_LOCKS
> WHERE HL_LOCK_EXT_ID < 333) LS
> INNER JOIN (
> SELECT HL_DB, HL_TABLE, HL_PARTITION, HL_LOCK_TYPE FROM HIVE_LOCKS WHERE
> HL_LOCK_EXT_ID = 333) LBC
> ON LS.HL_DB = LBC.HL_DB
> AND (LS.HL_TABLE IS NULL OR LBC.HL_TABLE IS NULL OR LS.HL_TABLE =
> LBC.HL_TABLE
> AND (LS.HL_PARTITION IS NULL OR LBC.HL_PARTITION IS NULL OR
> LS.HL_PARTITION = LBC.HL_PARTITION))
> WHERE (LBC.HL_LOCK_TYPE='e'
>AND NOT (LS.HL_TABLE IS NULL AND LS.HL_LOCK_TYPE='r' AND
> LBC.HL_TABLE IS NOT NULL )
> OR LBC.HL_LOCK_TYPE='w' AND LS.HL_LOCK_TYPE IN ('w','e')
> OR LBC.HL_LOCK_TYPE='r' AND LS.HL_LOCK_TYPE='e'
>AND NOT (LS.HL_TABLE IS NOT NULL AND LBC.HL_TABLE IS NULL))
> LIMIT 1;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22699) Mask UDFs should mask numeric value 0
[
https://issues.apache.org/jira/browse/HIVE-22699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043112#comment-17043112
]
Quanlong Huang commented on HIVE-22699:
---
[~madhan] could you help to review this?
> Mask UDFs should mask numeric value 0
> -
>
> Key: HIVE-22699
> URL: https://issues.apache.org/jira/browse/HIVE-22699
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Assignee: Quanlong Huang
>Priority: Major
> Attachments: HIVE-22699.1.patch, HIVE-22699.2.patch
>
>
> These queries all return 0:
> {code:java}
> select mask(0);
> select mask_first_n(0);
> select mask_last_n(0);
> select mask_show_first_n(0, 0);
> select mask_show_last_n(0, 0);{code}
> According to the doc, they should return 1.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HIVE-21187) OptimizedSql is not shown when the expression contains BETWEENs
[ https://issues.apache.org/jira/browse/HIVE-21187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved HIVE-21187. Fix Version/s: 4.0.0 Resolution: Fixed > OptimizedSql is not shown when the expression contains BETWEENs > --- > > Key: HIVE-21187 > URL: https://issues.apache.org/jira/browse/HIVE-21187 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 4.0.0 > > > in the patch for HIVE-21143 we see that a lot of optimized sql printouts are > going away because of this -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22711) yearValue of UDF mask() should not start from 1900
[
https://issues.apache.org/jira/browse/HIVE-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043110#comment-17043110
]
Madhan Neethiraj commented on HIVE-22711:
-
[~stigahuang] - thanks for the details. I agree that it will be
intuitive/helpful if mask() simply uses the given year value, instead of adding
to 1900. This issue is caused by use of constructor {{Date(year, month, day)}},
which adds 1900 to the given year value. The fix will be to replace this call
with {{Calendar.set(year, month, day, 0, 0)}}
> yearValue of UDF mask() should not start from 1900
> --
>
> Key: HIVE-22711
> URL: https://issues.apache.org/jira/browse/HIVE-22711
> Project: Hive
> Issue Type: Bug
>Reporter: Quanlong Huang
>Priority: Major
>
> Here's the description of the UDF mask():
> {code:java}
> masks the given value
> Examples:
>mask(ccn)
>mask(ccn, 'X', 'x', '0')
>mask(ccn, 'x', 'x', 'x')
> Arguments:
>mask(value, upperChar, lowerChar, digitChar, otherChar, numberChar,
> dayValue, monthValue, yearValue)
> value - value to mask. Supported types: TINYINT, SMALLINT, INT,
> BIGINT, STRING, VARCHAR, CHAR, DATE
> upperChar - character to replace upper-case characters with. Specify -1
> to retain original character. Default value: 'X'
> lowerChar - character to replace lower-case characters with. Specify -1
> to retain original character. Default value: 'x'
> digitChar - character to replace digit characters with. Specify -1 to
> retain original character. Default value: 'n'
> otherChar - character to replace all other characters with. Specify -1
> to retain original character. Default value: -1
> numberChar - character to replace digits in a number with. Valid values:
> 0-9. Default value: '1'
> dayValue - value to replace day field in a date with. Specify -1 to
> retain original value. Valid values: 1-31. Default value: 1
> monthValue - value to replace month field in a date with. Specify -1 to
> retain original value. Valid values: 0-11. Default value: 0
> yearValue - value to replace year field in a date with. Specify -1 to
> retain original value. Default value: 0 {code}
> Although it says 'yearValue' is the value to replace year field in a DATE
> with, it actually counts start at 1900. E.g. yearValue = 0 means masking the
> year field to 1900, yearValue=2000 means masking the year field to 3900,
> yearValue=-2 means masking the year field to 1988.
> Here are some query examples:
> {code:sql}
> beeline> select mask(cast('2019-02-03' as date), -1, -1, -1, -1, -1, -1, -1,
> 0);
> 1900-02-03
> beeline> select mask(cast('2019-02-03' as date), -1, -1, -1, -1, -1, -1, -1,
> 2000);
> 3900-02-03
> beeline> select mask(cast('2019-02-03' as date), -1, -1, -1, -1, -1, -1, -1,
> -2);
> 1898-02-03
> beeline> select mask(cast('2019-02-03' as date), -1, -1, -1, -1, -1, -1, -1,
> -100);
> 1800-02-03
> {code}
> The drawback of this behavior is that we can't mask year field to be 1899,
> since -1 already means retaining the original value.
> It'd be better to change the behavior to be intuitive that simply masking
> year filed to yearValue. And only accept yearValue from 0 to . Still use
> -1 to retain original value.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22888) Rewrite checkLock inner select with JOIN operator
[
https://issues.apache.org/jira/browse/HIVE-22888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043103#comment-17043103
]
Hive QA commented on HIVE-22888:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
19s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 21 new + 565 unchanged - 38 fixed = 586 total (was 603) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
26s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 184 unchanged - 1 fixed = 185 total (was 185) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(Connection,
long, long) passes a nonconstant String to an execute or addBatch method on an
SQL statement At TxnHandler.java:nonconstant String to an execute or addBatch
method on an SQL statement At TxnHandler.java:[line 4428] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20794/dev-support/hive-personality.sh
|
| git revision | master / 6c3ee53 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20794/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20794/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20794/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20794/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Rewrite checkLock inner select with JOIN operator
> -
>
> Key: HIVE-22888
> URL: https://issues.apache.org/jira/browse/HIVE-22888
> Project: Hive
> Issue Type: Improvement
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22888.1.patch, HIVE-22888.2.patch,
> HIVE-22888.3.patch, HIVE-22888.4.patch, HIVE-22888.5.patch, HIVE-22888.6.patch
>
>
> - Created extra (db, tbl, part) index on HIVE_LOCKS table;
> - Replaced inner select under checkLocks using multiple I
[jira] [Updated] (HIVE-22888) Rewrite checkLock inner select with JOIN operator
[
https://issues.apache.org/jira/browse/HIVE-22888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22888:
--
Attachment: HIVE-22888.6.patch
> Rewrite checkLock inner select with JOIN operator
> -
>
> Key: HIVE-22888
> URL: https://issues.apache.org/jira/browse/HIVE-22888
> Project: Hive
> Issue Type: Improvement
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22888.1.patch, HIVE-22888.2.patch,
> HIVE-22888.3.patch, HIVE-22888.4.patch, HIVE-22888.5.patch, HIVE-22888.6.patch
>
>
> - Created extra (db, tbl, part) index on HIVE_LOCKS table;
> - Replaced inner select under checkLocks using multiple IN statements with
> JOIN operator;
> generated query looks like :
> {code}
> SELECT LS.* FROM (
> SELECT HL_LOCK_EXT_ID, HL_DB, HL_TABLE, HL_PARTITION, HL_LOCK_STATE,
> HL_LOCK_TYPE FROM HIVE_LOCKS
> WHERE HL_LOCK_EXT_ID < 333) LS
> INNER JOIN (
> SELECT HL_DB, HL_TABLE, HL_PARTITION, HL_LOCK_TYPE FROM HIVE_LOCKS WHERE
> HL_LOCK_EXT_ID = 333) LBC
> ON LS.HL_DB = LBC.HL_DB
> AND (LS.HL_TABLE IS NULL OR LBC.HL_TABLE IS NULL OR LS.HL_TABLE =
> LBC.HL_TABLE
> AND (LS.HL_PARTITION IS NULL OR LBC.HL_PARTITION IS NULL OR
> LS.HL_PARTITION = LBC.HL_PARTITION))
> WHERE (LBC.HL_LOCK_TYPE='e'
>AND NOT (LS.HL_TABLE IS NULL AND LS.HL_LOCK_TYPE='r' AND
> LBC.HL_TABLE IS NOT NULL )
> OR LBC.HL_LOCK_TYPE='w' AND LS.HL_LOCK_TYPE IN ('w','e')
> OR LBC.HL_LOCK_TYPE='r' AND LS.HL_LOCK_TYPE='e'
>AND NOT (LS.HL_TABLE IS NOT NULL AND LBC.HL_TABLE IS NULL))
> LIMIT 1;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22888) Rewrite checkLock inner select with JOIN operator
[
https://issues.apache.org/jira/browse/HIVE-22888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043071#comment-17043071
]
Hive QA commented on HIVE-22888:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994254/HIVE-22888.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 45 failed/errored test(s), 18056 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[cbo_query31]
(batchId=305)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.showLocks (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testCheckLockAcquireAfterWaiting
(batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockDbLocksTable
(batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockEESR (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockEESW (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockESRE (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockESRSR (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockESRSW (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSRE (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSWSWSR (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSWSWSW (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSameDB (batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSamePartition
(batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockSameTable
(batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testLockTableLocksPartition
(batchId=316)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testMultipleLockWait
(batchId=316)
org.apache.hadoop.hive.ql.TestTxnCommands.testMultipleDelete (batchId=361)
org.apache.hadoop.hive.ql.TestTxnCommands.testUpdateDeleteOfInserts
(batchId=361)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMultipleDelete
(batchId=342)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testParallelTruncateAnalyzeStats
(batchId=342)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testUpdateDeleteOfInserts
(batchId=342)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.basicBlocking (batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.checkExpectedLocks
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.checkExpectedLocks2
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.checkExpectedReadLocksForNonAcidTables
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.lockConflictDbTable
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testFairness (batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testFairness2 (batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLockBlockedBy
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testLockRetryLimit
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMerge3Way01
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMerge3Way02
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergePartitioned01
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergePartitioned02
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMergeUnpartitioned01
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testMetadataOperationLocks
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking10
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking11
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking3
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking5
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking7
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking8
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking9
(batchId=360)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.updateSelectUpdate
(batchId=360)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20793/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20793/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20793/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Execut
[jira] [Commented] (HIVE-22888) Rewrite checkLock inner select with JOIN operator
[
https://issues.apache.org/jira/browse/HIVE-22888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043059#comment-17043059
]
Hive QA commented on HIVE-22888:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
20s{color} | {color:blue} standalone-metastore/metastore-server in master has
185 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
24s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 23 new + 564 unchanged - 39 fixed = 587 total (was 603) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
26s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 184 unchanged - 1 fixed = 185 total (was 185) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 15s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | org.apache.hadoop.hive.metastore.txn.TxnHandler.checkLock(Connection,
long, long) passes a nonconstant String to an execute or addBatch method on an
SQL statement At TxnHandler.java:nonconstant String to an execute or addBatch
method on an SQL statement At TxnHandler.java:[line 4430] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20793/dev-support/hive-personality.sh
|
| git revision | master / 6c3ee53 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20793/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20793/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20793/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20793/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Rewrite checkLock inner select with JOIN operator
> -
>
> Key: HIVE-22888
> URL: https://issues.apache.org/jira/browse/HIVE-22888
> Project: Hive
> Issue Type: Improvement
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22888.1.patch, HIVE-22888.2.patch,
> HIVE-22888.3.patch, HIVE-22888.4.patch, HIVE-22888.5.patch
>
>
> - Created extra (db, tbl, part) index on HIVE_LOCKS table;
> - Replaced inner select under checkLocks using multiple IN statements with
>
[jira] [Updated] (HIVE-22888) Rewrite checkLock inner select with JOIN operator
[
https://issues.apache.org/jira/browse/HIVE-22888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-22888:
--
Attachment: HIVE-22888.5.patch
> Rewrite checkLock inner select with JOIN operator
> -
>
> Key: HIVE-22888
> URL: https://issues.apache.org/jira/browse/HIVE-22888
> Project: Hive
> Issue Type: Improvement
> Components: Locking
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22888.1.patch, HIVE-22888.2.patch,
> HIVE-22888.3.patch, HIVE-22888.4.patch, HIVE-22888.5.patch
>
>
> - Created extra (db, tbl, part) index on HIVE_LOCKS table;
> - Replaced inner select under checkLocks using multiple IN statements with
> JOIN operator;
> generated query looks like :
> {code}
> SELECT LS.* FROM (
> SELECT HL_LOCK_EXT_ID, HL_DB, HL_TABLE, HL_PARTITION, HL_LOCK_STATE,
> HL_LOCK_TYPE FROM HIVE_LOCKS
> WHERE HL_LOCK_EXT_ID < 333) LS
> INNER JOIN (
> SELECT HL_DB, HL_TABLE, HL_PARTITION, HL_LOCK_TYPE FROM HIVE_LOCKS WHERE
> HL_LOCK_EXT_ID = 333) LBC
> ON LS.HL_DB = LBC.HL_DB
> AND (LS.HL_TABLE IS NULL OR LBC.HL_TABLE IS NULL OR LS.HL_TABLE =
> LBC.HL_TABLE
> AND (LS.HL_PARTITION IS NULL OR LBC.HL_PARTITION IS NULL OR
> LS.HL_PARTITION = LBC.HL_PARTITION))
> WHERE (LBC.HL_LOCK_TYPE='e'
>AND NOT (LS.HL_TABLE IS NULL AND LS.HL_LOCK_TYPE='r' AND
> LBC.HL_TABLE IS NOT NULL )
> OR LBC.HL_LOCK_TYPE='w' AND LS.HL_LOCK_TYPE IN ('w','e')
> OR LBC.HL_LOCK_TYPE='r' AND LS.HL_LOCK_TYPE='e'
>AND NOT (LS.HL_TABLE IS NOT NULL AND LBC.HL_TABLE IS NULL))
> LIMIT 1;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20359) Update protobuf version in pom.xml
[
https://issues.apache.org/jira/browse/HIVE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043013#comment-17043013
]
Hive QA commented on HIVE-20359:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994249/HIVE-20359.01.branch-3.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 144 failed/errored test(s), 14405 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=226)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestChainFilter - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestCloseableThreadLocal - did not produce a TEST-*.xml file (likely timed out)
(batchId=333)
TestCustomQueryFilter - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=237)
TestDatabaseName - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDateColumnVector - did not produce a TEST-*.xml file (likely timed out)
(batchId=195)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=274)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=229)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=230)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestGroupFilter - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=235)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=237)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=228)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=235)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=237)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=232)
TestHiveMetaToolCommandLine - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=226)
TestHmsServerAuthorization - did not produce a TEST-*.xml file (likely timed
out) (batchId=235)
TestHyperLogLog - did not p
[jira] [Commented] (HIVE-22923) Extract cumulative cost metadata from HiveRelMdDistinctRowCount metadata provider
[ https://issues.apache.org/jira/browse/HIVE-22923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043000#comment-17043000 ] Miklos Gergely commented on HIVE-22923: --- +1 > Extract cumulative cost metadata from HiveRelMdDistinctRowCount metadata > provider > -- > > Key: HIVE-22923 > URL: https://issues.apache.org/jira/browse/HIVE-22923 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22923.patch > > > It should not contained there. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-20359) Update protobuf version in pom.xml
[
https://issues.apache.org/jira/browse/HIVE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042996#comment-17042996
]
Hive QA commented on HIVE-20359:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 20s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-20792/patches/PreCommit-HIVE-Build-20792.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20792/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Update protobuf version in pom.xml
> --
>
> Key: HIVE-20359
> URL: https://issues.apache.org/jira/browse/HIVE-20359
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Affects Versions: 3.1.0
> Environment: ARM64 Server machines with CentOS/Debian OS.
>Reporter: Naresh Bhat
>Assignee: David Lavati
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-20359.01.branch-3.1.patch,
> HIVE-20359.01.branch-3.1.patch, HIVE-20359.01.patch, HIVE-20359.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The following protoc version's used in hive is very old i.e. 2.5.0
> [https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/] . The
> v2.5.0 does not have aarch64 support. But the AArch64 support started from
> v3.5.0 on-words in google's protobuf project release. The hive release
> version branch v3/3.0/3.1/master branch build fails on AArch64 because of old
> protoc version 2.5.0, in which there is no AArch64 support/exe available.
> After I replaced to latest available protoc version 3.6.1 the hive master
> branch pom.xml files, The hive compiled without any issues on ARM64 machine.
> This Jira ticket is used to propose a patch on Hive mailing list
> [https://git.linaro.org/people/naresh.bhat/apache/hive.git/] to update
> pom.xml files protobuf version. I will also cherry-pick patches for other
> branches v3/3.0/3.1, and create a pull request.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-20359) Update protobuf version in pom.xml
[ https://issues.apache.org/jira/browse/HIVE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Lavati updated HIVE-20359: Attachment: HIVE-20359.01.branch-3.1.patch HIVE-20359.01.patch Status: Patch Available (was: In Progress) > Update protobuf version in pom.xml > -- > > Key: HIVE-20359 > URL: https://issues.apache.org/jira/browse/HIVE-20359 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 > Environment: ARM64 Server machines with CentOS/Debian OS. >Reporter: Naresh Bhat >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0, 3.1.0 > > Attachments: HIVE-20359.01.branch-3.1.patch, > HIVE-20359.01.branch-3.1.patch, HIVE-20359.01.patch, HIVE-20359.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The following protoc version's used in hive is very old i.e. 2.5.0 > [https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/] . The > v2.5.0 does not have aarch64 support. But the AArch64 support started from > v3.5.0 on-words in google's protobuf project release. The hive release > version branch v3/3.0/3.1/master branch build fails on AArch64 because of old > protoc version 2.5.0, in which there is no AArch64 support/exe available. > After I replaced to latest available protoc version 3.6.1 the hive master > branch pom.xml files, The hive compiled without any issues on ARM64 machine. > This Jira ticket is used to propose a patch on Hive mailing list > [https://git.linaro.org/people/naresh.bhat/apache/hive.git/] to update > pom.xml files protobuf version. I will also cherry-pick patches for other > branches v3/3.0/3.1, and create a pull request. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21304) Show Bucketing version for ReduceSinkOp in explain extended plan
[
https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042955#comment-17042955
]
Hive QA commented on HIVE-21304:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12994244/HIVE-21304.17.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 18056 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[reduce_deduplicate]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket2]
(batchId=180)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket4]
(batchId=188)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[disable_merge_for_bucketing]
(batchId=188)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/20791/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20791/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20791/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12994244 - PreCommit-HIVE-Build
> Show Bucketing version for ReduceSinkOp in explain extended plan
>
>
> Key: HIVE-21304
> URL: https://issues.apache.org/jira/browse/HIVE-21304
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21304.01.patch, HIVE-21304.02.patch,
> HIVE-21304.03.patch, HIVE-21304.04.patch, HIVE-21304.05.patch,
> HIVE-21304.06.patch, HIVE-21304.07.patch, HIVE-21304.08.patch,
> HIVE-21304.09.patch, HIVE-21304.10.patch, HIVE-21304.11.patch,
> HIVE-21304.12.patch, HIVE-21304.13.patch, HIVE-21304.14.patch,
> HIVE-21304.15.patch, HIVE-21304.16.patch, HIVE-21304.17.patch
>
>
> Show Bucketing version for ReduceSinkOp in explain extended plan.
> This helps identify what hashing algorithm is being used by by ReduceSinkOp.
>
> cc [~vgarg]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-21304) Show Bucketing version for ReduceSinkOp in explain extended plan
[
https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042939#comment-17042939
]
Hive QA commented on HIVE-21304:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
5s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
31s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
55s{color} | {color:red} ql: The patch generated 2 new + 891 unchanged - 6
fixed = 893 total (was 897) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
15s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20791/dev-support/hive-personality.sh
|
| git revision | master / 6c3ee53 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20791/yetus/diff-checkstyle-ql.txt
|
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20791/yetus/patch-asflicense-problems.txt
|
| modules | C: ql itests/hive-blobstore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-20791/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Show Bucketing version for ReduceSinkOp in explain extended plan
>
>
> Key: HIVE-21304
> URL: https://issues.apache.org/jira/browse/HIVE-21304
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21304.01.patch, HIVE-21304.02.patch,
> HIVE-21304.03.patch, HIVE-21304.04.patch, HIVE-21304.05.patch,
> HIVE-21304.06.patch, HIVE-21304.07.patch, HIVE-21304.08.patch,
> HIVE-21304.09.patch, HIVE-21304.10.patch, HIVE-21304.11.patch,
> HIVE-21304.12.patch, HIVE-21304.13.patch, HIVE-21304.14.patch,
> HIVE-21304.15.patch, HIVE-21304.16.patch, HIVE-21304.17.patch
>
>
> Show Bucketing version for ReduceSinkOp in explain extended plan.
> This helps identify what hashing algorithm is being used by by ReduceSinkOp.
>
> cc [~vgarg]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-20359) Update protobuf version in pom.xml
[ https://issues.apache.org/jira/browse/HIVE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17042933#comment-17042933 ] Naresh Bhat commented on HIVE-20359: [~dlavati] Thank you very much. I will check the fix on ARM64. > Update protobuf version in pom.xml > -- > > Key: HIVE-20359 > URL: https://issues.apache.org/jira/browse/HIVE-20359 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0 > Environment: ARM64 Server machines with CentOS/Debian OS. >Reporter: Naresh Bhat >Assignee: David Lavati >Priority: Major > Labels: pull-request-available > Fix For: 3.1.0, 4.0.0 > > Attachments: HIVE-20359.01.branch-3.1.patch, HIVE-20359.01.patch > > Time Spent: 10m > Remaining Estimate: 0h > > The following protoc version's used in hive is very old i.e. 2.5.0 > [https://repo.maven.apache.org/maven2/com/google/protobuf/protoc/] . The > v2.5.0 does not have aarch64 support. But the AArch64 support started from > v3.5.0 on-words in google's protobuf project release. The hive release > version branch v3/3.0/3.1/master branch build fails on AArch64 because of old > protoc version 2.5.0, in which there is no AArch64 support/exe available. > After I replaced to latest available protoc version 3.6.1 the hive master > branch pom.xml files, The hive compiled without any issues on ARM64 machine. > This Jira ticket is used to propose a patch on Hive mailing list > [https://git.linaro.org/people/naresh.bhat/apache/hive.git/] to update > pom.xml files protobuf version. I will also cherry-pick patches for other > branches v3/3.0/3.1, and create a pull request. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21304) Show Bucketing version for ReduceSinkOp in explain extended plan
[ https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21304: Attachment: HIVE-21304.17.patch > Show Bucketing version for ReduceSinkOp in explain extended plan > > > Key: HIVE-21304 > URL: https://issues.apache.org/jira/browse/HIVE-21304 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21304.01.patch, HIVE-21304.02.patch, > HIVE-21304.03.patch, HIVE-21304.04.patch, HIVE-21304.05.patch, > HIVE-21304.06.patch, HIVE-21304.07.patch, HIVE-21304.08.patch, > HIVE-21304.09.patch, HIVE-21304.10.patch, HIVE-21304.11.patch, > HIVE-21304.12.patch, HIVE-21304.13.patch, HIVE-21304.14.patch, > HIVE-21304.15.patch, HIVE-21304.16.patch, HIVE-21304.17.patch > > > Show Bucketing version for ReduceSinkOp in explain extended plan. > This helps identify what hashing algorithm is being used by by ReduceSinkOp. > > cc [~vgarg] -- This message was sent by Atlassian Jira (v8.3.4#803005)
