[jira] [Commented] (HIVE-18866) Semijoin: Implement a Long -> Hash64 vector fast-path
[
https://issues.apache.org/jira/browse/HIVE-18866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483559#comment-16483559
]
Hive QA commented on HIVE-18866:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} storage-api in master has 48 extant Findbugs
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 13m
42s{color} | {color:red} branch/itests/hive-jmh cannot run convertXmlToText
from findbugs {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
51s{color} | {color:blue} standalone-metastore in master has 216 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} storage-api: The patch generated 4 new + 109 unchanged
- 0 fixed = 113 total (was 109) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
10s{color} | {color:red} itests/hive-jmh: The patch generated 8 new + 0
unchanged - 0 fixed = 8 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} standalone-metastore: The patch generated 1 new + 13
unchanged - 3 fixed = 14 total (was 16) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 10m
0s{color} | {color:red} patch/itests/hive-jmh cannot run convertXmlToText from
findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11129/dev-support/hive-personality.sh
|
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11129/yetus/branch-findbugs-itests_hive-jmh.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11129/yetus/diff-checkstyle-storage-api.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11129/yetus/diff-checkstyle-itests_hive-jmh.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11129/yetus/diff-checkstyle-standalone-metastore.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11129/y
[jira] [Commented] (HIVE-18079) Statistics: Allow HyperLogLog to be merged to the lowest-common-denominator bit-size
[
https://issues.apache.org/jira/browse/HIVE-18079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483520#comment-16483520
]
Hive QA commented on HIVE-18079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924388/HIVE-18079.18.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11128/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11128/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11128/
Messages:
{noformat}
This message was trimmed, see log for full details
error:
a/ql/src/test/results/clientpositive/encrypted/encryption_move_tbl.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/hll.q.out: does not exist in index
error: a/ql/src/test/results/clientpositive/llap/acid_no_buckets.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/autoColumnStats_2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/auto_join1.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/auto_join21.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/auto_join29.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/auto_join30.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/auto_sortmerge_join_6.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_groupby.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_map_join_tez1.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/check_constraint.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/correlationoptimizer2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/correlationoptimizer3.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/correlationoptimizer6.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/cross_join.q.out: does not
exist in index
error:
a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_reduction_sw.q.out:
does not exist in index
error:
a/ql/src/test/results/clientpositive/llap/dynamic_semijoin_user_level.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/explainanalyze_2.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_1.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_2.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/explainuser_4.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/groupby1.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/llap/groupby2.q.out: does not exist
in index
error: a/ql/src/test/results/clientpositive/llap/groupby_resolution.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_2.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/join1.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/llap/join32_lessSize.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/join_max_hashtable.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/limit_join_transpose.q.out:
does not exist in index
error: a/ql/src/test/results/clientpositive/llap/limit_pushdown.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/limit_pushdown3.q.out: does
not exist in index
error:
a/ql/src/test/results/clientpositive/llap/llap_vector_nohybridgrace.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/llapdecider.q.out: does not
exist in index
error: a/ql/src/test/results/clientpositive/llap/mapjoin_decimal.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/mapjoin_mapjoin.q.out: does
not exist in index
error: a/ql/src/test/results/clientpositive/llap/mrr.q.out: does not exist in
index
error: a/ql/src/test/results/clientpositive/llap/multiMapJoin2.q.out: does not
exist in index
error:
a/ql/src/test/results/clientpositive/llap/offset_limit_ppd_optimizer.q.out:
does not exist in index
error: a/ql/src/test
[jira] [Commented] (HIVE-19628) possible NPE in LLAP testSigning
[
https://issues.apache.org/jira/browse/HIVE-19628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483519#comment-16483519
]
Hive QA commented on HIVE-19628:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924387/HIVE-19628.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14390 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11127/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11127/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11127/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924387 - PreCommit-HIVE-Build
> possible NPE in LLAP testSigning
>
>
> Key: HIVE-19628
> URL: https://issues.apache.org/jira/browse/HIVE-19628
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19628.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19454) Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError
[
https://issues.apache.org/jira/browse/HIVE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483505#comment-16483505
]
Deepak Jaiswal commented on HIVE-19454:
---
Committed to master.
Thanks for the patch [~alishap]
> Test failure :
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails
> with java.lang.AssertionError
> --
>
> Key: HIVE-19454
> URL: https://issues.apache.org/jira/browse/HIVE-19454
> Project: Hive
> Issue Type: Bug
> Environment: uname -a
> Linux pts00433-vm40 3.10.0-862.el7.ppc64le #1 SMP Wed Mar 21 22:20:19 UTC
> 2018 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Attachments: HIVE-19454.1.patch, HIVE-19454.2.patch
>
>
> org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
> refers to the same test case.
> Maven command used is : mvn -Dtest=TestTxnCommands2 test
> Error:
> {code:java}
> [INFO] Running org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3, Time elapsed:
> 618.215 s <<< FAILURE! - in org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR]
> testNonAcidToAcidConversion1(org.apache.hadoop.hive.ql.TestTxnCommands2)
> Time elapsed: 17.557 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1(TestTxnCommands2.java:499)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] TestTxnCommands2.testNonAcidToAcidConversion1:499
> [INFO]
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19604) Incorrect Handling of Boolean in DruidSerde
[ https://issues.apache.org/jira/browse/HIVE-19604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-19604: --- Fix Version/s: (was: 3.0.0) > Incorrect Handling of Boolean in DruidSerde > --- > > Key: HIVE-19604 > URL: https://issues.apache.org/jira/browse/HIVE-19604 > Project: Hive > Issue Type: Bug > Components: Druid integration >Reporter: Nishant Bangarwa >Assignee: Nishant Bangarwa >Priority: Major > Attachments: HIVE-19604.patch > > > Results of boolean expressions from Druid are expressed in the form of > numeric 1 or 0. > When reading the results in DruidSerde both 1 and 0 are translated to String > and then we call Boolean.valueOf(stringForm), this leads to the boolean being > read always as false. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19628) possible NPE in LLAP testSigning
[
https://issues.apache.org/jira/browse/HIVE-19628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483494#comment-16483494
]
Hive QA commented on HIVE-19628:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} llap-server in master has 86 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11127/dev-support/hive-personality.sh
|
| git revision | master / 8fc0f72 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11127/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> possible NPE in LLAP testSigning
>
>
> Key: HIVE-19628
> URL: https://issues.apache.org/jira/browse/HIVE-19628
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19628.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19579) remove HBase transitive dependency that drags in some snapshot
[
https://issues.apache.org/jira/browse/HIVE-19579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483436#comment-16483436
]
Hive QA commented on HIVE-19579:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924383/HIVE-19579.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14390 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11120/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11120/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11120/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924383 - PreCommit-HIVE-Build
> remove HBase transitive dependency that drags in some snapshot
> --
>
> Key: HIVE-19579
> URL: https://issues.apache.org/jira/browse/HIVE-19579
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19579.01.patch, HIVE-19579.02.patch,
> HIVE-19579.03.patch, HIVE-19579.04.patch, HIVE-19579.patch
>
>
> This:
> {noformat}
> [INFO] org.apache.hive:hive-llap-server:jar:3.1.0-SNAPSHOT
> [INFO] \- org.apache.hbase:hbase-server:jar:2.0.0-alpha4:compile
> [INFO]\- org.glassfish.web:javax.servlet.jsp:jar:2.3.2:compile
> [INFO] \- org.glassfish:javax.el:jar:3.0.1-b10:compile (version
> selected from constraint [3.0.0,))
> {noformat}
> occasionally results in bizarre build failures like this:
> {noformat}
> Failed to read artifact descriptor for
> org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Could not transfer artifact
> org.glassfish:javax.el:pom:3.0.1-b06-SNAPSHOT from/to jvnet-nexus-snapshots
> (https://maven.java.net/content/repositories/snapshots): Failed to transfer
> file:
> https://maven.java.net/content/repositories/snapshots/org/glassfish/javax.el/3.0.1-b06-SNAPSHOT/javax.el-3.0.1-b06-SNAPSHOT.pom.
> Return code is: 402 , ReasonPhrase:Payment Required.
> {noformat}
> cc [~enis] [~stack] if you guys want to take a look at HBase side
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19499) Bootstrap REPL LOAD shall add tasks to create checkpoints for db/tables/partitions.
[ https://issues.apache.org/jira/browse/HIVE-19499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483487#comment-16483487 ] Sankar Hariappan edited comment on HIVE-19499 at 5/22/18 4:40 AM: -- I got a green build for the pre-commit test on 01.patch. But not sure why the test results are not commented here. https://builds.apache.org/job/PreCommit-HIVE-Build/6/ [https://builds.apache.org/job/PreCommit-HIVE-Build/6/testReport/] [~maheshk114], [~thejas], could you please take a look at the patch. was (Author: sankarh): I got a green build for the pre-commit test on 01.patch. But not sure why the test results are not commented here. [https://builds.apache.org/job/PreCommit-HIVE-Build/6/testReport/] [~maheshk114], [~thejas], could you please take a look at the patch. > Bootstrap REPL LOAD shall add tasks to create checkpoints for > db/tables/partitions. > --- > > Key: HIVE-19499 > URL: https://issues.apache.org/jira/browse/HIVE-19499 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 3.1.0 > > Attachments: HIVE-19499.01.patch > > > Currently. bootstrap REPL LOAD expect the target database to be empty or not > exist to start bootstrap load. > But, this adds overhead when there is a failure in between bootstrap load and > there is no way to resume it from where it fails. So, it is needed to create > checkpoints in table/partitions to skip the completely loaded objects. > Use the fully qualified path of the dump directory as a checkpoint > identifier. This should be added to the table / partition properties in hive > via a task, as the last task in the DAG for table / partition creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19258) add originals support to MM tables (and make the conversion a metadata only operation)
[
https://issues.apache.org/jira/browse/HIVE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483455#comment-16483455
]
Hive QA commented on HIVE-19258:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 62 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
54s{color} | {color:blue} ql in master has 2322 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
29s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
52s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} itests/hive-unit: The patch generated 5 new + 165
unchanged - 0 fixed = 170 total (was 165) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
45s{color} | {color:red} ql: The patch generated 13 new + 888 unchanged - 9
fixed = 901 total (was 897) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11121/dev-support/hive-personality.sh
|
| git revision | master / 8fc0f72 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11121/yetus/diff-checkstyle-itests_hive-unit.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11121/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11121/yetus/whitespace-eol.txt
|
| modules | C: common itests/hive-unit ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11121/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> add originals support to MM tables (and make the conversion a metadata only
> operation)
> --
>
> Key: HIVE-19258
> URL: https://issues.apache.org/jira/browse/HIVE-19258
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter:
[jira] [Commented] (HIVE-19499) Bootstrap REPL LOAD shall add tasks to create checkpoints for db/tables/partitions.
[ https://issues.apache.org/jira/browse/HIVE-19499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483487#comment-16483487 ] Sankar Hariappan commented on HIVE-19499: - I got a green build for the pre-commit test on 01.patch. But not sure why the test results are not commented here. [https://builds.apache.org/job/PreCommit-HIVE-Build/6/testReport/] [~maheshk114], [~thejas], could you please take a look at the patch. > Bootstrap REPL LOAD shall add tasks to create checkpoints for > db/tables/partitions. > --- > > Key: HIVE-19499 > URL: https://issues.apache.org/jira/browse/HIVE-19499 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, repl >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, pull-request-available, replication > Fix For: 3.1.0 > > Attachments: HIVE-19499.01.patch > > > Currently. bootstrap REPL LOAD expect the target database to be empty or not > exist to start bootstrap load. > But, this adds overhead when there is a failure in between bootstrap load and > there is no way to resume it from where it fails. So, it is needed to create > checkpoints in table/partitions to skip the completely loaded objects. > Use the fully qualified path of the dump directory as a checkpoint > identifier. This should be added to the table / partition properties in hive > via a task, as the last task in the DAG for table / partition creation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19614) GenericUDTFGetSplits does not honor ORDER BY
[
https://issues.apache.org/jira/browse/HIVE-19614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483485#comment-16483485
]
Hive QA commented on HIVE-19614:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924231/HIVE-19614.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11124/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11124/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11124/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12924231/HIVE-19614.1.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924231 - PreCommit-HIVE-Build
> GenericUDTFGetSplits does not honor ORDER BY
>
>
> Key: HIVE-19614
> URL: https://issues.apache.org/jira/browse/HIVE-19614
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Wohlstadter
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19614.1.patch
>
>
> GenericUDTFGetSplits handles ORDER BY by writing the results to temp table.
> However running select * on that temp table may create >1 splits which will
> lose the original ordering.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19578) HLL merges tempList on every add
[
https://issues.apache.org/jira/browse/HIVE-19578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483486#comment-16483486
]
Hive QA commented on HIVE-19578:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923998/HIVE-19578.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11125/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11125/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11125/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12923998/HIVE-19578.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12923998 - PreCommit-HIVE-Build
> HLL merges tempList on every add
>
>
> Key: HIVE-19578
> URL: https://issues.apache.org/jira/browse/HIVE-19578
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19578.1.patch, HIVE-19578.2.patch, Screen Shot
> 2018-05-16 at 15.29.12 .png
>
>
> See comments on HIVE-18866; this has significant perf overhead after the
> even bigger overhead from hashing is removed. !Screen Shot 2018-05-16 at
> 15.29.12 .png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19613) GenericUDTFGetSplits should handle fetch task with temp table rewrite
[
https://issues.apache.org/jira/browse/HIVE-19613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483482#comment-16483482
]
Hive QA commented on HIVE-19613:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924234/HIVE-19613.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11123/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11123/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11123/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12924234/HIVE-19613.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924234 - PreCommit-HIVE-Build
> GenericUDTFGetSplits should handle fetch task with temp table rewrite
> -
>
> Key: HIVE-19613
> URL: https://issues.apache.org/jira/browse/HIVE-19613
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Wohlstadter
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19613.1.patch, HIVE-19613.2.patch
>
>
> GenericUDTFGetSplits fails for fetch task only queries. Fetch task only
> queries can be handled same way as >1 task queries using temp tables.
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Was expecting a
> single TezTask.
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.createPlanFragment(GenericUDTFGetSplits.java:262)
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.process(GenericUDTFGetSplits.java:201)
> at org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116)
> at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:984)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:930)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:917)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:984)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:930)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:492)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:484)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> ... 16 more{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader
[
https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483481#comment-16483481
]
Hive QA commented on HIVE-19588:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924051/HIVE-19588.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11122/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11122/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11122/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12924051/HIVE-19588.2.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924051 - PreCommit-HIVE-Build
> Several invocation of file listing when creating
> VectorizedOrcAcidRowBatchReader
>
>
> Key: HIVE-19588
> URL: https://issues.apache.org/jira/browse/HIVE-19588
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Nita Dembla
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19588.1.patch, HIVE-19588.2.patch, Screen Shot
> 2018-05-16 at 2.23.25 PM.png
>
>
> Looks like we are doing file listing several times when creating one instance
> of VectorizedOrcAcidRowBatchReader
> AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when
> there are files with bucket_* prefix) just to get a single file out of a path
> to figure out if it has ACID schema (as part of HIVE-18190).
> There is full file listing where we populate
> 1) ColumnizedDeleteEventRegistry
> 2) SortMergedDeleteEventRegistry
> 3) Twice in computeOffsetAndBucket()
>
> Attaching profiles which [~gopalv] took while debugging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19258) add originals support to MM tables (and make the conversion a metadata only operation)
[
https://issues.apache.org/jira/browse/HIVE-19258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483480#comment-16483480
]
Hive QA commented on HIVE-19258:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924384/HIVE-19258.12.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14394 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11121/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11121/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11121/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924384 - PreCommit-HIVE-Build
> add originals support to MM tables (and make the conversion a metadata only
> operation)
> --
>
> Key: HIVE-19258
> URL: https://issues.apache.org/jira/browse/HIVE-19258
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19258.01.patch, HIVE-19258.02.patch,
> HIVE-19258.03.patch, HIVE-19258.04.patch, HIVE-19258.05.patch,
> HIVE-19258.06.patch, HIVE-19258.07.patch, HIVE-19258.08.patch,
> HIVE-19258.08.patch, HIVE-19258.09.patch, HIVE-19258.10.patch,
> HIVE-19258.11.patch, HIVE-19258.12.patch, HIVE-19258.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19579) remove HBase transitive dependency that drags in some snapshot
[
https://issues.apache.org/jira/browse/HIVE-19579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483435#comment-16483435
]
Hive QA commented on HIVE-19579:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
46s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
6s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
11s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11120/dev-support/hive-personality.sh
|
| git revision | master / 8fc0f72 |
| Default Java | 1.8.0_111 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11120/yetus/whitespace-eol.txt
|
| modules | C: . hbase-handler itests/hcatalog-unit itests/hive-minikdc
itests/hive-unit itests/hive-unit-hadoop2 itests/qtest itests/qtest-accumulo
itests/qtest-spark itests/util llap-server U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11120/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> remove HBase transitive dependency that drags in some snapshot
> --
>
> Key: HIVE-19579
> URL: https://issues.apache.org/jira/browse/HIVE-19579
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19579.01.patch, HIVE-19579.02.patch,
> HIVE-19579.03.patch, HIVE-19579.04.patch, HIVE-19579.patch
>
>
> This:
> {noformat}
> [INFO] org.apache.hive:hive-llap-server:jar:3.1.0-SNAPSHOT
> [INFO] \- org.apache.hbase:hbase-server:jar:2.0.0-alpha4:compile
> [INFO]\- org.glassfish.web:javax.servlet.jsp:jar:2.3.2:compile
> [INFO] \- org.glassfish:javax.el:jar:3.0.1-b10:compile (version
> selected from constraint [3.0.0,))
> {noformat}
> occasionally results in bizarre build failures like this:
> {noformat}
> Failed to read artifact descriptor for
> org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Could not transfer artifact
> org.glassfish:javax.el:pom:3.0.1-b06-SNAPSHOT from/to jvnet-nexus-snapshots
> (https://maven.java.net/content/repositories/snapshots): Failed to transfer
> file:
> https://maven.java.net/content/repositories/snapshots/org/glassfish/javax.el/3.0.1-b06-SNAPSHOT/javax.el-3.0.1-b06-SNAPSHOT.pom.
> Return code is: 402 , ReasonPhrase:Payment Required.
> {noformat}
> cc [~enis] [~stack] if you guys want to take a look at HBase side
--
This
[jira] [Commented] (HIVE-19454) Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError
[
https://issues.apache.org/jira/browse/HIVE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483465#comment-16483465
]
Alisha Prabhu commented on HIVE-19454:
--
Thanks. Waiting for the same.
> Test failure :
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails
> with java.lang.AssertionError
> --
>
> Key: HIVE-19454
> URL: https://issues.apache.org/jira/browse/HIVE-19454
> Project: Hive
> Issue Type: Bug
> Environment: uname -a
> Linux pts00433-vm40 3.10.0-862.el7.ppc64le #1 SMP Wed Mar 21 22:20:19 UTC
> 2018 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Attachments: HIVE-19454.1.patch, HIVE-19454.2.patch
>
>
> org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
> refers to the same test case.
> Maven command used is : mvn -Dtest=TestTxnCommands2 test
> Error:
> {code:java}
> [INFO] Running org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3, Time elapsed:
> 618.215 s <<< FAILURE! - in org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR]
> testNonAcidToAcidConversion1(org.apache.hadoop.hive.ql.TestTxnCommands2)
> Time elapsed: 17.557 s <<< FAILURE!
> java.lang.AssertionError
> at org.junit.Assert.fail(Assert.java:86)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertTrue(Assert.java:52)
> at
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1(TestTxnCommands2.java:499)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
> at
> org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR] TestTxnCommands2.testNonAcidToAcidConversion1:499
> [INFO]
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19623) Inconsistency with Hive Logging DRFA
[
https://issues.apache.org/jira/browse/HIVE-19623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483467#comment-16483467
]
seshukumar commented on HIVE-19623:
---
Additional info:
In the system, there are hundreds of such jobs configured to run at a regular
frequency. And this logs getting missed is happening randomly across all the
jobs.
All the while, the jobs get executed fine, but on certain days/instances, there
are no entries in the hive log.
> Inconsistency with Hive Logging DRFA
>
>
> Key: HIVE-19623
> URL: https://issues.apache.org/jira/browse/HIVE-19623
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.2.1
>Reporter: seshukumar
>Priority: Major
>
> We have Hive job with set of Hive queries like truncate, insert statement,
> which is scheduled as daily job via azkaban. IN order to debug and analyse
> any issues we rely on hive logging.
> In Hive, we have set up DRFA logging, with configuration something like this
> hive.log.threshold=ALL
> hive.root.logger=DEBUG,DRFA
> hive.log.dir=
> hive.log.file=hive.loglog4j.rootLogger=${hive.root.logger}, EventCounter
> log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender
> log4j.appender.DRFA.File=${hive.log.dir}/${hive.log.file}
> We have noticed two issues with hive logging.
> 1) On certain days, there are more than one file into which logging is being
> done. Once the log rolls over to a new file, some entries are being written
> into old log file also.
> 2) Log entries for many days are missing. On certain days, even when the job
> executes correctly, there are no log entries in the hive-log file. On other
> days, the entries are present.
> The hive version is 1.2.1. And the log4j jar is log4j-1.2.16.jar
> Due to the above we are not able to debug any issues in a consistant manner.
> If this a known issue? please provide any workaround or resolution for this.
> thanks
> seshu
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19334) Use actual file size rather than stats for fetch task optimization with external tables
[
https://issues.apache.org/jira/browse/HIVE-19334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483411#comment-16483411
]
Hive QA commented on HIVE-19334:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12922346/HIVE-19334.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/9/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12922346/HIVE-19334.3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12922346 - PreCommit-HIVE-Build
> Use actual file size rather than stats for fetch task optimization with
> external tables
> ---
>
> Key: HIVE-19334
> URL: https://issues.apache.org/jira/browse/HIVE-19334
> Project: Hive
> Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19334.1.patch, HIVE-19334.2.patch,
> HIVE-19334.3.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19332) Disable compute.query.using.stats for external table
[
https://issues.apache.org/jira/browse/HIVE-19332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483409#comment-16483409
]
Hive QA commented on HIVE-19332:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923109/HIVE-19332.6.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/8/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12923109/HIVE-19332.6.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12923109 - PreCommit-HIVE-Build
> Disable compute.query.using.stats for external table
>
>
> Key: HIVE-19332
> URL: https://issues.apache.org/jira/browse/HIVE-19332
> Project: Hive
> Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19332.1.patch, HIVE-19332.2.patch,
> HIVE-19332.3.patch, HIVE-19332.4.patch, HIVE-19332.5.patch, HIVE-19332.6.patch
>
>
> Hive can use statistics to answer queries like count(*). This can be
> problematic on external tables where another tool might add files that Hive
> doesn’t know about. In that case Hive will return incorrect results.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19592) TestWorkloadManager - add retry for now
[
https://issues.apache.org/jira/browse/HIVE-19592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483408#comment-16483408
]
Hive QA commented on HIVE-19592:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924382/HIVE-19592.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14390 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/7/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924382 - PreCommit-HIVE-Build
> TestWorkloadManager - add retry for now
> ---
>
> Key: HIVE-19592
> URL: https://issues.apache.org/jira/browse/HIVE-19592
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19592.patch, HIVE-19592.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19644) change WM syntax to avoid conflicts with identifiers starting with a number
[ https://issues.apache.org/jira/browse/HIVE-19644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19644: --- > change WM syntax to avoid conflicts with identifiers starting with a number > --- > > Key: HIVE-19644 > URL: https://issues.apache.org/jira/browse/HIVE-19644 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > Time/etc literals conflict with non-ANSI query column names starting with a > number that were previously supported without quotes (e.g. 30days). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19465) Upgrade ORC to 1.5.0
[
https://issues.apache.org/jira/browse/HIVE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483389#comment-16483389
]
Jesus Camacho Rodriguez edited comment on HIVE-19465 at 5/22/18 1:27 AM:
-
{{HIVE-19465.03.patch}} contains regenerated q files. Changes seem fine (though
I created ORC-365 as we print unnecessary/incorrect info for timestamp column).
However, there are two tests that still fail:
{code}
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testSeek
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testZeroCopySeek
{code}
Problem seems to be at reading time for boolean type (value always false).
Reverting ORC-187 (which was pushed in 1.5.0) from release fixes those tests,
hence I am wondering whether this is a bug introduced by that patch or maybe I
need to change something in Hive side that I am missing.
Cc [~gopalv] [~prasanth_j] [~ashutoshc]
was (Author: jcamachorodriguez):
{{HIVE-19465.03.patch}} contains regenerated q files. Changes seem fine (though
I created ORC-365 as we print unnecessary/incorrect info for timestamp column).
However, there are two tests that still fail:
{code}
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testSeek
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testZeroCopySeek
{code}
Problem seems to be at reading time for boolean type (value always false).
Reverting ORC-187 (which was pushed in 1.5.0) from release fixes those tests,
hence I am wondering whether this is a bug introduced by that patch or maybe I
need to change something in Hive side that I am missing.
Cc [~gopalv] [~prasanth_j]
> Upgrade ORC to 1.5.0
>
>
> Key: HIVE-19465
> URL: https://issues.apache.org/jira/browse/HIVE-19465
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
> Attachments: HIVE-19465.01.patch, HIVE-19465.02.patch,
> HIVE-19465.03.patch, HIVE-19465.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19465) Upgrade ORC to 1.5.0
[
https://issues.apache.org/jira/browse/HIVE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483389#comment-16483389
]
Jesus Camacho Rodriguez commented on HIVE-19465:
{{HIVE-19465.03.patch}} contains regenerated q files. Changes seem fine (though
I created ORC-365 as we print unnecessary/incorrect info for timestamp column).
However, there are two tests that still fail:
{code}
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testSeek
org.apache.hadoop.hive.ql.io.orc.TestOrcFile.testZeroCopySeek
{code}
Problem seems to be at reading time for boolean type (value always false).
Reverting ORC-187 (which was pushed in 1.5.0) from release fixes those tests,
hence I am wondering whether this is a bug introduced by that patch or maybe I
need to change something in Hive side that I am missing.
Cc [~gopalv] [~prasanth_j]
> Upgrade ORC to 1.5.0
>
>
> Key: HIVE-19465
> URL: https://issues.apache.org/jira/browse/HIVE-19465
> Project: Hive
> Issue Type: Improvement
> Components: File Formats
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
> Attachments: HIVE-19465.01.patch, HIVE-19465.02.patch,
> HIVE-19465.03.patch, HIVE-19465.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19643) MM table conversion doesn't need full ACID structure checks
[ https://issues.apache.org/jira/browse/HIVE-19643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19643: Status: Patch Available (was: Open) [~ekoifman] can you take a look? thnx > MM table conversion doesn't need full ACID structure checks > --- > > Key: HIVE-19643 > URL: https://issues.apache.org/jira/browse/HIVE-19643 > Project: Hive > Issue Type: Bug >Reporter: Jason Dere >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19643.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19643) MM table conversion doesn't need full ACID structure checks
[ https://issues.apache.org/jira/browse/HIVE-19643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19643: --- > MM table conversion doesn't need full ACID structure checks > --- > > Key: HIVE-19643 > URL: https://issues.apache.org/jira/browse/HIVE-19643 > Project: Hive > Issue Type: Bug >Reporter: Jason Dere >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19643.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19465) Upgrade ORC to 1.5.0
[ https://issues.apache.org/jira/browse/HIVE-19465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19465: --- Attachment: HIVE-19465.03.patch > Upgrade ORC to 1.5.0 > > > Key: HIVE-19465 > URL: https://issues.apache.org/jira/browse/HIVE-19465 > Project: Hive > Issue Type: Improvement > Components: File Formats >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Blocker > Attachments: HIVE-19465.01.patch, HIVE-19465.02.patch, > HIVE-19465.03.patch, HIVE-19465.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19643) MM table conversion doesn't need full ACID structure checks
[ https://issues.apache.org/jira/browse/HIVE-19643?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19643: Attachment: HIVE-19643.patch > MM table conversion doesn't need full ACID structure checks > --- > > Key: HIVE-19643 > URL: https://issues.apache.org/jira/browse/HIVE-19643 > Project: Hive > Issue Type: Bug >Reporter: Jason Dere >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19643.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19463) TezTask - getting groups may fail (PartialGroupNameException in some tests)
[
https://issues.apache.org/jira/browse/HIVE-19463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-19463:
Resolution: Fixed
Fix Version/s: 3.0.1
3.1.0
Status: Resolved (was: Patch Available)
Committed to master and branch-3; this is a test fix.
> TezTask - getting groups may fail (PartialGroupNameException in some tests)
> ---
>
> Key: HIVE-19463
> URL: https://issues.apache.org/jira/browse/HIVE-19463
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19463.01.patch, HIVE-19463.02.patch,
> HIVE-19463.03.patch, HIVE-19463.03.patch, HIVE-19463.04.patch,
> HIVE-19463.05.patch, HIVE-19463.06.patch, HIVE-19463.07.patch,
> HIVE-19463.patch
>
>
> {noformat}
> org.apache.hadoop.security.ShellBasedUnixGroupsMapping$PartialGroupNameException:
> The user name 'hive_test_user' is not found. id: hive_test_user: no such user
> id: hive_test_user: no such user
> at
> org.apache.hadoop.security.ShellBasedUnixGroupsMapping.resolvePartialGroupNames(ShellBasedUnixGroupsMapping.java:294)
> ~[hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getUnixGroups(ShellBasedUnixGroupsMapping.java:207)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.security.ShellBasedUnixGroupsMapping.getGroups(ShellBasedUnixGroupsMapping.java:97)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.getGroups(JniBasedUnixGroupsMappingWithFallback.java:51)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.security.Groups$GroupCacheLoader.fetchGroupList(Groups.java:384)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:319)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.security.Groups$GroupCacheLoader.load(Groups.java:269)
> [hadoop-common-3.1.0.jar:?]
> at
> com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3542)
> [guava-19.0.jar:?]
> at
> com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2323)
> [guava-19.0.jar:?]
> at
> com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2286)
> [guava-19.0.jar:?]
> at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2201)
> [guava-19.0.jar:?]
> at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
> [guava-19.0.jar:?]
> at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3957)
> [guava-19.0.jar:?]
> at
> com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4875)
> [guava-19.0.jar:?]
> at org.apache.hadoop.security.Groups.getGroups(Groups.java:227)
> [hadoop-common-3.1.0.jar:?]
> at
> org.apache.hadoop.security.UserGroupInformation.getGroups(UserGroupInformation.java:1540)
> [hadoop-common-3.1.0.jar:?]
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:163)
> [hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19639) a transactional Hive table cannot be imported as an external table
[ https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483236#comment-16483236 ] Thejas M Nair commented on HIVE-19639: -- +1 > a transactional Hive table cannot be imported as an external table > -- > > Key: HIVE-19639 > URL: https://issues.apache.org/jira/browse/HIVE-19639 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19639.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19639) a transactional Hive table cannot be imported as an external table
[ https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-19639: - Description: When transactional table is imported to a external table, the table should be imported as a non transactional table, as external tables cannot be transactional. > a transactional Hive table cannot be imported as an external table > -- > > Key: HIVE-19639 > URL: https://issues.apache.org/jira/browse/HIVE-19639 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19639.patch > > > When transactional table is imported to a external table, the table should be > imported as a non transactional table, as external tables cannot be > transactional. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19617) Rename test tables to avoid collisions during execution in batches
[ https://issues.apache.org/jira/browse/HIVE-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19617: --- Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Last run was blue. Closing as resolved. > Rename test tables to avoid collisions during execution in batches > -- > > Key: HIVE-19617 > URL: https://issues.apache.org/jira/browse/HIVE-19617 > Project: Hive > Issue Type: Test > Components: Testing Infrastructure >Affects Versions: 3.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Critical > Fix For: 3.1.0 > > Attachments: HIVE-19617.01.patch, HIVE-19617.02.patch, > HIVE-19617.03.patch, HIVE-19617.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19626) Change tmp staging mapred directory for CliDriver
[ https://issues.apache.org/jira/browse/HIVE-19626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-19626: --- Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Pushed to master. > Change tmp staging mapred directory for CliDriver > - > > Key: HIVE-19626 > URL: https://issues.apache.org/jira/browse/HIVE-19626 > Project: Hive > Issue Type: Test >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19626.02.patch > > > We do not see many failures anymore, but one of the intermittent ones is this: > https://builds.apache.org/job/PreCommit-HIVE-Build/11101/testReport/junit/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_localtimezone_/ > As with other tests, we can change the staging directory property value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19626) Change tmp staging mapred directory for CliDriver
[ https://issues.apache.org/jira/browse/HIVE-19626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483225#comment-16483225 ] Prasanth Jayachandran commented on HIVE-19626: -- +1 > Change tmp staging mapred directory for CliDriver > - > > Key: HIVE-19626 > URL: https://issues.apache.org/jira/browse/HIVE-19626 > Project: Hive > Issue Type: Test >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-19626.02.patch > > > We do not see many failures anymore, but one of the intermittent ones is this: > https://builds.apache.org/job/PreCommit-HIVE-Build/11101/testReport/junit/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_localtimezone_/ > As with other tests, we can change the staging directory property value. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19626) Change tmp staging mapred directory for CliDriver
[
https://issues.apache.org/jira/browse/HIVE-19626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483221#comment-16483221
]
Hive QA commented on HIVE-19626:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924379/HIVE-19626.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14390 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/5/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924379 - PreCommit-HIVE-Build
> Change tmp staging mapred directory for CliDriver
> -
>
> Key: HIVE-19626
> URL: https://issues.apache.org/jira/browse/HIVE-19626
> Project: Hive
> Issue Type: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19626.02.patch
>
>
> We do not see many failures anymore, but one of the intermittent ones is this:
> https://builds.apache.org/job/PreCommit-HIVE-Build/11101/testReport/junit/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_localtimezone_/
> As with other tests, we can change the staging directory property value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19577) CREATE TEMPORARY TABLE LIKE and INSERT generate output format mismatch errors
[ https://issues.apache.org/jira/browse/HIVE-19577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-19577: -- Status: Patch Available (was: Open) > CREATE TEMPORARY TABLE LIKE and INSERT generate output format mismatch errors > -- > > Key: HIVE-19577 > URL: https://issues.apache.org/jira/browse/HIVE-19577 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19577.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19577) CREATE TEMPORARY TABLE LIKE and INSERT generate output format mismatch errors
[ https://issues.apache.org/jira/browse/HIVE-19577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Yeom updated HIVE-19577: -- Attachment: HIVE-19577.01.patch > CREATE TEMPORARY TABLE LIKE and INSERT generate output format mismatch errors > -- > > Key: HIVE-19577 > URL: https://issues.apache.org/jira/browse/HIVE-19577 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19577.01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19642) add cache dump when LLAP cache is full on branch-2 (and fix retry issues)
[ https://issues.apache.org/jira/browse/HIVE-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483202#comment-16483202 ] Prasanth Jayachandran edited comment on HIVE-19642 at 5/21/18 11:31 PM: +1. left some nits was (Author: prasanth_j): +1 > add cache dump when LLAP cache is full on branch-2 (and fix retry issues) > - > > Key: HIVE-19642 > URL: https://issues.apache.org/jira/browse/HIVE-19642 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19642-branch-2.patch > > > LLAP cache doesn't have defragmentation on branch-2 so it retries when > failing to evict. > There are a couple of issues with retries (e.g. sleep is not applied properly > and time can go negative), also it would be helpful to make use of existing > cache dump logic used in LLAP IO view to dump log state into the log in such > cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19633) Remove/Migrate Minimr tests
[ https://issues.apache.org/jira/browse/HIVE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483201#comment-16483201 ] Sergey Shelukhin commented on HIVE-19633: - I think we can just remove the driver and move the tests not already on CliDriver to CliDriver, unless they test some MR cluster specific features. MR was deprecated in 2.X, so on 3.X we don't have to test it at all IMO. > Remove/Migrate Minimr tests > --- > > Key: HIVE-19633 > URL: https://issues.apache.org/jira/browse/HIVE-19633 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > MR has been deprecated for a long time. Minimr tests are incredibly slow. We > should remove the tests or migrate to faster options for coverage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19642) add cache dump when LLAP cache is full on branch-2 (and fix retry issues)
[ https://issues.apache.org/jira/browse/HIVE-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483202#comment-16483202 ] Prasanth Jayachandran commented on HIVE-19642: -- +1 > add cache dump when LLAP cache is full on branch-2 (and fix retry issues) > - > > Key: HIVE-19642 > URL: https://issues.apache.org/jira/browse/HIVE-19642 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19642-branch-2.patch > > > LLAP cache doesn't have defragmentation on branch-2 so it retries when > failing to evict. > There are a couple of issues with retries (e.g. sleep is not applied properly > and time can go negative), also it would be helpful to make use of existing > cache dump logic used in LLAP IO view to dump log state into the log in such > cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Comment Edited] (HIVE-19633) Remove/Migrate Minimr tests
[ https://issues.apache.org/jira/browse/HIVE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483201#comment-16483201 ] Sergey Shelukhin edited comment on HIVE-19633 at 5/21/18 11:30 PM: --- I think we can just remove the driver and move the tests not already on CliDriver to CliDriver, unless they test some MR cluster specific features. MR was deprecated in 2.X, so on 3.X we don't have to test it at all IMO, unless it was completely painless, which it isn't. was (Author: sershe): I think we can just remove the driver and move the tests not already on CliDriver to CliDriver, unless they test some MR cluster specific features. MR was deprecated in 2.X, so on 3.X we don't have to test it at all IMO. > Remove/Migrate Minimr tests > --- > > Key: HIVE-19633 > URL: https://issues.apache.org/jira/browse/HIVE-19633 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > MR has been deprecated for a long time. Minimr tests are incredibly slow. We > should remove the tests or migrate to faster options for coverage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19642) add cache dump when LLAP cache is full on branch-2 (and fix retry issues)
[ https://issues.apache.org/jira/browse/HIVE-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19642: Summary: add cache dump when LLAP cache is full on branch-2 (and fix retry issues) (was: add cache dump when LLAP cache has issues on branch-2 (and fix retry issues)) > add cache dump when LLAP cache is full on branch-2 (and fix retry issues) > - > > Key: HIVE-19642 > URL: https://issues.apache.org/jira/browse/HIVE-19642 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19642-branch-2.patch > > > LLAP cache doesn't have defragmentation on branch-2 so it retries when > failing to evict. > There are a couple of issues with retries (e.g. sleep is not applied properly > and time can go negative), also it would be helpful to make use of existing > cache dump logic used in LLAP IO view to dump log state into the log in such > cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19642) add cache dump when LLAP cache has issues on branch-2 (and fix retry issues)
[ https://issues.apache.org/jira/browse/HIVE-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19642: Status: Patch Available (was: Open) [~prasanth_j] can you review? > add cache dump when LLAP cache has issues on branch-2 (and fix retry issues) > > > Key: HIVE-19642 > URL: https://issues.apache.org/jira/browse/HIVE-19642 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19642-branch-2.patch > > > LLAP cache doesn't have defragmentation on branch-2 so it retries when > failing to evict. > There are a couple of issues with retries (e.g. sleep is not applied properly > and time can go negative), also it would be helpful to make use of existing > cache dump logic used in LLAP IO view to dump log state into the log in such > cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19642) add cache dump when LLAP cache has issues on branch-2 (and fix retry issues)
[ https://issues.apache.org/jira/browse/HIVE-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19642: Attachment: HIVE-19642-branch-2.patch > add cache dump when LLAP cache has issues on branch-2 (and fix retry issues) > > > Key: HIVE-19642 > URL: https://issues.apache.org/jira/browse/HIVE-19642 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19642-branch-2.patch > > > LLAP cache doesn't have defragmentation on branch-2 so it retries when > failing to evict. > There are a couple of issues with retries (e.g. sleep is not applied properly > and time can go negative), also it would be helpful to make use of existing > cache dump logic used in LLAP IO view to dump log state into the log in such > cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19642) add cache dump when LLAP cache has issues on branch-2 (and fix retry issues)
[ https://issues.apache.org/jira/browse/HIVE-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19642: Attachment: HIVE-19642.patch > add cache dump when LLAP cache has issues on branch-2 (and fix retry issues) > > > Key: HIVE-19642 > URL: https://issues.apache.org/jira/browse/HIVE-19642 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > LLAP cache doesn't have defragmentation on branch-2 so it retries when > failing to evict. > There are a couple of issues with retries (e.g. sleep is not applied properly > and time can go negative), also it would be helpful to make use of existing > cache dump logic used in LLAP IO view to dump log state into the log in such > cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19642) add cache dump when LLAP cache has issues on branch-2 (and fix retry issues)
[ https://issues.apache.org/jira/browse/HIVE-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19642: Attachment: (was: HIVE-19642.patch) > add cache dump when LLAP cache has issues on branch-2 (and fix retry issues) > > > Key: HIVE-19642 > URL: https://issues.apache.org/jira/browse/HIVE-19642 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > LLAP cache doesn't have defragmentation on branch-2 so it retries when > failing to evict. > There are a couple of issues with retries (e.g. sleep is not applied properly > and time can go negative), also it would be helpful to make use of existing > cache dump logic used in LLAP IO view to dump log state into the log in such > cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19527) Preparing for 2.4 development
[ https://issues.apache.org/jira/browse/HIVE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-19527: --- Attachment: HIVE-19527.2-branch-2.patch > Preparing for 2.4 development > - > > Key: HIVE-19527 > URL: https://issues.apache.org/jira/browse/HIVE-19527 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Major > Attachments: HIVE-19527.2-branch-2.patch > > > branch-2 is now being used for 2.4 development. > The build files will need to reflect this change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19527) Preparing for 2.4 development
[ https://issues.apache.org/jira/browse/HIVE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-19527: --- Attachment: (was: HIVE-19527.2.patch) > Preparing for 2.4 development > - > > Key: HIVE-19527 > URL: https://issues.apache.org/jira/browse/HIVE-19527 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Major > Attachments: HIVE-19527.2-branch-2.patch > > > branch-2 is now being used for 2.4 development. > The build files will need to reflect this change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19527) Preparing for 2.4 development
[ https://issues.apache.org/jira/browse/HIVE-19527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergio Peña updated HIVE-19527: --- Attachment: (was: HIVE-19527.1.patch) > Preparing for 2.4 development > - > > Key: HIVE-19527 > URL: https://issues.apache.org/jira/browse/HIVE-19527 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Sergio Peña >Assignee: Sergio Peña >Priority: Major > Attachments: HIVE-19527.2-branch-2.patch > > > branch-2 is now being used for 2.4 development. > The build files will need to reflect this change. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19642) add cache dump when LLAP cache has issues on branch-2 (and fix retry issues)
[ https://issues.apache.org/jira/browse/HIVE-19642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19642: --- > add cache dump when LLAP cache has issues on branch-2 (and fix retry issues) > > > Key: HIVE-19642 > URL: https://issues.apache.org/jira/browse/HIVE-19642 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > > LLAP cache doesn't have defragmentation on branch-2 so it retries when > failing to evict. > There are a couple of issues with retries (e.g. sleep is not applied properly > and time can go negative), also it would be helpful to make use of existing > cache dump logic used in LLAP IO view to dump log state into the log in such > cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19626) Change tmp staging mapred directory for CliDriver
[
https://issues.apache.org/jira/browse/HIVE-19626?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483176#comment-16483176
]
Hive QA commented on HIVE-19626:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
53s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-5/dev-support/hive-personality.sh
|
| git revision | master / e030170 |
| modules | C: . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-5/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Change tmp staging mapred directory for CliDriver
> -
>
> Key: HIVE-19626
> URL: https://issues.apache.org/jira/browse/HIVE-19626
> Project: Hive
> Issue Type: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19626.02.patch
>
>
> We do not see many failures anymore, but one of the intermittent ones is this:
> https://builds.apache.org/job/PreCommit-HIVE-Build/11101/testReport/junit/org.apache.hadoop.hive.cli/TestCliDriver/testCliDriver_localtimezone_/
> As with other tests, we can change the staging directory property value.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16307) add IO memory usage report to LLAP UI
[ https://issues.apache.org/jira/browse/HIVE-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483174#comment-16483174 ] Sergey Shelukhin commented on HIVE-16307: - Backported to branch-2 with the commit message corrected. > add IO memory usage report to LLAP UI > - > > Key: HIVE-16307 > URL: https://issues.apache.org/jira/browse/HIVE-16307 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0, 2.4.0 > > Attachments: HIVE-16307.01.patch, HIVE-16307.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16307) add IO memory usage report to LLAP UI
[ https://issues.apache.org/jira/browse/HIVE-16307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-16307: Fix Version/s: 2.4.0 > add IO memory usage report to LLAP UI > - > > Key: HIVE-16307 > URL: https://issues.apache.org/jira/browse/HIVE-16307 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Fix For: 3.0.0, 2.4.0 > > Attachments: HIVE-16307.01.patch, HIVE-16307.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19440) Make StorageBasedAuthorizer work with information schema
[
https://issues.apache.org/jira/browse/HIVE-19440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483168#comment-16483168
]
Hive QA commented on HIVE-19440:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924150/HIVE-19440.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 14391 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_functions]
(batchId=76)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=168)
org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testStatCachingMetaStore
(batchId=300)
org.apache.hadoop.hive.ql.security.TestHDFSPermissionPolicyProvider.org.apache.hadoop.hive.ql.security.TestHDFSPermissionPolicyProvider
(batchId=236)
org.apache.hive.beeline.TestSchemaToolCatalogOps.createCatalog (batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.createExistingCatalog
(batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.createExistingCatalogWithIfNotExists
(batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveDatabase (batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveDatabaseWithExistingDbOfSameNameAlreadyInTargetCatalog
(batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveDbToNonExistentCatalog
(batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveNonExistentDatabase
(batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveNonExistentTable
(batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveTable (batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveTableToNonExistentDb
(batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveTableWithExistingTableOfSameNameAlreadyInTargetDatabase
(batchId=237)
org.apache.hive.beeline.TestSchemaToolCatalogOps.moveTableWithinCatalog
(batchId=237)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConcurrentStatements (batchId=244)
org.apache.hive.service.server.TestInformationSchemaWithPrivilege.test
(batchId=239)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/4/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/4/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-4/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924150 - PreCommit-HIVE-Build
> Make StorageBasedAuthorizer work with information schema
>
>
> Key: HIVE-19440
> URL: https://issues.apache.org/jira/browse/HIVE-19440
> Project: Hive
> Issue Type: Improvement
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Blocker
> Fix For: 3.1.0
>
> Attachments: HIVE-19440.1.patch, HIVE-19440.2.patch,
> HIVE-19440.3.patch, HIVE-19440.4.patch, HIVE-19440.5.patch, HIVE-19440.6.patch
>
>
> With HIVE-19161, Hive information schema works with external authorizer (such
> as ranger). However, we also need to make StorageBasedAuthorizer
> synchronization work as it is also widely use.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19641) sync up hadoop version used by storage-api with hive
[ https://issues.apache.org/jira/browse/HIVE-19641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483161#comment-16483161 ] Vineet Garg commented on HIVE-19641: +1. > sync up hadoop version used by storage-api with hive > > > Key: HIVE-19641 > URL: https://issues.apache.org/jira/browse/HIVE-19641 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19641.1.patch > > > There is hadoop version mismatch between hive and storage-api and hence > different transitive dependency versions gets pulled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19641) sync up hadoop version used by storage-api with hive
[ https://issues.apache.org/jira/browse/HIVE-19641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19641: - Attachment: HIVE-19641.1.patch > sync up hadoop version used by storage-api with hive > > > Key: HIVE-19641 > URL: https://issues.apache.org/jira/browse/HIVE-19641 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19641.1.patch > > > There is hadoop version mismatch between hive and storage-api and hence > different transitive dependency versions gets pulled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19641) sync up hadoop version used by storage-api with hive
[ https://issues.apache.org/jira/browse/HIVE-19641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19641: - Status: Patch Available (was: Open) > sync up hadoop version used by storage-api with hive > > > Key: HIVE-19641 > URL: https://issues.apache.org/jira/browse/HIVE-19641 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19641.1.patch > > > There is hadoop version mismatch between hive and storage-api and hence > different transitive dependency versions gets pulled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19440) Make StorageBasedAuthorizer work with information schema
[
https://issues.apache.org/jira/browse/HIVE-19440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483145#comment-16483145
]
Hive QA commented on HIVE-19440:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} hcatalog/core in master has 33 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
43s{color} | {color:blue} itests/util in master has 55 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2322 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} service in master has 50 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
56s{color} | {color:blue} standalone-metastore in master has 216 extant
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
18s{color} | {color:red} hcatalog-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
29s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
48s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
17s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
32s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
19s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 32s{color}
| {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 19s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} itests/hive-unit: The patch generated 19 new + 0
unchanged - 0 fixed = 19 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 4 new + 104 unchanged - 0
fixed = 108 total (was 104) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
29s{color} | {color:red} standalone-metastore: The patch generated 6 new + 1524
unchanged - 3 fixed = 1530 total (was 1527) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 11 line(s) that end in whitespace. Use
git apply --whitespace=fix <>. Refer
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 59 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
30s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
57s{color} | {color:red} ql generated 2 new + 2322 unchanged - 0 fixed = 2324
total (was 2322) {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
18s{color} | {color:red} service in the patch failed.
[jira] [Commented] (HIVE-19599) Release Notes : Highlighting backwards incompatible changes
[ https://issues.apache.org/jira/browse/HIVE-19599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483144#comment-16483144 ] Eugene Koifman commented on HIVE-19599: --- The on-disk layout of Acid tables has changed in Hive 3.0. Any Acid table partition that had Update/Delete/Merge statement executed since the last Major compaction must execute Major compaction before upgrading to 3.0. No more Update/Delete/Merge may be executed against these tables since the start of Major compaction. Not following this may lead to data corruption. Tables/partitions that only contain results of Insert statements are fully compatible and don't need to be compacted. > Release Notes : Highlighting backwards incompatible changes > --- > > Key: HIVE-19599 > URL: https://issues.apache.org/jira/browse/HIVE-19599 > Project: Hive > Issue Type: Bug > Components: Documentation >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Vineet Garg >Priority: Blocker > > We need to highlight backwards incompatible changes. A list Jira titles > won't be sufficient. > For example, tables with Acid V1 (pre 3.0) data has to be major compacted > before upgrade and may not process any update/delete/merge until after > upgrade. Not doing so may result in data corruption/loss. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19641) sync up hadoop version used by storage-api with hive
[ https://issues.apache.org/jira/browse/HIVE-19641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-19641: > sync up hadoop version used by storage-api with hive > > > Key: HIVE-19641 > URL: https://issues.apache.org/jira/browse/HIVE-19641 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > There is hadoop version mismatch between hive and storage-api and hence > different transitive dependency versions gets pulled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19640) dependency version upgrades/fixes/convergence
[ https://issues.apache.org/jira/browse/HIVE-19640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-19640: > dependency version upgrades/fixes/convergence > - > > Key: HIVE-19640 > URL: https://issues.apache.org/jira/browse/HIVE-19640 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > There are several versioning jars, some are old, some have multiple versions, > transitive version conflicts etc. > This is umbrella jar to fix up all that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18629) copyValues in BytesColumnVector may be missing null checks
[
https://issues.apache.org/jira/browse/HIVE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-18629:
Resolution: Won't Fix
Status: Resolved (was: Patch Available)
> copyValues in BytesColumnVector may be missing null checks
> --
>
> Key: HIVE-18629
> URL: https://issues.apache.org/jira/browse/HIVE-18629
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18629.01.patch, HIVE-18629.02.patch,
> HIVE-18629.03.patch, HIVE-18629.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:333)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions..evaluate(IfExprStringGroupColumnStringGroupColumn.java:83)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:133)
> {noformat}
> IfExprStringGroupColumnStringGroupColumn code below the v1.isRepeating case
> has isNull checks for v2/v3 buffers that copySelected is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19638) Configuration not passed to ORC Reader.Options
[ https://issues.apache.org/jira/browse/HIVE-19638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483134#comment-16483134 ] Rentao Wu commented on HIVE-19638: -- Attached patch > Configuration not passed to ORC Reader.Options > -- > > Key: HIVE-19638 > URL: https://issues.apache.org/jira/browse/HIVE-19638 > Project: Hive > Issue Type: Bug > Components: Configuration, File Formats, ORC >Affects Versions: 2.3.0, 2.3.1, 2.3.2 >Reporter: Rentao Wu >Assignee: Rentao Wu >Priority: Major > Attachments: HIVE-19638.patch > > > Configuration is not passed to ORC's Reader.Option in OrcFileInputFormat > which causes some [ORC > configurations|https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/Reader.java#L170-L176] > to not be able to be picked up. > Related issues: > For example, the ORC upgrade in Hive 2.3.x changed schema evolution from > positional to column name matching. A backwards compatibility configuration > "orc.force.positional.evolution" could be set in ORC Reader.Options by > [ORC-120|https://issues.apache.org/jira/browse/ORC-120] however it could not > be picked up resulting in null values when querying ORC tables where the > column names do not match. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19638) Configuration not passed to ORC Reader.Options
[ https://issues.apache.org/jira/browse/HIVE-19638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rentao Wu updated HIVE-19638: - Status: Patch Available (was: Open) > Configuration not passed to ORC Reader.Options > -- > > Key: HIVE-19638 > URL: https://issues.apache.org/jira/browse/HIVE-19638 > Project: Hive > Issue Type: Bug > Components: Configuration, File Formats, ORC >Affects Versions: 2.3.2, 2.3.1, 2.3.0 >Reporter: Rentao Wu >Assignee: Rentao Wu >Priority: Major > Attachments: HIVE-19638.patch > > > Configuration is not passed to ORC's Reader.Option in OrcFileInputFormat > which causes some [ORC > configurations|https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/Reader.java#L170-L176] > to not be able to be picked up. > Related issues: > For example, the ORC upgrade in Hive 2.3.x changed schema evolution from > positional to column name matching. A backwards compatibility configuration > "orc.force.positional.evolution" could be set in ORC Reader.Options by > [ORC-120|https://issues.apache.org/jira/browse/ORC-120] however it could not > be picked up resulting in null values when querying ORC tables where the > column names do not match. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19638) Configuration not passed to ORC Reader.Options
[ https://issues.apache.org/jira/browse/HIVE-19638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rentao Wu updated HIVE-19638: - Description: Configuration is not passed to ORC's Reader.Option in OrcFileInputFormat which causes some [ORC configurations|https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/Reader.java#L170-L176] to not be able to be picked up. Related issues: For example, the ORC upgrade in Hive 2.3.x changed schema evolution from positional to column name matching. A backwards compatibility configuration "orc.force.positional.evolution" could be set in ORC Reader.Options by [ORC-120|https://issues.apache.org/jira/browse/ORC-120] however it could not be picked up resulting in null values when querying ORC tables where the column names do not match. was: Configuration is not passed to ORC's Reader.Option in OrcFileInputFormat which causes some [ORC configurations|https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/Reader.java#L170-L176] to not be able to be picked up. > Configuration not passed to ORC Reader.Options > -- > > Key: HIVE-19638 > URL: https://issues.apache.org/jira/browse/HIVE-19638 > Project: Hive > Issue Type: Bug > Components: Configuration, File Formats, ORC >Affects Versions: 2.3.0, 2.3.1, 2.3.2 >Reporter: Rentao Wu >Assignee: Rentao Wu >Priority: Major > Attachments: HIVE-19638.patch > > > Configuration is not passed to ORC's Reader.Option in OrcFileInputFormat > which causes some [ORC > configurations|https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/Reader.java#L170-L176] > to not be able to be picked up. > Related issues: > For example, the ORC upgrade in Hive 2.3.x changed schema evolution from > positional to column name matching. A backwards compatibility configuration > "orc.force.positional.evolution" could be set in ORC Reader.Options by > [ORC-120|https://issues.apache.org/jira/browse/ORC-120] however it could not > be picked up resulting in null values when querying ORC tables where the > column names do not match. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-6998) Select query can only support maximum 128 distinct expressions
[
https://issues.apache.org/jira/browse/HIVE-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-6998:
-
Attachment: HIVE-6998.03.patch
> Select query can only support maximum 128 distinct expressions
> --
>
> Key: HIVE-6998
> URL: https://issues.apache.org/jira/browse/HIVE-6998
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Serializers/Deserializers
>Affects Versions: 0.14.0
>Reporter: Chaoyu Tang
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-6998.01.patch, HIVE-6998.02.patch,
> HIVE-6998.03.patch
>
>
> Select query can only support maximum 128 distinct expressions. Otherwise,
> you will be thrown ArrayIndexOutOfBoundsException. For a query like:
> select count(distinct c1), count(distinct c2), count(distinct c3),
> count(distinct c4), count(distinct c5), count(distinct c6), ,
> count(distinct c128), count(distinct c129) from tbl_129columns;
> you will get error like:
> {code}
> java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while
> closing operators
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
> Caused by: java.lang.RuntimeException: Hive Runtime Error while closing
> operators
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:695)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1141)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:579)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
> ... 10 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1099)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1138)
> ... 15 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:327)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1064)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1082)
> ... 16 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: -128
> at java.util.ArrayList.get(ArrayList.java:324)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:838)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:600)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.toHiveKey(ReduceSinkOperator.java:401)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:320)
> ... 19 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18629) copyValues in BytesColumnVector may be missing null checks
[
https://issues.apache.org/jira/browse/HIVE-18629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483126#comment-16483126
]
Vihang Karajgaonkar commented on HIVE-18629:
Oh I see. In that case can we close this issue?
> copyValues in BytesColumnVector may be missing null checks
> --
>
> Key: HIVE-18629
> URL: https://issues.apache.org/jira/browse/HIVE-18629
> Project: Hive
> Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-18629.01.patch, HIVE-18629.02.patch,
> HIVE-18629.03.patch, HIVE-18629.patch
>
>
> {noformat}
> Caused by: java.lang.NullPointerException
> at java.lang.System.arraycopy(Native Method)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.setVal(BytesColumnVector.java:173)
> at
> org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector.copySelected(BytesColumnVector.java:333)
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions..evaluate(IfExprStringGroupColumnStringGroupColumn.java:83)
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:133)
> {noformat}
> IfExprStringGroupColumnStringGroupColumn code below the v1.isRepeating case
> has isNull checks for v2/v3 buffers that copySelected is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19638) Configuration not passed to ORC Reader.Options
[ https://issues.apache.org/jira/browse/HIVE-19638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rentao Wu updated HIVE-19638: - Attachment: HIVE-19638.patch > Configuration not passed to ORC Reader.Options > -- > > Key: HIVE-19638 > URL: https://issues.apache.org/jira/browse/HIVE-19638 > Project: Hive > Issue Type: Bug > Components: Configuration, File Formats, ORC >Affects Versions: 2.3.0, 2.3.1, 2.3.2 >Reporter: Rentao Wu >Assignee: Rentao Wu >Priority: Major > Attachments: HIVE-19638.patch > > > Configuration is not passed to ORC's Reader.Option in OrcFileInputFormat > which causes some [ORC > configurations|https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/Reader.java#L170-L176] > to not be able to be picked up. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19632) Remove webapps directory from standalone jar
[ https://issues.apache.org/jira/browse/HIVE-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483100#comment-16483100 ] Thejas M Nair commented on HIVE-19632: -- +1 > Remove webapps directory from standalone jar > > > Key: HIVE-19632 > URL: https://issues.apache.org/jira/browse/HIVE-19632 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19632.1.patch, HIVE-19632.2.patch > > > JDBC standalone jar contains webapps static files which just adds to the jar > size and are not required by the clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18117) Create TestCliDriver for HDFS EC
[
https://issues.apache.org/jira/browse/HIVE-18117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483080#comment-16483080
]
Hive QA commented on HIVE-18117:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924372/HIVE-18117.9.patch
{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 14312 tests
executed
*Failed tests:*
{noformat}
TestBeeLineExceptionHandling - did not produce a TEST-*.xml file (likely timed
out) (batchId=192)
TestBeeLineHistory - did not produce a TEST-*.xml file (likely timed out)
(batchId=192)
TestBeelineArgParsing - did not produce a TEST-*.xml file (likely timed out)
(batchId=192)
TestClientCommandHookFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=192)
TestHiveCli - did not produce a TEST-*.xml file (likely timed out) (batchId=192)
TestHiveSchemaTool - did not produce a TEST-*.xml file (likely timed out)
(batchId=192)
TestIncrementalRows - did not produce a TEST-*.xml file (likely timed out)
(batchId=192)
TestShutdownHook - did not produce a TEST-*.xml file (likely timed out)
(batchId=192)
TestTableOutputFormat - did not produce a TEST-*.xml file (likely timed out)
(batchId=192)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=69)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/3/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924372 - PreCommit-HIVE-Build
> Create TestCliDriver for HDFS EC
>
>
> Key: HIVE-18117
> URL: https://issues.apache.org/jira/browse/HIVE-18117
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sahil Takiar
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-18117.1.patch, HIVE-18117.2.patch,
> HIVE-18117.3.patch, HIVE-18117.4.patch, HIVE-18117.5.patch,
> HIVE-18117.6.patch, HIVE-18117.7.patch, HIVE-18117.8.patch, HIVE-18117.9.patch
>
>
> Should be able to do something similar to what we do for HDFS encryption.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19417) Modify metastore to have/access persistent tables for stats
[ https://issues.apache.org/jira/browse/HIVE-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483070#comment-16483070 ] Steve Yeom commented on HIVE-19417: --- Hey [~sershe] Thanks for the review. I am following up comments of Eugene and Vineet. > Modify metastore to have/access persistent tables for stats > --- > > Key: HIVE-19417 > URL: https://issues.apache.org/jira/browse/HIVE-19417 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Attachments: HIVE-19417.01.patch, HIVE-19417.02.patch, > HIVE-19417.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (HIVE-19637) Add slow test report script to testutils
[
https://issues.apache.org/jira/browse/HIVE-19637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran resolved HIVE-19637.
--
Resolution: Fixed
Fix Version/s: 3.1.0
Committed to master. Thanks for the review!
> Add slow test report script to testutils
>
>
> Key: HIVE-19637
> URL: https://issues.apache.org/jira/browse/HIVE-19637
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19637.1.patch
>
>
> Wrote the attached utility script to find top K slow tests from precommit
> test url. Would like to get that committed to testutils so that its useful
> for everyone.
> {code:title=ascii mode}
> $ python gen-report.py -b 11102 -a
> Processing 1073 test xml reports from
> http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time:
> 73882.661 seconds]
> ##
> ██ 20806 TestCliDriver
> ███ 9601
> TestMiniLlapLocalCliDriver
> ███ 8210 TestSparkCliDriver
> ██ 2744 TestMinimrCliDriver
> █2262
> TestEncryptedHDFSCliDriver
> 2021
> TestMiniSparkOnYarnCliDriver
> 1808 TestHiveCli
> ███ 1566
> TestMiniLlapCliDriver
> ███ 1345
> TestReplicationScenarios
> ██ 1238
> TestMiniDruidCliDriver
> ██940
> TestNegativeCliDriver
> ██865 TestHBaseCliDriver
> █ 681
> TestMiniTezCliDriver
> █ 555
> TestTxnCommands2WithSplitUpdateAndVectorization
> █ 543 TestCompactor
> █ 528 TestTxnCommands2
> 378 TestStreaming
> 374
> TestBlobstoreCliDriver
> 328
> TestNegativeMinimrCliDriver
> 302
> TestTxnCommandsWithSplitUpdateAndVectorization
> 301 TestHCatClient
> 299 TestTxnCommands
> 261 TestTxnLoadData
> 258 TestAcidOnTez
> 240
> TestHBaseNegativeCliDriver
> Top 25 testcases in terms of execution time (in seconds).. [Total time:
> 63102.607 seconds]
> ###
> ██ 680
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
> █ 623
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]
> ███ 429
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]
> ███ 374
> TestSparkCliDriver_testCliDriver[vectorization_short_regress]
> ███ 374
> TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]
> 330
> TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]
> █ 238
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]
> 227
> TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]
> ███ 214
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
> ███ 211
> TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]
[jira] [Updated] (HIVE-19639) a transactional Hive table cannot be imported as an external table
[ https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-19639: -- Component/s: Transactions > a transactional Hive table cannot be imported as an external table > -- > > Key: HIVE-19639 > URL: https://issues.apache.org/jira/browse/HIVE-19639 > Project: Hive > Issue Type: Bug > Components: Transactions >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19639.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19639) a transactional Hive table cannot be imported as an external table
[ https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19639: Attachment: HIVE-19639.patch > a transactional Hive table cannot be imported as an external table > -- > > Key: HIVE-19639 > URL: https://issues.apache.org/jira/browse/HIVE-19639 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19639.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19639) a transactional Hive table cannot be imported as an external table
[ https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19639: Attachment: (was: HIVE-19639.patch) > a transactional Hive table cannot be imported as an external table > -- > > Key: HIVE-19639 > URL: https://issues.apache.org/jira/browse/HIVE-19639 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19639.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19639) a transactional Hive table cannot be imported as an external table
[ https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19639: Attachment: HIVE-19639.patch > a transactional Hive table cannot be imported as an external table > -- > > Key: HIVE-19639 > URL: https://issues.apache.org/jira/browse/HIVE-19639 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19639.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19639) a transactional Hive table cannot be imported as an external table
[ https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-19639: Status: Patch Available (was: Open) [~thejas] can you take a look? thnx > a transactional Hive table cannot be imported as an external table > -- > > Key: HIVE-19639 > URL: https://issues.apache.org/jira/browse/HIVE-19639 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > Attachments: HIVE-19639.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19639) a transactional Hive table cannot be imported as an external table
[ https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-19639: --- Assignee: Sergey Shelukhin > a transactional Hive table cannot be imported as an external table > -- > > Key: HIVE-19639 > URL: https://issues.apache.org/jira/browse/HIVE-19639 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work started] (HIVE-19637) Add slow test report script to testutils
[
https://issues.apache.org/jira/browse/HIVE-19637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-19637 started by Prasanth Jayachandran.
> Add slow test report script to testutils
>
>
> Key: HIVE-19637
> URL: https://issues.apache.org/jira/browse/HIVE-19637
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19637.1.patch
>
>
> Wrote the attached utility script to find top K slow tests from precommit
> test url. Would like to get that committed to testutils so that its useful
> for everyone.
> {code:title=ascii mode}
> $ python gen-report.py -b 11102 -a
> Processing 1073 test xml reports from
> http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time:
> 73882.661 seconds]
> ##
> ██ 20806 TestCliDriver
> ███ 9601
> TestMiniLlapLocalCliDriver
> ███ 8210 TestSparkCliDriver
> ██ 2744 TestMinimrCliDriver
> █2262
> TestEncryptedHDFSCliDriver
> 2021
> TestMiniSparkOnYarnCliDriver
> 1808 TestHiveCli
> ███ 1566
> TestMiniLlapCliDriver
> ███ 1345
> TestReplicationScenarios
> ██ 1238
> TestMiniDruidCliDriver
> ██940
> TestNegativeCliDriver
> ██865 TestHBaseCliDriver
> █ 681
> TestMiniTezCliDriver
> █ 555
> TestTxnCommands2WithSplitUpdateAndVectorization
> █ 543 TestCompactor
> █ 528 TestTxnCommands2
> 378 TestStreaming
> 374
> TestBlobstoreCliDriver
> 328
> TestNegativeMinimrCliDriver
> 302
> TestTxnCommandsWithSplitUpdateAndVectorization
> 301 TestHCatClient
> 299 TestTxnCommands
> 261 TestTxnLoadData
> 258 TestAcidOnTez
> 240
> TestHBaseNegativeCliDriver
> Top 25 testcases in terms of execution time (in seconds).. [Total time:
> 63102.607 seconds]
> ###
> ██ 680
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
> █ 623
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]
> ███ 429
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]
> ███ 374
> TestSparkCliDriver_testCliDriver[vectorization_short_regress]
> ███ 374
> TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]
> 330
> TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]
> █ 238
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]
> 227
> TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]
> ███ 214
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
> ███ 211
> TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]
> ███ 210
> TestMiniSparkO
[jira] [Commented] (HIVE-19523) Decimal truncation for trailing zeros in Hive 1.2.1
[
https://issues.apache.org/jira/browse/HIVE-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483051#comment-16483051
]
Xuefu Zhang commented on HIVE-19523:
{{HiveDecimal}} doesn't carry precision and scale info even though it's backed
by a {{BigDecimal}} object. Internally, Hive normalizes the {{BigDecimal}}
input when storing it as a {{HiveDecimal}} instance. If desired, you can call
{{HiveDecimal.enforcePrecisionScale()}} to generate an instance that has
certain precision/scale.
> Decimal truncation for trailing zeros in Hive 1.2.1
> ---
>
> Key: HIVE-19523
> URL: https://issues.apache.org/jira/browse/HIVE-19523
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.2.1, 2.3.1
>Reporter: vinay kant garg
>Priority: Critical
>
> This issue is related to the unnecessary truncation of zero's while
> serializing BigDecimal object to HiveDecimal object.
> BigDecimal object has info about scale and precision still serializing ended
> up modifying scale and precision metadata as well truncation of zero's.
> Eg: if our BigDecimal val is : 47.6300, scale = 4 and precision = 6. If I am
> serializing this to HiveDecimal by using API hive expose :
> static HiveDecimal create(BigDecimal b)
> static HiveDecimal create(BigDecimal b, boolean allowRounding)
> our output will be : val = 47.63, scale = 2 and precision = 4.
> or if IP: val = 47.00, scale = 2 and precision = 4 then
> OP: val = 47, scale = 0 and precision = 2
> *In above example if we see there is no DATA CORRUPTION because 47.6300 is
> equivalent to 47.63 or 47.00 is equivalent to 47 but later on if this data is
> used to identify whether it is integer or decimal then we may run into weird
> issues because 47.00 is decimal but 47 is an integer*.
> I am able to reproduce this issue even with standalone program:
> *import java.math.BigDecimal;*
> *import org.apache.hadoop.hive.common.type.HiveDecimal;*
> *public class testHivedecimal {*
> *public static void main(String[] argv){*
> *BigDecimal bd = new BigDecimal ("47.6300");*
> *bd.setScale(4);*
> *System.out.println("bigdecimal object created and value is : " +
> bd.toString());*
> *System.out.println("precision : "+ bd.precision());*
> *System.out.println("scale : "+ bd.scale());*
> *HiveDecimal hv = HiveDecimal.create(bd);*
> *String str = hv.toString();*
> *System.out.println("value after serialization of bigdecimal to hivedecimal
> : " + str);*
> *System.out.println("precision after hivedecimal : "+ hv.precision());*
> *System.out.println("scale after hivedecimal : "+ hv.scale());*
> *}*
> *}*
> you can use above program to reproduce issue:
> To Compile:
> 1) use jdk8_u72
> 2) command : javac -cp hive-common-1.2.1000.2.6.0.3-8.jar:.
> testHivedecimal.java
> To Run:
> 1) use jdk8_u72
> 2) command : java -cp hive-common-1.2.1000.2.6.0.3-8.jar:. testHivedecimal
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19308) Provide an Arrow stream reader for external LLAP clients
[ https://issues.apache.org/jira/browse/HIVE-19308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-19308: -- Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Committed to master > Provide an Arrow stream reader for external LLAP clients > - > > Key: HIVE-19308 > URL: https://issues.apache.org/jira/browse/HIVE-19308 > Project: Hive > Issue Type: Task > Components: llap >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19308.1.patch, HIVE-19308.10.patch, > HIVE-19308.2.patch, HIVE-19308.3.patch, HIVE-19308.4.patch, > HIVE-19308.5.patch, HIVE-19308.6.patch, HIVE-19308.7.patch, > HIVE-19308.8.patch, HIVE-19308.9.patch > > > This is a sub-class of LlapBaseRecordReader that wraps the socket inputStream > and produces Arrow batches for an external client. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19599) Release Notes : Highlighting backwards incompatible changes
[ https://issues.apache.org/jira/browse/HIVE-19599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483045#comment-16483045 ] Vineet Garg commented on HIVE-19599: [~ekoifman] Can you provide the text which should be highlighted? > Release Notes : Highlighting backwards incompatible changes > --- > > Key: HIVE-19599 > URL: https://issues.apache.org/jira/browse/HIVE-19599 > Project: Hive > Issue Type: Bug > Components: Documentation >Affects Versions: 3.0.0 >Reporter: Eugene Koifman >Assignee: Vineet Garg >Priority: Blocker > > We need to highlight backwards incompatible changes. A list Jira titles > won't be sufficient. > For example, tables with Acid V1 (pre 3.0) data has to be major compacted > before upgrade and may not process any update/delete/merge until after > upgrade. Not doing so may result in data corruption/loss. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18117) Create TestCliDriver for HDFS EC
[
https://issues.apache.org/jira/browse/HIVE-18117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483043#comment-16483043
]
Hive QA commented on HIVE-18117:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
26s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
17s{color} | {color:blue} shims/common in master has 6 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
22s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} itests/util in master has 55 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2320 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} testutils/ptest2 in master has 24 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
28s{color} | {color:red} qtest in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} The patch common passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch 0.23 passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} itests/qtest: The patch generated 1 new + 0 unchanged
- 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} The patch util passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} ql: The patch generated 0 new + 76 unchanged - 30
fixed = 76 total (was 106) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} The patch ptest2 passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
12s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-3/dev-support/hive-personality.sh
|
| git revision | master / c6a5d6f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-3/yetus/patch-mvninstall-itests_qtest.txt
|
| c
[jira] [Commented] (HIVE-19417) Modify metastore to have/access persistent tables for stats
[ https://issues.apache.org/jira/browse/HIVE-19417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483037#comment-16483037 ] Sergey Shelukhin commented on HIVE-19417: - [~steveyeom2017] can you address above comments from [~ekoifman] and [~vgarg]? Other than that it makes sense to me > Modify metastore to have/access persistent tables for stats > --- > > Key: HIVE-19417 > URL: https://issues.apache.org/jira/browse/HIVE-19417 > Project: Hive > Issue Type: Sub-task > Components: Transactions >Affects Versions: 3.0.0 >Reporter: Steve Yeom >Assignee: Steve Yeom >Priority: Major > Attachments: HIVE-19417.01.patch, HIVE-19417.02.patch, > HIVE-19417.03.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19421) Upgrade version of Jetty to 9.3.20.v20170531
[ https://issues.apache.org/jira/browse/HIVE-19421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sahil Takiar updated HIVE-19421: Resolution: Fixed Fix Version/s: 3.1.0 Status: Resolved (was: Patch Available) Pushed to master. > Upgrade version of Jetty to 9.3.20.v20170531 > > > Key: HIVE-19421 > URL: https://issues.apache.org/jira/browse/HIVE-19421 > Project: Hive > Issue Type: Improvement >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani >Priority: Major > Fix For: 3.1.0 > > Attachments: HIVE-19421.1.patch, HIVE-19421.2.patch, > HIVE-19421.3.patch, HIVE-19421.3.patch, HIVE-19421.3.patch, > HIVE-19421.4.patch, HIVE-19421.4.patch, HIVE-19421.4.patch, HIVE-19421.4.patch > > > Move Jetty up to 9.3.20.v20170531 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19617) Rename test tables to avoid collisions during execution in batches
[
https://issues.apache.org/jira/browse/HIVE-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483019#comment-16483019
]
Jesus Camacho Rodriguez commented on HIVE-19617:
I've pushed this one to master. I have disabled {{sample2}} and {{sample4}},
and will create a follow-up for them. I will keep this issue open to monitor
the next few test runs.
> Rename test tables to avoid collisions during execution in batches
> --
>
> Key: HIVE-19617
> URL: https://issues.apache.org/jira/browse/HIVE-19617
> Project: Hive
> Issue Type: Test
> Components: Testing Infrastructure
>Affects Versions: 3.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-19617.01.patch, HIVE-19617.02.patch,
> HIVE-19617.03.patch, HIVE-19617.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19637) Add slow test report script to testutils
[
https://issues.apache.org/jira/browse/HIVE-19637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16483007#comment-16483007
]
Jesus Camacho Rodriguez commented on HIVE-19637:
Fantastic... +1
> Add slow test report script to testutils
>
>
> Key: HIVE-19637
> URL: https://issues.apache.org/jira/browse/HIVE-19637
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19637.1.patch
>
>
> Wrote the attached utility script to find top K slow tests from precommit
> test url. Would like to get that committed to testutils so that its useful
> for everyone.
> {code:title=ascii mode}
> $ python gen-report.py -b 11102 -a
> Processing 1073 test xml reports from
> http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time:
> 73882.661 seconds]
> ##
> ██ 20806 TestCliDriver
> ███ 9601
> TestMiniLlapLocalCliDriver
> ███ 8210 TestSparkCliDriver
> ██ 2744 TestMinimrCliDriver
> █2262
> TestEncryptedHDFSCliDriver
> 2021
> TestMiniSparkOnYarnCliDriver
> 1808 TestHiveCli
> ███ 1566
> TestMiniLlapCliDriver
> ███ 1345
> TestReplicationScenarios
> ██ 1238
> TestMiniDruidCliDriver
> ██940
> TestNegativeCliDriver
> ██865 TestHBaseCliDriver
> █ 681
> TestMiniTezCliDriver
> █ 555
> TestTxnCommands2WithSplitUpdateAndVectorization
> █ 543 TestCompactor
> █ 528 TestTxnCommands2
> 378 TestStreaming
> 374
> TestBlobstoreCliDriver
> 328
> TestNegativeMinimrCliDriver
> 302
> TestTxnCommandsWithSplitUpdateAndVectorization
> 301 TestHCatClient
> 299 TestTxnCommands
> 261 TestTxnLoadData
> 258 TestAcidOnTez
> 240
> TestHBaseNegativeCliDriver
> Top 25 testcases in terms of execution time (in seconds).. [Total time:
> 63102.607 seconds]
> ###
> ██ 680
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
> █ 623
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]
> ███ 429
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]
> ███ 374
> TestSparkCliDriver_testCliDriver[vectorization_short_regress]
> ███ 374
> TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]
> 330
> TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]
> █ 238
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]
> 227
> TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]
> ███ 214
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
> ███ 211
> TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]
> ███
[jira] [Updated] (HIVE-6998) Select query can only support maximum 128 distinct expressions
[
https://issues.apache.org/jira/browse/HIVE-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-6998:
-
Attachment: HIVE-6998.02.patch
> Select query can only support maximum 128 distinct expressions
> --
>
> Key: HIVE-6998
> URL: https://issues.apache.org/jira/browse/HIVE-6998
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Serializers/Deserializers
>Affects Versions: 0.14.0
>Reporter: Chaoyu Tang
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-6998.01.patch, HIVE-6998.02.patch
>
>
> Select query can only support maximum 128 distinct expressions. Otherwise,
> you will be thrown ArrayIndexOutOfBoundsException. For a query like:
> select count(distinct c1), count(distinct c2), count(distinct c3),
> count(distinct c4), count(distinct c5), count(distinct c6), ,
> count(distinct c128), count(distinct c129) from tbl_129columns;
> you will get error like:
> {code}
> java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while
> closing operators
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
> Caused by: java.lang.RuntimeException: Hive Runtime Error while closing
> operators
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
> at
> org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:223)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:695)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1141)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:579)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:591)
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
> ... 10 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1099)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1138)
> ... 15 more
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ArrayIndexOutOfBoundsException: -128
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:327)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:796)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.forward(GroupByOperator.java:1064)
> at
> org.apache.hadoop.hive.ql.exec.GroupByOperator.flush(GroupByOperator.java:1082)
> ... 16 more
> Caused by: java.lang.ArrayIndexOutOfBoundsException: -128
> at java.util.ArrayList.get(ArrayList.java:324)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:838)
> at
> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.serialize(BinarySortableSerDe.java:600)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.toHiveKey(ReduceSinkOperator.java:401)
> at
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:320)
> ... 19 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19523) Decimal truncation for trailing zeros in Hive 1.2.1
[
https://issues.apache.org/jira/browse/HIVE-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482991#comment-16482991
]
Ashutosh Chauhan commented on HIVE-19523:
-
{{HiveDecimal}} is a class internal to Hive. Its not suppose to be use outside
of Hive. [~vgarg47] Can you expand on usecase? Do you have a hive query which
exhibits the issue?
> Decimal truncation for trailing zeros in Hive 1.2.1
> ---
>
> Key: HIVE-19523
> URL: https://issues.apache.org/jira/browse/HIVE-19523
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.2.1, 2.3.1
>Reporter: vinay kant garg
>Priority: Critical
>
> This issue is related to the unnecessary truncation of zero's while
> serializing BigDecimal object to HiveDecimal object.
> BigDecimal object has info about scale and precision still serializing ended
> up modifying scale and precision metadata as well truncation of zero's.
> Eg: if our BigDecimal val is : 47.6300, scale = 4 and precision = 6. If I am
> serializing this to HiveDecimal by using API hive expose :
> static HiveDecimal create(BigDecimal b)
> static HiveDecimal create(BigDecimal b, boolean allowRounding)
> our output will be : val = 47.63, scale = 2 and precision = 4.
> or if IP: val = 47.00, scale = 2 and precision = 4 then
> OP: val = 47, scale = 0 and precision = 2
> *In above example if we see there is no DATA CORRUPTION because 47.6300 is
> equivalent to 47.63 or 47.00 is equivalent to 47 but later on if this data is
> used to identify whether it is integer or decimal then we may run into weird
> issues because 47.00 is decimal but 47 is an integer*.
> I am able to reproduce this issue even with standalone program:
> *import java.math.BigDecimal;*
> *import org.apache.hadoop.hive.common.type.HiveDecimal;*
> *public class testHivedecimal {*
> *public static void main(String[] argv){*
> *BigDecimal bd = new BigDecimal ("47.6300");*
> *bd.setScale(4);*
> *System.out.println("bigdecimal object created and value is : " +
> bd.toString());*
> *System.out.println("precision : "+ bd.precision());*
> *System.out.println("scale : "+ bd.scale());*
> *HiveDecimal hv = HiveDecimal.create(bd);*
> *String str = hv.toString();*
> *System.out.println("value after serialization of bigdecimal to hivedecimal
> : " + str);*
> *System.out.println("precision after hivedecimal : "+ hv.precision());*
> *System.out.println("scale after hivedecimal : "+ hv.scale());*
> *}*
> *}*
> you can use above program to reproduce issue:
> To Compile:
> 1) use jdk8_u72
> 2) command : javac -cp hive-common-1.2.1000.2.6.0.3-8.jar:.
> testHivedecimal.java
> To Run:
> 1) use jdk8_u72
> 2) command : java -cp hive-common-1.2.1000.2.6.0.3-8.jar:. testHivedecimal
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19632) Remove webapps directory from standalone jar
[ https://issues.apache.org/jira/browse/HIVE-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19632: - Status: Patch Available (was: Open) > Remove webapps directory from standalone jar > > > Key: HIVE-19632 > URL: https://issues.apache.org/jira/browse/HIVE-19632 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19632.1.patch, HIVE-19632.2.patch > > > JDBC standalone jar contains webapps static files which just adds to the jar > size and are not required by the clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19632) Remove webapps directory from standalone jar
[ https://issues.apache.org/jira/browse/HIVE-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19632: - Attachment: HIVE-19632.2.patch > Remove webapps directory from standalone jar > > > Key: HIVE-19632 > URL: https://issues.apache.org/jira/browse/HIVE-19632 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19632.1.patch, HIVE-19632.2.patch > > > JDBC standalone jar contains webapps static files which just adds to the jar > size and are not required by the clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19638) Configuration not passed to ORC Reader.Options
[ https://issues.apache.org/jira/browse/HIVE-19638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rentao Wu reassigned HIVE-19638: > Configuration not passed to ORC Reader.Options > -- > > Key: HIVE-19638 > URL: https://issues.apache.org/jira/browse/HIVE-19638 > Project: Hive > Issue Type: Bug > Components: Configuration, File Formats, ORC >Affects Versions: 2.3.2, 2.3.1, 2.3.0 >Reporter: Rentao Wu >Assignee: Rentao Wu >Priority: Major > > Configuration is not passed to ORC's Reader.Option in OrcFileInputFormat > which causes some [ORC > configurations|https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/Reader.java#L170-L176] > to not be able to be picked up. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19421) Upgrade version of Jetty to 9.3.20.v20170531
[
https://issues.apache.org/jira/browse/HIVE-19421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482977#comment-16482977
]
Hive QA commented on HIVE-19421:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924369/HIVE-19421.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 14386 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/2/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924369 - PreCommit-HIVE-Build
> Upgrade version of Jetty to 9.3.20.v20170531
>
>
> Key: HIVE-19421
> URL: https://issues.apache.org/jira/browse/HIVE-19421
> Project: Hive
> Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-19421.1.patch, HIVE-19421.2.patch,
> HIVE-19421.3.patch, HIVE-19421.3.patch, HIVE-19421.3.patch,
> HIVE-19421.4.patch, HIVE-19421.4.patch, HIVE-19421.4.patch, HIVE-19421.4.patch
>
>
> Move Jetty up to 9.3.20.v20170531
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19523) Decimal truncation for trailing zeros in Hive 1.2.1
[
https://issues.apache.org/jira/browse/HIVE-19523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482951#comment-16482951
]
Sergey Shelukhin commented on HIVE-19523:
-
[~ashutoshc] [~jcamachorodriguez] do you have any input? Seems like this would
be behavior change for Hive. I am not sure why Hive would change
precision/scale, but I'm assuming it's there for some reason... not changing it
seems more logical to me.
> Decimal truncation for trailing zeros in Hive 1.2.1
> ---
>
> Key: HIVE-19523
> URL: https://issues.apache.org/jira/browse/HIVE-19523
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.2.1, 2.3.1
>Reporter: vinay kant garg
>Priority: Critical
>
> This issue is related to the unnecessary truncation of zero's while
> serializing BigDecimal object to HiveDecimal object.
> BigDecimal object has info about scale and precision still serializing ended
> up modifying scale and precision metadata as well truncation of zero's.
> Eg: if our BigDecimal val is : 47.6300, scale = 4 and precision = 6. If I am
> serializing this to HiveDecimal by using API hive expose :
> static HiveDecimal create(BigDecimal b)
> static HiveDecimal create(BigDecimal b, boolean allowRounding)
> our output will be : val = 47.63, scale = 2 and precision = 4.
> or if IP: val = 47.00, scale = 2 and precision = 4 then
> OP: val = 47, scale = 0 and precision = 2
> *In above example if we see there is no DATA CORRUPTION because 47.6300 is
> equivalent to 47.63 or 47.00 is equivalent to 47 but later on if this data is
> used to identify whether it is integer or decimal then we may run into weird
> issues because 47.00 is decimal but 47 is an integer*.
> I am able to reproduce this issue even with standalone program:
> *import java.math.BigDecimal;*
> *import org.apache.hadoop.hive.common.type.HiveDecimal;*
> *public class testHivedecimal {*
> *public static void main(String[] argv){*
> *BigDecimal bd = new BigDecimal ("47.6300");*
> *bd.setScale(4);*
> *System.out.println("bigdecimal object created and value is : " +
> bd.toString());*
> *System.out.println("precision : "+ bd.precision());*
> *System.out.println("scale : "+ bd.scale());*
> *HiveDecimal hv = HiveDecimal.create(bd);*
> *String str = hv.toString();*
> *System.out.println("value after serialization of bigdecimal to hivedecimal
> : " + str);*
> *System.out.println("precision after hivedecimal : "+ hv.precision());*
> *System.out.println("scale after hivedecimal : "+ hv.scale());*
> *}*
> *}*
> you can use above program to reproduce issue:
> To Compile:
> 1) use jdk8_u72
> 2) command : javac -cp hive-common-1.2.1000.2.6.0.3-8.jar:.
> testHivedecimal.java
> To Run:
> 1) use jdk8_u72
> 2) command : java -cp hive-common-1.2.1000.2.6.0.3-8.jar:. testHivedecimal
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19421) Upgrade version of Jetty to 9.3.20.v20170531
[
https://issues.apache.org/jira/browse/HIVE-19421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482949#comment-16482949
]
Hive QA commented on HIVE-19421:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 5m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 4m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 5m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile
xml |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-2/dev-support/hive-personality.sh
|
| git revision | master / c6a5d6f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: . itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-2/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade version of Jetty to 9.3.20.v20170531
>
>
> Key: HIVE-19421
> URL: https://issues.apache.org/jira/browse/HIVE-19421
> Project: Hive
> Issue Type: Improvement
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
> Attachments: HIVE-19421.1.patch, HIVE-19421.2.patch,
> HIVE-19421.3.patch, HIVE-19421.3.patch, HIVE-19421.3.patch,
> HIVE-19421.4.patch, HIVE-19421.4.patch, HIVE-19421.4.patch, HIVE-19421.4.patch
>
>
> Move Jetty up to 9.3.20.v20170531
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19632) Remove webapps directory from standalone jar
[ https://issues.apache.org/jira/browse/HIVE-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482944#comment-16482944 ] Prasanth Jayachandran commented on HIVE-19632: -- [~thejas] can you please take a look? Small patch. This shouldn't affect any tests. > Remove webapps directory from standalone jar > > > Key: HIVE-19632 > URL: https://issues.apache.org/jira/browse/HIVE-19632 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19632.1.patch > > > JDBC standalone jar contains webapps static files which just adds to the jar > size and are not required by the clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19632) Remove webapps directory from standalone jar
[ https://issues.apache.org/jira/browse/HIVE-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-19632: - Attachment: HIVE-19632.1.patch > Remove webapps directory from standalone jar > > > Key: HIVE-19632 > URL: https://issues.apache.org/jira/browse/HIVE-19632 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > Attachments: HIVE-19632.1.patch > > > JDBC standalone jar contains webapps static files which just adds to the jar > size and are not required by the clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19617) Rename test tables to avoid collisions during execution in batches
[ https://issues.apache.org/jira/browse/HIVE-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482939#comment-16482939 ] Gunther Hagleitner commented on HIVE-19617: --- +1. The saga for blue builds continues. > Rename test tables to avoid collisions during execution in batches > -- > > Key: HIVE-19617 > URL: https://issues.apache.org/jira/browse/HIVE-19617 > Project: Hive > Issue Type: Test > Components: Testing Infrastructure >Affects Versions: 3.1.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Critical > Attachments: HIVE-19617.01.patch, HIVE-19617.02.patch, > HIVE-19617.03.patch, HIVE-19617.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19633) Remove/Migrate Minimr tests
[ https://issues.apache.org/jira/browse/HIVE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-19633: Assignee: Prasanth Jayachandran > Remove/Migrate Minimr tests > --- > > Key: HIVE-19633 > URL: https://issues.apache.org/jira/browse/HIVE-19633 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.1.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran >Priority: Major > > MR has been deprecated for a long time. Minimr tests are incredibly slow. We > should remove the tests or migrate to faster options for coverage. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19637) Add slow test report script to testutils
[
https://issues.apache.org/jira/browse/HIVE-19637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth Jayachandran updated HIVE-19637:
-
Attachment: HIVE-19637.1.patch
> Add slow test report script to testutils
>
>
> Key: HIVE-19637
> URL: https://issues.apache.org/jira/browse/HIVE-19637
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19637.1.patch
>
>
> Wrote the attached utility script to find top K slow tests from precommit
> test url. Would like to get that committed to testutils so that its useful
> for everyone.
> {code:title=ascii mode}
> $ python gen-report.py -b 11102 -a
> Processing 1073 test xml reports from
> http://104.198.109.242/logs/PreCommit-HIVE-Build-11102/test-results/..
> Top 25 testsuite in terms of execution time (in seconds).. [Total time:
> 73882.661 seconds]
> ##
> ██ 20806 TestCliDriver
> ███ 9601
> TestMiniLlapLocalCliDriver
> ███ 8210 TestSparkCliDriver
> ██ 2744 TestMinimrCliDriver
> █2262
> TestEncryptedHDFSCliDriver
> 2021
> TestMiniSparkOnYarnCliDriver
> 1808 TestHiveCli
> ███ 1566
> TestMiniLlapCliDriver
> ███ 1345
> TestReplicationScenarios
> ██ 1238
> TestMiniDruidCliDriver
> ██940
> TestNegativeCliDriver
> ██865 TestHBaseCliDriver
> █ 681
> TestMiniTezCliDriver
> █ 555
> TestTxnCommands2WithSplitUpdateAndVectorization
> █ 543 TestCompactor
> █ 528 TestTxnCommands2
> 378 TestStreaming
> 374
> TestBlobstoreCliDriver
> 328
> TestNegativeMinimrCliDriver
> 302
> TestTxnCommandsWithSplitUpdateAndVectorization
> 301 TestHCatClient
> 299 TestTxnCommands
> 261 TestTxnLoadData
> 258 TestAcidOnTez
> 240
> TestHBaseNegativeCliDriver
> Top 25 testcases in terms of execution time (in seconds).. [Total time:
> 63102.607 seconds]
> ###
> ██ 680
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_reducers_power_two]
> █ 623
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_map_operators]
> ███ 429
> TestMinimrCliDriver_testCliDriver[infer_bucket_sort_dyn_part]
> ███ 374
> TestSparkCliDriver_testCliDriver[vectorization_short_regress]
> ███ 374
> TestMiniLlapLocalCliDriver_testCliDriver[vectorization_short_regress]
> 330
> TestMiniDruidCliDriver_testCliDriver[druidmini_dynamic_partition]
> █ 238
> TestMiniLlapLocalCliDriver_testCliDriver[vector_outer_join5]
> 227
> TestMiniDruidCliDriver_testCliDriver[druidmini_test_insert]
> ███ 214
> TestEncryptedHDFSCliDriver_testCliDriver[encryption_auto_purge_tables]
> ███ 211
> TestMiniLlapCliDriver_testCliDriver[unionDistinct_1]
> ███ 210
> TestMiniSparkOnYarnCliDriver_te
