[jira] [Commented] (HIVE-20754) JDBC: Add some missing classes to jdbc standalone jar (follow up to HIVE-19801)
[
https://issues.apache.org/jira/browse/HIVE-20754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913953#comment-16913953
]
Hive QA commented on HIVE-20754:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 14 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18383/dev-support/hive-personality.sh
|
| git revision | master / db59ec6 |
| Default Java | 1.8.0_111 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18383/yetus/whitespace-tabs.txt
|
| modules | C: jdbc U: jdbc |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18383/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> JDBC: Add some missing classes to jdbc standalone jar (follow up to
> HIVE-19801)
> ---
>
> Key: HIVE-20754
> URL: https://issues.apache.org/jira/browse/HIVE-20754
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 3.1.0
>Reporter: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20754.1.patch
>
>
> Some more classes are needed in a secure cluster
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22139) Will not pad Decimal numbers with trailing zeros if select from value
[
https://issues.apache.org/jira/browse/HIVE-22139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang updated HIVE-22139:
---
Description:
How to reproduce:
{code:sql}
hive> SELECT CAST(1 AS decimal(38, 18));
OK
1
Time taken: 0.226 seconds, Fetched: 1 row(s)
hive> CREATE TABLE HIVE_22139 AS SELECT CAST(1 AS decimal(38, 18)) as c;
OK
Time taken: 2.278 seconds
hive> SELECT * FROM HIVE_22139;
OK
1.00
Time taken: 0.07 seconds, Fetched: 1 row(s)
{code}
was:
How to reproduce:
{code:sql}
// code placeholder
{code}
> Will not pad Decimal numbers with trailing zeros if select from value
> -
>
> Key: HIVE-22139
> URL: https://issues.apache.org/jira/browse/HIVE-22139
> Project: Hive
> Issue Type: Bug
> Components: SQL
>Affects Versions: 3.1.1
>Reporter: Yuming Wang
>Priority: Major
>
> How to reproduce:
> {code:sql}
> hive> SELECT CAST(1 AS decimal(38, 18));
> OK
> 1
> Time taken: 0.226 seconds, Fetched: 1 row(s)
> hive> CREATE TABLE HIVE_22139 AS SELECT CAST(1 AS decimal(38, 18)) as c;
> OK
> Time taken: 2.278 seconds
> hive> SELECT * FROM HIVE_22139;
> OK
> 1.00
> Time taken: 0.07 seconds, Fetched: 1 row(s)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22121) Turning on hive.tez.bucket.pruning produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-22121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913946#comment-16913946
]
Hive QA commented on HIVE-22121:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12978324/HIVE-22121.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16745 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=110)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18382/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18382/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18382/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12978324 - PreCommit-HIVE-Build
> Turning on hive.tez.bucket.pruning produce wrong results
>
>
> Key: HIVE-22121
> URL: https://issues.apache.org/jira/browse/HIVE-22121
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22121.1patch, HIVE-22121.2.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.query.results.cache.enabled=false;
> set hive.optimize.ppd.storage=true;
> set hive.optimize.index.filter=true;
> set hive.tez.bucket.pruning=true;
> CREATE TABLE `test_table`(
>`col_1` int,
>`col_2` string,
>`col_3` string)
> CLUSTERED BY (
>col_1)
> INTO 4 BUCKETS;
> insert into test_table values(1, 'one', 'ONE'), (2, 'two', 'TWO'),
> (3,'three','THREE'),(4,'four','FOUR');
> select * from test_table;
> explain select col_1, col_2, col_3 from test_table where col_1 <> 2 order by
> col_2;
> select col_1, col_2, col_3 from test_table where col_1 <> 2 order by col_2;
> {code}
> Above sql query produce zero rows.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22121) Turning on hive.tez.bucket.pruning produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-22121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913923#comment-16913923
]
Hive QA commented on HIVE-22121:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
37s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
8s{color} | {color:blue} ql in master has 2254 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 2 new + 13 unchanged - 0 fixed
= 15 total (was 13) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18382/dev-support/hive-personality.sh
|
| git revision | master / db59ec6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18382/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18382/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Turning on hive.tez.bucket.pruning produce wrong results
>
>
> Key: HIVE-22121
> URL: https://issues.apache.org/jira/browse/HIVE-22121
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22121.1patch, HIVE-22121.2.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.query.results.cache.enabled=false;
> set hive.optimize.ppd.storage=true;
> set hive.optimize.index.filter=true;
> set hive.tez.bucket.pruning=true;
> CREATE TABLE `test_table`(
>`col_1` int,
>`col_2` string,
>`col_3` string)
> CLUSTERED BY (
>col_1)
> INTO 4 BUCKETS;
> insert into test_table values(1, 'one', 'ONE'), (2, 'two', 'TWO'),
> (3,'three','THREE'),(4,'four','FOUR');
> select * from test_table;
> explain select col_1, col_2, col_3 from test_table where col_1 <> 2 order by
> col_2;
> select col_1, col_2, col_3 from test_table where col_1 <> 2 order by col_2;
> {code}
> Above sql query produce zero rows.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22126) hive-exec packaging should shade guava
[ https://issues.apache.org/jira/browse/HIVE-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dinesh Garg reassigned HIVE-22126: -- Assignee: (was: Vihang Karajgaonkar) > hive-exec packaging should shade guava > -- > > Key: HIVE-22126 > URL: https://issues.apache.org/jira/browse/HIVE-22126 > Project: Hive > Issue Type: Bug >Reporter: Vihang Karajgaonkar >Priority: Major > > The ql/pom.xml includes complete guava library into hive-exec.jar > https://github.com/apache/hive/blob/master/ql/pom.xml#L990 This causes a > problems for downstream clients of hive which have hive-exec.jar in their > classpath since they are pinned to the same guava version as that of hive. > We should shade guava classes so that other components which depend on > hive-exec can independently use a different version of guava as needed. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21698) TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while localizing resources
[
https://issues.apache.org/jira/browse/HIVE-21698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naresh P R updated HIVE-21698:
--
Attachment: HIVE-21698.1.patch
> TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while
> localizing resources
> --
>
> Key: HIVE-21698
> URL: https://issues.apache.org/jira/browse/HIVE-21698
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Tez
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-21698.1.patch, HIVE-21698.patch
>
>
> The cause of exception is due to [this loops
> over|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L633]
> {{newFilesNotFromConf}} and tries to access {{newResources.get(i)}} in that.
> Size of {{newResources}} is determined by [utils.localizeTempFiles()
> here|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L629]
> which may return a smaller list depending upon the {{skipFilesFromConf}} and
> this would result in {{newResources.size()}} < {{newFilesNotFromConf.length}}
> causing IndexOutOfBounds.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21698) TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while localizing resources
[
https://issues.apache.org/jira/browse/HIVE-21698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913864#comment-16913864
]
Hive QA commented on HIVE-21698:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12978320/HIVE-21698.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16745 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime
(batchId=283)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18381/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18381/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18381/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12978320 - PreCommit-HIVE-Build
> TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while
> localizing resources
> --
>
> Key: HIVE-21698
> URL: https://issues.apache.org/jira/browse/HIVE-21698
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Tez
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-21698.patch
>
>
> The cause of exception is due to [this loops
> over|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L633]
> {{newFilesNotFromConf}} and tries to access {{newResources.get(i)}} in that.
> Size of {{newResources}} is determined by [utils.localizeTempFiles()
> here|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L629]
> which may return a smaller list depending upon the {{skipFilesFromConf}} and
> this would result in {{newResources.size()}} < {{newFilesNotFromConf.length}}
> causing IndexOutOfBounds.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21698) TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while localizing resources
[
https://issues.apache.org/jira/browse/HIVE-21698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913847#comment-16913847
]
Hive QA commented on HIVE-21698:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
14s{color} | {color:blue} ql in master has 2254 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 3 new + 89 unchanged - 3 fixed
= 92 total (was 92) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18381/dev-support/hive-personality.sh
|
| git revision | master / db59ec6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18381/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18381/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while
> localizing resources
> --
>
> Key: HIVE-21698
> URL: https://issues.apache.org/jira/browse/HIVE-21698
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Tez
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-21698.patch
>
>
> The cause of exception is due to [this loops
> over|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L633]
> {{newFilesNotFromConf}} and tries to access {{newResources.get(i)}} in that.
> Size of {{newResources}} is determined by [utils.localizeTempFiles()
> here|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L629]
> which may return a smaller list depending upon the {{skipFilesFromConf}} and
> this would result in {{newResources.size()}} < {{newFilesNotFromConf.length}}
> causing IndexOutOfBounds.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22135) Remove Hive.get().getConf() call and instead pass the configuration
[
https://issues.apache.org/jira/browse/HIVE-22135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913835#comment-16913835
]
Rajesh Balamohan commented on HIVE-22135:
-
Thanks for creating this ticket [~vgarg] . Is this codepath expensive when
accessed from different threads or is it expensive due to any other reason?.
> Remove Hive.get().getConf() call and instead pass the configuration
> ---
>
> Key: HIVE-22135
> URL: https://issues.apache.org/jira/browse/HIVE-22135
> Project: Hive
> Issue Type: Task
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Priority: Major
>
> There are multiple places where {{Hive.get().getConf()}} is used to get to
> hive configuration. This static call could be expensive and should be
> avoided, instead configuration should be passed on to where ever it is
> required.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22074) Slow compilation due to IN to OR transformation
[
https://issues.apache.org/jira/browse/HIVE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913758#comment-16913758
]
Hive QA commented on HIVE-22074:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12978325/HIVE-22074.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16745 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18380/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18380/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18380/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12978325 - PreCommit-HIVE-Build
> Slow compilation due to IN to OR transformation
> ---
>
> Key: HIVE-22074
> URL: https://issues.apache.org/jira/browse/HIVE-22074
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22074.1.patch, HIVE-22074.2.patch,
> HIVE-22074.3.patch, HIVE-22074.4.patch, HIVE-22074.5.patch, HIVE-22074.6.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently Hive transform IN expressions to OR to apply various CBO rules.
> This incur significant performance hit if IN consist of large number of
> expressions.
> It is better to not transform IN expressions to OR in such cases because
> overall benefit of various optimizations/transformations is unrealized due to
> the compilation overhead
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22074) Slow compilation due to IN to OR transformation
[
https://issues.apache.org/jira/browse/HIVE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913718#comment-16913718
]
Hive QA commented on HIVE-22074:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 61 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 2254 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 2 new + 259 unchanged - 2
fixed = 261 total (was 261) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18380/dev-support/hive-personality.sh
|
| git revision | master / db59ec6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18380/yetus/diff-checkstyle-ql.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18380/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Slow compilation due to IN to OR transformation
> ---
>
> Key: HIVE-22074
> URL: https://issues.apache.org/jira/browse/HIVE-22074
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22074.1.patch, HIVE-22074.2.patch,
> HIVE-22074.3.patch, HIVE-22074.4.patch, HIVE-22074.5.patch, HIVE-22074.6.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Currently Hive transform IN expressions to OR to apply various CBO rules.
> This incur significant performance hit if IN consist of large number of
> expressions.
> It is better to not transform IN expressions to OR in such cases because
> overall benefit of various optimizations/transformations is unrealized due to
> the compilation overhead
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-19667) Remove distribution management tag from pom.xml
[ https://issues.apache.org/jira/browse/HIVE-19667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913710#comment-16913710 ] Alan Gates commented on HIVE-19667: --- Ported to branch-3 and branch-3.1 as well so we can make releases on those branches. > Remove distribution management tag from pom.xml > --- > > Key: HIVE-19667 > URL: https://issues.apache.org/jira/browse/HIVE-19667 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0, 3.2.0, 3.1.2 > > Attachments: HIVE-19667.1.patch > > > This tag overrides apaches configuration in maven settings file and makes it > impossible to publish maven artifacts. There is no way around to it either. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-19667) Remove distribution management tag from pom.xml
[ https://issues.apache.org/jira/browse/HIVE-19667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-19667: -- Fix Version/s: 3.1.2 3.2.0 > Remove distribution management tag from pom.xml > --- > > Key: HIVE-19667 > URL: https://issues.apache.org/jira/browse/HIVE-19667 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Fix For: 4.0.0, 3.2.0, 3.1.2 > > Attachments: HIVE-19667.1.patch > > > This tag overrides apaches configuration in maven settings file and makes it > impossible to publish maven artifacts. There is no way around to it either. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22138) Upgrade script for Hive version
[ https://issues.apache.org/jira/browse/HIVE-22138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam An reassigned HIVE-22138: - > Upgrade script for Hive version > --- > > Key: HIVE-22138 > URL: https://issues.apache.org/jira/browse/HIVE-22138 > Project: Hive > Issue Type: Task >Affects Versions: 3.1.0 >Reporter: Sam An >Assignee: Sam An >Priority: Major > > Hive metastore and Hive schema version will be bumped to 3.1.2000, and we > need to have corresponding sql script to handle that. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22121) Turning on hive.tez.bucket.pruning produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-22121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913609#comment-16913609
]
Gopal V commented on HIVE-22121:
[~vgarg]: add a positive test with =, to show the explain extended having the
buckets listed?
> Turning on hive.tez.bucket.pruning produce wrong results
>
>
> Key: HIVE-22121
> URL: https://issues.apache.org/jira/browse/HIVE-22121
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22121.1patch, HIVE-22121.2.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.query.results.cache.enabled=false;
> set hive.optimize.ppd.storage=true;
> set hive.optimize.index.filter=true;
> set hive.tez.bucket.pruning=true;
> CREATE TABLE `test_table`(
>`col_1` int,
>`col_2` string,
>`col_3` string)
> CLUSTERED BY (
>col_1)
> INTO 4 BUCKETS;
> insert into test_table values(1, 'one', 'ONE'), (2, 'two', 'TWO'),
> (3,'three','THREE'),(4,'four','FOUR');
> select * from test_table;
> explain select col_1, col_2, col_3 from test_table where col_1 <> 2 order by
> col_2;
> select col_1, col_2, col_3 from test_table where col_1 <> 2 order by col_2;
> {code}
> Above sql query produce zero rows.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[
https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913579#comment-16913579
]
Hive QA commented on HIVE-21828:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12978274/HIVE-21828.9.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16745 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18379/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18379/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18379/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12978274 - PreCommit-HIVE-Build
> Tez: Use a pre-parsed TezConfiguration from DagUtils
>
>
> Key: HIVE-21828
> URL: https://issues.apache.org/jira/browse/HIVE-21828
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21828.1.patch, HIVE-21828.2.patch,
> HIVE-21828.5.patch, HIVE-21828.6.patch, HIVE-21828.7.patch,
> HIVE-21828.8.patch, HIVE-21828.9.patch
>
>
> The HS2 tez-site.xml does not change dynamically - the XML parsed components
> of the config can be obtained statically and kept across sessions.
> This allows for the replacing of "new TezConfiguration()" with a HS2 local
> version instead.
> The configuration object however has to reference the right resource file
> (i.e location of tez-site.xml) without reparsing it for each query.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22074) Slow compilation due to IN to OR transformation
[ https://issues.apache.org/jira/browse/HIVE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22074: --- Attachment: HIVE-22074.6.patch > Slow compilation due to IN to OR transformation > --- > > Key: HIVE-22074 > URL: https://issues.apache.org/jira/browse/HIVE-22074 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22074.1.patch, HIVE-22074.2.patch, > HIVE-22074.3.patch, HIVE-22074.4.patch, HIVE-22074.5.patch, HIVE-22074.6.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently Hive transform IN expressions to OR to apply various CBO rules. > This incur significant performance hit if IN consist of large number of > expressions. > It is better to not transform IN expressions to OR in such cases because > overall benefit of various optimizations/transformations is unrealized due to > the compilation overhead -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22074) Slow compilation due to IN to OR transformation
[ https://issues.apache.org/jira/browse/HIVE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22074: --- Status: Patch Available (was: Open) > Slow compilation due to IN to OR transformation > --- > > Key: HIVE-22074 > URL: https://issues.apache.org/jira/browse/HIVE-22074 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22074.1.patch, HIVE-22074.2.patch, > HIVE-22074.3.patch, HIVE-22074.4.patch, HIVE-22074.5.patch, HIVE-22074.6.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently Hive transform IN expressions to OR to apply various CBO rules. > This incur significant performance hit if IN consist of large number of > expressions. > It is better to not transform IN expressions to OR in such cases because > overall benefit of various optimizations/transformations is unrealized due to > the compilation overhead -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22074) Slow compilation due to IN to OR transformation
[ https://issues.apache.org/jira/browse/HIVE-22074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-22074: --- Status: Open (was: Patch Available) > Slow compilation due to IN to OR transformation > --- > > Key: HIVE-22074 > URL: https://issues.apache.org/jira/browse/HIVE-22074 > Project: Hive > Issue Type: Improvement > Components: Logical Optimizer >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22074.1.patch, HIVE-22074.2.patch, > HIVE-22074.3.patch, HIVE-22074.4.patch, HIVE-22074.5.patch, HIVE-22074.6.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently Hive transform IN expressions to OR to apply various CBO rules. > This incur significant performance hit if IN consist of large number of > expressions. > It is better to not transform IN expressions to OR in such cases because > overall benefit of various optimizations/transformations is unrealized due to > the compilation overhead -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22121) Turning on hive.tez.bucket.pruning produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-22121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22121:
---
Attachment: HIVE-22121.2.patch
> Turning on hive.tez.bucket.pruning produce wrong results
>
>
> Key: HIVE-22121
> URL: https://issues.apache.org/jira/browse/HIVE-22121
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22121.1patch, HIVE-22121.2.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.query.results.cache.enabled=false;
> set hive.optimize.ppd.storage=true;
> set hive.optimize.index.filter=true;
> set hive.tez.bucket.pruning=true;
> CREATE TABLE `test_table`(
>`col_1` int,
>`col_2` string,
>`col_3` string)
> CLUSTERED BY (
>col_1)
> INTO 4 BUCKETS;
> insert into test_table values(1, 'one', 'ONE'), (2, 'two', 'TWO'),
> (3,'three','THREE'),(4,'four','FOUR');
> select * from test_table;
> explain select col_1, col_2, col_3 from test_table where col_1 <> 2 order by
> col_2;
> select col_1, col_2, col_3 from test_table where col_1 <> 2 order by col_2;
> {code}
> Above sql query produce zero rows.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22121) Turning on hive.tez.bucket.pruning produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-22121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22121:
---
Status: Patch Available (was: Open)
> Turning on hive.tez.bucket.pruning produce wrong results
>
>
> Key: HIVE-22121
> URL: https://issues.apache.org/jira/browse/HIVE-22121
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22121.1patch, HIVE-22121.2.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.query.results.cache.enabled=false;
> set hive.optimize.ppd.storage=true;
> set hive.optimize.index.filter=true;
> set hive.tez.bucket.pruning=true;
> CREATE TABLE `test_table`(
>`col_1` int,
>`col_2` string,
>`col_3` string)
> CLUSTERED BY (
>col_1)
> INTO 4 BUCKETS;
> insert into test_table values(1, 'one', 'ONE'), (2, 'two', 'TWO'),
> (3,'three','THREE'),(4,'four','FOUR');
> select * from test_table;
> explain select col_1, col_2, col_3 from test_table where col_1 <> 2 order by
> col_2;
> select col_1, col_2, col_3 from test_table where col_1 <> 2 order by col_2;
> {code}
> Above sql query produce zero rows.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22121) Turning on hive.tez.bucket.pruning produce wrong results
[
https://issues.apache.org/jira/browse/HIVE-22121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-22121:
---
Status: Open (was: Patch Available)
> Turning on hive.tez.bucket.pruning produce wrong results
>
>
> Key: HIVE-22121
> URL: https://issues.apache.org/jira/browse/HIVE-22121
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22121.1patch, HIVE-22121.2.patch
>
>
> *Reproducer*
> {code:sql}
> set hive.query.results.cache.enabled=false;
> set hive.optimize.ppd.storage=true;
> set hive.optimize.index.filter=true;
> set hive.tez.bucket.pruning=true;
> CREATE TABLE `test_table`(
>`col_1` int,
>`col_2` string,
>`col_3` string)
> CLUSTERED BY (
>col_1)
> INTO 4 BUCKETS;
> insert into test_table values(1, 'one', 'ONE'), (2, 'two', 'TWO'),
> (3,'three','THREE'),(4,'four','FOUR');
> select * from test_table;
> explain select col_1, col_2, col_3 from test_table where col_1 <> 2 order by
> col_2;
> select col_1, col_2, col_3 from test_table where col_1 <> 2 order by col_2;
> {code}
> Above sql query produce zero rows.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[
https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913537#comment-16913537
]
Hive QA commented on HIVE-21828:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
4s{color} | {color:blue} ql in master has 2254 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18379/dev-support/hive-personality.sh
|
| git revision | master / db59ec6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18379/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tez: Use a pre-parsed TezConfiguration from DagUtils
>
>
> Key: HIVE-21828
> URL: https://issues.apache.org/jira/browse/HIVE-21828
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Attila Magyar
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21828.1.patch, HIVE-21828.2.patch,
> HIVE-21828.5.patch, HIVE-21828.6.patch, HIVE-21828.7.patch,
> HIVE-21828.8.patch, HIVE-21828.9.patch
>
>
> The HS2 tez-site.xml does not change dynamically - the XML parsed components
> of the config can be obtained statically and kept across sessions.
> This allows for the replacing of "new TezConfiguration()" with a HS2 local
> version instead.
> The configuration object however has to reference the right resource file
> (i.e location of tez-site.xml) without reparsing it for each query.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21698) TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while localizing resources
[
https://issues.apache.org/jira/browse/HIVE-21698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naresh P R updated HIVE-21698:
--
Attachment: HIVE-21698.patch
Status: Patch Available (was: In Progress)
> TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while
> localizing resources
> --
>
> Key: HIVE-21698
> URL: https://issues.apache.org/jira/browse/HIVE-21698
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Tez
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-21698.patch
>
>
> The cause of exception is due to [this loops
> over|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L633]
> {{newFilesNotFromConf}} and tries to access {{newResources.get(i)}} in that.
> Size of {{newResources}} is determined by [utils.localizeTempFiles()
> here|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L629]
> which may return a smaller list depending upon the {{skipFilesFromConf}} and
> this would result in {{newResources.size()}} < {{newFilesNotFromConf.length}}
> causing IndexOutOfBounds.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work started] (HIVE-21698) TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while localizing resources
[
https://issues.apache.org/jira/browse/HIVE-21698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-21698 started by Naresh P R.
-
> TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while
> localizing resources
> --
>
> Key: HIVE-21698
> URL: https://issues.apache.org/jira/browse/HIVE-21698
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Tez
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Naresh P R
>Priority: Major
> Attachments: HIVE-21698.patch
>
>
> The cause of exception is due to [this loops
> over|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L633]
> {{newFilesNotFromConf}} and tries to access {{newResources.get(i)}} in that.
> Size of {{newResources}} is determined by [utils.localizeTempFiles()
> here|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L629]
> which may return a smaller list depending upon the {{skipFilesFromConf}} and
> this would result in {{newResources.size()}} < {{newFilesNotFromConf.length}}
> causing IndexOutOfBounds.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-21698) TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while localizing resources
[
https://issues.apache.org/jira/browse/HIVE-21698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naresh P R reassigned HIVE-21698:
-
Assignee: Naresh P R (was: Shubham Chaurasia)
> TezSessionState#ensureLocalResources() causes IndexOutOfBoundsException while
> localizing resources
> --
>
> Key: HIVE-21698
> URL: https://issues.apache.org/jira/browse/HIVE-21698
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Tez
>Affects Versions: 3.1.1
>Reporter: Shubham Chaurasia
>Assignee: Naresh P R
>Priority: Major
>
> The cause of exception is due to [this loops
> over|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L633]
> {{newFilesNotFromConf}} and tries to access {{newResources.get(i)}} in that.
> Size of {{newResources}} is determined by [utils.localizeTempFiles()
> here|https://github.com/apache/hive/blob/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java#L629]
> which may return a smaller list depending upon the {{skipFilesFromConf}} and
> this would result in {{newResources.size()}} < {{newFilesNotFromConf.length}}
> causing IndexOutOfBounds.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22118) Log the table name while skipping the compaction because it's sorted table/partitions
[ https://issues.apache.org/jira/browse/HIVE-22118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913473#comment-16913473 ] Dinesh Chitlangia commented on HIVE-22118: -- Very useful improvement. Thanks for filing this patch and the fix [~Rajkumar Singh] > Log the table name while skipping the compaction because it's sorted > table/partitions > - > > Key: HIVE-22118 > URL: https://issues.apache.org/jira/browse/HIVE-22118 > Project: Hive > Issue Type: Improvement > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Rajkumar Singh >Assignee: Rajkumar Singh >Priority: Minor > Attachments: HIVE-22118.patch > > > for debugging perspective it's good if we log the full table name while > skipping the table for compaction otherwise it's tedious to know why the > compaction is not happening for the target table. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22129) Hive 3.1 standalone JAR includes Microsoft SQL Server JDBC driver
[ https://issues.apache.org/jira/browse/HIVE-22129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam reassigned HIVE-22129: Assignee: Naveen Gangam > Hive 3.1 standalone JAR includes Microsoft SQL Server JDBC driver > - > > Key: HIVE-22129 > URL: https://issues.apache.org/jira/browse/HIVE-22129 > Project: Hive > Issue Type: Bug > Components: JDBC >Affects Versions: 3.1.0 >Reporter: N Campbell >Assignee: Naveen Gangam >Priority: Major > Attachments: ApacheHive.png > > > The Apache Hive 3.1 JDBC driver, specifically the standalone JAR has included > the Microsoft SQL Server JDBC driver. A mutual customer may be trying to use > their own copy of a Microsoft JDBC driver and the Apache Hive driver. > Potentially the JRE will pick up the code in the Apache Hive driver. > What is it included at all and can it be removed going forward? > !ApacheHive.png! -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22134) Hive 3.1 driver includes org.glassfish.jersey.* which can interfer with an application
[
https://issues.apache.org/jira/browse/HIVE-22134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naveen Gangam reassigned HIVE-22134:
Assignee: Naveen Gangam
> Hive 3.1 driver includes org.glassfish.jersey.* which can interfer with an
> application
> --
>
> Key: HIVE-22134
> URL: https://issues.apache.org/jira/browse/HIVE-22134
> Project: Hive
> Issue Type: Bug
> Components: JDBC
>Affects Versions: 3.1.0
>Reporter: N Campbell
>Assignee: Naveen Gangam
>Priority: Major
>
> An application that uses JAX-RS 1.1 can be broken by the Hive 3.1 standalone
> JAR.
> For example, an application is running in IBM Websphere Liberty Profile
> (WLP) which detects the classes packaged in the Apache Hive standalone JAR.
> This results in WLP assuming that the application is providing it's
> implementation and should not use the default in WLP.
> Can the Apache Hive JDBC team confirm why these classes are in the JAR.
> Can the Apache Hive JDBC team schedule to remove them if they are not
> mandatory.
> Can the Apache Hive JDBC team confirm which individual JAR files can be
> copied instead of the uber-standalone JAR which would not include these
> conflicting classes.
> This is the class which triggers the problem if all of the jersey stuff is
> deleted the issue will go away
> org.glassfish.jersey.server.internal.RuntimeDelegateImpl
> {{org.glassfish.jersey.*}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-17678) appendPartition in HiveMetaStoreClient does not conform to the IMetaStoreClient.
[
https://issues.apache.org/jira/browse/HIVE-17678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nithish updated HIVE-17678:
---
Target Version/s: 2.0.0, 1.1.0 (was: 1.1.0, 2.0.0)
Status: Open (was: Patch Available)
> appendPartition in HiveMetaStoreClient does not conform to the
> IMetaStoreClient.
>
>
> Key: HIVE-17678
> URL: https://issues.apache.org/jira/browse/HIVE-17678
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 1.1.0
>Reporter: Nithish
>Assignee: Nithish
>Priority: Major
> Labels: Metastore
> Fix For: 2.0.0, 1.1.0
>
> Attachments: HIVE-17678.1.patch
>
>
> {code:java}
> Partition appendPartition(String dbName, String tableName, String partName)
> {code}
> in HiveMetaStoreClient does not conform with the declaration
> {code:java}
> Partition appendPartition(String tableName, String dbName, String name)
> {code}
> in IMetaStoreClient
> *Positions for dbName and tableName are interchanged.*
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22137) Implement alter/rename partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-22137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter reassigned HIVE-22137:
> Implement alter/rename partition related methods on temporary tables
>
>
> Key: HIVE-22137
> URL: https://issues.apache.org/jira/browse/HIVE-22137
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
>
> IMetaStoreClient exposes the following methods related to altering of
> partitions:
> {code:java}
> void alter_partition(String dbName, String tblName, Partition newPart);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart);
> void alter_partition(String dbName, String tblName, Partition newPart,
> EnvironmentContext environmentContext);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext, String writeIdList);
> void alter_partition(String catName, String dbName, String tblName, Partition
> newPart, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext);
> void alter_partitions(String dbName, String tblName, List
> newParts, EnvironmentContext environmentContext,String writeIdList, long
> writeId);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts);
> void alter_partitions(String catName, String dbName, String tblName,
> List newParts, EnvironmentContext environmentContext, String
> writeIdList, long writeId);
> void renamePartition(final String dbname, final String tableName, final
> List part_vals, final Partition newPart);
> void renamePartition(String catName, String dbname, String tableName,
> List part_vals, Partition newPart, String validWriteIds){code}
> These should be implemented, in order to completely support partition on
> temporary tables.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21580) Introduce ISO 8601 week numbering SQL:2016 formats
[ https://issues.apache.org/jira/browse/HIVE-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913187#comment-16913187 ] Karen Coppage commented on HIVE-21580: -- Thank you, Marta! > Introduce ISO 8601 week numbering SQL:2016 formats > -- > > Key: HIVE-21580 > URL: https://issues.apache.org/jira/browse/HIVE-21580 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21580.01.patch, HIVE-21580.02.patch > > > Enable Hive to parse the following datetime formats when any > combination/subset of these or previously implemented patterns is provided in > one string. Also catch combinations that conflict. > * IYYY > * IYY > * IY > * I > * IW > [https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21580) Introduce ISO 8601 week numbering SQL:2016 formats
[ https://issues.apache.org/jira/browse/HIVE-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marta Kuczora updated HIVE-21580: - Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Introduce ISO 8601 week numbering SQL:2016 formats > -- > > Key: HIVE-21580 > URL: https://issues.apache.org/jira/browse/HIVE-21580 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21580.01.patch, HIVE-21580.02.patch > > > Enable Hive to parse the following datetime formats when any > combination/subset of these or previously implemented patterns is provided in > one string. Also catch combinations that conflict. > * IYYY > * IYY > * IY > * I > * IW > [https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21580) Introduce ISO 8601 week numbering SQL:2016 formats
[ https://issues.apache.org/jira/browse/HIVE-21580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913172#comment-16913172 ] Marta Kuczora commented on HIVE-21580: -- Pushed to master. > Introduce ISO 8601 week numbering SQL:2016 formats > -- > > Key: HIVE-21580 > URL: https://issues.apache.org/jira/browse/HIVE-21580 > Project: Hive > Issue Type: Improvement >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21580.01.patch, HIVE-21580.02.patch > > > Enable Hive to parse the following datetime formats when any > combination/subset of these or previously implemented patterns is provided in > one string. Also catch combinations that conflict. > * IYYY > * IYY > * IY > * I > * IW > [https://docs.google.com/document/d/1V7k6-lrPGW7_uhqM-FhKl3QsxwCRy69v2KIxPsGjc1k/edit] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22132) Upgrade commons-lang3 version to 3.9
[ https://issues.apache.org/jira/browse/HIVE-22132?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-22132: -- Fix Version/s: 4.0.0 Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master. Thanks for the review [~thejas]! > Upgrade commons-lang3 version to 3.9 > > > Key: HIVE-22132 > URL: https://issues.apache.org/jira/browse/HIVE-22132 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22132.02.patch, HIVE-22132.patch > > > Upgrade commons lang 3 to 3.9 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[ https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-21828: - Attachment: (was: HIVE-21828.9.patch) > Tez: Use a pre-parsed TezConfiguration from DagUtils > > > Key: HIVE-21828 > URL: https://issues.apache.org/jira/browse/HIVE-21828 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21828.1.patch, HIVE-21828.2.patch, > HIVE-21828.5.patch, HIVE-21828.6.patch, HIVE-21828.7.patch, > HIVE-21828.8.patch, HIVE-21828.9.patch > > > The HS2 tez-site.xml does not change dynamically - the XML parsed components > of the config can be obtained statically and kept across sessions. > This allows for the replacing of "new TezConfiguration()" with a HS2 local > version instead. > The configuration object however has to reference the right resource file > (i.e location of tez-site.xml) without reparsing it for each query. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[ https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-21828: - Attachment: HIVE-21828.9.patch > Tez: Use a pre-parsed TezConfiguration from DagUtils > > > Key: HIVE-21828 > URL: https://issues.apache.org/jira/browse/HIVE-21828 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21828.1.patch, HIVE-21828.2.patch, > HIVE-21828.5.patch, HIVE-21828.6.patch, HIVE-21828.7.patch, > HIVE-21828.8.patch, HIVE-21828.9.patch > > > The HS2 tez-site.xml does not change dynamically - the XML parsed components > of the config can be obtained statically and kept across sessions. > This allows for the replacing of "new TezConfiguration()" with a HS2 local > version instead. > The configuration object however has to reference the right resource file > (i.e location of tez-site.xml) without reparsing it for each query. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[ https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-21828: - Status: Patch Available (was: Open) > Tez: Use a pre-parsed TezConfiguration from DagUtils > > > Key: HIVE-21828 > URL: https://issues.apache.org/jira/browse/HIVE-21828 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21828.1.patch, HIVE-21828.2.patch, > HIVE-21828.5.patch, HIVE-21828.6.patch, HIVE-21828.7.patch, > HIVE-21828.8.patch, HIVE-21828.9.patch > > > The HS2 tez-site.xml does not change dynamically - the XML parsed components > of the config can be obtained statically and kept across sessions. > This allows for the replacing of "new TezConfiguration()" with a HS2 local > version instead. > The configuration object however has to reference the right resource file > (i.e location of tez-site.xml) without reparsing it for each query. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21828) Tez: Use a pre-parsed TezConfiguration from DagUtils
[ https://issues.apache.org/jira/browse/HIVE-21828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Attila Magyar updated HIVE-21828: - Status: Open (was: Patch Available) > Tez: Use a pre-parsed TezConfiguration from DagUtils > > > Key: HIVE-21828 > URL: https://issues.apache.org/jira/browse/HIVE-21828 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Attila Magyar >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21828.1.patch, HIVE-21828.2.patch, > HIVE-21828.5.patch, HIVE-21828.6.patch, HIVE-21828.7.patch, > HIVE-21828.8.patch, HIVE-21828.9.patch > > > The HS2 tez-site.xml does not change dynamically - the XML parsed components > of the config can be obtained statically and kept across sessions. > This allows for the replacing of "new TezConfiguration()" with a HS2 local > version instead. > The configuration object however has to reference the right resource file > (i.e location of tez-site.xml) without reparsing it for each query. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-14650) Select fails when ORC file has more columns than table schema
[
https://issues.apache.org/jira/browse/HIVE-14650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
xiepengjie reassigned HIVE-14650:
-
Assignee: xiepengjie
> Select fails when ORC file has more columns than table schema
> -
>
> Key: HIVE-14650
> URL: https://issues.apache.org/jira/browse/HIVE-14650
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.2.1
>Reporter: Jeff Mink
>Assignee: xiepengjie
>Priority: Minor
>
> When SELECTing from a Hive ORC table, the following IndexOutOfBoundsException
> is thrown if the underlying ORC file has 4 or more columns than the Hive
> schema (where N is the number of columns in the ORC file).
> {noformat}
> Failed with exception
> java.io.IOException:java.lang.IndexOutOfBoundsException: toIndex = N
> 16/08/25 15:22:19 ERROR CliDriver: Failed with exception
> java.io.IOException:java.lang.IndexOutOfBoundsException: toIndex = N
> java.io.IOException: java.lang.IndexOutOfBoundsException: toIndex = N
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:507)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:414)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:140)
> at org.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1686)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:165)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:376)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:736)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.lang.IndexOutOfBoundsException: toIndex = 6
> at java.util.ArrayList.subListRangeCheck(ArrayList.java:1004)
> at java.util.ArrayList.subList(ArrayList.java:996)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderFactory.getSchemaOnRead(RecordReaderFactory.java:161)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderFactory.createTreeReader(RecordReaderFactory.java:66)
> at
> org.apache.hadoop.hive.ql.io.orc.RecordReaderImpl.(RecordReaderImpl.java:202)
> at
> org.apache.hadoop.hive.ql.io.orc.ReaderImpl.rowsOptions(ReaderImpl.java:541)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$ReaderPair.(OrcRawRecordMerger.java:183)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger$OriginalReaderPair.(OrcRawRecordMerger.java:226)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcRawRecordMerger.(OrcRawRecordMerger.java:437)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getReader(OrcInputFormat.java:1216)
> at
> org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:1113)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator$FetchInputFormatSplit.getRecordReader(FetchOperator.java:673)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getRecordReader(FetchOperator.java:323)
> at
> org.apache.hadoop.hive.ql.exec.FetchOperator.getNextRow(FetchOperator.java:445)
> ... 15 more
> {noformat}
> This error appears to be related to the patch of HIVE-10591.
> Steps to reproduce (Hive QL):
> {noformat}
> DROP TABLE IF EXISTS orc_drop_column;
> CREATE TABLE orc_drop_column (`id` int, `name` string, `description` string,
> `somevalue` double, `someflag` boolean, `somedate` timestamp) STORED AS ORC;
> INSERT INTO TABLE orc_drop_column select * from (select 1, 'my_name',
> 'my_desc', 5.5, true, '2016-08-25 06:00:00') a;
> ALTER TABLE orc_drop_column SET SERDE
> 'org.apache.hadoop.hive.serde2.columnar.ColumnarSerDe';
> ALTER TABLE orc_drop_column REPLACE COLUMNS (
> `id` int,
> `name` string
> );
> ALTER TABLE orc_drop_column SET SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde';
> SELECT id, name FROM orc_drop_column;
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
