[jira] [Commented] (HIVE-23309) Lazy Initialization of Hadoop Shims
[
https://issues.apache.org/jira/browse/HIVE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097827#comment-17097827
]
Hive QA commented on HIVE-23309:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001829/HIVE-23309.03.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17195 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testReplTimeouts
(batchId=244)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22069/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22069/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22069/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001829 - PreCommit-HIVE-Build
> Lazy Initialization of Hadoop Shims
> ---
>
> Key: HIVE-23309
> URL: https://issues.apache.org/jira/browse/HIVE-23309
> Project: Hive
> Issue Type: Bug
>Reporter: Aasha Medhi
>Assignee: Aasha Medhi
>Priority: Major
> Attachments: HIVE-23309.01.patch, HIVE-23309.02.patch,
> HIVE-23309.03.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Initialize hadoop-shims only if CM is enabled
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23309) Lazy Initialization of Hadoop Shims
[
https://issues.apache.org/jira/browse/HIVE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097825#comment-17097825
]
Hive QA commented on HIVE-23309:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
59s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
36s{color} | {color:blue} standalone-metastore/metastore-common in master has
35 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
27s{color} | {color:blue} hcatalog/server-extensions in master has 3 extant
Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch metastore-common passed checkstyle {color}
|
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} hcatalog/server-extensions: The patch generated 0
new + 4 unchanged - 1 fixed = 4 total (was 5) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} The patch hive-unit passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
42s{color} | {color:green} metastore-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
55s{color} | {color:green} ql in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} hcatalog/server-extensions generated 0 new + 2
unchanged - 1 fixed = 2 total (was 3) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 43m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22069/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C:
[jira] [Commented] (HIVE-23124) Review of SQLOperation Class
[
https://issues.apache.org/jira/browse/HIVE-23124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097820#comment-17097820
]
Hive QA commented on HIVE-23124:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001828/HIVE-23124.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17195 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22068/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22068/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22068/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001828 - PreCommit-HIVE-Build
> Review of SQLOperation Class
>
>
> Key: HIVE-23124
> URL: https://issues.apache.org/jira/browse/HIVE-23124
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23124.1.patch, HIVE-23124.2.patch,
> HIVE-23124.3.patch, HIVE-23124.3.patch, HIVE-23124.3.patch, HIVE-23124.3.patch
>
>
> * Use ConcurrentHashMap instead of synchronized methods to improve
> multi-threaded access
> * Use JDK 8 facilities where applicable
> * General cleanup
> * Better log messages and Exception messages
> * Use {{switch}} statement instead of if/else blocks
> * Checkstyle fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23124) Review of SQLOperation Class
[
https://issues.apache.org/jira/browse/HIVE-23124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097804#comment-17097804
]
Hive QA commented on HIVE-23124:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} service in master has 51 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} service: The patch generated 0 new + 14 unchanged -
1 fixed = 14 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 56s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22068/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22068/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Review of SQLOperation Class
>
>
> Key: HIVE-23124
> URL: https://issues.apache.org/jira/browse/HIVE-23124
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23124.1.patch, HIVE-23124.2.patch,
> HIVE-23124.3.patch, HIVE-23124.3.patch, HIVE-23124.3.patch, HIVE-23124.3.patch
>
>
> * Use ConcurrentHashMap instead of synchronized methods to improve
> multi-threaded access
> * Use JDK 8 facilities where applicable
> * General cleanup
> * Better log messages and Exception messages
> * Use {{switch}} statement instead of if/else blocks
> * Checkstyle fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23345) INT64 Parquet timestamps cannot be read into bigint Hive type
[
https://issues.apache.org/jira/browse/HIVE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097797#comment-17097797
]
Hive QA commented on HIVE-23345:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001847/HIVE-23345.01.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17198 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22067/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22067/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22067/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001847 - PreCommit-HIVE-Build
> INT64 Parquet timestamps cannot be read into bigint Hive type
> -
>
> Key: HIVE-23345
> URL: https://issues.apache.org/jira/browse/HIVE-23345
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Marta Kuczora
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23345.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce:
> - create external table ts_pq (ts timestamp) stored as parquet;
> - insert into ts_pq values ('1998-10-03 09:58:31.231');
> - create external table ts_pq_2 (ts bigint) stored as parquet location
> '';
> - select * from ts_pq_2;
> The following exception occurs during the select:
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.io.TimestampWritableV2 cannot be cast to
> org.apache.hadoop.io.LongWritable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23315) Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-23315:
--
Attachment: (was: HIVE-23315.01.patch)
> Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED
> VIEWS
> --
>
> Key: HIVE-23315
> URL: https://issues.apache.org/jira/browse/HIVE-23315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23315.01.patch
>
>
> At the end of each SHOW EXTENDED TABLES; and SHOW MATERIALIZED VIEWS; command
> there is an empty line like this:
> {code:java}
> +++
> | tab_name | table_type |
> +++
> | sample_07 | MANAGED_TABLE |
> | sample_08 | MANAGED_TABLE |
> | web_logs | MANAGED_TABLE |
> | | NULL |
> +++
> {code}
> It should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23315) Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-23315:
--
Attachment: HIVE-23315.01.patch
> Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED
> VIEWS
> --
>
> Key: HIVE-23315
> URL: https://issues.apache.org/jira/browse/HIVE-23315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23315.01.patch
>
>
> At the end of each SHOW EXTENDED TABLES; and SHOW MATERIALIZED VIEWS; command
> there is an empty line like this:
> {code:java}
> +++
> | tab_name | table_type |
> +++
> | sample_07 | MANAGED_TABLE |
> | sample_08 | MANAGED_TABLE |
> | web_logs | MANAGED_TABLE |
> | | NULL |
> +++
> {code}
> It should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23345) INT64 Parquet timestamps cannot be read into bigint Hive type
[
https://issues.apache.org/jira/browse/HIVE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097789#comment-17097789
]
Hive QA commented on HIVE-23345:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
1s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
33s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 8 new + 15 unchanged - 0 fixed
= 23 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22067/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22067/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22067/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> INT64 Parquet timestamps cannot be read into bigint Hive type
> -
>
> Key: HIVE-23345
> URL: https://issues.apache.org/jira/browse/HIVE-23345
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Marta Kuczora
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23345.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce:
> - create external table ts_pq (ts timestamp) stored as parquet;
> - insert into ts_pq values ('1998-10-03 09:58:31.231');
> - create external table ts_pq_2 (ts bigint) stored as parquet location
> '';
> - select * from ts_pq_2;
> The following exception occurs during the select:
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.io.TimestampWritableV2 cannot be cast to
> org.apache.hadoop.io.LongWritable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23355) Handle STORED AS quoting behavior properly when masking is enabled
[
https://issues.apache.org/jira/browse/HIVE-23355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Sherman reassigned HIVE-23355:
---
> Handle STORED AS quoting behavior properly when masking is enabled
> --
>
> Key: HIVE-23355
> URL: https://issues.apache.org/jira/browse/HIVE-23355
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: John Sherman
>Assignee: John Sherman
>Priority: Major
> Fix For: 4.0.0
>
>
> Test framework repro:
> {code:java}
> --- a/ql/src/test/queries/clientpositive/masking_reserved.q
> +++ b/ql/src/test/queries/clientpositive/masking_reserved.q
> @@ -5,6 +5,7 @@ set
> hive.security.authorization.manager=org.apache.hadoop.hive.ql.security.autho
> create table keyword_test_off (id int, `etad` string, key int);
> create table keyword_test_on (id int, `date` string, key int);
> create table masking_test_n_masking_reserved (id int, value string, key int);
> +create temporary table masking_test_n_masking_temp stored as orc as select *
> from masking_test_n_masking_reserved; explain select a.`etad`, b.value from
> keyword_test_off a join masking_test_n_masking_reserved b on b.id = a.id;
> select a.`etad`, b.value from keyword_test_off a join
> masking_test_n_masking_reserved b on b.id = a.id; {code}
> fails with:
> {code:java}
> 2020-04-30T20:19:06,833 INFO [bba048b2-380a-46e2-b283-52c1090a0d5c main]
> parse.CalcitePlanner: Creating table default.masking_test_n_masking_temp
> position=23
> 2020-04-30T20:19:06,851 ERROR [bba048b2-380a-46e2-b283-52c1090a0d5c main]
> ql.Driver: FAILED: SemanticException Unrecognized file format in STORED AS
> clause: '`ORC`'
> org.apache.hadoop.hive.ql.parse.SemanticException: Unrecognized file format
> in STORED AS clause: '`ORC`'
> at
> org.apache.hadoop.hive.ql.parse.StorageFormat.processStorageFormat(StorageFormat.java:89)
> at
> org.apache.hadoop.hive.ql.parse.StorageFormat.fillStorageFormat(StorageFormat.java:74)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeCreateTable(SemanticAnalyzer.java:13351)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genResolvedParseTree(SemanticAnalyzer.java:12256)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12471)
> at
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:432)
> at
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Compiler.analyze(Compiler.java:220)
> at org.apache.hadoop.hive.ql.Compiler.compile(Compiler.java:104)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:184) {code}
> We should either not quote the STORED BY argument in this path OR strip
> quotes in processStorageFormat.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23354) Remove file size sanity checking from compareTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman updated HIVE-23354: Description: [https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010] compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to determine which file(s) to keep. I've seen instances where this function throws an exception due to the fact that the newer attemptId file size is less than the older attemptId (thus failing the query). I think this assumption is faulty, due to various factors such as file compression and the order in which values are written. It may be prudent to trust that the newest attemptId is in fact the best choice. was: https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010 compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to determine which file(s) to keep. I've seen instances where this function throws an exception due to the fact that the newer attemptId file size is less than the older attemptId (thus failing the query). I think this assumption is faulty, due to various factors such as file compression and the order in which values are written. It may be prudent to trust that the newest attemptId is in fact the best choice. > Remove file size sanity checking from compareTempOrDuplicateFiles > - > > Key: HIVE-23354 > URL: https://issues.apache.org/jira/browse/HIVE-23354 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > > [https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010] > compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to > determine which file(s) to keep. > I've seen instances where this function throws an exception due to the fact > that the newer attemptId file size is less than the older attemptId (thus > failing the query). > I think this assumption is faulty, due to various factors such as file > compression and the order in which values are written. It may be prudent to > trust that the newest attemptId is in fact the best choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23354) Remove file size sanity checking from compareTempOrDuplicateFiles
[ https://issues.apache.org/jira/browse/HIVE-23354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Sherman reassigned HIVE-23354: --- > Remove file size sanity checking from compareTempOrDuplicateFiles > - > > Key: HIVE-23354 > URL: https://issues.apache.org/jira/browse/HIVE-23354 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Reporter: John Sherman >Assignee: John Sherman >Priority: Major > > https://github.com/apache/hive/blob/cdd55aa319a3440963a886ebfff11cd2a240781d/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L1952-L2010 > compareTempOrDuplicateFiles uses a combination of attemptId and fileSize to > determine which file(s) to keep. > I've seen instances where this function throws an exception due to the fact > that the newer attemptId file size is less than the older attemptId (thus > failing the query). > I think this assumption is faulty, due to various factors such as file > compression and the order in which values are written. It may be prudent to > trust that the newest attemptId is in > fact the best choice. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23315) Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097784#comment-17097784
]
Hive QA commented on HIVE-23315:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001806/HIVE-23315.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 17195 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multi_insert_partitioned]
(batchId=98)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testCancelRenewTokenFlow
(batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testConnection
(batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValid (batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValidNeg
(batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeProxyAuth
(batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeTokenAuth
(batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testProxyAuth
(batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testRenewDelegationToken
(batchId=235)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testTokenAuth
(batchId=235)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22066/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22066/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22066/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001806 - PreCommit-HIVE-Build
> Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED
> VIEWS
> --
>
> Key: HIVE-23315
> URL: https://issues.apache.org/jira/browse/HIVE-23315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23315.01.patch
>
>
> At the end of each SHOW EXTENDED TABLES; and SHOW MATERIALIZED VIEWS; command
> there is an empty line like this:
> {code:java}
> +++
> | tab_name | table_type |
> +++
> | sample_07 | MANAGED_TABLE |
> | sample_08 | MANAGED_TABLE |
> | web_logs | MANAGED_TABLE |
> | | NULL |
> +++
> {code}
> It should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23315) Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097775#comment-17097775
]
Hive QA commented on HIVE-23315:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22066/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22066/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED
> VIEWS
> --
>
> Key: HIVE-23315
> URL: https://issues.apache.org/jira/browse/HIVE-23315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23315.01.patch
>
>
> At the end of each SHOW EXTENDED TABLES; and SHOW MATERIALIZED VIEWS; command
> there is an empty line like this:
> {code:java}
> +++
> | tab_name | table_type |
> +++
> | sample_07 | MANAGED_TABLE |
> | sample_08 | MANAGED_TABLE |
> | web_logs | MANAGED_TABLE |
> | | NULL |
> +++
> {code}
> It should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22769) Incorrect query results and query failure during split generation for compressed text files
[
https://issues.apache.org/jira/browse/HIVE-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097769#comment-17097769
]
Hive QA commented on HIVE-22769:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
5s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
47s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} llap-server in master has 89 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 4 new + 36 unchanged - 0 fixed
= 40 total (was 36) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} llap-server: The patch generated 3 new + 78 unchanged
- 1 fixed = 81 total (was 79) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
56s{color} | {color:red} root: The patch generated 7 new + 114 unchanged - 1
fixed = 121 total (was 115) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
49s{color} | {color:red} llap-server generated 1 new + 88 unchanged - 1 fixed =
89 total (was 89) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 13m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
44s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 75m 45s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:llap-server |
| | Redundant nullcheck of offsetReader, which is known to be non-null in
org.apache.hadoop.hive.llap.io.encoded.SerDeEncodedDataReader.startReadSplitFromFile(FileSplit,
boolean[], SerDeLowLevelCacheImpl$StripeData) Redundant null check at
SerDeEncodedDataReader.java:is known to be non-null in
org.apache.hadoop.hive.llap.io.encoded.SerDeEncodedDataReader.startReadSplitFromFile(FileSplit,
boolean[], SerDeLowLevelCacheImpl$StripeData) Redundant null check at
SerDeEncodedDataReader.java:[line 1468] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22065/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22065/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22065/yetus/diff-checkstyle-llap-server.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22065/yetus/diff-checkstyle-root.txt
|
| findbugs |
[jira] [Commented] (HIVE-22769) Incorrect query results and query failure during split generation for compressed text files
[
https://issues.apache.org/jira/browse/HIVE-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097762#comment-17097762
]
Hive QA commented on HIVE-22769:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001804/HIVE-22769.04.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17199 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22065/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22065/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22065/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001804 - PreCommit-HIVE-Build
> Incorrect query results and query failure during split generation for
> compressed text files
> ---
>
> Key: HIVE-22769
> URL: https://issues.apache.org/jira/browse/HIVE-22769
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Chiran Ravani
>Assignee: Panagiotis Garefalakis
>Priority: Critical
> Attachments: HIVE-22769.01.patch, HIVE-22769.02.patch,
> HIVE-22769.03.patch, HIVE-22769.04.patch, testcase1.csv.bz2, testcase2.csv.bz2
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Hive Query produces incorrect results when data is in text format and
> compressed and for certain data the query fails during split generation.
> This behavior is seen when skip.header.line.count and skip.footer.line.count
> are set for table.
> Case 1: Select count/aggregate query produces Incorrect row counts/displays
> all rows (when hive.fetch.task.conversion=none)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase1`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase1'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase1.csv.bz2 file to /user/hive/testcase1
> 3. Run count(*) on table.
> {code}
> > select * from testcase1;
> INFO : Compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:testcase1.id, type:string,
> comment:null), FieldSchema(name:testcase1.name, type:string, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.07 seconds
> INFO : Executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Completed executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.007 seconds
> INFO : OK
> +---+-+
> | testcase1.id | testcase1.name |
> +---+-+
> | 2 | 2019-12-31 |
> +---+-+
> 1 row selected (0.111 seconds)
> > select count(*) from testcase1
> INFO : Compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:bigint, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 0.073 seconds
> INFO : Executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Query ID = hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Subscribed to counters: [] for queryId:
> hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Session is already open
> INFO : Dag name: select count(*) from testcase1 (Stage-1)
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1579811438512_0046)
> .
> .
> .
> INFO : Completed executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 4.228 seconds
> INFO : OK
> +--+
> |
[jira] [Work logged] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[ https://issues.apache.org/jira/browse/HIVE-22256?focusedWorklogId=429738=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429738 ] ASF GitHub Bot logged work on HIVE-22256: - Author: ASF GitHub Bot Created on: 02/May/20 00:09 Start Date: 02/May/20 00:09 Worklog Time Spent: 10m Work Description: jcamachor commented on a change in pull request #1002: URL: https://github.com/apache/hive/pull/1002#discussion_r418784188 ## File path: ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java ## @@ -2463,6 +2473,15 @@ private RelNode applyPostJoinOrderingTransform(RelNode basePlan, RelMetadataProv HiveProjectSortExchangeTransposeRule.INSTANCE, HiveProjectMergeRule.INSTANCE); } + // 10. We need to expand IN/BETWEEN expressions when loading a materialized view Review comment: This is called only when we are loading the materialized view in the registry so we can keep the plan in the correct form (IN and BETWEEN expanded) and we do not have to do it at compilation time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 429738) Time Spent: 50m (was: 40m) > Rewriting fails when `IN` clause has items in different order in MV and query. > -- > > Key: HIVE-22256 > URL: https://issues.apache.org/jira/browse/HIVE-22256 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22256.01.patch, HIVE-22256.patch, expr2.sql > > Time Spent: 50m > Remaining Estimate: 0h > > Rewriting fails on following materialized view and query (script is also > attached): > create materialized view view2 stored as orc as (select prod_id, cust_id, > store_id, sale_date, qty, amt, descr from sales where cust_id in (1,2,3,4,5)); > explain extended select prod_id, cust_id from sales where cust_id in > (5,1,2,3,4); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[
https://issues.apache.org/jira/browse/HIVE-22256?focusedWorklogId=429737=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429737
]
ASF GitHub Bot logged work on HIVE-22256:
-
Author: ASF GitHub Bot
Created on: 02/May/20 00:04
Start Date: 02/May/20 00:04
Worklog Time Spent: 10m
Work Description: jcamachor commented on a change in pull request #1002:
URL: https://github.com/apache/hive/pull/1002#discussion_r418784851
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveInBetweenExpandRule.java
##
@@ -0,0 +1,198 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Filter;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.Project;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.rex.RexShuttle;
+import org.apache.calcite.rex.RexUtil;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories;
+import org.apache.hadoop.hive.ql.optimizer.calcite.translator.RexNodeConverter;
+
+/**
+ * This class contains rules to rewrite IN/BETWEEN clauses into their
+ * corresponding AND/OR versions.
+ * It is the counterpart to {@link HivePointLookupOptimizerRule}.
+ */
+public class HiveInBetweenExpandRule {
+
+ public static final FilterRule FILTER_INSTANCE = new FilterRule();
+ public static final JoinRule JOIN_INSTANCE = new JoinRule();
+ public static final ProjectRule PROJECT_INSTANCE = new ProjectRule();
+
+ /** Rule adapter to apply the transformation to Filter conditions. */
+ private static class FilterRule extends RelOptRule {
+
+FilterRule() {
+ super(operand(Filter.class, any()), HiveRelFactories.HIVE_BUILDER, null);
+}
+
+@Override
+public void onMatch(RelOptRuleCall call) {
+ final Filter filter = call.rel(0);
+ RexInBetweenExpander expander = new RexInBetweenExpander(
+ filter.getCluster().getRexBuilder());
+ RexNode condition = expander.apply(filter.getCondition());
+
+ if (!expander.modified) {
+return;
+ }
+
+ RelNode newFilter = filter.copy(filter.getTraitSet(),
+ filter.getInput(), condition);
+
+ call.transformTo(newFilter);
+}
+ }
+
+ /** Rule adapter to apply the transformation to Join conditions. */
+ private static class JoinRule extends RelOptRule {
+
+JoinRule() {
+ super(operand(Join.class, any()), HiveRelFactories.HIVE_BUILDER, null);
+}
+
+@Override
+public void onMatch(RelOptRuleCall call) {
+ final Join join = call.rel(0);
+ RexInBetweenExpander expander = new RexInBetweenExpander(
+ join.getCluster().getRexBuilder());
+ RexNode condition = expander.apply(join.getCondition());
+
+ if (!expander.modified) {
+return;
+ }
+
+ RelNode newJoin = join.copy(join.getTraitSet(),
+ condition,
+ join.getLeft(),
+ join.getRight(),
+ join.getJoinType(),
+ join.isSemiJoinDone());
+
+ call.transformTo(newJoin);
+}
+ }
+
+ /** Rule adapter to apply the transformation to Project expressions. */
+ private static class ProjectRule extends RelOptRule {
+
+ProjectRule() {
+ super(operand(Project.class, any()), HiveRelFactories.HIVE_BUILDER,
null);
+}
+
+@Override
+public void onMatch(RelOptRuleCall call) {
+ final Project project = call.rel(0);
+ RexInBetweenExpander expander = new RexInBetweenExpander(
+ project.getCluster().getRexBuilder());
+ List newProjects = new ArrayList<>();
+ for (RexNode expr : project.getProjects()) {
+newProjects.add(expander.apply(expr));
+ }
+
+ if (!expander.modified) {
+return;
+ }
+
+
[jira] [Work logged] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[ https://issues.apache.org/jira/browse/HIVE-22256?focusedWorklogId=429735=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429735 ] ASF GitHub Bot logged work on HIVE-22256: - Author: ASF GitHub Bot Created on: 02/May/20 00:01 Start Date: 02/May/20 00:01 Worklog Time Spent: 10m Work Description: jcamachor commented on a change in pull request #1002: URL: https://github.com/apache/hive/pull/1002#discussion_r418784188 ## File path: ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java ## @@ -2463,6 +2473,15 @@ private RelNode applyPostJoinOrderingTransform(RelNode basePlan, RelMetadataProv HiveProjectSortExchangeTransposeRule.INSTANCE, HiveProjectMergeRule.INSTANCE); } + // 10. We need to expand IN/BETWEEN expressions when loading a materialized view Review comment: This is call only when we are loading the materialized view in the registry so we can keep the plan in the correct form (IN and BETWEEN expanded) and we do not have to do it at compilation time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 429735) Time Spent: 0.5h (was: 20m) > Rewriting fails when `IN` clause has items in different order in MV and query. > -- > > Key: HIVE-22256 > URL: https://issues.apache.org/jira/browse/HIVE-22256 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22256.01.patch, HIVE-22256.patch, expr2.sql > > Time Spent: 0.5h > Remaining Estimate: 0h > > Rewriting fails on following materialized view and query (script is also > attached): > create materialized view view2 stored as orc as (select prod_id, cust_id, > store_id, sale_date, qty, amt, descr from sales where cust_id in (1,2,3,4,5)); > explain extended select prod_id, cust_id from sales where cust_id in > (5,1,2,3,4); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[
https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097740#comment-17097740
]
Hive QA commented on HIVE-23215:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001802/HIVE-23215.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17194 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22064/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22064/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22064/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001802 - PreCommit-HIVE-Build
> Make FilterContext and MutableFilterContext interfaces
> --
>
> Key: HIVE-23215
> URL: https://issues.apache.org/jira/browse/HIVE-23215
> Project: Hive
> Issue Type: Bug
> Components: storage-api
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23215.01.patch, HIVE-23215.02.patch,
> HIVE-23215.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> HIVE-22959 introduced FilterContext to support ORC-577. The duplication of
> fields between the FilterContext and VectorizedRowBatch seems likely to cause
> user confusion. This patch makes them interfaces that VectorizedRowBatch
> implements.
> Thus, there is a single copy of the data and no need to copy them back and
> forth. LLAP can make its own implementation of the interfaces if it doesn't
> want to use VectorizedRowBatch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[
https://issues.apache.org/jira/browse/HIVE-22256?focusedWorklogId=429726=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429726
]
ASF GitHub Bot logged work on HIVE-22256:
-
Author: ASF GitHub Bot
Created on: 01/May/20 23:38
Start Date: 01/May/20 23:38
Worklog Time Spent: 10m
Work Description: vineetgarg02 commented on a change in pull request
#1002:
URL: https://github.com/apache/hive/pull/1002#discussion_r418778365
##
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java
##
@@ -2463,6 +2473,15 @@ private RelNode applyPostJoinOrderingTransform(RelNode
basePlan, RelMetadataProv
HiveProjectSortExchangeTransposeRule.INSTANCE,
HiveProjectMergeRule.INSTANCE);
}
+ // 10. We need to expand IN/BETWEEN expressions when loading a
materialized view
Review comment:
Why do we need to call these rules again in post ordering?
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveInBetweenExpandRule.java
##
@@ -0,0 +1,198 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hive.ql.optimizer.calcite.rules;
+
+import java.util.ArrayList;
+import java.util.List;
+import org.apache.calcite.plan.RelOptRule;
+import org.apache.calcite.plan.RelOptRuleCall;
+import org.apache.calcite.rel.RelNode;
+import org.apache.calcite.rel.core.Filter;
+import org.apache.calcite.rel.core.Join;
+import org.apache.calcite.rel.core.Project;
+import org.apache.calcite.rex.RexBuilder;
+import org.apache.calcite.rex.RexCall;
+import org.apache.calcite.rex.RexNode;
+import org.apache.calcite.rex.RexShuttle;
+import org.apache.calcite.rex.RexUtil;
+import org.apache.calcite.sql.fun.SqlStdOperatorTable;
+import org.apache.hadoop.hive.ql.optimizer.calcite.HiveRelFactories;
+import org.apache.hadoop.hive.ql.optimizer.calcite.translator.RexNodeConverter;
+
+/**
+ * This class contains rules to rewrite IN/BETWEEN clauses into their
+ * corresponding AND/OR versions.
+ * It is the counterpart to {@link HivePointLookupOptimizerRule}.
+ */
+public class HiveInBetweenExpandRule {
+
+ public static final FilterRule FILTER_INSTANCE = new FilterRule();
+ public static final JoinRule JOIN_INSTANCE = new JoinRule();
+ public static final ProjectRule PROJECT_INSTANCE = new ProjectRule();
+
+ /** Rule adapter to apply the transformation to Filter conditions. */
+ private static class FilterRule extends RelOptRule {
+
+FilterRule() {
+ super(operand(Filter.class, any()), HiveRelFactories.HIVE_BUILDER, null);
+}
+
+@Override
+public void onMatch(RelOptRuleCall call) {
+ final Filter filter = call.rel(0);
+ RexInBetweenExpander expander = new RexInBetweenExpander(
+ filter.getCluster().getRexBuilder());
+ RexNode condition = expander.apply(filter.getCondition());
+
+ if (!expander.modified) {
+return;
+ }
+
+ RelNode newFilter = filter.copy(filter.getTraitSet(),
+ filter.getInput(), condition);
+
+ call.transformTo(newFilter);
+}
+ }
+
+ /** Rule adapter to apply the transformation to Join conditions. */
+ private static class JoinRule extends RelOptRule {
+
+JoinRule() {
+ super(operand(Join.class, any()), HiveRelFactories.HIVE_BUILDER, null);
+}
+
+@Override
+public void onMatch(RelOptRuleCall call) {
+ final Join join = call.rel(0);
+ RexInBetweenExpander expander = new RexInBetweenExpander(
+ join.getCluster().getRexBuilder());
+ RexNode condition = expander.apply(join.getCondition());
+
+ if (!expander.modified) {
+return;
+ }
+
+ RelNode newJoin = join.copy(join.getTraitSet(),
+ condition,
+ join.getLeft(),
+ join.getRight(),
+ join.getJoinType(),
+ join.isSemiJoinDone());
+
+ call.transformTo(newJoin);
+}
+ }
+
+ /** Rule adapter to apply the transformation to Project expressions. */
+ private static class ProjectRule extends RelOptRule {
+
+ProjectRule() {
+ super(operand(Project.class, any()),
[jira] [Updated] (HIVE-23274) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is different, batch 1
[ https://issues.apache.org/jira/browse/HIVE-23274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23274: -- Attachment: HIVE-23274.05.patch > Move q tests to TestMiniLlapLocal from TestCliDriver where the output is > different, batch 1 > --- > > Key: HIVE-23274 > URL: https://issues.apache.org/jira/browse/HIVE-23274 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23274.01.patch, HIVE-23274.02.patch, > HIVE-23274.03.patch, HIVE-23274.04.patch, HIVE-23274.05.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[
https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097728#comment-17097728
]
Hive QA commented on HIVE-23215:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
52s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
23s{color} | {color:blue} storage-api in master has 51 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} storage-api: The patch generated 0 new + 11
unchanged - 3 fixed = 11 total (was 14) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} The patch ql passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
32s{color} | {color:red} storage-api generated 4 new + 46 unchanged - 5 fixed =
50 total (was 51) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 38s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:storage-api |
| | org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch.getSelected()
may expose internal representation by returning VectorizedRowBatch.selected At
VectorizedRowBatch.java:by returning VectorizedRowBatch.selected At
VectorizedRowBatch.java:[line 373] |
| |
org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch.updateSelected(int)
may expose internal representation by returning VectorizedRowBatch.selected At
VectorizedRowBatch.java:by returning VectorizedRowBatch.selected At
VectorizedRowBatch.java:[line 406] |
| |
org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch.setFilterContext(boolean,
int[], int) may expose internal representation by storing an externally
mutable object into VectorizedRowBatch.selected At
VectorizedRowBatch.java:internal representation by storing an externally
mutable object into VectorizedRowBatch.selected At
VectorizedRowBatch.java:[line 384] |
| |
org.apache.hadoop.hive.ql.exec.vector.VectorizedRowBatch.setSelected(int[]) may
expose internal representation by storing an externally mutable object into
VectorizedRowBatch.selected At VectorizedRowBatch.java:by storing an
externally mutable object into VectorizedRowBatch.selected At
VectorizedRowBatch.java:[line 416] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
[jira] [Commented] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[ https://issues.apache.org/jira/browse/HIVE-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097720#comment-17097720 ] Jesus Camacho Rodriguez commented on HIVE-22256: [~vgarg], can you take a look? https://github.com/apache/hive/pull/1002 Finally I took a less intrusive approach rather than changing current optimization workflow. > Rewriting fails when `IN` clause has items in different order in MV and query. > -- > > Key: HIVE-22256 > URL: https://issues.apache.org/jira/browse/HIVE-22256 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22256.01.patch, HIVE-22256.patch, expr2.sql > > Time Spent: 10m > Remaining Estimate: 0h > > Rewriting fails on following materialized view and query (script is also > attached): > create materialized view view2 stored as orc as (select prod_id, cust_id, > store_id, sale_date, qty, amt, descr from sales where cust_id in (1,2,3,4,5)); > explain extended select prod_id, cust_id from sales where cust_id in > (5,1,2,3,4); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[ https://issues.apache.org/jira/browse/HIVE-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22256: -- Labels: pull-request-available (was: ) > Rewriting fails when `IN` clause has items in different order in MV and query. > -- > > Key: HIVE-22256 > URL: https://issues.apache.org/jira/browse/HIVE-22256 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22256.01.patch, HIVE-22256.patch, expr2.sql > > Time Spent: 10m > Remaining Estimate: 0h > > Rewriting fails on following materialized view and query (script is also > attached): > create materialized view view2 stored as orc as (select prod_id, cust_id, > store_id, sale_date, qty, amt, descr from sales where cust_id in (1,2,3,4,5)); > explain extended select prod_id, cust_id from sales where cust_id in > (5,1,2,3,4); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[ https://issues.apache.org/jira/browse/HIVE-22256?focusedWorklogId=429697=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429697 ] ASF GitHub Bot logged work on HIVE-22256: - Author: ASF GitHub Bot Created on: 01/May/20 22:34 Start Date: 01/May/20 22:34 Worklog Time Spent: 10m Work Description: jcamachor opened a new pull request #1002: URL: https://github.com/apache/hive/pull/1002 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 429697) Remaining Estimate: 0h Time Spent: 10m > Rewriting fails when `IN` clause has items in different order in MV and query. > -- > > Key: HIVE-22256 > URL: https://issues.apache.org/jira/browse/HIVE-22256 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22256.01.patch, HIVE-22256.patch, expr2.sql > > Time Spent: 10m > Remaining Estimate: 0h > > Rewriting fails on following materialized view and query (script is also > attached): > create materialized view view2 stored as orc as (select prod_id, cust_id, > store_id, sale_date, qty, amt, descr from sales where cust_id in (1,2,3,4,5)); > explain extended select prod_id, cust_id from sales where cust_id in > (5,1,2,3,4); -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[
https://issues.apache.org/jira/browse/HIVE-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097717#comment-17097717
]
Hive QA commented on HIVE-22256:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001791/HIVE-22256.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17196 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22063/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22063/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22063/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001791 - PreCommit-HIVE-Build
> Rewriting fails when `IN` clause has items in different order in MV and query.
> --
>
> Key: HIVE-22256
> URL: https://issues.apache.org/jira/browse/HIVE-22256
> Project: Hive
> Issue Type: Sub-task
> Components: CBO, Materialized views
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22256.01.patch, HIVE-22256.patch, expr2.sql
>
>
> Rewriting fails on following materialized view and query (script is also
> attached):
> create materialized view view2 stored as orc as (select prod_id, cust_id,
> store_id, sale_date, qty, amt, descr from sales where cust_id in (1,2,3,4,5));
> explain extended select prod_id, cust_id from sales where cust_id in
> (5,1,2,3,4);
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23316) Add tests to cover database managed location related DDL and fix minor issues
[ https://issues.apache.org/jira/browse/HIVE-23316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-23316: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Add tests to cover database managed location related DDL and fix minor issues > - > > Key: HIVE-23316 > URL: https://issues.apache.org/jira/browse/HIVE-23316 > Project: Hive > Issue Type: Bug >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Attachments: HIVE-23316.01.patch > > > Database managed location was recently introduced, but no tests were added to > cover it. also the following issues were fixed: > * ALTER DATABASE ... SET MANAGEDLOCATION ... commands were not handled in a > separate path as it should, as in DDL each command type have their own > Analyzer, Desc, and Operation class > * in case of setting the LOCATION or the MANAGEDLOCATION the location was > not getting qualified as in the CREATE DATABASE command > * in case of setting the LOCATION or the MANAGEDLOCATION it was not checked > if this modification makes the two the same > * some minor checkstyle issues were fixed as well > Also the DDL documentation was not modified. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[
https://issues.apache.org/jira/browse/HIVE-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097691#comment-17097691
]
Hive QA commented on HIVE-22256:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
34s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 12 new + 119 unchanged - 0
fixed = 131 total (was 119) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22063/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22063/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22063/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Rewriting fails when `IN` clause has items in different order in MV and query.
> --
>
> Key: HIVE-22256
> URL: https://issues.apache.org/jira/browse/HIVE-22256
> Project: Hive
> Issue Type: Sub-task
> Components: CBO, Materialized views
>Affects Versions: 3.1.2
>Reporter: Steve Carlin
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22256.01.patch, HIVE-22256.patch, expr2.sql
>
>
> Rewriting fails on following materialized view and query (script is also
> attached):
> create materialized view view2 stored as orc as (select prod_id, cust_id,
> store_id, sale_date, qty, amt, descr from sales where cust_id in (1,2,3,4,5));
> explain extended select prod_id, cust_id from sales where cust_id in
> (5,1,2,3,4);
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23352) More user friendly StorageAuthorizationProvider log messages
[
https://issues.apache.org/jira/browse/HIVE-23352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097685#comment-17097685
]
Thejas Nair commented on HIVE-23352:
[~pgaref]
That is the intended behavior of SBA with external tables as well. Ie,
authorize metadata access based on file permissions. Sure we would improve the
error/log messages. What change are you suggesting ?
> More user friendly StorageAuthorizationProvider log messages
>
>
> Key: HIVE-23352
> URL: https://issues.apache.org/jira/browse/HIVE-23352
> Project: Hive
> Issue Type: Improvement
> Components: Security
>Affects Versions: 4.0.0
>Reporter: Panagiotis Garefalakis
>Priority: Minor
>
> Currently *StorageBasedAuthorizationProvider* returns messages (like below)
> about data paths even for _External_ tables where a drop command would just
> remove metadata. Lets make those messages more user-friendly.
> {code:java}
> Permission Denied: User hive can't delete hdfs://XXX.com:8020/tmp/testuser
> because sticky bit is set on the parent dir and user does not own this file
> or its parent)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23266) Remove QueryWrapper from ObjectStore
[
https://issues.apache.org/jira/browse/HIVE-23266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097681#comment-17097681
]
Hive QA commented on HIVE-23266:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001786/HIVE-23266.7.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 17195 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestGetPartitionsUsingProjectionAndFilterSpecs.testGetPartitions
(batchId=156)
org.apache.hadoop.hive.metastore.TestGetPartitionsUsingProjectionAndFilterSpecs.testPartitionProjectionEmptySpec
(batchId=156)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=163)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheInSession
(batchId=163)
org.apache.hadoop.hive.metastore.tools.metatool.TestHiveMetaTool.testExecuteJDOQL
(batchId=174)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22062/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22062/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22062/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001786 - PreCommit-HIVE-Build
> Remove QueryWrapper from ObjectStore
>
>
> Key: HIVE-23266
> URL: https://issues.apache.org/jira/browse/HIVE-23266
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23266.1.patch, HIVE-23266.2.patch,
> HIVE-23266.2.patch, HIVE-23266.3.patch, HIVE-23266.4.patch,
> HIVE-23266.5.patch, HIVE-23266.6.patch, HIVE-23266.7.patch
>
>
> There is currently a utility called {{QueryWrapper}} that makes a normal
> {{Query}} auto-closable. However, {{Query}} is now in fact already
> auto-closing, so there is no need for this class. In trying to remove it, I
> realized that this wrapper was being passed around in pretty convoluted ways
> and also it was sometimes being created in a {{try-with-resources}} block but
> then never actually used in any way.
> Remove the {{QueryWrapper}} from the class and simplify some of the DB
> interactions.
> https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/ObjectStore.java#L178
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23345) INT64 Parquet timestamps cannot be read into bigint Hive type
[
https://issues.apache.org/jira/browse/HIVE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-23345:
--
Attachment: HIVE-23345.01.patch
> INT64 Parquet timestamps cannot be read into bigint Hive type
> -
>
> Key: HIVE-23345
> URL: https://issues.apache.org/jira/browse/HIVE-23345
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Marta Kuczora
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23345.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce:
> - create external table ts_pq (ts timestamp) stored as parquet;
> - insert into ts_pq values ('1998-10-03 09:58:31.231');
> - create external table ts_pq_2 (ts bigint) stored as parquet location
> '';
> - select * from ts_pq_2;
> The following exception occurs during the select:
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.io.TimestampWritableV2 cannot be cast to
> org.apache.hadoop.io.LongWritable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23345) INT64 Parquet timestamps cannot be read into bigint Hive type
[
https://issues.apache.org/jira/browse/HIVE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-23345:
--
Attachment: (was: HIVE-23345.01.patch)
> INT64 Parquet timestamps cannot be read into bigint Hive type
> -
>
> Key: HIVE-23345
> URL: https://issues.apache.org/jira/browse/HIVE-23345
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Marta Kuczora
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23345.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce:
> - create external table ts_pq (ts timestamp) stored as parquet;
> - insert into ts_pq values ('1998-10-03 09:58:31.231');
> - create external table ts_pq_2 (ts bigint) stored as parquet location
> '';
> - select * from ts_pq_2;
> The following exception occurs during the select:
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.io.TimestampWritableV2 cannot be cast to
> org.apache.hadoop.io.LongWritable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23266) Remove QueryWrapper from ObjectStore
[
https://issues.apache.org/jira/browse/HIVE-23266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097664#comment-17097664
]
Hive QA commented on HIVE-23266:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
187 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
21s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 29 new + 277 unchanged - 28 fixed = 306 total (was 305) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
25s{color} | {color:red} standalone-metastore/metastore-server generated 2 new
+ 186 unchanged - 1 fixed = 188 total (was 187) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 49s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | Impossible cast from String to
org.apache.hadoop.hive.metastore.model.MPartition in
org.apache.hadoop.hive.metastore.ObjectStore.listPartitionsPsWithAuth(String,
String, String, List, short, String, List) At
ObjectStore.java:org.apache.hadoop.hive.metastore.model.MPartition in
org.apache.hadoop.hive.metastore.ObjectStore.listPartitionsPsWithAuth(String,
String, String, List, short, String, List) At ObjectStore.java:[line 3176] |
| |
org.apache.hadoop.hive.metastore.ObjectStore.getMPartitionColumnStatistics(Table,
List, List, String) concatenates strings using + in a loop At
ObjectStore.java:strings using + in a loop At ObjectStore.java:[line 9130] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22062/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22062/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22062/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22062/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove QueryWrapper from ObjectStore
>
>
> Key: HIVE-23266
> URL: https://issues.apache.org/jira/browse/HIVE-23266
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23266.1.patch,
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097649#comment-17097649
]
Hive QA commented on HIVE-22737:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001782/HIVE-22737.6.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17186 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.beeline.TestBeeLineWithArgs.testRowsAffected (batchId=208)
org.apache.hive.beeline.TestBeelinePasswordOption.org.apache.hive.beeline.TestBeelinePasswordOption
(batchId=208)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22061/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22061/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22061/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001782 - PreCommit-HIVE-Build
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.3.patch, HIVE-22737.4.patch, HIVE-22737.5.patch,
> HIVE-22737.6.patch, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22737) Concurrency: FunctionRegistry::getFunctionInfo is static object locked
[
https://issues.apache.org/jira/browse/HIVE-22737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097630#comment-17097630
]
Hive QA commented on HIVE-22737:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
35s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
40s{color} | {color:red} ql: The patch generated 19 new + 27 unchanged - 0
fixed = 46 total (was 27) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 35s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22061/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22061/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22061/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Concurrency: FunctionRegistry::getFunctionInfo is static object locked
> --
>
> Key: HIVE-22737
> URL: https://issues.apache.org/jira/browse/HIVE-22737
> Project: Hive
> Issue Type: Bug
> Components: Logical Optimizer, UDF
>Reporter: Gopal Vijayaraghavan
>Assignee: Ashutosh Chauhan
>Priority: Major
> Attachments: FunctionRegistry-lock.png, HIVE-22737.2.patch,
> HIVE-22737.3.patch, HIVE-22737.4.patch, HIVE-22737.5.patch,
> HIVE-22737.6.patch, HIVE-22737.patch
>
>
> The lock is inside a HS2-wide static object
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java#L191
> {code}
> // registry for system functions
> private static final Registry system = new Registry(true);
> {code}
> And this is the lock itself
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/Registry.java#L332
> {code}
> public FunctionInfo getFunctionInfo(String functionName) throws
> SemanticException {
> lock.lock();
> {code}
> !FunctionRegistry-lock.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23274) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is different, batch 1
[
https://issues.apache.org/jira/browse/HIVE-23274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097603#comment-17097603
]
Hive QA commented on HIVE-23274:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001781/HIVE-23274.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 17195 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[char_serde]
(batchId=92)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[distinct_windowing]
(batchId=60)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[distinct_windowing_no_cbo]
(batchId=100)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[gby_star]
(batchId=64)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22060/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22060/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22060/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001781 - PreCommit-HIVE-Build
> Move q tests to TestMiniLlapLocal from TestCliDriver where the output is
> different, batch 1
> ---
>
> Key: HIVE-23274
> URL: https://issues.apache.org/jira/browse/HIVE-23274
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23274.01.patch, HIVE-23274.02.patch,
> HIVE-23274.03.patch, HIVE-23274.04.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23206) Project not defined correctly after reordering a join
[ https://issues.apache.org/jira/browse/HIVE-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097565#comment-17097565 ] Vineet Garg commented on HIVE-23206: [~kkasa] So only concern left to address is why shared optimizer is unable to merge joins with this change when it was doing before? If so we can address that in a separate follow-up jira. Can you also address the checkstyle violation errors? > Project not defined correctly after reordering a join > - > > Key: HIVE-23206 > URL: https://issues.apache.org/jira/browse/HIVE-23206 > Project: Hive > Issue Type: Bug > Components: CBO >Reporter: Steve Carlin >Assignee: Krisztian Kasa >Priority: Major > Attachments: HIVE-23206.1.patch, HIVE-23206.2.patch, > HIVE-23206.3.patch, HIVE-23206.4.patch > > > The following highlighted line seems to be incorrect in the test suite: > [https://github.com/apache/hive/blob/master/ql/src/test/results/clientpositive/perf/tez/cbo_query95.q.out#L89] > Note that the project takes all the columns from the table scan, yet it only > needs a couple of them. > I did some very small debugging on this. When I removed the > applyJoinOrderingTransform here: > [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1897] > ... the problem goes away. So presumably one of the rules in there is > causing the problem. > Here is a slightly simplified version of the query which has the same problem > (using the same tpc-ds database): > explain cbo with ws_wh as > (select ws1.ws_order_number > from web_sales ws1,web_returns wr2 > where ws1.ws_order_number = wr2.wr_order_number) > select > ws_order_number > from > web_sales ws1 > where > ws1.ws_order_number in (select wr_order_number > from web_returns,ws_wh > where wr_order_number = ws_wh.ws_order_number) > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23274) Move q tests to TestMiniLlapLocal from TestCliDriver where the output is different, batch 1
[
https://issues.apache.org/jira/browse/HIVE-23274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097557#comment-17097557
]
Hive QA commented on HIVE-23274:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
11s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 3m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22060/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22060/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Move q tests to TestMiniLlapLocal from TestCliDriver where the output is
> different, batch 1
> ---
>
> Key: HIVE-23274
> URL: https://issues.apache.org/jira/browse/HIVE-23274
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23274.01.patch, HIVE-23274.02.patch,
> HIVE-23274.03.patch, HIVE-23274.04.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23348) Add API timing metric for fire_listener_event API
[
https://issues.apache.org/jira/browse/HIVE-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097548#comment-17097548
]
Hive QA commented on HIVE-23348:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001767/HIVE-23348.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17181 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc
(batchId=235)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22059/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22059/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22059/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001767 - PreCommit-HIVE-Build
> Add API timing metric for fire_listener_event API
> -
>
> Key: HIVE-23348
> URL: https://issues.apache.org/jira/browse/HIVE-23348
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Trivial
> Attachments: HIVE-23348.01.patch
>
>
> Currently metastore does not have any metric to report the time taken to
> execute {{fire_listener_event}} API. It would be useful to add such a metric.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23350) Upgrade DBCP To DBCP 2.7.0
[
https://issues.apache.org/jira/browse/HIVE-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23350:
--
Attachment: HIVE-23350.1.patch
> Upgrade DBCP To DBCP 2.7.0
> --
>
> Key: HIVE-23350
> URL: https://issues.apache.org/jira/browse/HIVE-23350
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23350.1.patch
>
>
> In {{hive-standalone-metastore-common}} the current version is 1.4 which is
> very antiquated and apparently (emphasis mine):
> bq. DBCP 1.4 compiles and runs under Java 6 *only* (JDBC 4)
> Hive is on Java 8
> {code:xml}
> 2.6.0
> 1.4
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23353) Atlas metadata replication scheduling
[ https://issues.apache.org/jira/browse/HIVE-23353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] PRAVIN KUMAR SINHA reassigned HIVE-23353: - > Atlas metadata replication scheduling > - > > Key: HIVE-23353 > URL: https://issues.apache.org/jira/browse/HIVE-23353 > Project: Hive > Issue Type: Task >Reporter: PRAVIN KUMAR SINHA >Assignee: PRAVIN KUMAR SINHA >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23348) Add API timing metric for fire_listener_event API
[
https://issues.apache.org/jira/browse/HIVE-23348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097515#comment-17097515
]
Hive QA commented on HIVE-23348:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
14s{color} | {color:blue} standalone-metastore/metastore-server in master has
187 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 43 new + 420 unchanged - 3 fixed = 463 total (was 423) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22059/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22059/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22059/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add API timing metric for fire_listener_event API
> -
>
> Key: HIVE-23348
> URL: https://issues.apache.org/jira/browse/HIVE-23348
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Trivial
> Attachments: HIVE-23348.01.patch
>
>
> Currently metastore does not have any metric to report the time taken to
> execute {{fire_listener_event}} API. It would be useful to add such a metric.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22476) Hive datediff function provided inconsistent results when hive.fetch.task.conversion is set to none
[
https://issues.apache.org/jira/browse/HIVE-22476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097493#comment-17097493
]
Hive QA commented on HIVE-22476:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001766/HIVE-22476.10.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17196 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testReplTimeouts
(batchId=244)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22058/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22058/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22058/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001766 - PreCommit-HIVE-Build
> Hive datediff function provided inconsistent results when
> hive.fetch.task.conversion is set to none
> ---
>
> Key: HIVE-22476
> URL: https://issues.apache.org/jira/browse/HIVE-22476
> Project: Hive
> Issue Type: Bug
>Reporter: Slim Bouguerra
>Assignee: Slim Bouguerra
>Priority: Major
> Attachments: HIVE-22476.10.patch, HIVE-22476.2.patch,
> HIVE-22476.3.patch, HIVE-22476.5.patch, HIVE-22476.6.patch,
> HIVE-22476.7.patch, HIVE-22476.7.patch, HIVE-22476.8.patch,
> HIVE-22476.8.patch, HIVE-22476.8.patch, HIVE-22476.9.patch
>
>
> The actual issue stems to the different date parser used by various part of
> the engine.
> Fetch task uses udfdatediff via {code}
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate{code} while the
> vectorized llap execution uses {code}VectorUDFDateDiffScalarCol{code}.
> This fix is meant to be not very intrusive and will add more support to the
> GenericUDFToDate by enhancing the parser.
> For the longer term will be better to use one parser for all the operators.
> Thanks [~Rajkumar Singh] for the repro example
> {code}
> create external table testdatediff(datetimecol string) stored as orc;
> insert into testdatediff values ('2019-09-09T10:45:49+02:00'),('2019-07-24');
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> set hive.ferch.task.conversion=none;
> select datetimecol from testdatediff where datediff(cast(current_timestamp as
> string), datetimecol)<183;
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-22476) Hive datediff function provided inconsistent results when hive.fetch.task.conversion is set to none
[
https://issues.apache.org/jira/browse/HIVE-22476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097472#comment-17097472
]
Hive QA commented on HIVE-22476:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
33s{color} | {color:blue} common in master has 63 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22058/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22058/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive datediff function provided inconsistent results when
> hive.fetch.task.conversion is set to none
> ---
>
> Key: HIVE-22476
> URL: https://issues.apache.org/jira/browse/HIVE-22476
> Project: Hive
> Issue Type: Bug
>Reporter: Slim Bouguerra
>Assignee: Slim Bouguerra
>Priority: Major
> Attachments: HIVE-22476.10.patch, HIVE-22476.2.patch,
> HIVE-22476.3.patch, HIVE-22476.5.patch, HIVE-22476.6.patch,
> HIVE-22476.7.patch, HIVE-22476.7.patch, HIVE-22476.8.patch,
> HIVE-22476.8.patch, HIVE-22476.8.patch, HIVE-22476.9.patch
>
>
> The actual issue stems to the different date parser used by various part of
> the engine.
> Fetch task uses udfdatediff via {code}
> org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate{code} while the
> vectorized llap execution uses {code}VectorUDFDateDiffScalarCol{code}.
> This fix is meant to be not very intrusive and will add more support to the
> GenericUDFToDate by enhancing the parser.
> For the longer term will be better to use one parser for all the operators.
> Thanks [~Rajkumar Singh] for the repro example
>
[jira] [Updated] (HIVE-23240) loadDynamicPartition complains about static partitions even when they are provided in the description
[ https://issues.apache.org/jira/browse/HIVE-23240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi updated HIVE-23240: - Attachment: HIVE-23240.3.patch > loadDynamicPartition complains about static partitions even when they are > provided in the description > -- > > Key: HIVE-23240 > URL: https://issues.apache.org/jira/browse/HIVE-23240 > Project: Hive > Issue Type: Bug >Reporter: Reza Safi >Assignee: Reza Safi >Priority: Minor > Attachments: HIVE-23240.2.patch, HIVE-23240.3.patch, HIVE-23240.patch > > > Hive is computing valid dynamic partitions here: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2853 > However it later uses the specification provided by client here: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2879 > (partSpec is exactly what client has provided and partSpec.keySet() contains > both static and dynamic partitions key) > As a result the makeSpecFromName here will expect both static and dynamic > partitions in requiredKeys: > https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/Warehouse.java#L580 > However since the curPath that is passed to the above method is just like > "somePath/dynamicPart=value" which miss the static partitions and a result > the method will ignore static partition keys then complains in log a warning > that the static partition keys are missing. Returning false to Hive.java, a > log warning that "dynamicPart=value" is an invalid partition will be issued, > despite the fact that the dynamic partition has been validated before: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2880 > > This will cause a silent data corruption in some clients. As an example spark > will suffer from this when working with hive metastore in master branch. > It seems that if the goal was just to warn the client, there is no need to > ignore the valid dynamic partition. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23240) loadDynamicPartition complains about static partitions even when they are provided in the description
[ https://issues.apache.org/jira/browse/HIVE-23240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097460#comment-17097460 ] Reza Safi commented on HIVE-23240: -- [~soundhearer] I think I've more context about this issue. I also was working on this for sometime and was testing another patch last night. I'll attach another patch soon. Sorry for reassignment to myself. > loadDynamicPartition complains about static partitions even when they are > provided in the description > -- > > Key: HIVE-23240 > URL: https://issues.apache.org/jira/browse/HIVE-23240 > Project: Hive > Issue Type: Bug >Reporter: Reza Safi >Assignee: Leping Huang >Priority: Minor > Attachments: HIVE-23240.2.patch, HIVE-23240.patch > > > Hive is computing valid dynamic partitions here: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2853 > However it later uses the specification provided by client here: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2879 > (partSpec is exactly what client has provided and partSpec.keySet() contains > both static and dynamic partitions key) > As a result the makeSpecFromName here will expect both static and dynamic > partitions in requiredKeys: > https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/Warehouse.java#L580 > However since the curPath that is passed to the above method is just like > "somePath/dynamicPart=value" which miss the static partitions and a result > the method will ignore static partition keys then complains in log a warning > that the static partition keys are missing. Returning false to Hive.java, a > log warning that "dynamicPart=value" is an invalid partition will be issued, > despite the fact that the dynamic partition has been validated before: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2880 > > This will cause a silent data corruption in some clients. As an example spark > will suffer from this when working with hive metastore in master branch. > It seems that if the goal was just to warn the client, there is no need to > ignore the valid dynamic partition. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23240) loadDynamicPartition complains about static partitions even when they are provided in the description
[ https://issues.apache.org/jira/browse/HIVE-23240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reza Safi reassigned HIVE-23240: Assignee: Reza Safi (was: Leping Huang) > loadDynamicPartition complains about static partitions even when they are > provided in the description > -- > > Key: HIVE-23240 > URL: https://issues.apache.org/jira/browse/HIVE-23240 > Project: Hive > Issue Type: Bug >Reporter: Reza Safi >Assignee: Reza Safi >Priority: Minor > Attachments: HIVE-23240.2.patch, HIVE-23240.patch > > > Hive is computing valid dynamic partitions here: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2853 > However it later uses the specification provided by client here: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2879 > (partSpec is exactly what client has provided and partSpec.keySet() contains > both static and dynamic partitions key) > As a result the makeSpecFromName here will expect both static and dynamic > partitions in requiredKeys: > https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/Warehouse.java#L580 > However since the curPath that is passed to the above method is just like > "somePath/dynamicPart=value" which miss the static partitions and a result > the method will ignore static partition keys then complains in log a warning > that the static partition keys are missing. Returning false to Hive.java, a > log warning that "dynamicPart=value" is an invalid partition will be issued, > despite the fact that the dynamic partition has been validated before: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2880 > > This will cause a silent data corruption in some clients. As an example spark > will suffer from this when working with hive metastore in master branch. > It seems that if the goal was just to warn the client, there is no need to > ignore the valid dynamic partition. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21304) Make bucketing version usage more robust
[ https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21304: Attachment: HIVE-21304.35.patch > Make bucketing version usage more robust > > > Key: HIVE-21304 > URL: https://issues.apache.org/jira/browse/HIVE-21304 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21304.01.patch, HIVE-21304.02.patch, > HIVE-21304.03.patch, HIVE-21304.04.patch, HIVE-21304.05.patch, > HIVE-21304.06.patch, HIVE-21304.07.patch, HIVE-21304.08.patch, > HIVE-21304.09.patch, HIVE-21304.10.patch, HIVE-21304.11.patch, > HIVE-21304.12.patch, HIVE-21304.13.patch, HIVE-21304.14.patch, > HIVE-21304.15.patch, HIVE-21304.16.patch, HIVE-21304.17.patch, > HIVE-21304.18.patch, HIVE-21304.19.patch, HIVE-21304.20.patch, > HIVE-21304.21.patch, HIVE-21304.22.patch, HIVE-21304.23.patch, > HIVE-21304.24.patch, HIVE-21304.25.patch, HIVE-21304.26.patch, > HIVE-21304.27.patch, HIVE-21304.28.patch, HIVE-21304.29.patch, > HIVE-21304.30.patch, HIVE-21304.31.patch, HIVE-21304.32.patch, > HIVE-21304.33.patch, HIVE-21304.33.patch, HIVE-21304.33.patch, > HIVE-21304.34.patch, HIVE-21304.34.patch, HIVE-21304.35.patch, > HIVE-21304.35.patch > > Time Spent: 50m > Remaining Estimate: 0h > > * Show Bucketing version for ReduceSinkOp in explain extended plan - this > helps identify what hashing algorithm is being used by by ReduceSinkOp. > * move the actually selected version to the "conf" so that it doesn't get lost > * replace trait related logic with a separate optimizer rule > * do version selection based on a group of operator - this is more reliable > * skip bucketingversion selection for tables with 1 buckets > * prefer to use version 2 if possible > * fix operator creations which didn't set a new conf -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23323) Add qsplits profile
[
https://issues.apache.org/jira/browse/HIVE-23323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097438#comment-17097438
]
Hive QA commented on HIVE-23323:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001735/HIVE-23323.02.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17199 tests
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation3 (batchId=218)
org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthBinary.testAuthorization1
(batchId=235)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22056/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22056/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22056/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001735 - PreCommit-HIVE-Build
> Add qsplits profile
> ---
>
> Key: HIVE-23323
> URL: https://issues.apache.org/jira/browse/HIVE-23323
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23323.01.patch, HIVE-23323.02.patch,
> HIVE-23323.02.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> provide an in-maven way to split-up big testclasses into smaller ones
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23350) Upgrade DBCP To DBCP 2.7.0
[
https://issues.apache.org/jira/browse/HIVE-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23350:
--
Description:
In {{hive-standalone-metastore-common}} the current version is 1.4 which is
very antiquated and apparently (emphasis mine):
bq. DBCP 1.4 compiles and runs under Java 6 *only* (JDBC 4)
Hive is on Java 8
{code:xml}
2.6.0
1.4
{code}
was:
In {{hive-standalone-metastore-common}} the current version is 1.4 which is
very antiquated and apparently (emphasis mine):
bq. DBCP 1.4 compiles and runs under Java 6 *only* (JDBC 4)
Hive is on Java 8
> Upgrade DBCP To DBCP 2.7.0
> --
>
> Key: HIVE-23350
> URL: https://issues.apache.org/jira/browse/HIVE-23350
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
>
> In {{hive-standalone-metastore-common}} the current version is 1.4 which is
> very antiquated and apparently (emphasis mine):
> bq. DBCP 1.4 compiles and runs under Java 6 *only* (JDBC 4)
> Hive is on Java 8
> {code:xml}
> 2.6.0
> 1.4
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23323) Add qsplits profile
[
https://issues.apache.org/jira/browse/HIVE-23323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097405#comment-17097405
]
Hive QA commented on HIVE-23323:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
56s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
49s{color} | {color:blue} itests/util in master has 53 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
34s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
13s{color} | {color:red} itests/util: The patch generated 10 new + 0 unchanged
- 0 fixed = 10 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/qtest: The patch generated 1 new + 8 unchanged
- 0 fixed = 9 total (was 8) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 10 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
54s{color} | {color:red} itests/util generated 1 new + 53 unchanged - 0 fixed =
54 total (was 53) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 4s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:itests/util |
| | Redundant nullcheck of c, which is known to be non-null in
org.apache.hadoop.hive.cli.control.SplitSupport.isSplit0ClassExistsFor(Class)
Redundant null check at SplitSupport.java:is known to be non-null in
org.apache.hadoop.hive.cli.control.SplitSupport.isSplit0ClassExistsFor(Class)
Redundant null check at SplitSupport.java:[line 63] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile findbugs
checkstyle |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22056/dev-support/hive-personality.sh
|
| git revision | master / cdd55aa |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22056/yetus/diff-checkstyle-itests_util.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22056/yetus/diff-checkstyle-itests_qtest.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22056/yetus/whitespace-tabs.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22056/yetus/new-findbugs-itests_util.html
|
| modules | C: itests itests/util itests/qtest itests/qtest-spark U: itests |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22056/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Add qsplits profile
> ---
>
> Key: HIVE-23323
>
[jira] [Assigned] (HIVE-23351) Ranger Replication Scheduling
[ https://issues.apache.org/jira/browse/HIVE-23351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi reassigned HIVE-23351: -- > Ranger Replication Scheduling > - > > Key: HIVE-23351 > URL: https://issues.apache.org/jira/browse/HIVE-23351 > Project: Hive > Issue Type: Task >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21304) Make bucketing version usage more robust
[
https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097394#comment-17097394
]
Hive QA commented on HIVE-21304:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001818/HIVE-21304.35.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17196 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_reducers_power_two]
(batchId=7)
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testReplTimeouts
(batchId=244)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22055/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22055/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22055/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001818 - PreCommit-HIVE-Build
> Make bucketing version usage more robust
>
>
> Key: HIVE-21304
> URL: https://issues.apache.org/jira/browse/HIVE-21304
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21304.01.patch, HIVE-21304.02.patch,
> HIVE-21304.03.patch, HIVE-21304.04.patch, HIVE-21304.05.patch,
> HIVE-21304.06.patch, HIVE-21304.07.patch, HIVE-21304.08.patch,
> HIVE-21304.09.patch, HIVE-21304.10.patch, HIVE-21304.11.patch,
> HIVE-21304.12.patch, HIVE-21304.13.patch, HIVE-21304.14.patch,
> HIVE-21304.15.patch, HIVE-21304.16.patch, HIVE-21304.17.patch,
> HIVE-21304.18.patch, HIVE-21304.19.patch, HIVE-21304.20.patch,
> HIVE-21304.21.patch, HIVE-21304.22.patch, HIVE-21304.23.patch,
> HIVE-21304.24.patch, HIVE-21304.25.patch, HIVE-21304.26.patch,
> HIVE-21304.27.patch, HIVE-21304.28.patch, HIVE-21304.29.patch,
> HIVE-21304.30.patch, HIVE-21304.31.patch, HIVE-21304.32.patch,
> HIVE-21304.33.patch, HIVE-21304.33.patch, HIVE-21304.33.patch,
> HIVE-21304.34.patch, HIVE-21304.34.patch, HIVE-21304.35.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> * Show Bucketing version for ReduceSinkOp in explain extended plan - this
> helps identify what hashing algorithm is being used by by ReduceSinkOp.
> * move the actually selected version to the "conf" so that it doesn't get lost
> * replace trait related logic with a separate optimizer rule
> * do version selection based on a group of operator - this is more reliable
> * skip bucketingversion selection for tables with 1 buckets
> * prefer to use version 2 if possible
> * fix operator creations which didn't set a new conf
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23350) Upgrade DBCP To DBCP 2.7.0
[
https://issues.apache.org/jira/browse/HIVE-23350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor reassigned HIVE-23350:
-
> Upgrade DBCP To DBCP 2.7.0
> --
>
> Key: HIVE-23350
> URL: https://issues.apache.org/jira/browse/HIVE-23350
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
>
> In {{hive-standalone-metastore-common}} the current version is 1.4 which is
> very antiquated and apparently (emphasis mine):
> bq. DBCP 1.4 compiles and runs under Java 6 *only* (JDBC 4)
> Hive is on Java 8
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23309) Lazy Initialization of Hadoop Shims
[ https://issues.apache.org/jira/browse/HIVE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23309: --- Attachment: HIVE-23309.03.patch Status: Patch Available (was: In Progress) > Lazy Initialization of Hadoop Shims > --- > > Key: HIVE-23309 > URL: https://issues.apache.org/jira/browse/HIVE-23309 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: HIVE-23309.01.patch, HIVE-23309.02.patch, > HIVE-23309.03.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Initialize hadoop-shims only if CM is enabled -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23309) Lazy Initialization of Hadoop Shims
[ https://issues.apache.org/jira/browse/HIVE-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aasha Medhi updated HIVE-23309: --- Status: In Progress (was: Patch Available) > Lazy Initialization of Hadoop Shims > --- > > Key: HIVE-23309 > URL: https://issues.apache.org/jira/browse/HIVE-23309 > Project: Hive > Issue Type: Bug >Reporter: Aasha Medhi >Assignee: Aasha Medhi >Priority: Major > Attachments: HIVE-23309.01.patch, HIVE-23309.02.patch, > HIVE-23309.03.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Initialize hadoop-shims only if CM is enabled -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-21304) Make bucketing version usage more robust
[
https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097373#comment-17097373
]
Hive QA commented on HIVE-21304:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
54s{color} | {color:red} ql: The patch generated 6 new + 1312 unchanged - 12
fixed = 1318 total (was 1324) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
46s{color} | {color:red} ql generated 1 new + 1530 unchanged - 0 fixed = 1531
total (was 1530) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 29m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Nullcheck of table_desc at line 8064 of value previously dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.createFileSinkDesc(String,
TableDesc, Partition, Path, int, boolean, boolean, boolean, Path,
SemanticAnalyzer$SortBucketRSCtx, DynamicPartitionCtx, ListBucketingCtx,
RowSchema, boolean, Table, Long, boolean, Integer, QB, boolean) At
SemanticAnalyzer.java:8064 of value previously dereferenced in
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.createFileSinkDesc(String,
TableDesc, Partition, Path, int, boolean, boolean, boolean, Path,
SemanticAnalyzer$SortBucketRSCtx, DynamicPartitionCtx, ListBucketingCtx,
RowSchema, boolean, Table, Long, boolean, Integer, QB, boolean) At
SemanticAnalyzer.java:[line 8057] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22055/dev-support/hive-personality.sh
|
| git revision | master / 0897480 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22055/yetus/diff-checkstyle-ql.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22055/yetus/new-findbugs-ql.html
|
| modules | C: ql itests itests/hive-blobstore U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22055/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Make bucketing version usage more robust
>
>
> Key: HIVE-21304
> URL: https://issues.apache.org/jira/browse/HIVE-21304
> Project: Hive
> Issue Type: Bug
>Reporter: Deepak Jaiswal
>
[jira] [Commented] (HIVE-23108) Cleanup HiveBaseResultSet.java
[
https://issues.apache.org/jira/browse/HIVE-23108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097372#comment-17097372
]
David Mollitor commented on HIVE-23108:
---
[~pvary] Please review once more :) I added some unit tests and fixed the
'wasNull' situation you pointed out
> Cleanup HiveBaseResultSet.java
> --
>
> Key: HIVE-23108
> URL: https://issues.apache.org/jira/browse/HIVE-23108
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23108.1.patch, HIVE-23108.2.patch,
> HIVE-23108.3.patch, HIVE-23108.4.patch, HIVE-23108.5.patch,
> HIVE-23108.5.patch, HIVE-23108.5.patch
>
>
> * Unify the code (there are several implementations of the same thing)
> * Added better error messages
> * In some cases, the code was throwing RuntimeExceptions which is against
> the JDBC Spec
> * Make findColumn a bit more streamlined
> * Remove non-javadoc comments
> * Add {{@Override}} annotations where appropriate
> * Formatting changes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23124) Review of SQLOperation Class
[
https://issues.apache.org/jira/browse/HIVE-23124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23124:
--
Attachment: HIVE-23124.3.patch
> Review of SQLOperation Class
>
>
> Key: HIVE-23124
> URL: https://issues.apache.org/jira/browse/HIVE-23124
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23124.1.patch, HIVE-23124.2.patch,
> HIVE-23124.3.patch, HIVE-23124.3.patch, HIVE-23124.3.patch, HIVE-23124.3.patch
>
>
> * Use ConcurrentHashMap instead of synchronized methods to improve
> multi-threaded access
> * Use JDK 8 facilities where applicable
> * General cleanup
> * Better log messages and Exception messages
> * Use {{switch}} statement instead of if/else blocks
> * Checkstyle fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23345) INT64 Parquet timestamps cannot be read into bigint Hive type
[
https://issues.apache.org/jira/browse/HIVE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-23345:
--
Status: Patch Available (was: In Progress)
> INT64 Parquet timestamps cannot be read into bigint Hive type
> -
>
> Key: HIVE-23345
> URL: https://issues.apache.org/jira/browse/HIVE-23345
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Marta Kuczora
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23345.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce:
> - create external table ts_pq (ts timestamp) stored as parquet;
> - insert into ts_pq values ('1998-10-03 09:58:31.231');
> - create external table ts_pq_2 (ts bigint) stored as parquet location
> '';
> - select * from ts_pq_2;
> The following exception occurs during the select:
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.io.TimestampWritableV2 cannot be cast to
> org.apache.hadoop.io.LongWritable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work started] (HIVE-23345) INT64 Parquet timestamps cannot be read into bigint Hive type
[
https://issues.apache.org/jira/browse/HIVE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-23345 started by Panagiotis Garefalakis.
-
> INT64 Parquet timestamps cannot be read into bigint Hive type
> -
>
> Key: HIVE-23345
> URL: https://issues.apache.org/jira/browse/HIVE-23345
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Marta Kuczora
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23345.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce:
> - create external table ts_pq (ts timestamp) stored as parquet;
> - insert into ts_pq values ('1998-10-03 09:58:31.231');
> - create external table ts_pq_2 (ts bigint) stored as parquet location
> '';
> - select * from ts_pq_2;
> The following exception occurs during the select:
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.io.TimestampWritableV2 cannot be cast to
> org.apache.hadoop.io.LongWritable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23117) Review of HiveStatement Class
[
https://issues.apache.org/jira/browse/HIVE-23117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Mollitor updated HIVE-23117:
--
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks for the review [~pvary]!
> Review of HiveStatement Class
> -
>
> Key: HIVE-23117
> URL: https://issues.apache.org/jira/browse/HIVE-23117
> Project: Hive
> Issue Type: Improvement
> Components: JDBC
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-23117.1.patch, HIVE-23117.2.patch,
> HIVE-23117.2.patch, HIVE-23117.3.patch
>
>
> * Remove unused instance variable(s)
> * Remove non-JavaDoc comments
> * Make inPlaceUpdateStream Optional (and remove NO-OP class) (inconsistent
> behavior with 'null' values)
> * {{getQueryTimeout()}} returns incorrect value
> * Unify and improve Exception messages
> * Checkstyle fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23345) INT64 Parquet timestamps cannot be read into bigint Hive type
[
https://issues.apache.org/jira/browse/HIVE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-23345:
--
Attachment: HIVE-23345.01.patch
> INT64 Parquet timestamps cannot be read into bigint Hive type
> -
>
> Key: HIVE-23345
> URL: https://issues.apache.org/jira/browse/HIVE-23345
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Marta Kuczora
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Attachments: HIVE-23345.01.patch
>
>
> How to reproduce:
> - create external table ts_pq (ts timestamp) stored as parquet;
> - insert into ts_pq values ('1998-10-03 09:58:31.231');
> - create external table ts_pq_2 (ts bigint) stored as parquet location
> '';
> - select * from ts_pq_2;
> The following exception occurs during the select:
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.io.TimestampWritableV2 cannot be cast to
> org.apache.hadoop.io.LongWritable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23345) INT64 Parquet timestamps cannot be read into bigint Hive type
[
https://issues.apache.org/jira/browse/HIVE-23345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-23345:
--
Labels: pull-request-available (was: )
> INT64 Parquet timestamps cannot be read into bigint Hive type
> -
>
> Key: HIVE-23345
> URL: https://issues.apache.org/jira/browse/HIVE-23345
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Marta Kuczora
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-23345.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce:
> - create external table ts_pq (ts timestamp) stored as parquet;
> - insert into ts_pq values ('1998-10-03 09:58:31.231');
> - create external table ts_pq_2 (ts bigint) stored as parquet location
> '';
> - select * from ts_pq_2;
> The following exception occurs during the select:
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.io.TimestampWritableV2 cannot be cast to
> org.apache.hadoop.io.LongWritable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-23345) INT64 Parquet timestamps cannot be read into bigint Hive type
[
https://issues.apache.org/jira/browse/HIVE-23345?focusedWorklogId=429477=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429477
]
ASF GitHub Bot logged work on HIVE-23345:
-
Author: ASF GitHub Bot
Created on: 01/May/20 13:26
Start Date: 01/May/20 13:26
Worklog Time Spent: 10m
Work Description: pgaref opened a new pull request #1001:
URL: https://github.com/apache/hive/pull/1001
Enable Parquet timestamps types (INT64 and INT96) conversion to Hive BIGINT
type
Adding test cases
Change-Id: I8666a95cc7ff7495a86b960c2ea173cd875bfa4f
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429477)
Remaining Estimate: 0h
Time Spent: 10m
> INT64 Parquet timestamps cannot be read into bigint Hive type
> -
>
> Key: HIVE-23345
> URL: https://issues.apache.org/jira/browse/HIVE-23345
> Project: Hive
> Issue Type: Bug
>Affects Versions: 4.0.0
>Reporter: Marta Kuczora
>Assignee: Panagiotis Garefalakis
>Priority: Major
> Attachments: HIVE-23345.01.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> How to reproduce:
> - create external table ts_pq (ts timestamp) stored as parquet;
> - insert into ts_pq values ('1998-10-03 09:58:31.231');
> - create external table ts_pq_2 (ts bigint) stored as parquet location
> '';
> - select * from ts_pq_2;
> The following exception occurs during the select:
> Failed with exception
> java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException:
> java.lang.ClassCastException:
> org.apache.hadoop.hive.serde2.io.TimestampWritableV2 cannot be cast to
> org.apache.hadoop.io.LongWritable
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23321) Tolerate in tests that metastore is not removing rows from the skewed_string_list_values table
[
https://issues.apache.org/jira/browse/HIVE-23321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097358#comment-17097358
]
Hive QA commented on HIVE-23321:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001736/HIVE-23321.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17195 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schq_ingest]
(batchId=102)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22054/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22054/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22054/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001736 - PreCommit-HIVE-Build
> Tolerate in tests that metastore is not removing rows from the
> skewed_string_list_values table
> --
>
> Key: HIVE-23321
> URL: https://issues.apache.org/jira/browse/HIVE-23321
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23321.01.patch, HIVE-23321.01.patch
>
>
> I've found this issue; and filed: HIVE-23289 - however fixing it is not easy
> at all..
> * foreign keys were not used
> * there were 0 effort to remove these rows before...I don't think when the
> feature was added the removal was considered...
> * unfortunately designed table hierarchy makes it hard to work with it...
> * unclear contracts (at least to me) between skewed_values and sd stuff
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-21304) Make bucketing version usage more robust
[ https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21304: Attachment: HIVE-21304.35.patch > Make bucketing version usage more robust > > > Key: HIVE-21304 > URL: https://issues.apache.org/jira/browse/HIVE-21304 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21304.01.patch, HIVE-21304.02.patch, > HIVE-21304.03.patch, HIVE-21304.04.patch, HIVE-21304.05.patch, > HIVE-21304.06.patch, HIVE-21304.07.patch, HIVE-21304.08.patch, > HIVE-21304.09.patch, HIVE-21304.10.patch, HIVE-21304.11.patch, > HIVE-21304.12.patch, HIVE-21304.13.patch, HIVE-21304.14.patch, > HIVE-21304.15.patch, HIVE-21304.16.patch, HIVE-21304.17.patch, > HIVE-21304.18.patch, HIVE-21304.19.patch, HIVE-21304.20.patch, > HIVE-21304.21.patch, HIVE-21304.22.patch, HIVE-21304.23.patch, > HIVE-21304.24.patch, HIVE-21304.25.patch, HIVE-21304.26.patch, > HIVE-21304.27.patch, HIVE-21304.28.patch, HIVE-21304.29.patch, > HIVE-21304.30.patch, HIVE-21304.31.patch, HIVE-21304.32.patch, > HIVE-21304.33.patch, HIVE-21304.33.patch, HIVE-21304.33.patch, > HIVE-21304.34.patch, HIVE-21304.34.patch, HIVE-21304.35.patch > > Time Spent: 50m > Remaining Estimate: 0h > > * Show Bucketing version for ReduceSinkOp in explain extended plan - this > helps identify what hashing algorithm is being used by by ReduceSinkOp. > * move the actually selected version to the "conf" so that it doesn't get lost > * replace trait related logic with a separate optimizer rule > * do version selection based on a group of operator - this is more reliable > * skip bucketingversion selection for tables with 1 buckets > * prefer to use version 2 if possible > * fix operator creations which didn't set a new conf -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-21304) Make bucketing version usage more robust
[ https://issues.apache.org/jira/browse/HIVE-21304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21304: Attachment: HIVE-21304.34.patch > Make bucketing version usage more robust > > > Key: HIVE-21304 > URL: https://issues.apache.org/jira/browse/HIVE-21304 > Project: Hive > Issue Type: Bug >Reporter: Deepak Jaiswal >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21304.01.patch, HIVE-21304.02.patch, > HIVE-21304.03.patch, HIVE-21304.04.patch, HIVE-21304.05.patch, > HIVE-21304.06.patch, HIVE-21304.07.patch, HIVE-21304.08.patch, > HIVE-21304.09.patch, HIVE-21304.10.patch, HIVE-21304.11.patch, > HIVE-21304.12.patch, HIVE-21304.13.patch, HIVE-21304.14.patch, > HIVE-21304.15.patch, HIVE-21304.16.patch, HIVE-21304.17.patch, > HIVE-21304.18.patch, HIVE-21304.19.patch, HIVE-21304.20.patch, > HIVE-21304.21.patch, HIVE-21304.22.patch, HIVE-21304.23.patch, > HIVE-21304.24.patch, HIVE-21304.25.patch, HIVE-21304.26.patch, > HIVE-21304.27.patch, HIVE-21304.28.patch, HIVE-21304.29.patch, > HIVE-21304.30.patch, HIVE-21304.31.patch, HIVE-21304.32.patch, > HIVE-21304.33.patch, HIVE-21304.33.patch, HIVE-21304.33.patch, > HIVE-21304.34.patch, HIVE-21304.34.patch > > Time Spent: 50m > Remaining Estimate: 0h > > * Show Bucketing version for ReduceSinkOp in explain extended plan - this > helps identify what hashing algorithm is being used by by ReduceSinkOp. > * move the actually selected version to the "conf" so that it doesn't get lost > * replace trait related logic with a separate optimizer rule > * do version selection based on a group of operator - this is more reliable > * skip bucketingversion selection for tables with 1 buckets > * prefer to use version 2 if possible > * fix operator creations which didn't set a new conf -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23321) Tolerate in tests that metastore is not removing rows from the skewed_string_list_values table
[
https://issues.apache.org/jira/browse/HIVE-23321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097330#comment-17097330
]
Hive QA commented on HIVE-23321:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
39s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22054/dev-support/hive-personality.sh
|
| git revision | master / 0897480 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22054/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tolerate in tests that metastore is not removing rows from the
> skewed_string_list_values table
> --
>
> Key: HIVE-23321
> URL: https://issues.apache.org/jira/browse/HIVE-23321
> Project: Hive
> Issue Type: Sub-task
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23321.01.patch, HIVE-23321.01.patch
>
>
> I've found this issue; and filed: HIVE-23289 - however fixing it is not easy
> at all..
> * foreign keys were not used
> * there were 0 effort to remove these rows before...I don't think when the
> feature was added the removal was considered...
> * unfortunately designed table hierarchy makes it hard to work with it...
> * unclear contracts (at least to me) between skewed_values and sd stuff
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23252) Change spark related tests to be optional
[
https://issues.apache.org/jira/browse/HIVE-23252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097325#comment-17097325
]
Hive QA commented on HIVE-23252:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001734/HIVE-23252.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17188 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACID - did not produce a TEST-*.xml file (likely
timed out) (batchId=184)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[infer_bucket_sort_reducers_power_two]
(batchId=7)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22053/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22053/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22053/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001734 - PreCommit-HIVE-Build
> Change spark related tests to be optional
> -
>
> Key: HIVE-23252
> URL: https://issues.apache.org/jira/browse/HIVE-23252
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23252.01.patch, HIVE-23252.01.patch,
> HIVE-23252.01.patch, HIVE-23252.01.patch, HIVE-23252.01.patch,
> HIVE-23252.01.patch, HIVE-23252.01.patch, HIVE-23252.01.patch
>
>
> HIVE-23137 have disabled the execution of some spark related tests; but they
> would be still considered by a plain maven command - and the spark artifacts
> are (unneccessarily) still downloaded
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (HIVE-23240) loadDynamicPartition complains about static partitions even when they are provided in the description
[ https://issues.apache.org/jira/browse/HIVE-23240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Leping Huang reassigned HIVE-23240: --- Assignee: Leping Huang > loadDynamicPartition complains about static partitions even when they are > provided in the description > -- > > Key: HIVE-23240 > URL: https://issues.apache.org/jira/browse/HIVE-23240 > Project: Hive > Issue Type: Bug >Reporter: Reza Safi >Assignee: Leping Huang >Priority: Minor > Attachments: HIVE-23240.2.patch, HIVE-23240.patch > > > Hive is computing valid dynamic partitions here: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2853 > However it later uses the specification provided by client here: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2879 > (partSpec is exactly what client has provided and partSpec.keySet() contains > both static and dynamic partitions key) > As a result the makeSpecFromName here will expect both static and dynamic > partitions in requiredKeys: > https://github.com/apache/hive/blob/master/standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/Warehouse.java#L580 > However since the curPath that is passed to the above method is just like > "somePath/dynamicPart=value" which miss the static partitions and a result > the method will ignore static partition keys then complains in log a warning > that the static partition keys are missing. Returning false to Hive.java, a > log warning that "dynamicPart=value" is an invalid partition will be issued, > despite the fact that the dynamic partition has been validated before: > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L2880 > > This will cause a silent data corruption in some clients. As an example spark > will suffer from this when working with hive metastore in master branch. > It seems that if the goal was just to warn the client, there is no need to > ignore the valid dynamic partition. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23252) Change spark related tests to be optional
[
https://issues.apache.org/jira/browse/HIVE-23252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097318#comment-17097318
]
Hive QA commented on HIVE-23252:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
3s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 47m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22053/dev-support/hive-personality.sh
|
| git revision | master / 4a82ccf |
| Default Java | 1.8.0_111 |
| modules | C: . itests itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22053/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Change spark related tests to be optional
> -
>
> Key: HIVE-23252
> URL: https://issues.apache.org/jira/browse/HIVE-23252
> Project: Hive
> Issue Type: Sub-task
> Components: Test
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-23252.01.patch, HIVE-23252.01.patch,
> HIVE-23252.01.patch, HIVE-23252.01.patch, HIVE-23252.01.patch,
> HIVE-23252.01.patch, HIVE-23252.01.patch, HIVE-23252.01.patch
>
>
> HIVE-23137 have disabled the execution of some spark related tests; but they
> would be still considered by a plain maven command - and the spark artifacts
> are (unneccessarily) still downloaded
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23344) Remove org.scala-lang:scala-compiler 2.11.0 transitive dependency
[
https://issues.apache.org/jira/browse/HIVE-23344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097291#comment-17097291
]
Hive QA commented on HIVE-23344:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001727/HIVE-23344.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 17192 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.spark.TestHiveSparkClient.testSetJobGroupAndDescription
(batchId=290)
org.apache.hadoop.hive.ql.exec.spark.TestLocalHiveSparkClient.testMultiSessionSparkContextReUse
(batchId=289)
org.apache.hadoop.hive.ql.exec.spark.TestSparkInvalidFileFormat.readTextFileAsParquet
(batchId=290)
org.apache.hadoop.hive.ql.exec.spark.TestSparkPlan.testSetRDDCallSite
(batchId=290)
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testForceConfCloning
(batchId=290)
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testGetHiveException
(batchId=290)
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testGetSessionId
(batchId=290)
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testMultiSessionMultipleUse
(batchId=290)
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testServerPortAssignment
(batchId=290)
org.apache.hadoop.hive.ql.exec.spark.session.TestSparkSessionManagerImpl.testSingleSessionMultipleUse
(batchId=290)
org.apache.hive.spark.client.TestSparkClient.testAddJarsAndFiles (batchId=303)
org.apache.hive.spark.client.TestSparkClient.testCounters (batchId=303)
org.apache.hive.spark.client.TestSparkClient.testErrorJob (batchId=303)
org.apache.hive.spark.client.TestSparkClient.testErrorJobNotSerializable
(batchId=303)
org.apache.hive.spark.client.TestSparkClient.testJobSubmission (batchId=303)
org.apache.hive.spark.client.TestSparkClient.testMetricsCollection (batchId=303)
org.apache.hive.spark.client.TestSparkClient.testSimpleSparkJob (batchId=303)
org.apache.hive.spark.client.TestSparkClient.testSyncRpc (batchId=303)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22052/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22052/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22052/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001727 - PreCommit-HIVE-Build
> Remove org.scala-lang:scala-compiler 2.11.0 transitive dependency
> -
>
> Key: HIVE-23344
> URL: https://issues.apache.org/jira/browse/HIVE-23344
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23344.01.patch
>
>
> Lots of
> org.scala-lang:scala-compiler:jar:2.11.0:compile ->
> org.scala-lang:scalap:jar:2.11.0:compile ->
> org.json4s:json4s-core_2.11:jar:3.2.11:compile ->
> org.json4s:json4s-jackson_2.11:jar:3.2.11:compile ->
> org.apache.spark:spark-core_2.11:jar:2.3.0:compile
> Where scala-compiler 2.11.0 is a security vulnerability.
> Excluding org.json4s:json4s-jackson_2.11:jar:3.2.11 from spark-core_2
> dependencies.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23344) Remove org.scala-lang:scala-compiler 2.11.0 transitive dependency
[
https://issues.apache.org/jira/browse/HIVE-23344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097288#comment-17097288
]
Hive QA commented on HIVE-23344:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
49s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 58m 36s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22052/dev-support/hive-personality.sh
|
| git revision | master / 4a82ccf |
| Default Java | 1.8.0_111 |
| modules | C: spark-client ql kryo-registrator . itests/hive-unit
itests/qtest-spark U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22052/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove org.scala-lang:scala-compiler 2.11.0 transitive dependency
> -
>
> Key: HIVE-23344
> URL: https://issues.apache.org/jira/browse/HIVE-23344
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-23344.01.patch
>
>
> Lots of
> org.scala-lang:scala-compiler:jar:2.11.0:compile ->
> org.scala-lang:scalap:jar:2.11.0:compile ->
> org.json4s:json4s-core_2.11:jar:3.2.11:compile ->
> org.json4s:json4s-jackson_2.11:jar:3.2.11:compile ->
> org.apache.spark:spark-core_2.11:jar:2.3.0:compile
> Where scala-compiler 2.11.0 is a security vulnerability.
> Excluding org.json4s:json4s-jackson_2.11:jar:3.2.11 from spark-core_2
> dependencies.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23315) Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-23315:
--
Attachment: (was: HIVE-23315.01.patch)
> Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED
> VIEWS
> --
>
> Key: HIVE-23315
> URL: https://issues.apache.org/jira/browse/HIVE-23315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23315.01.patch
>
>
> At the end of each SHOW EXTENDED TABLES; and SHOW MATERIALIZED VIEWS; command
> there is an empty line like this:
> {code:java}
> +++
> | tab_name | table_type |
> +++
> | sample_07 | MANAGED_TABLE |
> | sample_08 | MANAGED_TABLE |
> | web_logs | MANAGED_TABLE |
> | | NULL |
> +++
> {code}
> It should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23315) Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED VIEWS
[
https://issues.apache.org/jira/browse/HIVE-23315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Gergely updated HIVE-23315:
--
Attachment: HIVE-23315.01.patch
> Remove empty line from the end of SHOW EXTENDED TABLES and SHOW MATERIALIZED
> VIEWS
> --
>
> Key: HIVE-23315
> URL: https://issues.apache.org/jira/browse/HIVE-23315
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Attachments: HIVE-23315.01.patch
>
>
> At the end of each SHOW EXTENDED TABLES; and SHOW MATERIALIZED VIEWS; command
> there is an empty line like this:
> {code:java}
> +++
> | tab_name | table_type |
> +++
> | sample_07 | MANAGED_TABLE |
> | sample_08 | MANAGED_TABLE |
> | web_logs | MANAGED_TABLE |
> | | NULL |
> +++
> {code}
> It should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22769) Incorrect query results and query failure during split generation for compressed text files
[
https://issues.apache.org/jira/browse/HIVE-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-22769:
--
Attachment: HIVE-22769.04.patch
> Incorrect query results and query failure during split generation for
> compressed text files
> ---
>
> Key: HIVE-22769
> URL: https://issues.apache.org/jira/browse/HIVE-22769
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Chiran Ravani
>Assignee: Panagiotis Garefalakis
>Priority: Critical
> Attachments: HIVE-22769.01.patch, HIVE-22769.02.patch,
> HIVE-22769.03.patch, HIVE-22769.04.patch, testcase1.csv.bz2, testcase2.csv.bz2
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Hive Query produces incorrect results when data is in text format and
> compressed and for certain data the query fails during split generation.
> This behavior is seen when skip.header.line.count and skip.footer.line.count
> are set for table.
> Case 1: Select count/aggregate query produces Incorrect row counts/displays
> all rows (when hive.fetch.task.conversion=none)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase1`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase1'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase1.csv.bz2 file to /user/hive/testcase1
> 3. Run count(*) on table.
> {code}
> > select * from testcase1;
> INFO : Compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:testcase1.id, type:string,
> comment:null), FieldSchema(name:testcase1.name, type:string, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.07 seconds
> INFO : Executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Completed executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.007 seconds
> INFO : OK
> +---+-+
> | testcase1.id | testcase1.name |
> +---+-+
> | 2 | 2019-12-31 |
> +---+-+
> 1 row selected (0.111 seconds)
> > select count(*) from testcase1
> INFO : Compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:bigint, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 0.073 seconds
> INFO : Executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Query ID = hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Subscribed to counters: [] for queryId:
> hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Session is already open
> INFO : Dag name: select count(*) from testcase1 (Stage-1)
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1579811438512_0046)
> .
> .
> .
> INFO : Completed executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 4.228 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 3|
> +--+
> 1 row selected (4.335 seconds)
> {code}
> Case 2: Select count/aggregate query fails with java.lang.ClassCastException:
> java.io.PushbackInputStream cannot be cast to org.apache.hadoop.fs.Seekable
> The issue is only seen when there is a space in a field (eg:- "3,2019-12-31
> 01" second column has a space)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase2`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase2'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase2.csv.bz2 file to /user/hive/testcase2
> 3. Run count(*) on table.
> {code}
> 0: > select * from testcase2;
> INFO : Compiling
>
[jira] [Updated] (HIVE-22769) Incorrect query results and query failure during split generation for compressed text files
[
https://issues.apache.org/jira/browse/HIVE-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-22769:
--
Attachment: HIVE-22769.04.patch
> Incorrect query results and query failure during split generation for
> compressed text files
> ---
>
> Key: HIVE-22769
> URL: https://issues.apache.org/jira/browse/HIVE-22769
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Chiran Ravani
>Assignee: Panagiotis Garefalakis
>Priority: Critical
> Attachments: HIVE-22769.01.patch, HIVE-22769.02.patch,
> HIVE-22769.03.patch, testcase1.csv.bz2, testcase2.csv.bz2
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Hive Query produces incorrect results when data is in text format and
> compressed and for certain data the query fails during split generation.
> This behavior is seen when skip.header.line.count and skip.footer.line.count
> are set for table.
> Case 1: Select count/aggregate query produces Incorrect row counts/displays
> all rows (when hive.fetch.task.conversion=none)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase1`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase1'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase1.csv.bz2 file to /user/hive/testcase1
> 3. Run count(*) on table.
> {code}
> > select * from testcase1;
> INFO : Compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:testcase1.id, type:string,
> comment:null), FieldSchema(name:testcase1.name, type:string, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.07 seconds
> INFO : Executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Completed executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.007 seconds
> INFO : OK
> +---+-+
> | testcase1.id | testcase1.name |
> +---+-+
> | 2 | 2019-12-31 |
> +---+-+
> 1 row selected (0.111 seconds)
> > select count(*) from testcase1
> INFO : Compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:bigint, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 0.073 seconds
> INFO : Executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Query ID = hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Subscribed to counters: [] for queryId:
> hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Session is already open
> INFO : Dag name: select count(*) from testcase1 (Stage-1)
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1579811438512_0046)
> .
> .
> .
> INFO : Completed executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 4.228 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 3|
> +--+
> 1 row selected (4.335 seconds)
> {code}
> Case 2: Select count/aggregate query fails with java.lang.ClassCastException:
> java.io.PushbackInputStream cannot be cast to org.apache.hadoop.fs.Seekable
> The issue is only seen when there is a space in a field (eg:- "3,2019-12-31
> 01" second column has a space)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase2`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase2'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase2.csv.bz2 file to /user/hive/testcase2
> 3. Run count(*) on table.
> {code}
> 0: > select * from testcase2;
> INFO : Compiling
> command(queryId=hive_20200124053159_5d8ce56a-183d-4359-a147-bd470d82e134):
> select
[jira] [Updated] (HIVE-22769) Incorrect query results and query failure during split generation for compressed text files
[
https://issues.apache.org/jira/browse/HIVE-22769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-22769:
--
Attachment: (was: HIVE-22769.04.patch)
> Incorrect query results and query failure during split generation for
> compressed text files
> ---
>
> Key: HIVE-22769
> URL: https://issues.apache.org/jira/browse/HIVE-22769
> Project: Hive
> Issue Type: Bug
> Components: File Formats
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Chiran Ravani
>Assignee: Panagiotis Garefalakis
>Priority: Critical
> Attachments: HIVE-22769.01.patch, HIVE-22769.02.patch,
> HIVE-22769.03.patch, testcase1.csv.bz2, testcase2.csv.bz2
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Hive Query produces incorrect results when data is in text format and
> compressed and for certain data the query fails during split generation.
> This behavior is seen when skip.header.line.count and skip.footer.line.count
> are set for table.
> Case 1: Select count/aggregate query produces Incorrect row counts/displays
> all rows (when hive.fetch.task.conversion=none)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase1`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase1'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase1.csv.bz2 file to /user/hive/testcase1
> 3. Run count(*) on table.
> {code}
> > select * from testcase1;
> INFO : Compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema:
> Schema(fieldSchemas:[FieldSchema(name:testcase1.id, type:string,
> comment:null), FieldSchema(name:testcase1.name, type:string, comment:null)],
> properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.07 seconds
> INFO : Executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f):
> select * from testcase1
> INFO : Completed executing
> command(queryId=hive_20200124053854_454b03c1-d4c5-4dba-a2c2-91c09f4b670f);
> Time taken: 0.007 seconds
> INFO : OK
> +---+-+
> | testcase1.id | testcase1.name |
> +---+-+
> | 2 | 2019-12-31 |
> +---+-+
> 1 row selected (0.111 seconds)
> > select count(*) from testcase1
> INFO : Compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0,
> type:bigint, comment:null)], properties:null)
> INFO : Completed compiling
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 0.073 seconds
> INFO : Executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7):
> select count(*) from testcase1
> INFO : Query ID = hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Total jobs = 1
> INFO : Launching Job 1 out of 1
> INFO : Starting task [Stage-1:MAPRED] in serial mode
> INFO : Subscribed to counters: [] for queryId:
> hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7
> INFO : Session is already open
> INFO : Dag name: select count(*) from testcase1 (Stage-1)
> INFO : Status: Running (Executing on YARN cluster with App id
> application_1579811438512_0046)
> .
> .
> .
> INFO : Completed executing
> command(queryId=hive_20200124053645_a7d699b7-c7e1-4d92-8d99-666b0a010ba7);
> Time taken: 4.228 seconds
> INFO : OK
> +--+
> | _c0 |
> +--+
> | 3|
> +--+
> 1 row selected (4.335 seconds)
> {code}
> Case 2: Select count/aggregate query fails with java.lang.ClassCastException:
> java.io.PushbackInputStream cannot be cast to org.apache.hadoop.fs.Seekable
> The issue is only seen when there is a space in a field (eg:- "3,2019-12-31
> 01" second column has a space)
> Steps to reproduce:
> 1. Create table as below
> {code}
> CREATE EXTERNAL TABLE `testcase2`(id int, name string) ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.OpenCSVSerde' LOCATION '/user/hive/testcase2'
> TBLPROPERTIES ("skip.header.line.count"="1", "skip.footer.line.count"="1");
> {code}
> 2. Upload attached testcase2.csv.bz2 file to /user/hive/testcase2
> 3. Run count(*) on table.
> {code}
> 0: > select * from testcase2;
> INFO : Compiling
> command(queryId=hive_20200124053159_5d8ce56a-183d-4359-a147-bd470d82e134):
[jira] [Assigned] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-23215: - Assignee: Owen O'Malley (was: Panagiotis Garefalakis) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Owen O'Malley >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.02.patch, > HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis updated HIVE-23215: -- Attachment: HIVE-23215.02.patch > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.02.patch, > HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (HIVE-23215) Make FilterContext and MutableFilterContext interfaces
[ https://issues.apache.org/jira/browse/HIVE-23215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Panagiotis Garefalakis reassigned HIVE-23215: - Assignee: Panagiotis Garefalakis (was: Owen O'Malley) > Make FilterContext and MutableFilterContext interfaces > -- > > Key: HIVE-23215 > URL: https://issues.apache.org/jira/browse/HIVE-23215 > Project: Hive > Issue Type: Bug > Components: storage-api >Reporter: Owen O'Malley >Assignee: Panagiotis Garefalakis >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23215.01.patch, HIVE-23215.02.patch, > HIVE-23215.patch > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HIVE-22959 introduced FilterContext to support ORC-577. The duplication of > fields between the FilterContext and VectorizedRowBatch seems likely to cause > user confusion. This patch makes them interfaces that VectorizedRowBatch > implements. > Thus, there is a single copy of the data and no need to copy them back and > forth. LLAP can make its own implementation of the interfaces if it doesn't > want to use VectorizedRowBatch. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23206) Project not defined correctly after reordering a join
[
https://issues.apache.org/jira/browse/HIVE-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097267#comment-17097267
]
Hive QA commented on HIVE-23206:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001723/HIVE-23206.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17193 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.parse.TestScheduledReplicationScenarios.testAcidTablesReplLoadBootstrapIncr
(batchId=205)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22051/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22051/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22051/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001723 - PreCommit-HIVE-Build
> Project not defined correctly after reordering a join
> -
>
> Key: HIVE-23206
> URL: https://issues.apache.org/jira/browse/HIVE-23206
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Steve Carlin
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-23206.1.patch, HIVE-23206.2.patch,
> HIVE-23206.3.patch, HIVE-23206.4.patch
>
>
> The following highlighted line seems to be incorrect in the test suite:
> [https://github.com/apache/hive/blob/master/ql/src/test/results/clientpositive/perf/tez/cbo_query95.q.out#L89]
> Note that the project takes all the columns from the table scan, yet it only
> needs a couple of them.
> I did some very small debugging on this. When I removed the
> applyJoinOrderingTransform here:
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1897]
> ... the problem goes away. So presumably one of the rules in there is
> causing the problem.
> Here is a slightly simplified version of the query which has the same problem
> (using the same tpc-ds database):
> explain cbo with ws_wh as
> (select ws1.ws_order_number
> from web_sales ws1,web_returns wr2
> where ws1.ws_order_number = wr2.wr_order_number)
> select
> ws_order_number
> from
> web_sales ws1
> where
> ws1.ws_order_number in (select wr_order_number
> from web_returns,ws_wh
> where wr_order_number = ws_wh.ws_order_number)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23206) Project not defined correctly after reordering a join
[
https://issues.apache.org/jira/browse/HIVE-23206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097250#comment-17097250
]
Hive QA commented on HIVE-23206:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
30s{color} | {color:blue} ql in master has 1530 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
37s{color} | {color:red} ql: The patch generated 3 new + 0 unchanged - 3 fixed
= 3 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22051/dev-support/hive-personality.sh
|
| git revision | master / 4a82ccf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22051/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22051/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Project not defined correctly after reordering a join
> -
>
> Key: HIVE-23206
> URL: https://issues.apache.org/jira/browse/HIVE-23206
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Steve Carlin
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-23206.1.patch, HIVE-23206.2.patch,
> HIVE-23206.3.patch, HIVE-23206.4.patch
>
>
> The following highlighted line seems to be incorrect in the test suite:
> [https://github.com/apache/hive/blob/master/ql/src/test/results/clientpositive/perf/tez/cbo_query95.q.out#L89]
> Note that the project takes all the columns from the table scan, yet it only
> needs a couple of them.
> I did some very small debugging on this. When I removed the
> applyJoinOrderingTransform here:
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java#L1897]
> ... the problem goes away. So presumably one of the rules in there is
> causing the problem.
> Here is a slightly simplified version of the query which has the same problem
> (using the same tpc-ds database):
> explain cbo with ws_wh as
> (select
[jira] [Work logged] (HIVE-23323) Add qsplits profile
[ https://issues.apache.org/jira/browse/HIVE-23323?focusedWorklogId=429405=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429405 ] ASF GitHub Bot logged work on HIVE-23323: - Author: ASF GitHub Bot Created on: 01/May/20 08:22 Start Date: 01/May/20 08:22 Worklog Time Spent: 10m Work Description: kgyrtkirk commented on pull request #1000: URL: https://github.com/apache/hive/pull/1000#issuecomment-622297049 . This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 429405) Time Spent: 20m (was: 10m) > Add qsplits profile > --- > > Key: HIVE-23323 > URL: https://issues.apache.org/jira/browse/HIVE-23323 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-23323.01.patch, HIVE-23323.02.patch, > HIVE-23323.02.patch > > Time Spent: 20m > Remaining Estimate: 0h > > provide an in-maven way to split-up big testclasses into smaller ones -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (HIVE-23323) Add qsplits profile
[ https://issues.apache.org/jira/browse/HIVE-23323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-23323: -- Labels: pull-request-available (was: ) > Add qsplits profile > --- > > Key: HIVE-23323 > URL: https://issues.apache.org/jira/browse/HIVE-23323 > Project: Hive > Issue Type: Sub-task >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Labels: pull-request-available > Attachments: HIVE-23323.01.patch, HIVE-23323.02.patch, > HIVE-23323.02.patch > > Time Spent: 20m > Remaining Estimate: 0h > > provide an in-maven way to split-up big testclasses into smaller ones -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23124) Review of SQLOperation Class
[
https://issues.apache.org/jira/browse/HIVE-23124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097240#comment-17097240
]
Hive QA commented on HIVE-23124:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001720/HIVE-23124.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 17192 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.txn.TestTxnHandler.testReplTimeouts
(batchId=244)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22050/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22050/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22050/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001720 - PreCommit-HIVE-Build
> Review of SQLOperation Class
>
>
> Key: HIVE-23124
> URL: https://issues.apache.org/jira/browse/HIVE-23124
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23124.1.patch, HIVE-23124.2.patch,
> HIVE-23124.3.patch, HIVE-23124.3.patch, HIVE-23124.3.patch
>
>
> * Use ConcurrentHashMap instead of synchronized methods to improve
> multi-threaded access
> * Use JDK 8 facilities where applicable
> * General cleanup
> * Better log messages and Exception messages
> * Use {{switch}} statement instead of if/else blocks
> * Checkstyle fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-23347) MSCK REPAIR cannot discover partitions with upper case directory names.
[ https://issues.apache.org/jira/browse/HIVE-23347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-23347: Component/s: (was: HiveServer2) Standalone Metastore > MSCK REPAIR cannot discover partitions with upper case directory names. > --- > > Key: HIVE-23347 > URL: https://issues.apache.org/jira/browse/HIVE-23347 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.0 >Reporter: Sankar Hariappan >Assignee: Adesh Kumar Rao >Priority: Minor > > For the following scenario, we expect MSCK REPAIR to discover partitions but > it couldn't. > 1. Have partitioned data path as follows. > hdfs://mycluster/datapath/t1/Year=2020/Month=03/Day=10 > hdfs://mycluster/datapath/t1/Year=2020/Month=03/Day=11 > 2. create external table t1 (key int, value string) partitioned by (Year int, > Month int, Day int) stored as orc location hdfs://mycluster/datapath/t1''; > 3. msck repair table t1; > 4. show partitions t1; --> Returns zero partitions > 5. select * from t1; --> Returns empty data. > When the partition directory names are changed to lower case, this works fine. > hdfs://mycluster/datapath/t1/year=2020/month=03/day=10 > hdfs://mycluster/datapath/t1/year=2020/month=03/day=11 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HIVE-23124) Review of SQLOperation Class
[
https://issues.apache.org/jira/browse/HIVE-23124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097221#comment-17097221
]
Hive QA commented on HIVE-23124:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} service in master has 51 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} service: The patch generated 0 new + 14 unchanged -
1 fixed = 14 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22050/dev-support/hive-personality.sh
|
| git revision | master / 4a82ccf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: service U: service |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22050/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Review of SQLOperation Class
>
>
> Key: HIVE-23124
> URL: https://issues.apache.org/jira/browse/HIVE-23124
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Major
> Attachments: HIVE-23124.1.patch, HIVE-23124.2.patch,
> HIVE-23124.3.patch, HIVE-23124.3.patch, HIVE-23124.3.patch
>
>
> * Use ConcurrentHashMap instead of synchronized methods to improve
> multi-threaded access
> * Use JDK 8 facilities where applicable
> * General cleanup
> * Better log messages and Exception messages
> * Use {{switch}} statement instead of if/else blocks
> * Checkstyle fixes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23108) Cleanup HiveBaseResultSet.java
[
https://issues.apache.org/jira/browse/HIVE-23108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097209#comment-17097209
]
Hive QA commented on HIVE-23108:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13001730/HIVE-23108.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 17201 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22049/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22049/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22049/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13001730 - PreCommit-HIVE-Build
> Cleanup HiveBaseResultSet.java
> --
>
> Key: HIVE-23108
> URL: https://issues.apache.org/jira/browse/HIVE-23108
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23108.1.patch, HIVE-23108.2.patch,
> HIVE-23108.3.patch, HIVE-23108.4.patch, HIVE-23108.5.patch,
> HIVE-23108.5.patch, HIVE-23108.5.patch
>
>
> * Unify the code (there are several implementations of the same thing)
> * Added better error messages
> * In some cases, the code was throwing RuntimeExceptions which is against
> the JDBC Spec
> * Make findColumn a bit more streamlined
> * Remove non-javadoc comments
> * Add {{@Override}} annotations where appropriate
> * Formatting changes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HIVE-23108) Cleanup HiveBaseResultSet.java
[
https://issues.apache.org/jira/browse/HIVE-23108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097191#comment-17097191
]
Hive QA commented on HIVE-23108:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} jdbc in master has 16 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} jdbc: The patch generated 0 new + 0 unchanged - 23
fixed = 0 total (was 23) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} jdbc generated 0 new + 14 unchanged - 2 fixed = 14
total (was 16) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-22049/dev-support/hive-personality.sh
|
| git revision | master / 4a82ccf |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: jdbc U: jdbc |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-22049/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Cleanup HiveBaseResultSet.java
> --
>
> Key: HIVE-23108
> URL: https://issues.apache.org/jira/browse/HIVE-23108
> Project: Hive
> Issue Type: Improvement
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-23108.1.patch, HIVE-23108.2.patch,
> HIVE-23108.3.patch, HIVE-23108.4.patch, HIVE-23108.5.patch,
> HIVE-23108.5.patch, HIVE-23108.5.patch
>
>
> * Unify the code (there are several implementations of the same thing)
> * Added better error messages
> * In some cases, the code was throwing RuntimeExceptions which is against
> the JDBC Spec
> * Make findColumn a bit more streamlined
> * Remove non-javadoc comments
> * Add {{@Override}} annotations where appropriate
> * Formatting changes
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HIVE-22256) Rewriting fails when `IN` clause has items in different order in MV and query.
[ https://issues.apache.org/jira/browse/HIVE-22256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22256: --- Attachment: HIVE-22256.01.patch > Rewriting fails when `IN` clause has items in different order in MV and query. > -- > > Key: HIVE-22256 > URL: https://issues.apache.org/jira/browse/HIVE-22256 > Project: Hive > Issue Type: Sub-task > Components: CBO, Materialized views >Affects Versions: 3.1.2 >Reporter: Steve Carlin >Assignee: Jesus Camacho Rodriguez >Priority: Major > Attachments: HIVE-22256.01.patch, HIVE-22256.patch, expr2.sql > > > Rewriting fails on following materialized view and query (script is also > attached): > create materialized view view2 stored as orc as (select prod_id, cust_id, > store_id, sale_date, qty, amt, descr from sales where cust_id in (1,2,3,4,5)); > explain extended select prod_id, cust_id from sales where cust_id in > (5,1,2,3,4); -- This message was sent by Atlassian Jira (v8.3.4#803005)
