[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.1.patch
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Open (was: Patch Available)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available (was: Open)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22164) Vectorized Limit operator returns wrong number of results with offset
[
https://issues.apache.org/jira/browse/HIVE-22164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923929#comment-16923929
]
Hive QA commented on HIVE-22164:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
35s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18469/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18469/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorized Limit operator returns wrong number of results with offset
> -
>
> Key: HIVE-22164
> URL: https://issues.apache.org/jira/browse/HIVE-22164
> Project: Hive
> Issue Type: Bug
> Components: Hive, llap, Vectorization
>Affects Versions: 4.0.0
>Reporter: Ramesh Kumar Thangarajan
>Assignee: Ramesh Kumar Thangarajan
>Priority: Major
> Attachments: HIVE-22164.1.patch
>
>
> Vectorized Limit operator returns wrong number of results with offset
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22168) remove excessive logging by llap cache.
[
https://issues.apache.org/jira/browse/HIVE-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923922#comment-16923922
]
Hive QA commented on HIVE-22168:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979566/HIVE-22168.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 16746 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18468/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18468/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18468/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979566 - PreCommit-HIVE-Build
> remove excessive logging by llap cache.
> ---
>
> Key: HIVE-22168
> URL: https://issues.apache.org/jira/browse/HIVE-22168
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logging
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-22168.patch, HIVE-22168.patch
>
>
> Llap cache logging is very expensive when it comes to log every request
> buffers range.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22168) remove excessive logging by llap cache.
[
https://issues.apache.org/jira/browse/HIVE-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923905#comment-16923905
]
Hive QA commented on HIVE-22168:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
35s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 5s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18468/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18468/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> remove excessive logging by llap cache.
> ---
>
> Key: HIVE-22168
> URL: https://issues.apache.org/jira/browse/HIVE-22168
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logging
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-22168.patch, HIVE-22168.patch
>
>
> Llap cache logging is very expensive when it comes to log every request
> buffers range.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21942) Remove useless MetastoreType enum from AbstractCliConfig
[
https://issues.apache.org/jira/browse/HIVE-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923895#comment-16923895
]
Hive QA commented on HIVE-21942:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979563/HIVE-21942.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16746 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18467/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18467/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18467/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979563 - PreCommit-HIVE-Build
> Remove useless MetastoreType enum from AbstractCliConfig
>
>
> Key: HIVE-21942
> URL: https://issues.apache.org/jira/browse/HIVE-21942
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Laszlo Bodor
>Assignee: Imre Molnar
>Priority: Major
> Labels: newbie
> Fix For: 4.0.0
>
> Attachments: HIVE-21942.1.patch
>
>
> I'm not 100% aware of historical reasons, but this single-value enum seems
> totally useless at the moment.
> https://github.com/apache/hive/blob/86a15600ae610e1c6017019883ef9344585572ef/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/AbstractCliConfig.java#L50-L52
> According to HIVE-14854, it may have made sense when there was an initiative
> of hbase metastore, but it's not in the scope anymore AFAIK.
> https://github.com/apache/hive/commit/474425aa62e3f25b119419439373aa684c6c2121
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22150) HS2 allows setting system properties
[
https://issues.apache.org/jira/browse/HIVE-22150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hui An updated HIVE-22150:
--
Affects Version/s: 4.0.0
> HS2 allows setting system properties
>
>
> Key: HIVE-22150
> URL: https://issues.apache.org/jira/browse/HIVE-22150
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0, 3.1.1
>Reporter: Craig Condit
>Assignee: Hui An
>Priority: Major
> Attachments: HIVE-22150.patch.1, HIVE-22150.patch.2
>
>
> HiveServer2 currently allows setting system properties, which is a problem
> when used in a multi-user environment.
> Connecting via beeline and executing the following demonstrates the issue:
> {noformat}
> 0: jdbc:hive2://serv1000.example.com:2181,serv> SET system:java.io.tmpdir;
> +-+
> | set |
> +-+
> | system:java.io.tmpdir=/tmp |
> +-+
> 1 row selected (0.018 seconds)
> 0: jdbc:hive2://serv1000.example.com:2181,serv> SET
> system:java.io.tmpdir=/tmp/attacker-dir;
> No rows affected (0.013 seconds)
> 0: jdbc:hive2://serv1000.example.com:2181,serv> SET system:java.io.tmpdir;
> +--+
> | set|
> +--+
> | system:java.io.tmpdir=/tmp/attacker-dir |
> +--+
> 1 row selected (0.019 seconds)
> {noformat}
> Any changes persist until HS2 is restarted, and affect all connected users.
> At the very least, this is a denial-of-service vector (verified by setting
> line.separator to a random string).
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21942) Remove useless MetastoreType enum from AbstractCliConfig
[
https://issues.apache.org/jira/browse/HIVE-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923870#comment-16923870
]
Hive QA commented on HIVE-21942:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/util in master has 44 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} itests/util: The patch generated 0 new + 44
unchanged - 1 fixed = 44 total (was 45) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 38s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18467/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests/util U: itests/util |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18467/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove useless MetastoreType enum from AbstractCliConfig
>
>
> Key: HIVE-21942
> URL: https://issues.apache.org/jira/browse/HIVE-21942
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Laszlo Bodor
>Assignee: Imre Molnar
>Priority: Major
> Labels: newbie
> Fix For: 4.0.0
>
> Attachments: HIVE-21942.1.patch
>
>
> I'm not 100% aware of historical reasons, but this single-value enum seems
> totally useless at the moment.
> https://github.com/apache/hive/blob/86a15600ae610e1c6017019883ef9344585572ef/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/AbstractCliConfig.java#L50-L52
> According to HIVE-14854, it may have made sense when there was an initiative
> of hbase metastore, but it's not in the scope anymore AFAIK.
> https://github.com/apache/hive/commit/474425aa62e3f25b119419439373aa684c6c2121
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21397) BloomFilter for hive Managed [ACID] table does not work as expected
[
https://issues.apache.org/jira/browse/HIVE-21397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923862#comment-16923862
]
Hive QA commented on HIVE-21397:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979560/HIVE-21397.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16746 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18466/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18466/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18466/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979560 - PreCommit-HIVE-Build
> BloomFilter for hive Managed [ACID] table does not work as expected
> ---
>
> Key: HIVE-21397
> URL: https://issues.apache.org/jira/browse/HIVE-21397
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2, Transactions
>Affects Versions: 3.1.1
>Reporter: vaibhav
>Assignee: Denys Kuzmenko
>Priority: Blocker
> Attachments: HIVE-21397.1.patch, HIVE-21397.2.patch,
> HIVE-21397.3.patch, HIVE-21397.4.patch, HIVE-21397.5.patch, HIVE-21397.patch
>
>
> Steps to Reproduce this issue :
> -
> 1. Create a HIveManaged table as below :
> -
> {code:java}
> CREATE TABLE `bloomTest`(
> `msisdn` string,
> `imsi` varchar(20),
> `imei` bigint,
> `cell_id` bigint)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION
>
> 'hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest;
>
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'orc.bloom.filter.columns'='msisdn,cell_id,imsi',
> 'orc.bloom.filter.fpp'='0.02',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1551206683') {code}
> -
> 2. Insert a few rows.
> -
> -
> 3. Check if bloom filter or active : [ It does not show bloom filters for
> hive managed tables ]
> -
> {code:java}
> [hive@c1162-node2 root]$ hive --orcfiledump
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest/delta_001_001_
> | grep -i bloom
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory]
> Processing data file
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest/delta_001_001_/bucket_0
> [length: 791]
> Structure for
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest/delta_001_001_/bucket_0
> {code}
> -
> On Another hand: For hive External tables it works :
> -
> {code:java}
> CREATE external TABLE `ext_bloomTest`(
> `msisdn` string,
> `imsi` varchar(20),
> `imei` bigint,
> `cell_id` bigint)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'orc.bloom.filter.columns'='msisdn,cell_id,imsi',
> 'orc.bloom.filter.fpp'='0.02') {code}
> -
> {code:java}
> [hive@c1162-node2 root]$ hive --orcfiledump
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/external/hive/ext_bloomTest/00_0
> | grep -i bloom
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
>
[jira] [Commented] (HIVE-21397) BloomFilter for hive Managed [ACID] table does not work as expected
[
https://issues.apache.org/jira/browse/HIVE-21397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923849#comment-16923849
]
Hive QA commented on HIVE-21397:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18466/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18466/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> BloomFilter for hive Managed [ACID] table does not work as expected
> ---
>
> Key: HIVE-21397
> URL: https://issues.apache.org/jira/browse/HIVE-21397
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2, Transactions
>Affects Versions: 3.1.1
>Reporter: vaibhav
>Assignee: Denys Kuzmenko
>Priority: Blocker
> Attachments: HIVE-21397.1.patch, HIVE-21397.2.patch,
> HIVE-21397.3.patch, HIVE-21397.4.patch, HIVE-21397.5.patch, HIVE-21397.patch
>
>
> Steps to Reproduce this issue :
> -
> 1. Create a HIveManaged table as below :
> -
> {code:java}
> CREATE TABLE `bloomTest`(
> `msisdn` string,
> `imsi` varchar(20),
> `imei` bigint,
> `cell_id` bigint)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION
>
> 'hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest;
>
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'orc.bloom.filter.columns'='msisdn,cell_id,imsi',
> 'orc.bloom.filter.fpp'='0.02',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1551206683') {code}
> -
> 2. Insert a few rows.
> -
> -
> 3. Check if bloom filter or active : [ It does not show bloom filters for
> hive managed tables ]
> -
> {code:java}
> [hive@c1162-node2 root]$ hive --orcfiledump
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest/delta_001_001_
> | grep -i bloom
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory]
> Processing data file
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest/delta_001_001_/bucket_0
> [length: 791]
> Structure for
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest/delta_001_001_/bucket_0
> {code}
> -
> On Another hand: For hive External tables it works :
> -
> {code:java}
> CREATE external TABLE `ext_bloomTest`(
> `msisdn` string,
> `imsi` varchar(20),
> `imei` bigint,
> `cell_id` bigint)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
>
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923844#comment-16923844
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979557/HIVE-21844.04.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18465/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18465/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18465/
Messages:
{noformat}
This message was trimmed, see log for full details
error: patch failed:
ql/src/test/results/clientpositive/perf/tez/query6.q.out:109
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/perf/tez/query6.q.out'
cleanly.
error: patch failed:
ql/src/test/results/clientpositive/tez/explainanalyze_3.q.out:958
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/tez/explainanalyze_3.q.out' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/tez/explainanalyze_4.q.out:354
Falling back to three-way merge...
Applied patch to
'ql/src/test/results/clientpositive/tez/explainanalyze_4.q.out' cleanly.
error: patch failed:
ql/src/test/results/clientpositive/tez/explainuser_3.q.out:792
Falling back to three-way merge...
Applied patch to 'ql/src/test/results/clientpositive/tez/explainuser_3.q.out'
cleanly.
error: patch failed:
service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java:82
Falling back to three-way merge...
Applied patch to
'service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java'
cleanly.
error: patch failed:
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java:690
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java'
cleanly.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:27: trailing whitespace.
# +-- X: p#1
/data/hiveptest/working/scratch/build.patch:418: trailing whitespace.
SELECT 'abc123' AS ORIG_VALUE, CAST('abc123' AS BIGINT) AS CAST_AS_BIGINT, CASE
WHEN CAST('abc123' AS BIGINT) IS NULL THEN 'YES' ELSE 'NO' END AS IS_IT_NULL;
/data/hiveptest/working/scratch/build.patch:1696: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:1706: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:1717: trailing whitespace.
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:1736
Falling back to three-way merge...
Applied patch to 'common/src/java/org/apache/hadoop/hive/conf/HiveConf.java'
cleanly.
error: patch failed: jdbc/pom.xml:213
Falling back to three-way merge...
Applied patch to 'jdbc/pom.xml' cleanly.
error: patch failed: metastore/pom.xml:111
Falling back to three-way merge...
Applied patch to 'metastore/pom.xml' cleanly.
error: patch failed: pom.xml:171
Falling back to three-way merge...
Applied patch to 'pom.xml' cleanly.
error: patch failed: ql/pom.xml:971
Falling back to three-way merge...
Applied patch to 'ql/pom.xml' cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java:1029
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java'
cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java:608
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionState.java' cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:244
Falling back to three-way merge...
Applied patch to 'ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java'
cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java:575
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java'
cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java:639
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java'
cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/FixedBucketPruningOptimizer.java:120
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/FixedBucketPruningOptimizer.java'
cleanly.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java:177
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkMapJoinProc.java'
cleanly.
error: patch
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923843#comment-16923843
]
Hive QA commented on HIVE-21508:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979556/HIVE-21508.branch-3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18464/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18464/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18464/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12979556/HIVE-21508.branch-3.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979556 - PreCommit-HIVE-Build
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923842#comment-16923842
]
Hive QA commented on HIVE-21508:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979556/HIVE-21508.branch-3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 135 failed/errored test(s), 14435 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=275)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCloseableThreadLocal - did not produce a TEST-*.xml file (likely timed out)
(batchId=334)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=275)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=231)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=239)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=230)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=238)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=233)
TestHiveMetaToolCommandLine - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHmsServerAuthorization - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogSparse - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestJSONMessageDeserializer - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestKuduCliDriver - did not
[jira] [Commented] (HIVE-22173) HiveServer2: Query with multiple lateral view hung forever during compile stage
[
https://issues.apache.org/jira/browse/HIVE-22173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923834#comment-16923834
]
Vineet Garg commented on HIVE-22173:
Is CBO off in this case?
> HiveServer2: Query with multiple lateral view hung forever during compile
> stage
> ---
>
> Key: HIVE-22173
> URL: https://issues.apache.org/jira/browse/HIVE-22173
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.1
> Environment: Hive-3.1.1, Java-8
>Reporter: Rajkumar Singh
>Priority: Critical
> Attachments: thread-progress.log
>
>
> Steps To Repro:
> {code:java}
> -- create table
> CREATE EXTERNAL TABLE `jsontable`(
> `json_string` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' ;
> -- Run explain of the query
> explain SELECT
> *
> FROM jsontable
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.city'), "\\[|\\]|\"", ""),',')) t1 as c1
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.country'), "\\[|\\]|\"", ""),',')) t2 as c2
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr'), "\\[|\\]|\"", ""),',')) t3 as c3
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.postalCode'), "\\[|\\]|\"", ""),',')) t4 as c4
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.state'), "\\[|\\]|\"", ""),',')) t5 as c5
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.streetAddressLine'), "\\[|\\]|\"", ""),',')) t6 as c6
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield'), "\\[|\\]|\"", ""),',')) t7 as c7
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield'), "\\[|\\]|\"", ""),',')) t8 as c8
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield.name.suffix'), "\\[|\\]|\"", ""),',')) t9 as c9
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.id.extension'), "\\[|\\]|\"", ""),',')) t10 as c10
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.id'), "\\[|\\]|\"", ""),',')) t11 as c11
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.id.root'), "\\[|\\]|\"", ""),',')) t12 as c12
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.telecom.'), "\\[|\\]|\"", ""),',')) t13 as c13
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield1.use'), "\\[|\\]|\"", ""),',')) t14 as c14
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield1.value'), "\\[|\\]|\"", ""),',')) t15 as c15
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield1.dummyfield1.code'), "\\[|\\]|\"", ""),',')) t16 as c16
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield1.dummyfield1.value'), "\\[|\\]|\"", ""),',')) t17 as c17
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.city'), "\\[|\\]|\"", ""),',')) t18 as c18
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.city'), "\\[|\\]|\"", ""),',')) t19 as c19
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.country'), "\\[|\\]|\"", ""),',')) t20 as c20
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.country'), "\\[|\\]|\"", ""),',')) t21 as c21
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield'), "\\[|\\]|\"", ""),',')) t22 as c22
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.postalCode'), "\\[|\\]|\"", ""),',')) t23 as c23
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.postalCode'), "\\[|\\]|\"", ""),',')) t24 as c24
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.state'), "\\[|\\]|\"", ""),',')) t25 as c25
>
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923807#comment-16923807
]
Hive QA commented on HIVE-21508:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 13s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-18463/patches/PreCommit-HIVE-Build-18463.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18463/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21987) Hive is unable to read Parquet int32 annotated with decimal
[
https://issues.apache.org/jira/browse/HIVE-21987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923802#comment-16923802
]
Hive QA commented on HIVE-21987:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979531/HIVE-21987.4.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16747 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=111)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18462/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18462/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18462/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979531 - PreCommit-HIVE-Build
> Hive is unable to read Parquet int32 annotated with decimal
> ---
>
> Key: HIVE-21987
> URL: https://issues.apache.org/jira/browse/HIVE-21987
> Project: Hive
> Issue Type: Improvement
>Reporter: Nandor Kollar
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-21987.1.patch, HIVE-21987.2.patch,
> HIVE-21987.3.patch, HIVE-21987.4.patch,
> part-0-e5287735-8dcf-4dda-9c6e-4d5c98dc15f2-c000.snappy.parquet
>
>
> When I tried to read a Parquet file from a Hive (with Tez execution engine)
> table with a small decimal column, I got the following exception:
> {code}
> Caused by: java.lang.UnsupportedOperationException:
> org.apache.hadoop.hive.ql.io.parquet.convert.ETypeConverter$8$1
> at
> org.apache.parquet.io.api.PrimitiveConverter.addInt(PrimitiveConverter.java:98)
> at
> org.apache.parquet.column.impl.ColumnReaderImpl$2$3.writeValue(ColumnReaderImpl.java:248)
> at
> org.apache.parquet.column.impl.ColumnReaderImpl.writeCurrentValueToConverter(ColumnReaderImpl.java:367)
> at
> org.apache.parquet.io.RecordReaderImplementation.read(RecordReaderImplementation.java:406)
> at
> org.apache.parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:226)
> ... 28 more
> {code}
> Steps to reproduce:
> - Create a Hive table with a single decimal(4, 2) column
> - Create a Parquet file with int32 column annotated with decimal(4, 2)
> logical type, put it into the previously created table location (or use the
> attached parquet file, in this case the column should be named as 'd', to
> match the Hive schema with the Parquet schema in the file)
> - Execute a {{select *}} on this table
> Also, I'm afraid that similar problems can happen with int64 decimals too.
> [Parquet specification |
> https://github.com/apache/parquet-format/blob/master/LogicalTypes.md] allows
> both of these cases.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?focusedWorklogId=307525=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307525 ] ASF GitHub Bot logged work on HIVE-22143: - Author: ASF GitHub Bot Created on: 05/Sep/19 23:08 Start Date: 05/Sep/19 23:08 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #757: HIVE-22143 Break up DDLSemanticAnalyzer - extract Database related analyzers URL: https://github.com/apache/hive/pull/757#discussion_r321520256 ## File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/LoadDatabase.java ## @@ -21,9 +21,9 @@ import org.apache.hadoop.hive.metastore.api.Database; import org.apache.hadoop.hive.metastore.api.InvalidOperationException; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.database.AlterDatabaseSetOwnerDesc; -import org.apache.hadoop.hive.ql.ddl.database.AlterDatabaseSetPropertiesDesc; -import org.apache.hadoop.hive.ql.ddl.database.CreateDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.create.CreateDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.setowner.AlterDatabaseSetOwnerDesc; +import org.apache.hadoop.hive.ql.ddl.database.setproperties.AlterDatabaseSetPropertiesDesc; Review comment: Same as above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 307525) Time Spent: 0.5h (was: 20m) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?focusedWorklogId=307524=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307524 ] ASF GitHub Bot logged work on HIVE-22143: - Author: ASF GitHub Bot Created on: 05/Sep/19 23:08 Start Date: 05/Sep/19 23:08 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #757: HIVE-22143 Break up DDLSemanticAnalyzer - extract Database related analyzers URL: https://github.com/apache/hive/pull/757#discussion_r321518498 ## File path: hcatalog/core/src/main/java/org/apache/hive/hcatalog/cli/SemanticAnalysis/HCatSemanticAnalyzer.java ## @@ -21,10 +21,10 @@ import org.apache.hadoop.hive.metastore.api.Database; import org.apache.hadoop.hive.ql.ddl.DDLDesc; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.database.DescDatabaseDesc; -import org.apache.hadoop.hive.ql.ddl.database.DropDatabaseDesc; -import org.apache.hadoop.hive.ql.ddl.database.ShowDatabasesDesc; -import org.apache.hadoop.hive.ql.ddl.database.SwitchDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.desc.DescDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.drop.DropDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.show.ShowDatabasesDesc; +import org.apache.hadoop.hive.ql.ddl.database.switchdb.SwitchDatabaseDesc; Review comment: minor: switchdb -> switch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 307524) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch > > Time Spent: 20m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22143) Break up DDLSemanticAnalyzer - extract Database related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22143?focusedWorklogId=307523=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307523 ] ASF GitHub Bot logged work on HIVE-22143: - Author: ASF GitHub Bot Created on: 05/Sep/19 23:08 Start Date: 05/Sep/19 23:08 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #757: HIVE-22143 Break up DDLSemanticAnalyzer - extract Database related analyzers URL: https://github.com/apache/hive/pull/757#discussion_r321519922 ## File path: ql/src/java/org/apache/hadoop/hive/ql/exec/repl/bootstrap/load/LoadDatabase.java ## @@ -21,9 +21,9 @@ import org.apache.hadoop.hive.metastore.api.Database; import org.apache.hadoop.hive.metastore.api.InvalidOperationException; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.database.AlterDatabaseSetOwnerDesc; -import org.apache.hadoop.hive.ql.ddl.database.AlterDatabaseSetPropertiesDesc; -import org.apache.hadoop.hive.ql.ddl.database.CreateDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.create.CreateDatabaseDesc; +import org.apache.hadoop.hive.ql.ddl.database.setowner.AlterDatabaseSetOwnerDesc; Review comment: setowner -> owner? or should we combine all set operations into a single one? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 307523) Time Spent: 20m (was: 10m) > Break up DDLSemanticAnalyzer - extract Database related analyzers > - > > Key: HIVE-22143 > URL: https://issues.apache.org/jira/browse/HIVE-22143 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22143.01.patch, HIVE-22143.02.patch, > HIVE-22143.04.patch > > Time Spent: 20m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #1: extract all the database related analyzers from DDLSemanticAnalyzer, > and move them under the new package. Also create the new internal framework. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21987) Hive is unable to read Parquet int32 annotated with decimal
[
https://issues.apache.org/jira/browse/HIVE-21987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923796#comment-16923796
]
Hive QA commented on HIVE-21987:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
43s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
27s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
44s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 6m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 6m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
34s{color} | {color:red} ql: The patch generated 1 new + 15 unchanged - 0 fixed
= 16 total (was 15) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
48s{color} | {color:red} root: The patch generated 1 new + 15 unchanged - 0
fixed = 16 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 6m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 57m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18462/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18462/yetus/diff-checkstyle-ql.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18462/yetus/diff-checkstyle-root.txt
|
| modules | C: ql . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18462/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Hive is unable to read Parquet int32 annotated with decimal
> ---
>
> Key: HIVE-21987
> URL: https://issues.apache.org/jira/browse/HIVE-21987
> Project: Hive
> Issue Type: Improvement
>Reporter: Nandor Kollar
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-21987.1.patch, HIVE-21987.2.patch,
> HIVE-21987.3.patch, HIVE-21987.4.patch,
> part-0-e5287735-8dcf-4dda-9c6e-4d5c98dc15f2-c000.snappy.parquet
>
>
> When I tried to read a Parquet file from a Hive (with Tez execution engine)
> table with a small decimal column, I got the following exception:
> {code}
> Caused by: java.lang.UnsupportedOperationException:
> org.apache.hadoop.hive.ql.io.parquet.convert.ETypeConverter$8$1
> at
> org.apache.parquet.io.api.PrimitiveConverter.addInt(PrimitiveConverter.java:98)
> at
>
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Status: Open (was: Patch Available) > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, HIVE-20113.5.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Status: Patch Available (was: Open) > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, HIVE-20113.5.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-20113: --- Attachment: HIVE-20113.5.patch > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, HIVE-20113.5.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22115) Prevent the creation of query-router logger in HS2 as per property
[
https://issues.apache.org/jira/browse/HIVE-22115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22115:
---
Fix Version/s: 4.0.0
> Prevent the creation of query-router logger in HS2 as per property
> --
>
> Key: HIVE-22115
> URL: https://issues.apache.org/jira/browse/HIVE-22115
> Project: Hive
> Issue Type: Improvement
> Components: Logging
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22115.patch, HIVE-22115.patch, HIVE-22115.patch
>
>
> Avoid the creation and registration of query-router logger if the Hive server
> Property is set to false by the user
> {code}
> HiveConf.ConfVars.HIVE_SERVER2_LOGGING_OPERATION_ENABLED
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923761#comment-16923761
]
Hive QA commented on HIVE-22099:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979530/HIVE-22099.7.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18461/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18461/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18461/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12979530/HIVE-22099.7.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979530 - PreCommit-HIVE-Build
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923759#comment-16923759
]
Hive QA commented on HIVE-22099:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979530/HIVE-22099.7.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16747 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18460/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18460/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18460/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979530 - PreCommit-HIVE-Build
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-22169:
--
Assignee: Gopal V (was: Gopal V)
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923727#comment-16923727
]
Hive QA commented on HIVE-22099:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} ql: The patch generated 0 new + 206 unchanged - 3
fixed = 206 total (was 209) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18460/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18460/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22173) HiveServer2: Query with multiple lateral view hung forever during compile stage
[
https://issues.apache.org/jira/browse/HIVE-22173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-22173:
--
Attachment: thread-progress.log
> HiveServer2: Query with multiple lateral view hung forever during compile
> stage
> ---
>
> Key: HIVE-22173
> URL: https://issues.apache.org/jira/browse/HIVE-22173
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 3.1.1
> Environment: Hive-3.1.1, Java-8
>Reporter: Rajkumar Singh
>Priority: Critical
> Attachments: thread-progress.log
>
>
> Steps To Repro:
> {code:java}
> -- create table
> CREATE EXTERNAL TABLE `jsontable`(
> `json_string` string)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.mapred.TextInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' ;
> -- Run explain of the query
> explain SELECT
> *
> FROM jsontable
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.city'), "\\[|\\]|\"", ""),',')) t1 as c1
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.country'), "\\[|\\]|\"", ""),',')) t2 as c2
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr'), "\\[|\\]|\"", ""),',')) t3 as c3
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.postalCode'), "\\[|\\]|\"", ""),',')) t4 as c4
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.state'), "\\[|\\]|\"", ""),',')) t5 as c5
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.addr.streetAddressLine'), "\\[|\\]|\"", ""),',')) t6 as c6
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield'), "\\[|\\]|\"", ""),',')) t7 as c7
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield'), "\\[|\\]|\"", ""),',')) t8 as c8
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield.name.suffix'), "\\[|\\]|\"", ""),',')) t9 as c9
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.id.extension'), "\\[|\\]|\"", ""),',')) t10 as c10
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.id'), "\\[|\\]|\"", ""),',')) t11 as c11
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.id.root'), "\\[|\\]|\"", ""),',')) t12 as c12
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.telecom.'), "\\[|\\]|\"", ""),',')) t13 as c13
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield1.use'), "\\[|\\]|\"", ""),',')) t14 as c14
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield.dummyfield1.value'), "\\[|\\]|\"", ""),',')) t15 as c15
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield1.dummyfield1.code'), "\\[|\\]|\"", ""),',')) t16 as c16
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield1.dummyfield1.value'), "\\[|\\]|\"", ""),',')) t17 as c17
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.city'), "\\[|\\]|\"", ""),',')) t18 as c18
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.city'), "\\[|\\]|\"", ""),',')) t19 as c19
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.country'), "\\[|\\]|\"", ""),',')) t20 as c20
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.country'), "\\[|\\]|\"", ""),',')) t21 as c21
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield'), "\\[|\\]|\"", ""),',')) t22 as c22
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.postalCode'), "\\[|\\]|\"", ""),',')) t23 as c23
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.postalCode'), "\\[|\\]|\"", ""),',')) t24 as c24
> lateral view
> explode(split(regexp_replace(get_json_object(jsontable.json_string,
> '$.jsonfield2.state'), "\\[|\\]|\"", ""),',')) t25 as c25
> lateral view
>
[jira] [Updated] (HIVE-22173) HiveServer2: Query with multiple lateral view hung forever during compile stage
[
https://issues.apache.org/jira/browse/HIVE-22173?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajkumar Singh updated HIVE-22173:
--
Description:
Steps To Repro:
{code:java}
-- create table
CREATE EXTERNAL TABLE `jsontable`(
`json_string` string)
ROW FORMAT SERDE
'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'
STORED AS INPUTFORMAT
'org.apache.hadoop.mapred.TextInputFormat'
OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' ;
-- Run explain of the query
explain SELECT
*
FROM jsontable
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.addr.city'), "\\[|\\]|\"", ""),',')) t1 as c1
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.addr.country'), "\\[|\\]|\"", ""),',')) t2 as c2
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.addr'), "\\[|\\]|\"", ""),',')) t3 as c3
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.addr.postalCode'), "\\[|\\]|\"", ""),',')) t4 as c4
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.addr.state'), "\\[|\\]|\"", ""),',')) t5 as c5
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.addr.streetAddressLine'), "\\[|\\]|\"", ""),',')) t6 as c6
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.dummyfield'), "\\[|\\]|\"", ""),',')) t7 as c7
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.dummyfield'), "\\[|\\]|\"", ""),',')) t8 as c8
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.dummyfield.name.suffix'), "\\[|\\]|\"", ""),',')) t9 as c9
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.id.extension'), "\\[|\\]|\"", ""),',')) t10 as c10
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.id'), "\\[|\\]|\"", ""),',')) t11 as c11
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.id.root'), "\\[|\\]|\"", ""),',')) t12 as c12
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.telecom.'), "\\[|\\]|\"", ""),',')) t13 as c13
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.dummyfield1.use'), "\\[|\\]|\"", ""),',')) t14 as c14
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield.dummyfield1.value'), "\\[|\\]|\"", ""),',')) t15 as c15
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield1.dummyfield1.code'), "\\[|\\]|\"", ""),',')) t16 as c16
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield1.dummyfield1.value'), "\\[|\\]|\"", ""),',')) t17 as c17
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.city'), "\\[|\\]|\"", ""),',')) t18 as c18
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.city'), "\\[|\\]|\"", ""),',')) t19 as c19
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.country'), "\\[|\\]|\"", ""),',')) t20 as c20
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.country'), "\\[|\\]|\"", ""),',')) t21 as c21
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield'), "\\[|\\]|\"", ""),',')) t22 as c22
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.postalCode'), "\\[|\\]|\"", ""),',')) t23 as c23
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.postalCode'), "\\[|\\]|\"", ""),',')) t24 as c24
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.state'), "\\[|\\]|\"", ""),',')) t25 as c25
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.state'), "\\[|\\]|\"", ""),',')) t26 as c26
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2'), "\\[|\\]|\"", ""),',')) t27 as c27
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.streetAddressLine'), "\\[|\\]|\"", ""),',')) t28 as c28
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield2.use'), "\\[|\\]|\"", ""),',')) t29 as c29
lateral view
explode(split(regexp_replace(get_json_object(jsontable.json_string,
'$.jsonfield3'), "\\[|\\]|\"", ""),',')) t30 as c30
lateral view
[jira] [Updated] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[ https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-22149: Attachment: HIVE-22149.01.patch > Metastore: Unify codahale metrics.log json structure between hiveserver2 and > metastore services > --- > > Key: HIVE-22149 > URL: https://issues.apache.org/jira/browse/HIVE-22149 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch, > HIVE-22149.01.patch, metrics_hiveserver2.log, metrics_metastore.log > > > While fixing HIVE-22140 I found some really annoying differences between the > codahale metric file structures between hiveserver2 and metastore, e.g. > open_connections: can be found in "counters" for hs2, but in "gauges" for ms > threads count: it's a proper "threads.count" for hs2, but a really ambiguous > "count" for ms > so I realized that "memory." and "threads." prefix is completely absent in ms > metrics file, which is misleading -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[ https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923722#comment-16923722 ] Laszlo Bodor commented on HIVE-22149: - tests seem to pass locally, reuploading 01.patch > Metastore: Unify codahale metrics.log json structure between hiveserver2 and > metastore services > --- > > Key: HIVE-22149 > URL: https://issues.apache.org/jira/browse/HIVE-22149 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch, > HIVE-22149.01.patch, metrics_hiveserver2.log, metrics_metastore.log > > > While fixing HIVE-22140 I found some really annoying differences between the > codahale metric file structures between hiveserver2 and metastore, e.g. > open_connections: can be found in "counters" for hs2, but in "gauges" for ms > threads count: it's a proper "threads.count" for hs2, but a really ambiguous > "count" for ms > so I realized that "memory." and "threads." prefix is completely absent in ms > metrics file, which is misleading -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-22169:
--
Assignee: Jesus Camacho Rodriguez (was: Gopal V)
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-22169:
--
Assignee: Gopal V (was: Jesus Camacho Rodriguez)
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-22169:
---
Attachment: HIVE-22169.1.patch
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923715#comment-16923715
]
Hive QA commented on HIVE-22055:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979524/HIVE-22055.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16746 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18459/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18459/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18459/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979524 - PreCommit-HIVE-Build
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.1.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923700#comment-16923700
]
Hive QA commented on HIVE-22055:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
42s{color} | {color:blue} llap-server in master has 83 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
12s{color} | {color:red} llap-server: The patch generated 1 new + 34 unchanged
- 0 fixed = 35 total (was 34) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18459/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18459/yetus/diff-checkstyle-llap-server.txt
|
| modules | C: llap-server U: llap-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18459/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.1.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923698#comment-16923698
]
Hive QA commented on HIVE-22149:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979523/HIVE-22149.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 192 failed/errored test(s), 15052 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestAuthorizationPreEventListener - did not produce a TEST-*.xml file (likely
timed out) (batchId=276)
TestAuthzApiEmbedAuthorizerInRemote - did not produce a TEST-*.xml file (likely
timed out) (batchId=246)
TestBeelineConnectionUsingHiveSite - did not produce a TEST-*.xml file (likely
timed out) (batchId=277)
TestBeelineWithUserHs2ConnectionFile - did not produce a TEST-*.xml file
(likely timed out) (batchId=277)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=223)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestClientSideAuthorizationProvider - did not produce a TEST-*.xml file (likely
timed out) (batchId=277)
TestCommands - did not produce a TEST-*.xml file (likely timed out)
(batchId=207)
TestCopyUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestCreateUdfEntities - did not produce a TEST-*.xml file (likely timed out)
(batchId=246)
TestDDLWithRemoteMetastoreSecondNamenode - did not produce a TEST-*.xml file
(likely timed out) (batchId=246)
TestDFSErrorHandling - did not produce a TEST-*.xml file (likely timed out)
(batchId=281)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestEximReplicationTasks - did not produce a TEST-*.xml file (likely timed out)
(batchId=207)
TestExportImport - did not produce a TEST-*.xml file (likely timed out)
(batchId=274)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=225)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestHCatClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=207)
TestHDFSPermissionPolicyProvider - did not produce a TEST-*.xml file (likely
timed out) (batchId=277)
TestHS2ImpersonationWithRemoteMS - did not produce a TEST-*.xml file (likely
timed out) (batchId=281)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestHiveAuthFactory - did not produce a TEST-*.xml file (likely timed out)
(batchId=298)
TestHiveAuthorizerCheckInvocation - did not produce a TEST-*.xml file (likely
timed out) (batchId=277)
TestHiveAuthorizerShowFilters - did not produce a TEST-*.xml file (likely timed
out) (batchId=277)
TestHiveHistory - did not produce a TEST-*.xml file (likely timed out)
(batchId=277)
TestHiveMetaStoreAlterColumnPar - did not produce a TEST-*.xml file (likely
timed out) (batchId=246)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=225)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=233)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=223)
TestHiveProtoEventsCleanerTask - did not produce a TEST-*.xml file (likely
timed out) (batchId=246)
TestHiveRemote - did not produce a TEST-*.xml file (likely timed out)
(batchId=331)
TestHs2Hooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=246)
TestHs2HooksWithMiniKdc - did not produce a TEST-*.xml file (likely timed out)
(batchId=298)
TestJdbcGenericUDTFGetSplits - did not produce a TEST-*.xml file (likely timed
out) (batchId=281)
TestJdbcNonKrbSASLWithMiniKdc - did not produce a TEST-*.xml file (likely timed
out)
[jira] [Updated] (HIVE-22164) Vectorized Limit operator returns wrong number of results with offset
[ https://issues.apache.org/jira/browse/HIVE-22164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan updated HIVE-22164: Attachment: HIVE-22164.1.patch Status: Patch Available (was: Open) > Vectorized Limit operator returns wrong number of results with offset > - > > Key: HIVE-22164 > URL: https://issues.apache.org/jira/browse/HIVE-22164 > Project: Hive > Issue Type: Bug > Components: Hive, llap, Vectorization >Affects Versions: 4.0.0 >Reporter: Ramesh Kumar Thangarajan >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-22164.1.patch > > > Vectorized Limit operator returns wrong number of results with offset -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22149) Metastore: Unify codahale metrics.log json structure between hiveserver2 and metastore services
[
https://issues.apache.org/jira/browse/HIVE-22149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923620#comment-16923620
]
Hive QA commented on HIVE-22149:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
13s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 3s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18458/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18458/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Metastore: Unify codahale metrics.log json structure between hiveserver2 and
> metastore services
> ---
>
> Key: HIVE-22149
> URL: https://issues.apache.org/jira/browse/HIVE-22149
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-22149.01.patch, HIVE-22149.01.patch,
> metrics_hiveserver2.log, metrics_metastore.log
>
>
> While fixing HIVE-22140 I found some really annoying differences between the
> codahale metric file structures between hiveserver2 and metastore, e.g.
> open_connections: can be found in "counters" for hs2, but in "gauges" for ms
> threads count: it's a proper "threads.count" for hs2, but a really ambiguous
> "count" for ms
> so I realized that "memory." and "threads." prefix is completely absent in ms
> metrics file, which is misleading
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923601#comment-16923601
]
Hive QA commented on HIVE-22163:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979516/HIVE-22163.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18457/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18457/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18457/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-05 16:41:48.913
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18457/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-05 16:41:48.916
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
62676b0..0213afb master -> origin/master
+ git reset --hard HEAD
HEAD is now at 62676b0 HIVE-21996 Remove unused code from Driver (Miklos
Gergely reviewd by Jesus Camacho Rodriguez)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 0213afb HIVE-22028 Clean up Add Partition (Miklos Gergely
reviewd by Jesus Camacho Rodriguez)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-05 16:41:50.395
+ rm -rf ../yetus_PreCommit-HIVE-Build-18457
+ mkdir ../yetus_PreCommit-HIVE-Build-18457
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18457
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18457/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:100: trailing whitespace.
null sort order:
/data/hiveptest/working/scratch/build.patch:101: trailing whitespace.
sort order:
/data/hiveptest/working/scratch/build.patch:120: trailing whitespace.
columns.comments
/data/hiveptest/working/scratch/build.patch:133: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:141: trailing whitespace.
columns.comments
warning: squelched 6 whitespace errors
warning: 11 lines add whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc6436894967047510409.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc6436894967047510409.exe,
-I/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-common/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator Version 3.5.2
protoc-jar: executing: [/tmp/protoc698709551353323597.exe, --version]
libprotoc 2.5.0
ANTLR Parser Generator Version 3.5.2
Output file
/data/hiveptest/working/apache-github-source-source/standalone-metastore/metastore-server/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923599#comment-16923599
]
Hive QA commented on HIVE-22170:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979501/HIVE-22170.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18456/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18456/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18456/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12979501/HIVE-22170.01.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979501 - PreCommit-HIVE-Build
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22170.01.patch
>
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923597#comment-16923597
]
Hive QA commented on HIVE-22170:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979501/HIVE-22170.01.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 16742 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=254)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[udf_unix_timestamp]
(batchId=297)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[date_udf] (batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp_udf]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_folder_constants]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_to_unix_timestamp]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_unix_timestamp]
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_date_funcs]
(batchId=84)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[current_date_timestamp]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_3]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_on_aggcol]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[groupby_groupingset_bug]
(batchId=185)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[reopt_semijoin]
(batchId=186)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_date_funcs]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction2]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction]
(batchId=162)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[date_udf]
(batchId=127)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[timestamp_udf]
(batchId=149)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_timestamp_funcs]
(batchId=127)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorDateExpressions.testVectorUDFUnixTimeStamp
(batchId=336)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorTimestampExpressions.testVectorUDFUnixTimeStampString
(batchId=337)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorTimestampExpressions.testVectorUDFUnixTimeStampTimestamp
(batchId=337)
org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFToUnixTimestamp.testDate
(batchId=312)
org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFToUnixTimestamp.testString
(batchId=312)
org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFToUnixTimestamp.testTimestamp
(batchId=312)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18455/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18455/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18455/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 27 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979501 - PreCommit-HIVE-Build
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22170.01.patch
>
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22106) PCR: Remove cross-query synchronization for the partition-eval
[
https://issues.apache.org/jira/browse/HIVE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923575#comment-16923575
]
Gopal V commented on HIVE-22106:
LGTM - +1
> PCR: Remove cross-query synchronization for the partition-eval
> ---
>
> Key: HIVE-22106
> URL: https://issues.apache.org/jira/browse/HIVE-22106
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-22106.patch
>
>
> {code}
> HiveServer2-Handler-Pool: Thread-492 Blocked CPU usage on sample: 0ms
>
> org.apache.hadoop.hive.ql.optimizer.ppr.PartExprEvalUtils.evalExprWithPart(ExprNodeDesc,
> Partition, List, StructObjectInspector) PartExprEvalUtils.java:58
>
> org.apache.hadoop.hive.ql.optimizer.pcr.PcrExprProcFactory.evalExprWithPart(ExprNodeDesc,
> Partition, List) PcrExprProcFactory.java:83
>
> org.apache.hadoop.hive.ql.optimizer.pcr.PcrExprProcFactory$GenericFuncExprProcessor.handleDeterministicUdf(PcrExprProcCtx,
> ExprNodeGenericFuncDesc, Object[]) PcrExprProcFactory.java:317
>
> org.apache.hadoop.hive.ql.optimizer.pcr.PcrExprProcFactory$GenericFuncExprProcessor.process(Node,
> Stack, NodeProcessorCtx, Object[]) PcrExprProcFactory.java:298
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(Node, Stack,
> Object[]) DefaultRuleDispatcher.java:90
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(Node,
> Stack) DefaultGraphWalker.java:105
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(Node, Stack)
> DefaultGraphWalker.java:89
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(Node)
> DefaultGraphWalker.java:158
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(Collection,
> HashMap) DefaultGraphWalker.java:120
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22028) Clean up Add Partition
[ https://issues.apache.org/jira/browse/HIVE-22028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22028: --- Fix Version/s: 4.0.0 > Clean up Add Partition > -- > > Key: HIVE-22028 > URL: https://issues.apache.org/jira/browse/HIVE-22028 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22028.01.patch, HIVE-22028.02.patch, > HIVE-22028.03.patch, HIVE-22028.04.patch > > Time Spent: 20m > Remaining Estimate: 0h > > AlterTableAddPartitionDesc should be immutable, like the rest of the desc > classes. This can not be done 100% right now, as it requires the refactoring > of the ImportSemanticAnalyzer, so the task will be finished only then. > Add Partition logic should be moved from Hive.java to > AlterTableAddPartitionOperation.java, only the metastore calls should remain > in Hive.java. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923556#comment-16923556
]
Hive QA commented on HIVE-22170:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 17 new + 654 unchanged - 3
fixed = 671 total (was 657) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
9s{color} | {color:red} ql generated 2 new + 2243 unchanged - 3 fixed = 2245
total (was 2246) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 39s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Class
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFUnixTimeStampDate
defines non-transient non-serializable instance field date In
VectorUDFUnixTimeStampDate.java:instance field date In
VectorUDFUnixTimeStampDate.java |
| | Class
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFUnixTimeStampTimestamp
defines non-transient non-serializable instance field timestamp In
VectorUDFUnixTimeStampTimestamp.java:instance field timestamp In
VectorUDFUnixTimeStampTimestamp.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18455/dev-support/hive-personality.sh
|
| git revision | master / 0213afb |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18455/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18455/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18455/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18455/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-22170.01.patch
>
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously).
[jira] [Updated] (HIVE-21996) Remove unused code from Driver
[ https://issues.apache.org/jira/browse/HIVE-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-21996: --- Fix Version/s: 4.0.0 > Remove unused code from Driver > -- > > Key: HIVE-21996 > URL: https://issues.apache.org/jira/browse/HIVE-21996 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-driver > Fix For: 4.0.0 > > Attachments: HIVE-21996.01.patch, HIVE-21996.02.patch > > Time Spent: 1h > Remaining Estimate: 0h > > In the IDriver interface there is an "int compile(String command)' function > which is only used by tests. As this is production code, it should be removed > from the interface. Also there are several other unused codes and > functions/fields with visibility that should be reduced. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22028) Clean up Add Partition
[ https://issues.apache.org/jira/browse/HIVE-22028?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22028: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Clean up Add Partition > -- > > Key: HIVE-22028 > URL: https://issues.apache.org/jira/browse/HIVE-22028 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22028.01.patch, HIVE-22028.02.patch, > HIVE-22028.03.patch, HIVE-22028.04.patch > > Time Spent: 20m > Remaining Estimate: 0h > > AlterTableAddPartitionDesc should be immutable, like the rest of the desc > classes. This can not be done 100% right now, as it requires the refactoring > of the ImportSemanticAnalyzer, so the task will be finished only then. > Add Partition logic should be moved from Hive.java to > AlterTableAddPartitionOperation.java, only the metastore calls should remain > in Hive.java. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22028) Clean up Add Partition
[
https://issues.apache.org/jira/browse/HIVE-22028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923527#comment-16923527
]
Hive QA commented on HIVE-22028:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979482/HIVE-22028.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16746 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18454/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18454/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18454/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979482 - PreCommit-HIVE-Build
> Clean up Add Partition
> --
>
> Key: HIVE-22028
> URL: https://issues.apache.org/jira/browse/HIVE-22028
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22028.01.patch, HIVE-22028.02.patch,
> HIVE-22028.03.patch, HIVE-22028.04.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> AlterTableAddPartitionDesc should be immutable, like the rest of the desc
> classes. This can not be done 100% right now, as it requires the refactoring
> of the ImportSemanticAnalyzer, so the task will be finished only then.
> Add Partition logic should be moved from Hive.java to
> AlterTableAddPartitionOperation.java, only the metastore calls should remain
> in Hive.java.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-21996) Remove unused code from Driver
[ https://issues.apache.org/jira/browse/HIVE-21996?focusedWorklogId=307234=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-307234 ] ASF GitHub Bot logged work on HIVE-21996: - Author: ASF GitHub Bot Created on: 05/Sep/19 15:15 Start Date: 05/Sep/19 15:15 Worklog Time Spent: 10m Work Description: miklosgergely commented on pull request #739: HIVE-21996 Remove unused code from Driver URL: https://github.com/apache/hive/pull/739 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 307234) Time Spent: 1h (was: 50m) > Remove unused code from Driver > -- > > Key: HIVE-21996 > URL: https://issues.apache.org/jira/browse/HIVE-21996 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-driver > Attachments: HIVE-21996.01.patch, HIVE-21996.02.patch > > Time Spent: 1h > Remaining Estimate: 0h > > In the IDriver interface there is an "int compile(String command)' function > which is only used by tests. As this is production code, it should be removed > from the interface. Also there are several other unused codes and > functions/fields with visibility that should be reduced. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923520#comment-16923520
]
Jesus Camacho Rodriguez commented on HIVE-22099:
+1 (pending tests)
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22028) Clean up Add Partition
[
https://issues.apache.org/jira/browse/HIVE-22028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923485#comment-16923485
]
Hive QA commented on HIVE-22028:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
53s{color} | {color:blue} ql in master has 2248 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
26s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} streaming in master has 2 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 4 new + 480 unchanged - 17
fixed = 484 total (was 497) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} hcatalog/streaming: The patch generated 0 new + 95
unchanged - 6 fixed = 95 total (was 101) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} The patch streaming passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
11s{color} | {color:green} ql generated 0 new + 2246 unchanged - 2 fixed = 2246
total (was 2248) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} streaming in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} streaming in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 2s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18454/dev-support/hive-personality.sh
|
| git revision | master / 62676b0 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18454/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql hcatalog/streaming streaming U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18454/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Clean up Add Partition
> --
>
> Key: HIVE-22028
> URL: https://issues.apache.org/jira/browse/HIVE-22028
> Project: Hive
>
[jira] [Commented] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923459#comment-16923459
]
Hive QA commented on HIVE-22169:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979484/HIVE-22169.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16742 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18453/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18453/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18453/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979484 - PreCommit-HIVE-Build
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21378) Support TeraData in JDBC StorageHandler
[ https://issues.apache.org/jira/browse/HIVE-21378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923457#comment-16923457 ] Justin Leet commented on HIVE-21378: I believe this patch is ready for review. The attached patch adds some capability for using TD with the JdbcStorageHandler. There's a couple notes about the patch for reviewers: * To the best of my knowledge, TD doesn't support the `LIMIT ... OFFSET` style query. This throws an exception in the patch. This means that you either must a) provide a split column or b) have a single partition. * A config is added for the query band in TD. This is handled as a connection level setup query handled up front. If there's a cleaner or different way we'd like to handle the coding/configuration of that, let me know, and I'm happy to change it up as needed. * I've run some queries using this and it seems fine, but any testing suggestions/improvements are definitely welcome. For unit testing, I'm not sure how much can be done without TD, since the limitations and queries tend to be based on TD idioms / quirks. > Support TeraData in JDBC StorageHandler > --- > > Key: HIVE-21378 > URL: https://issues.apache.org/jira/browse/HIVE-21378 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0, 4.0.0 >Reporter: Donald FOSSOUO >Assignee: Justin Leet >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21378.1.patch > > > Make TeraData a first class member of JdbcStorageHandler. It doesn't work > even using POSTGRES, MSSQL or any existing available storage handler. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21996) Remove unused code from Driver
[ https://issues.apache.org/jira/browse/HIVE-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-21996: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Remove unused code from Driver > -- > > Key: HIVE-21996 > URL: https://issues.apache.org/jira/browse/HIVE-21996 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-driver > Attachments: HIVE-21996.01.patch, HIVE-21996.02.patch > > Time Spent: 50m > Remaining Estimate: 0h > > In the IDriver interface there is an "int compile(String command)' function > which is only used by tests. As this is production code, it should be removed > from the interface. Also there are several other unused codes and > functions/fields with visibility that should be reduced. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21378) Support TeraData in JDBC StorageHandler
[ https://issues.apache.org/jira/browse/HIVE-21378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Leet updated HIVE-21378: --- Attachment: HIVE-21378.1.patch > Support TeraData in JDBC StorageHandler > --- > > Key: HIVE-21378 > URL: https://issues.apache.org/jira/browse/HIVE-21378 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0, 4.0.0 >Reporter: Donald FOSSOUO >Assignee: Justin Leet >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21378.1.patch > > > Make TeraData a first class member of JdbcStorageHandler. It doesn't work > even using POSTGRES, MSSQL or any existing available storage handler. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-21378) Support TeraData in JDBC StorageHandler
[ https://issues.apache.org/jira/browse/HIVE-21378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Justin Leet reassigned HIVE-21378: -- Assignee: Justin Leet > Support TeraData in JDBC StorageHandler > --- > > Key: HIVE-21378 > URL: https://issues.apache.org/jira/browse/HIVE-21378 > Project: Hive > Issue Type: Improvement > Components: Hive >Affects Versions: 3.1.0, 4.0.0 >Reporter: Donald FOSSOUO >Assignee: Justin Leet >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21378.1.patch > > > Make TeraData a first class member of JdbcStorageHandler. It doesn't work > even using POSTGRES, MSSQL or any existing available storage handler. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923415#comment-16923415
]
Hive QA commented on HIVE-22169:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
49s{color} | {color:blue} ql in master has 2248 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
6s{color} | {color:green} ql generated 0 new + 2247 unchanged - 1 fixed = 2247
total (was 2248) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18453/dev-support/hive-personality.sh
|
| git revision | master / ebcc9bc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18453/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923402#comment-16923402
]
Ana Jalba commented on HIVE-21508:
--
[~kgyrtkirk] Please can you confirm if those are normal test results?
I also didn't add any test for this change because this is mostly a syntax
change.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22168) remove excessive logging by llap cache.
[ https://issues.apache.org/jira/browse/HIVE-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] slim bouguerra updated HIVE-22168: -- Attachment: HIVE-22168.patch > remove excessive logging by llap cache. > --- > > Key: HIVE-22168 > URL: https://issues.apache.org/jira/browse/HIVE-22168 > Project: Hive > Issue Type: Improvement > Components: llap, Logging >Reporter: slim bouguerra >Assignee: slim bouguerra >Priority: Major > Attachments: HIVE-22168.patch, HIVE-22168.patch > > > Llap cache logging is very expensive when it comes to log every request > buffers range. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21996) Remove unused code from Driver
[
https://issues.apache.org/jira/browse/HIVE-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923373#comment-16923373
]
Hive QA commented on HIVE-21996:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979483/HIVE-21996.02.patch
{color:green}SUCCESS:{color} +1 due to 12 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16714 tests
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18452/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18452/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18452/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979483 - PreCommit-HIVE-Build
> Remove unused code from Driver
> --
>
> Key: HIVE-21996
> URL: https://issues.apache.org/jira/browse/HIVE-21996
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-driver
> Attachments: HIVE-21996.01.patch, HIVE-21996.02.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In the IDriver interface there is an "int compile(String command)' function
> which is only used by tests. As this is production code, it should be removed
> from the interface. Also there are several other unused codes and
> functions/fields with visibility that should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21942) Remove useless MetastoreType enum from AbstractCliConfig
[ https://issues.apache.org/jira/browse/HIVE-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imre Molnar updated HIVE-21942: --- Status: Patch Available (was: Open) > Remove useless MetastoreType enum from AbstractCliConfig > > > Key: HIVE-21942 > URL: https://issues.apache.org/jira/browse/HIVE-21942 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Laszlo Bodor >Assignee: Imre Molnar >Priority: Major > Labels: newbie > Fix For: 4.0.0 > > Attachments: HIVE-21942.1.patch > > > I'm not 100% aware of historical reasons, but this single-value enum seems > totally useless at the moment. > https://github.com/apache/hive/blob/86a15600ae610e1c6017019883ef9344585572ef/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/AbstractCliConfig.java#L50-L52 > According to HIVE-14854, it may have made sense when there was an initiative > of hbase metastore, but it's not in the scope anymore AFAIK. > https://github.com/apache/hive/commit/474425aa62e3f25b119419439373aa684c6c2121 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21942) Remove useless MetastoreType enum from AbstractCliConfig
[ https://issues.apache.org/jira/browse/HIVE-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imre Molnar updated HIVE-21942: --- Attachment: HIVE-21942.1.patch > Remove useless MetastoreType enum from AbstractCliConfig > > > Key: HIVE-21942 > URL: https://issues.apache.org/jira/browse/HIVE-21942 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Laszlo Bodor >Assignee: Imre Molnar >Priority: Major > Labels: newbie > Fix For: 4.0.0 > > Attachments: HIVE-21942.1.patch > > > I'm not 100% aware of historical reasons, but this single-value enum seems > totally useless at the moment. > https://github.com/apache/hive/blob/86a15600ae610e1c6017019883ef9344585572ef/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/AbstractCliConfig.java#L50-L52 > According to HIVE-14854, it may have made sense when there was an initiative > of hbase metastore, but it's not in the scope anymore AFAIK. > https://github.com/apache/hive/commit/474425aa62e3f25b119419439373aa684c6c2121 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-21942) Remove useless MetastoreType enum from AbstractCliConfig
[ https://issues.apache.org/jira/browse/HIVE-21942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imre Molnar reassigned HIVE-21942: -- Assignee: Imre Molnar > Remove useless MetastoreType enum from AbstractCliConfig > > > Key: HIVE-21942 > URL: https://issues.apache.org/jira/browse/HIVE-21942 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Laszlo Bodor >Assignee: Imre Molnar >Priority: Major > Labels: newbie > Fix For: 4.0.0 > > Attachments: HIVE-21942.1.patch > > > I'm not 100% aware of historical reasons, but this single-value enum seems > totally useless at the moment. > https://github.com/apache/hive/blob/86a15600ae610e1c6017019883ef9344585572ef/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/AbstractCliConfig.java#L50-L52 > According to HIVE-14854, it may have made sense when there was an initiative > of hbase metastore, but it's not in the scope anymore AFAIK. > https://github.com/apache/hive/commit/474425aa62e3f25b119419439373aa684c6c2121 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21397) BloomFilter for hive Managed [ACID] table does not work as expected
[
https://issues.apache.org/jira/browse/HIVE-21397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-21397:
--
Attachment: HIVE-21397.5.patch
> BloomFilter for hive Managed [ACID] table does not work as expected
> ---
>
> Key: HIVE-21397
> URL: https://issues.apache.org/jira/browse/HIVE-21397
> Project: Hive
> Issue Type: Bug
> Components: Hive, HiveServer2, Transactions
>Affects Versions: 3.1.1
>Reporter: vaibhav
>Assignee: Denys Kuzmenko
>Priority: Blocker
> Attachments: HIVE-21397.1.patch, HIVE-21397.2.patch,
> HIVE-21397.3.patch, HIVE-21397.4.patch, HIVE-21397.5.patch, HIVE-21397.patch
>
>
> Steps to Reproduce this issue :
> -
> 1. Create a HIveManaged table as below :
> -
> {code:java}
> CREATE TABLE `bloomTest`(
> `msisdn` string,
> `imsi` varchar(20),
> `imei` bigint,
> `cell_id` bigint)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> LOCATION
>
> 'hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest;
>
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'orc.bloom.filter.columns'='msisdn,cell_id,imsi',
> 'orc.bloom.filter.fpp'='0.02',
> 'transactional'='true',
> 'transactional_properties'='default',
> 'transient_lastDdlTime'='1551206683') {code}
> -
> 2. Insert a few rows.
> -
> -
> 3. Check if bloom filter or active : [ It does not show bloom filters for
> hive managed tables ]
> -
> {code:java}
> [hive@c1162-node2 root]$ hive --orcfiledump
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest/delta_001_001_
> | grep -i bloom
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory]
> Processing data file
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest/delta_001_001_/bucket_0
> [length: 791]
> Structure for
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/managed/hive/bloomTest/delta_001_001_/bucket_0
> {code}
> -
> On Another hand: For hive External tables it works :
> -
> {code:java}
> CREATE external TABLE `ext_bloomTest`(
> `msisdn` string,
> `imsi` varchar(20),
> `imei` bigint,
> `cell_id` bigint)
> ROW FORMAT SERDE
> 'org.apache.hadoop.hive.ql.io.orc.OrcSerde'
> STORED AS INPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat'
> OUTPUTFORMAT
> 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat'
> TBLPROPERTIES (
> 'bucketing_version'='2',
> 'orc.bloom.filter.columns'='msisdn,cell_id,imsi',
> 'orc.bloom.filter.fpp'='0.02') {code}
> -
> {code:java}
> [hive@c1162-node2 root]$ hive --orcfiledump
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/external/hive/ext_bloomTest/00_0
> | grep -i bloom
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: Found binding in
> [jar:file:/usr/hdp/3.1.0.0-78/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
>
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
> explanation.
> SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory]
> Processing data file
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/external/hive/ext_bloomTest/00_0
> [length: 755]
> Structure for
> hdfs://c1162-node2.squadron-labs.com:8020/warehouse/tablespace/external/hive/ext_bloomTest/00_0
>
> Stream: column 1 section BLOOM_FILTER_UTF8 start: 41 length 110
> Stream: column 2 section BLOOM_FILTER_UTF8 start: 178 length 114
> Stream: column 4 section BLOOM_FILTER_UTF8 start:
[jira] [Commented] (HIVE-21996) Remove unused code from Driver
[
https://issues.apache.org/jira/browse/HIVE-21996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923310#comment-16923310
]
Hive QA commented on HIVE-21996:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2248 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} ql: The patch generated 0 new + 200 unchanged - 8
fixed = 200 total (was 208) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} itests/hive-unit: The patch generated 0 new + 24
unchanged - 4 fixed = 24 total (was 28) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18452/dev-support/hive-personality.sh
|
| git revision | master / ebcc9bc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql itests/hive-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18452/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove unused code from Driver
> --
>
> Key: HIVE-21996
> URL: https://issues.apache.org/jira/browse/HIVE-21996
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-driver
> Attachments: HIVE-21996.01.patch, HIVE-21996.02.patch
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In the IDriver interface there is an "int compile(String command)' function
> which is only used by tests. As this is production code, it should be removed
> from the interface. Also there are several other unused codes and
> functions/fields with visibility that should be reduced.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21844.04.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21884.01.patch, > HIVE-21884.02.patch, HIVE-21884.03.patch, Scheduled queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: HIVE-21508.branch-3.patch
HIVE-21508.branch-3.1.patch
HIVE-21508.branch-2.patch
HIVE-21508.branch-2.3.patch
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923290#comment-16923290
]
Hive QA commented on HIVE-21508:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979417/HIVE-21508.branch-3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 142 failed/errored test(s), 14408 tests
executed
*Failed tests:*
{noformat}
TestAddPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestAddPartitionsFromPartSpec - did not produce a TEST-*.xml file (likely timed
out) (batchId=230)
TestAdminUser - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAggregateStatsCache - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestAlterPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestAppendPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestAutoPurgeTables - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=275)
TestCachedStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogCaching - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestCatalogNonDefaultClient - did not produce a TEST-*.xml file (likely timed
out) (batchId=228)
TestCatalogNonDefaultSvr - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestCatalogOldClient - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCatalogs - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestCheckConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestCloseableThreadLocal - did not produce a TEST-*.xml file (likely timed out)
(batchId=334)
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestDatabases - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDeadline - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestDefaultConstraint - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestDropPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=275)
TestEmbeddedHiveMetaStore - did not produce a TEST-*.xml file (likely timed
out) (batchId=231)
TestExchangePartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFMSketchSerialization - did not produce a TEST-*.xml file (likely timed
out) (batchId=239)
TestFilterHooks - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestForeignKey - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestFunctions - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestGetPartitions - did not produce a TEST-*.xml file (likely timed out)
(batchId=230)
TestGetPartitionsUsingProjectionAndFilterSpecs - did not produce a TEST-*.xml
file (likely timed out) (batchId=230)
TestGetTableMeta - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHLLNoBias - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHLLSerialization - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHdfsUtils - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
TestHiveAlterHandler - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHiveMetaStoreGetMetaConf - did not produce a TEST-*.xml file (likely timed
out) (batchId=238)
TestHiveMetaStorePartitionSpecs - did not produce a TEST-*.xml file (likely
timed out) (batchId=230)
TestHiveMetaStoreSchemaMethods - did not produce a TEST-*.xml file (likely
timed out) (batchId=238)
TestHiveMetaStoreTimeout - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreTxns - did not produce a TEST-*.xml file (likely timed out)
(batchId=238)
TestHiveMetaStoreWithEnvironmentContext - did not produce a TEST-*.xml file
(likely timed out) (batchId=233)
TestHiveMetaToolCommandLine - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHiveMetastoreCli - did not produce a TEST-*.xml file (likely timed out)
(batchId=228)
TestHmsServerAuthorization - did not produce a TEST-*.xml file (likely timed
out) (batchId=234)
TestHyperLogLog - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogDense - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogMerge - did not produce a TEST-*.xml file (likely timed out)
(batchId=239)
TestHyperLogLogSparse - did not produce
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923273#comment-16923273
]
Ana Jalba commented on HIVE-21508:
--
The test used the patch for branch-3 on master (according to
http://104.198.109.242/logs/PreCommit-HIVE-Build-18451/patches/PreCommit-HIVE-Build-18451.patch)
so I deleted the previous patches and then submitted only the master. After
the tests run, I'll add the other patches.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: HIVE-21508.patch
Status: Patch Available (was: In Progress)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 2.3.4, 3.2.0
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: (was: HIVE-21508.branch-2.patch)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: (was: HIVE-21508.branch-3.1.patch)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: (was: HIVE-21508.branch-2.3.patch)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: (was: HIVE-21508.patch)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: (was: HIVE-21508.branch-3.patch)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: (was: HIVE-21508.1.patch)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Status: In Progress (was: Patch Available)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 2.3.4, 3.2.0
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: HIVE-21508.1.patch
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.branch-2.3.patch,
> HIVE-21508.branch-2.patch, HIVE-21508.branch-3.1.patch,
> HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: HIVE-21508.patch
Status: Patch Available (was: In Progress)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 2.3.4, 3.2.0
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Status: In Progress (was: Patch Available)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 2.3.4, 3.2.0
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: (was: HIVE-21508.patch)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923243#comment-16923243
]
Hive QA commented on HIVE-21508:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 11s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-18451/patches/PreCommit-HIVE-Build-18451.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18451/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923241#comment-16923241
]
Ana Jalba commented on HIVE-21508:
--
Are any tests supposed to run on these patches?
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 2.4.0, 3.2.0, 2.3.7
>
> Attachments: HIVE-21508.branch-2.3.patch, HIVE-21508.branch-2.patch,
> HIVE-21508.branch-3.1.patch, HIVE-21508.branch-3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22106) PCR: Remove cross-query synchronization for the partition-eval
[
https://issues.apache.org/jira/browse/HIVE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923236#comment-16923236
]
Hive QA commented on HIVE-22106:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979383/HIVE-22106.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16746 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18450/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18450/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18450/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979383 - PreCommit-HIVE-Build
> PCR: Remove cross-query synchronization for the partition-eval
> ---
>
> Key: HIVE-22106
> URL: https://issues.apache.org/jira/browse/HIVE-22106
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-22106.patch
>
>
> {code}
> HiveServer2-Handler-Pool: Thread-492 Blocked CPU usage on sample: 0ms
>
> org.apache.hadoop.hive.ql.optimizer.ppr.PartExprEvalUtils.evalExprWithPart(ExprNodeDesc,
> Partition, List, StructObjectInspector) PartExprEvalUtils.java:58
>
> org.apache.hadoop.hive.ql.optimizer.pcr.PcrExprProcFactory.evalExprWithPart(ExprNodeDesc,
> Partition, List) PcrExprProcFactory.java:83
>
> org.apache.hadoop.hive.ql.optimizer.pcr.PcrExprProcFactory$GenericFuncExprProcessor.handleDeterministicUdf(PcrExprProcCtx,
> ExprNodeGenericFuncDesc, Object[]) PcrExprProcFactory.java:317
>
> org.apache.hadoop.hive.ql.optimizer.pcr.PcrExprProcFactory$GenericFuncExprProcessor.process(Node,
> Stack, NodeProcessorCtx, Object[]) PcrExprProcFactory.java:298
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(Node, Stack,
> Object[]) DefaultRuleDispatcher.java:90
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(Node,
> Stack) DefaultGraphWalker.java:105
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatch(Node, Stack)
> DefaultGraphWalker.java:89
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.walk(Node)
> DefaultGraphWalker.java:158
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.startWalking(Collection,
> HashMap) DefaultGraphWalker.java:120
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22106) PCR: Remove cross-query synchronization for the partition-eval
[
https://issues.apache.org/jira/browse/HIVE-22106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923210#comment-16923210
]
Hive QA commented on HIVE-22106:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2248 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
36s{color} | {color:red} ql: The patch generated 3 new + 7 unchanged - 3 fixed
= 10 total (was 10) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18450/dev-support/hive-personality.sh
|
| git revision | master / ebcc9bc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18450/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18450/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> PCR: Remove cross-query synchronization for the partition-eval
> ---
>
> Key: HIVE-22106
> URL: https://issues.apache.org/jira/browse/HIVE-22106
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-22106.patch
>
>
> {code}
> HiveServer2-Handler-Pool: Thread-492 Blocked CPU usage on sample: 0ms
>
> org.apache.hadoop.hive.ql.optimizer.ppr.PartExprEvalUtils.evalExprWithPart(ExprNodeDesc,
> Partition, List, StructObjectInspector) PartExprEvalUtils.java:58
>
> org.apache.hadoop.hive.ql.optimizer.pcr.PcrExprProcFactory.evalExprWithPart(ExprNodeDesc,
> Partition, List) PcrExprProcFactory.java:83
>
> org.apache.hadoop.hive.ql.optimizer.pcr.PcrExprProcFactory$GenericFuncExprProcessor.handleDeterministicUdf(PcrExprProcCtx,
> ExprNodeGenericFuncDesc, Object[]) PcrExprProcFactory.java:317
>
> org.apache.hadoop.hive.ql.optimizer.pcr.PcrExprProcFactory$GenericFuncExprProcessor.process(Node,
> Stack, NodeProcessorCtx, Object[]) PcrExprProcFactory.java:298
> org.apache.hadoop.hive.ql.lib.DefaultRuleDispatcher.dispatch(Node, Stack,
> Object[]) DefaultRuleDispatcher.java:90
> org.apache.hadoop.hive.ql.lib.DefaultGraphWalker.dispatchAndReturn(Node,
> Stack) DefaultGraphWalker.java:105
>
[jira] [Updated] (HIVE-21987) Hive is unable to read Parquet int32 annotated with decimal
[
https://issues.apache.org/jira/browse/HIVE-21987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marta Kuczora updated HIVE-21987:
-
Attachment: HIVE-21987.4.patch
> Hive is unable to read Parquet int32 annotated with decimal
> ---
>
> Key: HIVE-21987
> URL: https://issues.apache.org/jira/browse/HIVE-21987
> Project: Hive
> Issue Type: Improvement
>Reporter: Nandor Kollar
>Assignee: Marta Kuczora
>Priority: Major
> Attachments: HIVE-21987.1.patch, HIVE-21987.2.patch,
> HIVE-21987.3.patch, HIVE-21987.4.patch,
> part-0-e5287735-8dcf-4dda-9c6e-4d5c98dc15f2-c000.snappy.parquet
>
>
> When I tried to read a Parquet file from a Hive (with Tez execution engine)
> table with a small decimal column, I got the following exception:
> {code}
> Caused by: java.lang.UnsupportedOperationException:
> org.apache.hadoop.hive.ql.io.parquet.convert.ETypeConverter$8$1
> at
> org.apache.parquet.io.api.PrimitiveConverter.addInt(PrimitiveConverter.java:98)
> at
> org.apache.parquet.column.impl.ColumnReaderImpl$2$3.writeValue(ColumnReaderImpl.java:248)
> at
> org.apache.parquet.column.impl.ColumnReaderImpl.writeCurrentValueToConverter(ColumnReaderImpl.java:367)
> at
> org.apache.parquet.io.RecordReaderImplementation.read(RecordReaderImplementation.java:406)
> at
> org.apache.parquet.hadoop.InternalParquetRecordReader.nextKeyValue(InternalParquetRecordReader.java:226)
> ... 28 more
> {code}
> Steps to reproduce:
> - Create a Hive table with a single decimal(4, 2) column
> - Create a Parquet file with int32 column annotated with decimal(4, 2)
> logical type, put it into the previously created table location (or use the
> attached parquet file, in this case the column should be named as 'd', to
> match the Hive schema with the Parquet schema in the file)
> - Execute a {{select *}} on this table
> Also, I'm afraid that similar problems can happen with int64 decimals too.
> [Parquet specification |
> https://github.com/apache/parquet-format/blob/master/LogicalTypes.md] allows
> both of these cases.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21987) Hive is unable to read Parquet int32 annotated with decimal
[
https://issues.apache.org/jira/browse/HIVE-21987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923179#comment-16923179
]
Hive QA commented on HIVE-21987:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979387/HIVE-21987.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18449/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18449/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18449/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-05 08:22:37.248
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18449/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-05 08:22:37.251
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at ebcc9bc HIVE-22161: UDF: FunctionRegistry synchronizes on
org.apache.hadoop.hive.ql.udf.UDFType class (Gopal V, reviewed by Ashutosh
Chauhan)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at ebcc9bc HIVE-22161: UDF: FunctionRegistry synchronizes on
org.apache.hadoop.hive.ql.udf.UDFType class (Gopal V, reviewed by Ashutosh
Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-05 08:22:38.293
+ rm -rf ../yetus_PreCommit-HIVE-Build-18449
+ mkdir ../yetus_PreCommit-HIVE-Build-18449
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18449
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18449/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: cannot apply binary patch to 'data/files/parquet_int_decimal_1.parquet'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/parquet_int_decimal_1.parquet'
without full index line
error: data/files/parquet_int_decimal_1.parquet: patch does not apply
error: cannot apply binary patch to 'data/files/parquet_int_decimal_2.parquet'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/parquet_int_decimal_2.parquet'
without full index line
error: data/files/parquet_int_decimal_2.parquet: patch does not apply
error: cannot apply binary patch to 'files/parquet_int_decimal_1.parquet'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/parquet_int_decimal_1.parquet'
without full index line
error: files/parquet_int_decimal_1.parquet: patch does not apply
error: cannot apply binary patch to 'files/parquet_int_decimal_2.parquet'
without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/parquet_int_decimal_2.parquet'
without full index line
error: files/parquet_int_decimal_2.parquet: patch does not apply
error:
src/java/org/apache/hadoop/hive/ql/io/parquet/convert/ETypeConverter.java: does
not exist in index
error:
src/java/org/apache/hadoop/hive/ql/io/parquet/vector/ParquetDataColumnReaderFactory.java:
does not exist in index
error: src/test/results/clientpositive/type_change_test_fraction.q.out: does
not exist in index
error: cannot apply binary patch to 'parquet_int_decimal_1.parquet' without
full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'parquet_int_decimal_1.parquet' without
full index line
error: parquet_int_decimal_1.parquet: patch does not apply
error: cannot apply binary patch to
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Status: In Progress (was: Patch Available)
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Attachment: HIVE-22099.7.patch
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22168) remove excessive logging by llap cache.
[
https://issues.apache.org/jira/browse/HIVE-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923176#comment-16923176
]
Hive QA commented on HIVE-22168:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979384/HIVE-22168.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16742 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=254)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18448/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18448/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18448/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979384 - PreCommit-HIVE-Build
> remove excessive logging by llap cache.
> ---
>
> Key: HIVE-22168
> URL: https://issues.apache.org/jira/browse/HIVE-22168
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logging
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-22168.patch
>
>
> Llap cache logging is very expensive when it comes to log every request
> buffers range.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22099) Several date related UDFs can't handle Julian dates properly since HIVE-20007
[
https://issues.apache.org/jira/browse/HIVE-22099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Szita updated HIVE-22099:
--
Status: Patch Available (was: In Progress)
> Several date related UDFs can't handle Julian dates properly since HIVE-20007
> -
>
> Key: HIVE-22099
> URL: https://issues.apache.org/jira/browse/HIVE-22099
> Project: Hive
> Issue Type: Bug
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-22099.0.patch, HIVE-22099.1.patch,
> HIVE-22099.2.patch, HIVE-22099.3.patch, HIVE-22099.4.patch,
> HIVE-22099.5.patch, HIVE-22099.6.patch, HIVE-22099.7.patch
>
>
> Currently dates that belong to Julian calendar (before Oct 15, 1582) are
> handled improperly by date/timestamp UDFs.
> E.g. DateFormat UDF:
> Although the dates are in Julian calendar, the formatter insists to print
> these according to Gregorian calendar causing multiple days of difference in
> some cases:
>
> {code:java}
> beeline> select date_format('1001-01-05','dd---MM--');
> ++
> | _c0 |
> ++
> | 30---12--1000 |
> ++{code}
> I've observed similar problems in the following UDFs:
> * add_months
> * date_format
> * day
> * month
> * months_between
> * weekofyear
> * year
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21737) Upgrade Avro to version 1.9.1
[
https://issues.apache.org/jira/browse/HIVE-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923164#comment-16923164
]
Nandor Kollar commented on HIVE-21737:
--
[~Fokko] FYI, precommit job failed, with the expected issue. Here are the
results: https://builds.apache.org/job/PreCommit-HIVE-Build/18437/
For example,
org.apache.hadoop.hive.serde2.avro.TestTypeInfoToSchema.createAvroUnionSchema
failed with:
{code}
org.apache.avro.AvroRuntimeException: Unknown datum class: class
org.codehaus.jackson.node.NullNode
at
org.apache.avro.util.internal.JacksonUtils.toJson(JacksonUtils.java:87)
at
org.apache.avro.util.internal.JacksonUtils.toJsonNode(JacksonUtils.java:48)
at org.apache.avro.Schema$Field.(Schema.java:560)
at
org.apache.hadoop.hive.serde2.avro.TypeInfoToSchema.getFields(TypeInfoToSchema.java:244)
at
org.apache.hadoop.hive.serde2.avro.TypeInfoToSchema.convert(TypeInfoToSchema.java:63)
at
org.apache.hadoop.hive.serde2.avro.TestTypeInfoToSchema.getAvroSchemaString(TestTypeInfoToSchema.java:89)
at
org.apache.hadoop.hive.serde2.avro.TestTypeInfoToSchema.createAvroUnionSchema(TestTypeInfoToSchema.java:314)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
> Upgrade Avro to version 1.9.1
> -
>
> Key: HIVE-21737
> URL: https://issues.apache.org/jira/browse/HIVE-21737
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Ismaël Mejía
>Assignee: Fokko Driesprong
>Priority: Minor
> Labels: pull-request-available
> Attachments: 0001-HIVE-21737-Bump-Apache-Avro-to-1.9.1.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Avro 1.9.0 was released recently. It brings a lot of fixes including a leaner
> version of Avro without Jackson in the public API. Worth the update.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-21431) Vectorization: ltrim throws ArrayIndexOutOfBounds in corner cases
[
https://issues.apache.org/jira/browse/HIVE-21431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Bodor reassigned HIVE-21431:
---
Assignee: (was: Laszlo Bodor)
> Vectorization: ltrim throws ArrayIndexOutOfBounds in corner cases
> -
>
> Key: HIVE-21431
> URL: https://issues.apache.org/jira/browse/HIVE-21431
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 2.3.4
>Reporter: Rajesh Balamohan
>Priority: Major
>
> In corner cases, {{ltrim}} with string columns throws
> arraryindexoutofboundsexception with vectorization enabled. {{HIVE-19565}}
> seem to fix corner cases. But in another corner case, {{length[]}} was all
> {{0}} and this causes {{-1}} to be returned in the length to be set in the
> target vector. I will check if i can get a easier repro for this.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-22055:
-
Status: Patch Available (was: Open)
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.1.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-22055:
-
Attachment: HIVE-22055.1.patch
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
> Attachments: HIVE-22055.1.patch
>
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-22055:
-
Attachment: (was: HIVE-22055.1.patch)
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22168) remove excessive logging by llap cache.
[
https://issues.apache.org/jira/browse/HIVE-22168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923135#comment-16923135
]
Hive QA commented on HIVE-22168:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
46s{color} | {color:blue} ql in master has 2248 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18448/dev-support/hive-personality.sh
|
| git revision | master / ebcc9bc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18448/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> remove excessive logging by llap cache.
> ---
>
> Key: HIVE-22168
> URL: https://issues.apache.org/jira/browse/HIVE-22168
> Project: Hive
> Issue Type: Improvement
> Components: llap, Logging
>Reporter: slim bouguerra
>Assignee: slim bouguerra
>Priority: Major
> Attachments: HIVE-22168.patch
>
>
> Llap cache logging is very expensive when it comes to log every request
> buffers range.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22055) select count gives incorrect result after loading data from text file
[
https://issues.apache.org/jira/browse/HIVE-22055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Magyar updated HIVE-22055:
-
Status: Open (was: Patch Available)
> select count gives incorrect result after loading data from text file
> -
>
> Key: HIVE-22055
> URL: https://issues.apache.org/jira/browse/HIVE-22055
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Attila Magyar
>Assignee: Attila Magyar
>Priority: Major
>
> Add one more load to mm_loaddata.q:
> Load data 3 times (both kv1.txt and kv2.txt contains 500 records)
> {code:java}
> create table load0_mm (key string, value string) stored as textfile
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> load data local inpath '../../data/files/kv1.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;
> load data local inpath '../../data/files/kv2.txt' into table load0_mm;
> select count(1) from load0_mm;{code}
> Expected output
> {code:java}
> PREHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> PREHOOK: type: LOAD
> A masked pattern was here
> PREHOOK: Output: default@load0_mm
> POSTHOOK: query: load data local inpath '../../data/files/kv2.txt' into table
> load0_mm
> POSTHOOK: type: LOAD
> A masked pattern was here
> POSTHOOK: Output: default@load0_mm
> PREHOOK: query: select count(1) from load0_mm
> PREHOOK: type: QUERY
> PREHOOK: Input: default@load0_mm
> A masked pattern was here
> POSTHOOK: query: select count(1) from load0_mm
> POSTHOOK: type: QUERY
> POSTHOOK: Input: default@load0_mm
> A masked pattern was here
> 1500{code}
> Got:
> [ERROR] TestMiniLlapLocalCliDriver.testCliDriver:59 Client Execution
> succeeded but contained differences (error code = 1) after executing
> mm_loaddata.q
> 63c63
> < 1480
> —
> > 1500
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
