[jira] [Commented] (HIVE-21656) Vectorize UDF mask
[
https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830806#comment-16830806
]
Hive QA commented on HIVE-21656:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967537/HIVE-21656.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15936 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17087/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17087/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17087/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967537 - PreCommit-HIVE-Build
> Vectorize UDF mask
> --
>
> Key: HIVE-21656
> URL: https://issues.apache.org/jira/browse/HIVE-21656
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch,
> HIVE-21656.3.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21656) Vectorize UDF mask
[
https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830784#comment-16830784
]
Hive QA commented on HIVE-21656:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 7 new + 42 unchanged - 0 fixed
= 49 total (was 42) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 9 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17087/dev-support/hive-personality.sh
|
| git revision | master / afdda58 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17087/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17087/yetus/whitespace-tabs.txt
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17087/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorize UDF mask
> --
>
> Key: HIVE-21656
> URL: https://issues.apache.org/jira/browse/HIVE-21656
> Project: Hive
> Issue Type: Improvement
> Components: Vectorization
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch,
> HIVE-21656.3.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[ https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830777#comment-16830777 ] Vihang Karajgaonkar commented on HIVE-21586: Addendum patch merged into master branch. It is not needed in branch-3 > Thrift generated cpp files for metastore do not compile > --- > > Key: HIVE-21586 > URL: https://issues.apache.org/jira/browse/HIVE-21586 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch, > HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch, > HIVE-21586.05.patch, HIVE-21586.nothrift.patch > > > The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics > are defined in hive_metastore.thrift is that these structs are used before > they are defined. While this works for the java code which is generated, it > does not work for the generated cpp code since Thrift does not use > pointer/references to the forward declared classes. > The easy fix for this would be to reorder the struct definitions in the > hive_metastore.thrift so that they are always defined before they are used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21586) Thrift generated cpp files for metastore do not compile
[ https://issues.apache.org/jira/browse/HIVE-21586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-21586: --- Resolution: Fixed Status: Resolved (was: Patch Available) > Thrift generated cpp files for metastore do not compile > --- > > Key: HIVE-21586 > URL: https://issues.apache.org/jira/browse/HIVE-21586 > Project: Hive > Issue Type: Bug > Components: Standalone Metastore >Affects Versions: 3.1.0 >Reporter: Vihang Karajgaonkar >Assignee: Vihang Karajgaonkar >Priority: Major > Fix For: 4.0.0, 3.2.0 > > Attachments: HIVE-21586.01.patch, HIVE-21586.02.patch, > HIVE-21586.03.patch, HIVE-21586.04.branch-3.patch, HIVE-21586.04.patch, > HIVE-21586.05.patch, HIVE-21586.nothrift.patch > > > The way some structs like CreationMetadata, CompactionInfo, ColumnStatistics > are defined in hive_metastore.thrift is that these structs are used before > they are defined. While this works for the java code which is generated, it > does not work for the generated cpp code since Thrift does not use > pointer/references to the forward declared classes. > The easy fix for this would be to reorder the struct definitions in the > hive_metastore.thrift so that they are always defined before they are used. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21656: --- Status: Patch Available (was: Open) > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch, > HIVE-21656.3.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21656: --- Attachment: HIVE-21656.3.patch > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch, > HIVE-21656.3.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21656) Vectorize UDF mask
[ https://issues.apache.org/jira/browse/HIVE-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-21656: --- Status: Open (was: Patch Available) > Vectorize UDF mask > -- > > Key: HIVE-21656 > URL: https://issues.apache.org/jira/browse/HIVE-21656 > Project: Hive > Issue Type: Improvement > Components: Vectorization >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Labels: pull-request-available > Attachments: HIVE-21656.1.patch, HIVE-21656.2.patch, > HIVE-21656.3.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830643#comment-16830643
]
Hive QA commented on HIVE-21660:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967507/HIVE-21660.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 15935 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=231)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17086/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17086/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17086/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967507 - PreCommit-HIVE-Build
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t2( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20479) Update content/people.mdtext in cms
[
https://issues.apache.org/jira/browse/HIVE-20479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-20479:
--
Affects Version/s: 3.0.0
> Update content/people.mdtext in cms
>
>
> Key: HIVE-20479
> URL: https://issues.apache.org/jira/browse/HIVE-20479
> Project: Hive
> Issue Type: Task
>Affects Versions: 3.0.0
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
>Priority: Major
>
> I added myself to the committers list.
>
> {code:java}
> asherman
> Andrew Sherman
> href="http://cloudera.com/";>Cloudera
>
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18119) show partitions should say whether a partition is stored via EC
[ https://issues.apache.org/jira/browse/HIVE-18119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18119: -- Fix Version/s: 3.0.0 > show partitions should say whether a partition is stored via EC > --- > > Key: HIVE-18119 > URL: https://issues.apache.org/jira/browse/HIVE-18119 > Project: Hive > Issue Type: Sub-task >Reporter: Sahil Takiar >Assignee: Andrew Sherman >Priority: Major > Fix For: 3.0.0 > > > Not sure what the criteria should be here because technically any single file > in a directory can be stored via EC. So a partition may contain both EC files > and regular files. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19581) view do not support unicode characters well
[
https://issues.apache.org/jira/browse/HIVE-19581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Sherman updated HIVE-19581:
--
Fix Version/s: 3.2.0
> view do not support unicode characters well
> ---
>
> Key: HIVE-19581
> URL: https://issues.apache.org/jira/browse/HIVE-19581
> Project: Hive
> Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: kai
>Assignee: Andrew Sherman
>Priority: Major
> Fix For: 3.2.0
>
> Attachments: HIVE-19581.1.patch, HIVE-19581.2.patch,
> HIVE-19581.3.patch, HIVE-19581.4.patch, HIVE-19581.5.patch,
> HIVE-19581.6.patch, explain.png, metastore.png
>
>
> create table t_test (name ,string) ;
> insert into table t_test VALUES ('李四');
> create view t_view_test as select * from t_test where name='李四';
> when select * from t_view_test no records return
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830606#comment-16830606
]
Hive QA commented on HIVE-21660:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
19s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17086/dev-support/hive-personality.sh
|
| git revision | master / ae82715 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17086/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t2( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode f
[jira] [Commented] (HIVE-21668) Remove tomcat dependencies even from tests
[
https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830579#comment-16830579
]
Hive QA commented on HIVE-21668:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967493/HIVE-21668.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15935 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17085/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17085/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17085/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967493 - PreCommit-HIVE-Build
> Remove tomcat dependencies even from tests
> --
>
> Key: HIVE-21668
> URL: https://issues.apache.org/jira/browse/HIVE-21668
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-21668.2.patch, HIVE-21668.3.patch, HIVE-21668.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-21672) HiveServer2 needs to support sidecar's 'ldap.xml' file
[ https://issues.apache.org/jira/browse/HIVE-21672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21672 started by Sam An. - > HiveServer2 needs to support sidecar's 'ldap.xml' file > -- > > Key: HIVE-21672 > URL: https://issues.apache.org/jira/browse/HIVE-21672 > Project: Hive > Issue Type: New Feature > Components: Hive >Reporter: Sam An >Assignee: Sam An >Priority: Major > Original Estimate: 96h > Remaining Estimate: 96h > > SDX *sidecar* will publish freeIPA ldap credentials in a shared location that > will be added to classpath. > Hive needs to be able to load LDAP settings from that file for LDAP > authentication. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21672) HiveServer2 needs to support sidecar's 'ldap.xml' file
[ https://issues.apache.org/jira/browse/HIVE-21672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam An reassigned HIVE-21672: - > HiveServer2 needs to support sidecar's 'ldap.xml' file > -- > > Key: HIVE-21672 > URL: https://issues.apache.org/jira/browse/HIVE-21672 > Project: Hive > Issue Type: New Feature > Components: Hive >Reporter: Sam An >Assignee: Sam An >Priority: Major > Original Estimate: 96h > Remaining Estimate: 96h > > SDX *sidecar* will publish freeIPA ldap credentials in a shared location that > will be added to classpath. > Hive needs to be able to load LDAP settings from that file for LDAP > authentication. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21668) Remove tomcat dependencies even from tests
[
https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830538#comment-16830538
]
Hive QA commented on HIVE-21668:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
27s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 22s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17085/dev-support/hive-personality.sh
|
| git revision | master / ae82715 |
| Default Java | 1.8.0_111 |
| modules | C: hcatalog hcatalog/core hcatalog/server-extensions
upgrade-acid/pre-upgrade itests/hcatalog-unit U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17085/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove tomcat dependencies even from tests
> --
>
> Key: HIVE-21668
> URL: https://issues.apache.org/jira/browse/HIVE-21668
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-21668.2.patch, HIVE-21668.3.patch, HIVE-21668.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21670) Replacing mockito-all with mockito-core dependency
[
https://issues.apache.org/jira/browse/HIVE-21670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830533#comment-16830533
]
Hive QA commented on HIVE-21670:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 14m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 13m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 16m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 16m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
22s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 25m
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
20s{color} | {color:red} The patch generated 3 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 97m 41s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17084/dev-support/hive-personality.sh
|
| git revision | master / ae82715 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17084/yetus/patch-asflicense-problems.txt
|
| modules | C: standalone-metastore serde metastore llap-common llap-client
llap-tez spark-client ql llap-server service accumulo-handler jdbc cli
jdbc-handler hcatalog llap-ext-client . itests/hive-minikdc itests/hive-unit
itests/hive-unit-hadoop2 itests/qtest-spark testutils/ptest2 U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17084/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Replacing mockito-all with mockito-core dependency
> --
>
> Key: HIVE-21670
> URL: https://issues.apache.org/jira/browse/HIVE-21670
> Project: Hive
> Issue Type: Bug
>Reporter: Ivan Suller
>Assignee: Ivan Suller
>Priority: Major
> Attachments: HIVE-21670.1.patch
>
>
> The mockito-all dependency contains an old version of Hamcrest core which can
> collide with other Hamcrest dependencies. Replacint it with mockito-core
> should be straightforward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21670) Replacing mockito-all with mockito-core dependency
[
https://issues.apache.org/jira/browse/HIVE-21670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830496#comment-16830496
]
Hive QA commented on HIVE-21670:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967500/HIVE-21670.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 15935 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMergeOnTezEdges
(batchId=318)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testComplexQuery (batchId=271)
org.apache.hive.jdbc.TestJdbcWithMiniLlapArrow.testKillQuery (batchId=271)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitions
(batchId=271)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomCreatedDynamicPartitionsUnionAll
(batchId=271)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerCustomNonExistent
(batchId=271)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighBytesRead
(batchId=271)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerHighShuffleBytes
(batchId=271)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryElapsedTime
(batchId=271)
org.apache.hive.jdbc.TestTriggersTezSessionPoolManager.testTriggerSlowQueryExecutionTime
(batchId=271)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17084/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17084/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17084/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967500 - PreCommit-HIVE-Build
> Replacing mockito-all with mockito-core dependency
> --
>
> Key: HIVE-21670
> URL: https://issues.apache.org/jira/browse/HIVE-21670
> Project: Hive
> Issue Type: Bug
>Reporter: Ivan Suller
>Assignee: Ivan Suller
>Priority: Major
> Attachments: HIVE-21670.1.patch
>
>
> The mockito-all dependency contains an old version of Hamcrest core which can
> collide with other Hamcrest dependencies. Replacint it with mockito-core
> should be straightforward.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830427#comment-16830427
]
Hive QA commented on HIVE-21660:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967499/HIVE-21660.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15935 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17083/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17083/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17083/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967499 - PreCommit-HIVE-Build
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t2( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21530) Replicate Streaming ingestion with transactions batch size greater than 1.
[ https://issues.apache.org/jira/browse/HIVE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21530: Summary: Replicate Streaming ingestion with transactions batch size greater than 1. (was: Replicate Streaming ingest on ACID tables.) > Replicate Streaming ingestion with transactions batch size greater than 1. > -- > > Key: HIVE-21530 > URL: https://issues.apache.org/jira/browse/HIVE-21530 > Project: Hive > Issue Type: Bug > Components: repl, Transactions >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > Attachments: Hive ACID Replication_ Streaming Ingest Tables.pdf > > > implement replication of hive streaming ingest of tables as per [^Hive ACID > Replication_ Streaming Ingest Tables.pdf] . > changes to txn_commit to include information about transaction batch. > changes to copy task to only copy if there is a difference in file size or > checksum, seems specific to transaction batch shouldnt be used for normal > transactions. > copy the correct sequence of files w.r.t data file + side file. > remove side files ( which looks like are suffixed as _flush in file names) > when the batch is committed. > how do we determine the idempotent nature of the events here, update the > corresponding table + partition and not copy new version of the file. > validate if partial copied data files are handled on the target warehouse > given correct side file. can we leave the side file file forever, in case > during transaction batch copy after certain transactions are copied over then > primary warehouse fails. we wont be able to remove _flush file, on failover > do we have to handle this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21660:
--
Description:
There is a data loss when the data is inserted to a partitioned table using
union all and lateral view with explode.
*Steps to reproduce:*
{code:java}
create table t1 (id int, dt string);
insert into t1 values (2, '2019-04-01');
create table t2( id int, dates array);
insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03') as
dates;
create table dst (id int) partitioned by (dt string);
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.exec.dynamic.partition=true;
insert overwrite table dst partition (dt)
select t.id, t.dt from (
select id, dt from t1
union all
select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts ) t;
select * from dst_hdfs;
{code}
*Actual Result:*
{code:java}
+--+--+
| 2| 2019-04-01 |
+--+--+{code}
*Expected Result* (Run only the select part from the above insert query)*:*
{code:java}
+---++
| 2 | 2019-04-01 |
| 1 | 2019-01-01 |
| 1 | 2019-01-02 |
| 1 | 2019-01-03 |
+---++{code}
Data retrieved using union all and lateral view with explode from second table
is missing.
was:
There is a data loss when the data is inserted to a partitioned table using
union all and lateral view with explode.
*Steps to reproduce:*
{code:java}
create table t1 (id int, dt string);
insert into t1 values (2, '2019-04-01');
create table t3( id int, dates array);
insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03') as
dates;
create table dst (id int) partitioned by (dt string);
set hive.exec.dynamic.partition.mode=nonstrict;
set hive.exec.dynamic.partition=true;
insert overwrite table dst partition (dt)
select t.id, t.dt from (
select id, dt from t1
union all
select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts ) t;
select * from dst_hdfs;
{code}
*Actual Result:*
{code:java}
+--+--+
| 2| 2019-04-01 |
+--+--+{code}
*Expected Result* (Run only the select part from the above insert query)*:*
{code:java}
+---++
| 2 | 2019-04-01 |
| 1 | 2019-01-01 |
| 1 | 2019-01-02 |
| 1 | 2019-01-03 |
+---++{code}
Data retrieved using union all and lateral view with explode from second table
is missing.
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t2( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21671) Replicate Streaming ingestion with transactional batch size as 1.
[ https://issues.apache.org/jira/browse/HIVE-21671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan reassigned HIVE-21671: --- > Replicate Streaming ingestion with transactional batch size as 1. > - > > Key: HIVE-21671 > URL: https://issues.apache.org/jira/browse/HIVE-21671 > Project: Hive > Issue Type: Sub-task > Components: repl, Streaming, Transactions >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, replication > > Replication streaming ingest HiveStreamingConnection on ACID tables with > transaction batch size as 1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18923) ValidWriteIdList snapshot per table can be cached for multi-statement transactions.
[
https://issues.apache.org/jira/browse/HIVE-18923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sankar Hariappan updated HIVE-18923:
Issue Type: Bug (was: Sub-task)
Parent: (was: HIVE-18320)
> ValidWriteIdList snapshot per table can be cached for multi-statement
> transactions.
> ---
>
> Key: HIVE-18923
> URL: https://issues.apache.org/jira/browse/HIVE-18923
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Minor
> Labels: ACID
> Fix For: 3.2.0
>
>
> Currently, for each query within a multi-statement transaction, it would
> request metastore/TxnHandler to build ValidWriteIdList snapshot for the given
> table. This is costly as it need to talk to metastore RDBMS. But, the
> snapshot won't change within the duration of transaction. So, it make sense
> to cache it within QueryTxnManager for better performance.
> However, each txn should be able to view their own written rows. So, when a
> transaction allocates writeId to write on a table, then the cached
> ValidWriteIdList on this table should be recalculated as follows.
> *Original ValidWriteIdList:* \{hwm=10, open/aborted=5,6} – (10 is allocated
> by txn < current txn_id).
> *Allocated writeId for this txn:* 13 – (11 and 12 are taken by some other txn
> > current txn_id)
> *New ValidWriteIdList:* \{hwm=12, open/aborted=5,6,11, 12} – (11, 12 are
> added to invalid list, so the snapshot remains same).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18752) HiveEndPoint, HiveStreamingConnection: Optimise metastore calls to open transactions batch and allocate write Ids.
[ https://issues.apache.org/jira/browse/HIVE-18752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-18752: Issue Type: Bug (was: Sub-task) Parent: (was: HIVE-18320) > HiveEndPoint, HiveStreamingConnection: Optimise metastore calls to open > transactions batch and allocate write Ids. > -- > > Key: HIVE-18752 > URL: https://issues.apache.org/jira/browse/HIVE-18752 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Metastore >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: ACID, Metastore, Streaming > Fix For: 3.2.0 > > > Per table write ID implementation (HIVE-18192) have introduced write ID and > maps it against the txn. > Now, for streaming ingest, we need to open txns batch and then allocate write > id for each txn in the batch which is 2 metastore calls. > This can be optimised to use only one metastore api. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21530) Replicate Streaming ingest on ACID tables.
[ https://issues.apache.org/jira/browse/HIVE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21530: Issue Type: Bug (was: Sub-task) Parent: (was: HIVE-18320) > Replicate Streaming ingest on ACID tables. > -- > > Key: HIVE-21530 > URL: https://issues.apache.org/jira/browse/HIVE-21530 > Project: Hive > Issue Type: Bug > Components: repl, Transactions >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > Attachments: Hive ACID Replication_ Streaming Ingest Tables.pdf > > > implement replication of hive streaming ingest of tables as per [^Hive ACID > Replication_ Streaming Ingest Tables.pdf] . > changes to txn_commit to include information about transaction batch. > changes to copy task to only copy if there is a difference in file size or > checksum, seems specific to transaction batch shouldnt be used for normal > transactions. > copy the correct sequence of files w.r.t data file + side file. > remove side files ( which looks like are suffixed as _flush in file names) > when the batch is committed. > how do we determine the idempotent nature of the events here, update the > corresponding table + partition and not copy new version of the file. > validate if partial copied data files are handled on the target warehouse > given correct side file. can we leave the side file file forever, in case > during transaction batch copy after certain transactions are copied over then > primary warehouse fails. we wont be able to remove _flush file, on failover > do we have to handle this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21530) Replicate Streaming ingest on ACID tables.
[ https://issues.apache.org/jira/browse/HIVE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830414#comment-16830414 ] Sankar Hariappan commented on HIVE-21530: - Thanks [~gopalv]! > Replicate Streaming ingest on ACID tables. > -- > > Key: HIVE-21530 > URL: https://issues.apache.org/jira/browse/HIVE-21530 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > Attachments: Hive ACID Replication_ Streaming Ingest Tables.pdf > > > implement replication of hive streaming ingest of tables as per [^Hive ACID > Replication_ Streaming Ingest Tables.pdf] . > changes to txn_commit to include information about transaction batch. > changes to copy task to only copy if there is a difference in file size or > checksum, seems specific to transaction batch shouldnt be used for normal > transactions. > copy the correct sequence of files w.r.t data file + side file. > remove side files ( which looks like are suffixed as _flush in file names) > when the batch is committed. > how do we determine the idempotent nature of the events here, update the > corresponding table + partition and not copy new version of the file. > validate if partial copied data files are handled on the target warehouse > given correct side file. can we leave the side file file forever, in case > during transaction batch copy after certain transactions are copied over then > primary warehouse fails. we wont be able to remove _flush file, on failover > do we have to handle this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21660:
--
Attachment: HIVE-21660.patch
Status: Patch Available (was: In Progress)
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21660:
--
Status: In Progress (was: Patch Available)
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21660:
--
Attachment: (was: HIVE-21660.patch)
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21660:
--
Attachment: HIVE-21660.patch
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21660:
--
Attachment: (was: HIVE-21660.patch)
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830388#comment-16830388
]
Hive QA commented on HIVE-21660:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 2253 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17083/dev-support/hive-personality.sh
|
| git revision | master / ae82715 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17083/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17083/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04
[jira] [Commented] (HIVE-21663) Hive Metastore Translation Layer
[ https://issues.apache.org/jira/browse/HIVE-21663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830383#comment-16830383 ] Naveen Gangam commented on HIVE-21663: -- [~thejas] [~daijy] Could you please start review on this? Thanks > Hive Metastore Translation Layer > > > Key: HIVE-21663 > URL: https://issues.apache.org/jira/browse/HIVE-21663 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HMS Translation Layer_v1.0.pdf > > > This task is for the implementation of the default provider for translation, > that is extensible if needed for a custom translator. Please refer the spec > for additional details on the translation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21663) Hive Metastore Translation Layer
[ https://issues.apache.org/jira/browse/HIVE-21663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830375#comment-16830375 ] Naveen Gangam commented on HIVE-21663: -- Initial patch posted to review board at https://reviews.apache.org/r/70572/ > Hive Metastore Translation Layer > > > Key: HIVE-21663 > URL: https://issues.apache.org/jira/browse/HIVE-21663 > Project: Hive > Issue Type: New Feature > Components: Standalone Metastore >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HMS Translation Layer_v1.0.pdf > > > This task is for the implementation of the default provider for translation, > that is extensible if needed for a custom translator. Please refer the spec > for additional details on the translation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[
https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830361#comment-16830361
]
Hive QA commented on HIVE-21662:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967489/HIVE-21662.02.patch
{color:green}SUCCESS:{color} +1 due to 15 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15935 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17082/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17082/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17082/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967489 - PreCommit-HIVE-Build
> Refactor some functionality from QTestUtil to separate classes
> --
>
> Key: HIVE-21662
> URL: https://issues.apache.org/jira/browse/HIVE-21662
> Project: Hive
> Issue Type: Improvement
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21662.01.patch, HIVE-21662.02.patch
>
>
> This may be a follow-up of HIVE-18560:
> new classes: QTestSystemProperties, QFileVersionHandler, QTestMiniClusters,
> QTestDatasetHandler, QTestResultProcessor
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21670) Replacing mockito-all with mockito-core dependency
[ https://issues.apache.org/jira/browse/HIVE-21670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21670: --- Status: Patch Available (was: Open) > Replacing mockito-all with mockito-core dependency > -- > > Key: HIVE-21670 > URL: https://issues.apache.org/jira/browse/HIVE-21670 > Project: Hive > Issue Type: Bug >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21670.1.patch > > > The mockito-all dependency contains an old version of Hamcrest core which can > collide with other Hamcrest dependencies. Replacint it with mockito-core > should be straightforward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21670) Replacing mockito-all with mockito-core dependency
[ https://issues.apache.org/jira/browse/HIVE-21670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-21670: --- Attachment: HIVE-21670.1.patch > Replacing mockito-all with mockito-core dependency > -- > > Key: HIVE-21670 > URL: https://issues.apache.org/jira/browse/HIVE-21670 > Project: Hive > Issue Type: Bug >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > Attachments: HIVE-21670.1.patch > > > The mockito-all dependency contains an old version of Hamcrest core which can > collide with other Hamcrest dependencies. Replacint it with mockito-core > should be straightforward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21670) Replacing mockito-all with mockito-core dependency
[ https://issues.apache.org/jira/browse/HIVE-21670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller reassigned HIVE-21670: -- Assignee: Ivan Suller > Replacing mockito-all with mockito-core dependency > -- > > Key: HIVE-21670 > URL: https://issues.apache.org/jira/browse/HIVE-21670 > Project: Hive > Issue Type: Bug >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Major > > The mockito-all dependency contains an old version of Hamcrest core which can > collide with other Hamcrest dependencies. Replacint it with mockito-core > should be straightforward. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21660:
--
Attachment: HIVE-21660.patch
Status: Patch Available (was: In Progress)
When lateral view is used along with union all, the same object of
FileSinkOperator type is getting visited twice in removeUnionOperators while
looking for objects of FileSinkOperator type from all root operators (Ref:
[source
code|[https://github.com/apache/hive/blame/rel/release-3.1.1/ql/src/java/org/apache/hadoop/hive/ql/parse/GenTezUtils.java#L293]]).
It first sets the directory for the FileSinkOperator object as
*tablePath+UNION_SUDBIR_PREFIX_2* (linked size is 2 because its the second
subquery of union all query).
When the same object is visited again, it resets the directory of that object
to *(tablePath+UNION_SUDBIR_PREFIX_2)+(**UNION_SUDBIR_PREFIX_1)*. **
So the data getting written in
*tablePath+UNION_SUDBIR_PREFIX_2+**UNION_SUDBIR_PREFIX_1* is not getting moved
to the final path.
This issue will be solved if we avoid setting the directory for the same object
again.
The operator tree for the second subquery having lateral view join is getting
formed as below:
{code:java}
TS17
|
LVF18
/\
SEL19 SEL20
| |
\ UDTF22
\/
LVJ21
|
SEL23
|
FS25{code}
FS25 object is getting visited twice here which is leading to this issue.
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
> Attachments: HIVE-21660.patch
>
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[
https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830323#comment-16830323
]
Hive QA commented on HIVE-21662:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
44s{color} | {color:blue} itests/util in master has 46 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
15s{color} | {color:red} itests/util: The patch generated 32 new + 194
unchanged - 42 fixed = 226 total (was 236) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
53s{color} | {color:red} itests/util generated 2 new + 42 unchanged - 4 fixed =
44 total (was 46) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 3s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:itests/util |
| | Exceptional return value of java.io.File.delete() ignored in
org.apache.hadoop.hive.ql.QTestResultProcessor.executeDiffCommand(String,
String, boolean, String) At QTestResultProcessor.java:ignored in
org.apache.hadoop.hive.ql.QTestResultProcessor.executeDiffCommand(String,
String, boolean, String) At QTestResultProcessor.java:[line 141] |
| | org.apache.hadoop.hive.ql.QTestUtil.DEBUG_HINT isn't final but should be
At QTestUtil.java:be At QTestUtil.java:[line 94] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17082/dev-support/hive-personality.sh
|
| git revision | master / ae82715 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17082/yetus/diff-checkstyle-itests_util.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17082/yetus/new-findbugs-itests_util.html
|
| modules | C: itests/util itests/hive-unit U: itests |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17082/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Refactor some functionality from QTestUtil to separate classes
> --
>
> Key: HIVE-21662
> URL: https://issues.apache.org/jira/browse/HIVE-21662
> Project: Hive
> Issue Type: Improvement
>Reporter: Las
[jira] [Updated] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21660:
--
Attachment: HIVE-21660.patch
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara updated HIVE-21660:
--
Attachment: (was: HIVE-21660.patch)
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21668) Remove tomcat dependencies even from tests
[ https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-21668: -- Attachment: HIVE-21668.3.patch > Remove tomcat dependencies even from tests > -- > > Key: HIVE-21668 > URL: https://issues.apache.org/jira/browse/HIVE-21668 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-21668.2.patch, HIVE-21668.3.patch, HIVE-21668.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21668) Remove tomcat dependencies even from tests
[ https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830301#comment-16830301 ] Peter Vary commented on HIVE-21668: --- Found 1 more place :( > Remove tomcat dependencies even from tests > -- > > Key: HIVE-21668 > URL: https://issues.apache.org/jira/browse/HIVE-21668 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-21668.2.patch, HIVE-21668.3.patch, HIVE-21668.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21668) Remove tomcat dependencies even from tests
[ https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830296#comment-16830296 ] Peter Vary commented on HIVE-21668: --- [~szita], or [~kuczoram]: Could you please review? Thanks, Peter > Remove tomcat dependencies even from tests > -- > > Key: HIVE-21668 > URL: https://issues.apache.org/jira/browse/HIVE-21668 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-21668.2.patch, HIVE-21668.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[ https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21662: Attachment: HIVE-21662.02.patch > Refactor some functionality from QTestUtil to separate classes > -- > > Key: HIVE-21662 > URL: https://issues.apache.org/jira/browse/HIVE-21662 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-21662.01.patch, HIVE-21662.02.patch > > > This may be a follow-up of HIVE-18560: > new classes: QTestSystemProperties, QFileVersionHandler, QTestMiniClusters, > QTestDatasetHandler, QTestResultProcessor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21668) Remove tomcat dependencies even from tests
[
https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830284#comment-16830284
]
Hive QA commented on HIVE-21668:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967475/HIVE-21668.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15935 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17081/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17081/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17081/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967475 - PreCommit-HIVE-Build
> Remove tomcat dependencies even from tests
> --
>
> Key: HIVE-21668
> URL: https://issues.apache.org/jira/browse/HIVE-21668
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-21668.2.patch, HIVE-21668.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21668) Remove tomcat dependencies even from tests
[
https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830238#comment-16830238
]
Hive QA commented on HIVE-21668:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
44s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 40s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17081/dev-support/hive-personality.sh
|
| git revision | master / ae82715 |
| Default Java | 1.8.0_111 |
| modules | C: hcatalog hcatalog/core hcatalog/server-extensions
upgrade-acid/pre-upgrade U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17081/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove tomcat dependencies even from tests
> --
>
> Key: HIVE-21668
> URL: https://issues.apache.org/jira/browse/HIVE-21668
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-21668.2.patch, HIVE-21668.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21669) HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed concurrently.
[ https://issues.apache.org/jira/browse/HIVE-21669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21669: Resolution: Fixed Fix Version/s: 4.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks [~maheshk114] for the review! > HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed > concurrently. > - > > Key: HIVE-21669 > URL: https://issues.apache.org/jira/browse/HIVE-21669 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21669.01.patch > > > HS2 throws NullPointerException if HiveStatement.getQueryId invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[
https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830214#comment-16830214
]
Hive QA commented on HIVE-21662:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967462/HIVE-21662.01.patch
{color:green}SUCCESS:{color} +1 due to 15 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 214 failed/errored test(s), 10572 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.org.apache.hadoop.hive.cli.TestAccumuloCliDriver
(batchId=275)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.org.apache.hadoop.hive.cli.TestBlobstoreCliDriver
(batchId=286)
org.apache.hadoop.hive.cli.TestBlobstoreNegativeCliDriver.org.apache.hadoop.hive.cli.TestBlobstoreNegativeCliDriver
(batchId=286)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=10)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=14)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=18)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=19)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=2)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=20)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=24)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=26)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=27)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=28)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=3)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=33)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=38)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=4)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=40)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=41)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=42)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=46)
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.org.
[jira] [Commented] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[
https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830208#comment-16830208
]
Hive QA commented on HIVE-21662:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/util in master has 46 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} itests/util: The patch generated 32 new + 194
unchanged - 42 fixed = 226 total (was 236) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
50s{color} | {color:red} itests/util generated 2 new + 42 unchanged - 4 fixed =
44 total (was 46) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 50s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:itests/util |
| | Exceptional return value of java.io.File.delete() ignored in
org.apache.hadoop.hive.ql.QTestResultProcessor.executeDiffCommand(String,
String, boolean, String) At QTestResultProcessor.java:ignored in
org.apache.hadoop.hive.ql.QTestResultProcessor.executeDiffCommand(String,
String, boolean, String) At QTestResultProcessor.java:[line 141] |
| | org.apache.hadoop.hive.ql.QTestUtil.DEBUG_HINT isn't final but should be
At QTestUtil.java:be At QTestUtil.java:[line 94] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17080/dev-support/hive-personality.sh
|
| git revision | master / 0f8119f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17080/yetus/diff-checkstyle-itests_util.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17080/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17080/yetus/new-findbugs-itests_util.html
|
| modules | C: itests/util itests/hive-unit U: itests |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17080/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Refactor some functionality from QTestUtil to separate classes
> --
>
>
[jira] [Work started] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-21660 started by Ganesha Shreedhara.
-
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830195#comment-16830195
]
Ganesha Shreedhara commented on HIVE-21660:
---
[~NavyaS] I think I got the issue. Let me submit the patch.
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Navya Sruthi Sunkarapalli
>Priority: Major
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-21660) Wrong result when union all and later view with explode is used
[
https://issues.apache.org/jira/browse/HIVE-21660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ganesha Shreedhara reassigned HIVE-21660:
-
Assignee: Ganesha Shreedhara (was: Navya Sruthi Sunkarapalli)
> Wrong result when union all and later view with explode is used
> ---
>
> Key: HIVE-21660
> URL: https://issues.apache.org/jira/browse/HIVE-21660
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Ganesha Shreedhara
>Assignee: Ganesha Shreedhara
>Priority: Major
>
> There is a data loss when the data is inserted to a partitioned table using
> union all and lateral view with explode.
>
> *Steps to reproduce:*
>
> {code:java}
> create table t1 (id int, dt string);
> insert into t1 values (2, '2019-04-01');
> create table t3( id int, dates array);
> insert into t2 select 1 as id, array('2019-01-01','2019-01-02','2019-01-03')
> as dates;
> create table dst (id int) partitioned by (dt string);
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.exec.dynamic.partition=true;
> insert overwrite table dst partition (dt)
> select t.id, t.dt from (
> select id, dt from t1
> union all
> select id, dts as dt from t2 tt2 lateral view explode(tt2.dates) dd as dts )
> t;
> select * from dst_hdfs;
> {code}
>
>
> *Actual Result:*
> {code:java}
> +--+--+
> | 2| 2019-04-01 |
> +--+--+{code}
>
> *Expected Result* (Run only the select part from the above insert query)*:*
> {code:java}
> +---++
> | 2 | 2019-04-01 |
> | 1 | 2019-01-01 |
> | 1 | 2019-01-02 |
> | 1 | 2019-01-03 |
> +---++{code}
>
> Data retrieved using union all and lateral view with explode from second
> table is missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21669) HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed concurrently.
[
https://issues.apache.org/jira/browse/HIVE-21669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830190#comment-16830190
]
Hive QA commented on HIVE-21669:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967470/HIVE-21669.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15935 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17079/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17079/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17079/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967470 - PreCommit-HIVE-Build
> HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed
> concurrently.
> -
>
> Key: HIVE-21669
> URL: https://issues.apache.org/jira/browse/HIVE-21669
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Attachments: HIVE-21669.01.patch
>
>
> HS2 throws NullPointerException if HiveStatement.getQueryId invoked without
> executing any query or query is closed. It should instead return null so that
> caller would check it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21661) Add option to check q file syntax before execution
[
https://issues.apache.org/jira/browse/HIVE-21661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830177#comment-16830177
]
Laszlo Bodor commented on HIVE-21661:
-
+1
> Add option to check q file syntax before execution
> --
>
> Key: HIVE-21661
> URL: https://issues.apache.org/jira/browse/HIVE-21661
> Project: Hive
> Issue Type: Improvement
> Components: Testing Infrastructure
>Reporter: Bruno Pusztahazi
>Assignee: Bruno Pusztahazi
>Priority: Major
> Attachments: HIVE-21661.1.patch, HIVE-21661.2.patch
>
>
> Currently q file tests are executed line-by-line. This could lead to
> problems, e.g. when a q file has some long running queries and the last line
> contains a syntax error. In this case everything before the wrong line will
> be executed first, which could take a lot of time and only then will the test
> fail due to the syntax error.
> I propose a simple syntax checker that will check the q file for errors
> before executing any statements. This check can will be turned off by default
> and can be enabled with the following option:
> {code:java}
> -Dtest.check.syntax=true{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21669) HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed concurrently.
[
https://issues.apache.org/jira/browse/HIVE-21669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830158#comment-16830158
]
Hive QA commented on HIVE-21669:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
28s{color} | {color:blue} jdbc in master has 16 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17079/dev-support/hive-personality.sh
|
| git revision | master / 0f8119f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: service jdbc U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17079/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed
> concurrently.
> -
>
> Key: HIVE-21669
> URL: https://issues.apache.org/jira/browse/HIVE-21669
> Project: Hive
> Issue Type: Bug
> Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Sankar Hariappan
>Assignee: Sankar Hariappan
>Priority: Major
> Attachments: HIVE-21669.01.patch
>
>
> HS2 throws NullPointerException if HiveStatement.getQueryId invoked without
> executing any query or query is closed. It should instead return null so that
> caller would check it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21668) Remove tomcat dependencies even from tests
[ https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-21668: -- Attachment: HIVE-21668.2.patch > Remove tomcat dependencies even from tests > -- > > Key: HIVE-21668 > URL: https://issues.apache.org/jira/browse/HIVE-21668 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-21668.2.patch, HIVE-21668.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21668) Remove tomcat dependencies even from tests
[
https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830138#comment-16830138
]
Hive QA commented on HIVE-21668:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967453/HIVE-21668.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17078/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17078/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17078/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12967453/HIVE-21668.patch was
found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967453 - PreCommit-HIVE-Build
> Remove tomcat dependencies even from tests
> --
>
> Key: HIVE-21668
> URL: https://issues.apache.org/jira/browse/HIVE-21668
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-21668.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21668) Remove tomcat dependencies even from tests
[
https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830136#comment-16830136
]
Hive QA commented on HIVE-21668:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12967453/HIVE-21668.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 15935 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=231)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=231)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/17077/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17077/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17077/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12967453 - PreCommit-HIVE-Build
> Remove tomcat dependencies even from tests
> --
>
> Key: HIVE-21668
> URL: https://issues.apache.org/jira/browse/HIVE-21668
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-21668.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21669) HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed concurrently.
[ https://issues.apache.org/jira/browse/HIVE-21669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830127#comment-16830127 ] mahesh kumar behera commented on HIVE-21669: code looks fine to me +1 > HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed > concurrently. > - > > Key: HIVE-21669 > URL: https://issues.apache.org/jira/browse/HIVE-21669 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Attachments: HIVE-21669.01.patch > > > HS2 throws NullPointerException if HiveStatement.getQueryId invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21669) HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed concurrently.
[ https://issues.apache.org/jira/browse/HIVE-21669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21669: Attachment: HIVE-21669.01.patch > HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed > concurrently. > - > > Key: HIVE-21669 > URL: https://issues.apache.org/jira/browse/HIVE-21669 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Attachments: HIVE-21669.01.patch > > > HS2 throws NullPointerException if HiveStatement.getQueryId invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21669) HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed concurrently.
[ https://issues.apache.org/jira/browse/HIVE-21669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21669: Status: Patch Available (was: Open) > HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed > concurrently. > - > > Key: HIVE-21669 > URL: https://issues.apache.org/jira/browse/HIVE-21669 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Attachments: HIVE-21669.01.patch > > > HS2 throws NullPointerException if HiveStatement.getQueryId invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[ https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21662: Status: Patch Available (was: Open) > Refactor some functionality from QTestUtil to separate classes > -- > > Key: HIVE-21662 > URL: https://issues.apache.org/jira/browse/HIVE-21662 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-21662.01.patch > > > This may be a follow-up of HIVE-18560: > new classes: QTestSystemProperties, QFileVersionHandler, QTestMiniClusters, > QTestDatasetHandler, QTestResultProcessor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21530) Replicate Streaming ingest on ACID tables.
[ https://issues.apache.org/jira/browse/HIVE-21530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan reassigned HIVE-21530: --- Assignee: Sankar Hariappan (was: mahesh kumar behera) > Replicate Streaming ingest on ACID tables. > -- > > Key: HIVE-21530 > URL: https://issues.apache.org/jira/browse/HIVE-21530 > Project: Hive > Issue Type: Sub-task > Components: repl, Transactions >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > Labels: DR, Replication > Attachments: Hive ACID Replication_ Streaming Ingest Tables.pdf > > > implement replication of hive streaming ingest of tables as per [^Hive ACID > Replication_ Streaming Ingest Tables.pdf] . > changes to txn_commit to include information about transaction batch. > changes to copy task to only copy if there is a difference in file size or > checksum, seems specific to transaction batch shouldnt be used for normal > transactions. > copy the correct sequence of files w.r.t data file + side file. > remove side files ( which looks like are suffixed as _flush in file names) > when the batch is committed. > how do we determine the idempotent nature of the events here, update the > corresponding table + partition and not copy new version of the file. > validate if partial copied data files are handled on the target warehouse > given correct side file. can we leave the side file file forever, in case > during transaction batch copy after certain transactions are copied over then > primary warehouse fails. we wont be able to remove _flush file, on failover > do we have to handle this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[ https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21662: Attachment: HIVE-21662.01.patch > Refactor some functionality from QTestUtil to separate classes > -- > > Key: HIVE-21662 > URL: https://issues.apache.org/jira/browse/HIVE-21662 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-21662.01.patch > > > This may be a follow-up of HIVE-18560: > new classes: QTestSystemProperties, QFileVersionHandler, QTestMiniClusters, > QTestDatasetHandler, QTestResultProcessor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21668) Remove tomcat dependencies even from tests
[
https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830077#comment-16830077
]
Hive QA commented on HIVE-21668:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
22s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
5s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 17m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-17077/dev-support/hive-personality.sh
|
| git revision | master / 0f8119f |
| Default Java | 1.8.0_111 |
| modules | C: hcatalog hcatalog/core hcatalog/server-extensions
upgrade-acid/pre-upgrade U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-17077/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove tomcat dependencies even from tests
> --
>
> Key: HIVE-21668
> URL: https://issues.apache.org/jira/browse/HIVE-21668
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-21668.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21662) Refactor some functionality from QTestUtil to separate classes
[ https://issues.apache.org/jira/browse/HIVE-21662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21662: Description: This may be a follow-up of HIVE-18560: new classes: QTestSystemProperties, QFileVersionHandler, QTestMiniClusters, QTestDatasetHandler, QTestResultProcessor was: This may be a follow-up of HIVE-18560: new classes: QTestSystemProperties, QFileVersionHandler, QTestMiniClusters, QTestDatasetHandler > Refactor some functionality from QTestUtil to separate classes > -- > > Key: HIVE-21662 > URL: https://issues.apache.org/jira/browse/HIVE-21662 > Project: Hive > Issue Type: Improvement >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > > This may be a follow-up of HIVE-18560: > new classes: QTestSystemProperties, QFileVersionHandler, QTestMiniClusters, > QTestDatasetHandler, QTestResultProcessor -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21669) HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed concurrently.
[ https://issues.apache.org/jira/browse/HIVE-21669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-21669: Component/s: HiveServer2 > HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed > concurrently. > - > > Key: HIVE-21669 > URL: https://issues.apache.org/jira/browse/HIVE-21669 > Project: Hive > Issue Type: Bug > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > > HS2 throws NullPointerException if HiveStatement.getQueryId invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21669) HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed concurrently.
[ https://issues.apache.org/jira/browse/HIVE-21669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan reassigned HIVE-21669: --- > HS2 throws NPE when HiveStatement.getQueryId is invoked and query is closed > concurrently. > - > > Key: HIVE-21669 > URL: https://issues.apache.org/jira/browse/HIVE-21669 > Project: Hive > Issue Type: Bug >Affects Versions: 4.0.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan >Priority: Major > > HS2 throws NullPointerException if HiveStatement.getQueryId invoked without > executing any query or query is closed. It should instead return null so that > caller would check it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21668) Remove tomcat dependencies even from tests
[ https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-21668: -- Status: Patch Available (was: Open) > Remove tomcat dependencies even from tests > -- > > Key: HIVE-21668 > URL: https://issues.apache.org/jira/browse/HIVE-21668 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-21668.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21668) Remove tomcat dependencies even from tests
[ https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary updated HIVE-21668: -- Attachment: HIVE-21668.patch > Remove tomcat dependencies even from tests > -- > > Key: HIVE-21668 > URL: https://issues.apache.org/jira/browse/HIVE-21668 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > Attachments: HIVE-21668.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21668) Remove tomcat dependencies even from tests
[ https://issues.apache.org/jira/browse/HIVE-21668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Vary reassigned HIVE-21668: - > Remove tomcat dependencies even from tests > -- > > Key: HIVE-21668 > URL: https://issues.apache.org/jira/browse/HIVE-21668 > Project: Hive > Issue Type: Task >Reporter: Peter Vary >Assignee: Peter Vary >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
