[jira] [Commented] (HIVE-22205) Upgrade zookeeper and curator versions

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933075#comment-16933075
 ] 

Hive QA commented on HIVE-22205:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980637/HIVE-22205.2.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18644/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18644/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18644/

Messages:
{noformat}
 This message was trimmed, see log for full details 
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/ServletRequest.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/ServletResponse.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/annotation/WebFilter.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/http/HttpServletRequest.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-runner/9.3.27.v20190418/jetty-runner-9.3.27.v20190418.jar(javax/servlet/http/HttpServletResponse.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceAudience$LimitedPrivate.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceStability$Unstable.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/ByteArrayOutputStream.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/OutputStream.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Closeable.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/lang/AutoCloseable.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Flushable.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(javax/xml/bind/annotation/XmlRootElement.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar(org/apache/commons/exec/ExecuteException.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/security/PrivilegedExceptionAction.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/ExecutionException.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/TimeoutException.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.1.0/hadoop-common-3.1.0.jar(org/apache/hadoop/fs/FileSystem.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShimsSecure.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/ShimLoader.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShims$WebHCatJTShim.class)]]
[loading 
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.1.0/hadoop-common-3.1.0.jar(org/apache/hadoop/util/ToolRunner.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/CancellationException.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/RejectedExecutionException.class)]]
[loading 
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/SynchronousQueue.class)]]
[loading 

[jira] [Commented] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933073#comment-16933073
 ] 

Hive QA commented on HIVE-22213:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980633/HIVE-22213.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 16833 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18643/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18643/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18643/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980633 - PreCommit-HIVE-Build

> TxnHander cleanupRecords should only clean records belonging to default 
> catalog
> ---
>
> Key: HIVE-22213
> URL: https://issues.apache.org/jira/browse/HIVE-22213
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22213.1.patch, HIVE-22213.2.patch
>
>
> Currently it removes record for given database and given table without 
> checking for the catalog, as a result it can end up removing records when it 
> shouldn't. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected

2019-09-18 Thread Himanshu Mishra (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Himanshu Mishra updated HIVE-22201:
---
Attachment: HIVE-22201.04.patch
Status: Patch Available  (was: Open)

> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws 
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch, 
> HIVE-22201.03.patch, HIVE-22201.04.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end 
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1, 
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected

2019-09-18 Thread Himanshu Mishra (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Himanshu Mishra updated HIVE-22201:
---
Status: Open  (was: Patch Available)

> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws 
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch, 
> HIVE-22201.03.patch, HIVE-22201.04.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end 
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1, 
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933049#comment-16933049
 ] 

Hive QA commented on HIVE-22213:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  1m 
11s{color} | {color:blue} standalone-metastore/metastore-server in master has 
178 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
25s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
23s{color} | {color:red} standalone-metastore/metastore-server: The patch 
generated 12 new + 533 unchanged - 0 fixed = 545 total (was 533) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 14s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18643/dev-support/hive-personality.sh
 |
| git revision | master / f7d5ef2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18643/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
 |
| modules | C: standalone-metastore/metastore-server U: 
standalone-metastore/metastore-server |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18643/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> TxnHander cleanupRecords should only clean records belonging to default 
> catalog
> ---
>
> Key: HIVE-22213
> URL: https://issues.apache.org/jira/browse/HIVE-22213
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22213.1.patch, HIVE-22213.2.patch
>
>
> Currently it removes record for given database and given table without 
> checking for the catalog, as a result it can end up removing records when it 
> shouldn't. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22163:
--
Status: Open  (was: Patch Available)

> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats: 
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567 
> Basic stats: COMPLETE Column stats: NONE'
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available  (was: Open)

> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats: 
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567 
> Basic stats: COMPLETE Column stats: NONE'
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.5.patch

> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats: 
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567 
> Basic stats: COMPLETE Column stats: NONE'
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22197) Common Merge join throwing class cast exception

2019-09-18 Thread mahesh kumar behera (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-22197:
---
Attachment: HIVE-22197.04.patch

> Common Merge join throwing class cast exception 
> 
>
> Key: HIVE-22197
> URL: https://issues.apache.org/jira/browse/HIVE-22197
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch, 
> HIVE-22197.03.patch, HIVE-22197.04.patch
>
>
> In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed 
> and stored in the writable format, but the object inspector is initialized to 
> default. So when join operator is fetching the data from dummy store 
> operator, its getting the OI is Long and the row as LongWritable. This is 
> causing the class cast exception.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22197) Common Merge join throwing class cast exception

2019-09-18 Thread mahesh kumar behera (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-22197:
---
Status: Patch Available  (was: Open)

> Common Merge join throwing class cast exception 
> 
>
> Key: HIVE-22197
> URL: https://issues.apache.org/jira/browse/HIVE-22197
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch, 
> HIVE-22197.03.patch, HIVE-22197.04.patch
>
>
> In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed 
> and stored in the writable format, but the object inspector is initialized to 
> default. So when join operator is fetching the data from dummy store 
> operator, its getting the OI is Long and the row as LongWritable. This is 
> causing the class cast exception.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22197) Common Merge join throwing class cast exception

2019-09-18 Thread mahesh kumar behera (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-22197:
---
Status: Open  (was: Patch Available)

> Common Merge join throwing class cast exception 
> 
>
> Key: HIVE-22197
> URL: https://issues.apache.org/jira/browse/HIVE-22197
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch, 
> HIVE-22197.03.patch
>
>
> In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed 
> and stored in the writable format, but the object inspector is initialized to 
> default. So when join operator is fetching the data from dummy store 
> operator, its getting the OI is Long and the row as LongWritable. This is 
> causing the class cast exception.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22203) Break up DDLSemanticAnalyzer - extract Process related analyzers

2019-09-18 Thread Jesus Camacho Rodriguez (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933037#comment-16933037
 ] 

Jesus Camacho Rodriguez commented on HIVE-22203:


+1

> Break up DDLSemanticAnalyzer - extract Process related analyzers
> 
>
> Key: HIVE-22203
> URL: https://issues.apache.org/jira/browse/HIVE-22203
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
>  Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22203.01.patch, HIVE-22203.02.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is 
> to refactor it in order to have everything cut into more handleable classes 
> under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each analyzers
>  * have a package for each operation, containing an analyzer, a description, 
> and an operation, so the amount of classes under a package is more manageable
> Step #4: extract all the process related analyzers from DDLSemanticAnalyzer, 
> and move them under the new package.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22217) Better Logging for Hive JAR Reload

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933027#comment-16933027
 ] 

Hive QA commented on HIVE-22217:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980631/HIVE-22217.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 16833 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18642/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18642/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18642/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980631 - PreCommit-HIVE-Build

> Better Logging for Hive JAR Reload
> --
>
> Key: HIVE-22217
> URL: https://issues.apache.org/jira/browse/HIVE-22217
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 3.2.0, 2.3.6
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22217.1.patch
>
>
> Troubleshooting Hive Reloadable Auxiliary JARs has always been difficult.
> Add logging to at least confirm which JAR files are being loaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22217) Better Logging for Hive JAR Reload

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933002#comment-16933002
 ] 

Hive QA commented on HIVE-22217:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  9m 
 0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
1s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
19s{color} | {color:green} ql generated 0 new + 1570 unchanged - 1 fixed = 1570 
total (was 1571) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 14s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18642/dev-support/hive-personality.sh
 |
| git revision | master / f7d5ef2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18642/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Better Logging for Hive JAR Reload
> --
>
> Key: HIVE-22217
> URL: https://issues.apache.org/jira/browse/HIVE-22217
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 3.2.0, 2.3.6
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22217.1.patch
>
>
> Troubleshooting Hive Reloadable Auxiliary JARs has always been difficult.
> Add logging to at least confirm which JAR files are being loaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22084) Implement exchange partitions related methods on temporary tables

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932985#comment-16932985
 ] 

Hive QA commented on HIVE-22084:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980621/HIVE-22084.01.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16989 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking10 
(batchId=350)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18641/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18641/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18641/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980621 - PreCommit-HIVE-Build

> Implement exchange partitions related methods on temporary tables
> -
>
> Key: HIVE-22084
> URL: https://issues.apache.org/jira/browse/HIVE-22084
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22084.01.patch
>
>
> IMetaStoreClient exposes the following methods related to exchanging 
> partitions:
> {code:java}
> Partition exchange_partition(Map partitionSpecs, String 
> sourceDb, String sourceTable, String destdb, String destTableName);
> Partition exchange_partition(Map partitionSpecs, String 
> sourceCat, String sourceDb, String sourceTable, String destCat, String 
> destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs, 
> String sourceDb, String sourceTable, String destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs, 
> String sourceCat, String sourceDb, String sourceTable, String destCat, String 
> destdb, String destTableName);{code}
> In order to support partitions on temporary tables, these methods must be 
> implemented. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22084) Implement exchange partitions related methods on temporary tables

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932961#comment-16932961
 ] 

Hive QA commented on HIVE-22084:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  2m  
2s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  1m 
11s{color} | {color:blue} standalone-metastore/metastore-server in master has 
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m 
11s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
16s{color} | {color:red} standalone-metastore/metastore-server: The patch 
generated 3 new + 0 unchanged - 0 fixed = 3 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
38s{color} | {color:red} ql: The patch generated 4 new + 17 unchanged - 0 fixed 
= 21 total (was 17) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  4m  
6s{color} | {color:red} ql generated 1 new + 1571 unchanged - 0 fixed = 1572 
total (was 1571) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
14s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 57s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
|  |  Redundant nullcheck of partitions, which is known to be non-null in 
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.exchange_partition(Map,
 String, String, String, String, String, String)  Redundant null check at 
SessionHiveMetaStoreClient.java:is known to be non-null in 
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.exchange_partition(Map,
 String, String, String, String, String, String)  Redundant null check at 
SessionHiveMetaStoreClient.java:[line 1590] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18641/dev-support/hive-personality.sh
 |
| git revision | master / f7d5ef2 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18641/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18641/yetus/diff-checkstyle-ql.txt
 |
| findbugs | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18641/yetus/new-findbugs-ql.html
 |
| asflicense | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18641/yetus/patch-asflicense-problems.txt
 |
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18641/yetus.txt 

[jira] [Commented] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932944#comment-16932944
 ] 

Hive QA commented on HIVE-22201:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980599/HIVE-22201.03.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16833 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18640/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18640/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18640/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980599 - PreCommit-HIVE-Build

> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws 
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch, 
> HIVE-22201.03.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end 
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1, 
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected

2019-09-18 Thread Ashutosh Chauhan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-20983:

Fix Version/s: 4.0.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

Pushed to master. Thanks, Gopal and Mustafa!

> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Mustafa Iman
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, 
> HIVE-20983.3.patch, HIVE-20983.4.patch, HIVE-20983.5.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in 
> place, but an accurate estimate does not always result in a low number of 
> collisions.
> The hashtables which contain a very small number of items tend to lose their 
> O(1) lookup performance where there are collisions. Since collisions are easy 
> to detect within the fast hashtable implementation, a rehashing to a higher 
> size will help these small hashtables avoid collisions and go back to O(1) 
> perf.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22214) Explain vectorization should disable user level explain

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932911#comment-16932911
 ] 

Hive QA commented on HIVE-22214:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980594/HIVE-22214.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 16801 tests 
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed 
out) (batchId=233)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out) 
(batchId=233)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[topnkey] 
(batchId=111)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vector_join_part_col_char]
 (batchId=111)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vector_topnkey] 
(batchId=111)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18639/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18639/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18639/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980594 - PreCommit-HIVE-Build

> Explain vectorization should disable user level explain
> ---
>
> Key: HIVE-22214
> URL: https://issues.apache.org/jira/browse/HIVE-22214
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22214.1.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default

2019-09-18 Thread John Sherman (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sherman updated HIVE-22042:

Attachment: HIVE-22042.1.patch

> Set hive.exec.dynamic.partition.mode=nonstrict by default
> -
>
> Key: HIVE-22042
> URL: https://issues.apache.org/jira/browse/HIVE-22042
> Project: Hive
>  Issue Type: Bug
>Reporter: Jesus Camacho Rodriguez
>Assignee: John Sherman
>Priority: Minor
> Attachments: HIVE-22042.1.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22214) Explain vectorization should disable user level explain

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932889#comment-16932889
 ] 

Hive QA commented on HIVE-22214:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
2s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 31s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18639/dev-support/hive-personality.sh
 |
| git revision | master / c05a32c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18639/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Explain vectorization should disable user level explain
> ---
>
> Key: HIVE-22214
> URL: https://issues.apache.org/jira/browse/HIVE-22214
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22214.1.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-22209) Creating a materialized view with no tables should be handled more gracefully

2019-09-18 Thread Steve Carlin (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Carlin reassigned HIVE-22209:
---

Assignee: Steve Carlin  (was: John Sherman)

> Creating a materialized view with no tables should be handled more gracefully
> -
>
> Key: HIVE-22209
> URL: https://issues.apache.org/jira/browse/HIVE-22209
> Project: Hive
>  Issue Type: Bug
>  Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Steve Carlin
>Priority: Minor
>
> Currently, materialized views without a table reference are not supported. 
> However, instead of printing a clear message about it, when a materialized 
> view is created without a table reference, we fail with an unclear message.
> {code}
> > create materialized view mv_test1 as select 5;
> (...)
> ERROR : FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Add request 
> failed :
> INSERT INTO MV_TABLES_USED (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) )
> INFO : Completed executing 
> command(queryId=hive_20190916203511_b609cccf-f5e3-45dd-abfd-6e869d94e39a); 
> Time taken: 10.469 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaExcep
> tion(message:Add request failed : INSERT INTO MV_TABLES_USED 
> (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) ) (state=08S01,code=1)
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932808#comment-16932808
 ] 

Hive QA commented on HIVE-22163:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980593/HIVE-22163.5.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16834 tests 
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestActivePassiveHA.testClientConnectionsOnFailover 
(batchId=284)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18638/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18638/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18638/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980593 - PreCommit-HIVE-Build

> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats: 
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567 
> Basic stats: COMPLETE Column stats: NONE'
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932775#comment-16932775
 ] 

Hive QA commented on HIVE-22163:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
39s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
5s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 47s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18638/dev-support/hive-personality.sh
 |
| git revision | master / c05a32c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18638/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> '

[jira] [Updated] (HIVE-22205) Upgrade zookeeper and curator versions

2019-09-18 Thread Naveen Gangam (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Naveen Gangam updated HIVE-22205:
-
Attachment: HIVE-22205.2.patch

> Upgrade zookeeper and curator versions
> --
>
> Key: HIVE-22205
> URL: https://issues.apache.org/jira/browse/HIVE-22205
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-22205.2.patch, HIVE-22205.patch
>
>
> Other components like hadoop have switched to using new ZK versions. So these 
> jars end up in classpath for hive services and could cause issues due to 
> in-compatible curator versions that hive uses.
> So it makes sense for hive to upgrade the ZK and curator versions to try to 
> keep up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22205) Upgrade zookeeper and curator versions

2019-09-18 Thread Naveen Gangam (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Naveen Gangam updated HIVE-22205:
-
Status: Patch Available  (was: Open)

> Upgrade zookeeper and curator versions
> --
>
> Key: HIVE-22205
> URL: https://issues.apache.org/jira/browse/HIVE-22205
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-22205.2.patch, HIVE-22205.patch
>
>
> Other components like hadoop have switched to using new ZK versions. So these 
> jars end up in classpath for hive services and could cause issues due to 
> in-compatible curator versions that hive uses.
> So it makes sense for hive to upgrade the ZK and curator versions to try to 
> keep up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22205) Upgrade zookeeper and curator versions

2019-09-18 Thread Naveen Gangam (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Naveen Gangam updated HIVE-22205:
-
Status: Open  (was: Patch Available)

> Upgrade zookeeper and curator versions
> --
>
> Key: HIVE-22205
> URL: https://issues.apache.org/jira/browse/HIVE-22205
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 4.0.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
>Priority: Minor
> Attachments: HIVE-22205.patch
>
>
> Other components like hadoop have switched to using new ZK versions. So these 
> jars end up in classpath for hive services and could cause issues due to 
> in-compatible curator versions that hive uses.
> So it makes sense for hive to upgrade the ZK and curator versions to try to 
> keep up.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-21924) Split text files even if header/footer exists

2019-09-18 Thread Mustafa Iman (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-21924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mustafa Iman reassigned HIVE-21924:
---

Assignee: (was: Mustafa Iman)

> Split text files even if header/footer exists
> -
>
> Key: HIVE-21924
> URL: https://issues.apache.org/jira/browse/HIVE-21924
> Project: Hive
>  Issue Type: Improvement
>  Components: File Formats
>Affects Versions: 2.4.0, 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Priority: Major
>
> https://github.com/apache/hive/blob/967a1cc98beede8e6568ce750ebeb6e0d048b8ea/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L494-L503
>  
> {code}
> int headerCount = 0;
> int footerCount = 0;
> if (table != null) {
>   headerCount = Utilities.getHeaderCount(table);
>   footerCount = Utilities.getFooterCount(table, conf);
>   if (headerCount != 0 || footerCount != 0) {
> // Input file has header or footer, cannot be splitted.
> HiveConf.setLongVar(conf, ConfVars.MAPREDMINSPLITSIZE, 
> Long.MAX_VALUE);
>   }
> }
> {code}
> this piece of code makes the CSV (or any text files with header/footer) files 
> not splittable if header or footer is present. 
> If only header is present, we can find the offset after first line break and 
> use that to split. Similarly for footer, may be read few KB's of data at the 
> end and find the last line break offset. Use that to determine the data range 
> which can be used for splitting. Few reads during split generation are 
> cheaper than not splitting the file at all.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-21924) Split text files even if header/footer exists

2019-09-18 Thread Mustafa Iman (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-21924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mustafa Iman reassigned HIVE-21924:
---

Assignee: Mustafa Iman

> Split text files even if header/footer exists
> -
>
> Key: HIVE-21924
> URL: https://issues.apache.org/jira/browse/HIVE-21924
> Project: Hive
>  Issue Type: Improvement
>  Components: File Formats
>Affects Versions: 2.4.0, 4.0.0, 3.2.0
>Reporter: Prasanth Jayachandran
>Assignee: Mustafa Iman
>Priority: Major
>
> https://github.com/apache/hive/blob/967a1cc98beede8e6568ce750ebeb6e0d048b8ea/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java#L494-L503
>  
> {code}
> int headerCount = 0;
> int footerCount = 0;
> if (table != null) {
>   headerCount = Utilities.getHeaderCount(table);
>   footerCount = Utilities.getFooterCount(table, conf);
>   if (headerCount != 0 || footerCount != 0) {
> // Input file has header or footer, cannot be splitted.
> HiveConf.setLongVar(conf, ConfVars.MAPREDMINSPLITSIZE, 
> Long.MAX_VALUE);
>   }
> }
> {code}
> this piece of code makes the CSV (or any text files with header/footer) files 
> not splittable if header or footer is present. 
> If only header is present, we can find the offset after first line break and 
> use that to split. Similarly for footer, may be read few KB's of data at the 
> end and find the last line break offset. Use that to determine the data range 
> which can be used for splitting. Few reads during split generation are 
> cheaper than not splitting the file at all.  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog

2019-09-18 Thread Vineet Garg (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg updated HIVE-22213:
---
Status: Open  (was: Patch Available)

> TxnHander cleanupRecords should only clean records belonging to default 
> catalog
> ---
>
> Key: HIVE-22213
> URL: https://issues.apache.org/jira/browse/HIVE-22213
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22213.1.patch, HIVE-22213.2.patch
>
>
> Currently it removes record for given database and given table without 
> checking for the catalog, as a result it can end up removing records when it 
> shouldn't. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog

2019-09-18 Thread Vineet Garg (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg updated HIVE-22213:
---
Status: Patch Available  (was: Open)

> TxnHander cleanupRecords should only clean records belonging to default 
> catalog
> ---
>
> Key: HIVE-22213
> URL: https://issues.apache.org/jira/browse/HIVE-22213
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22213.1.patch, HIVE-22213.2.patch
>
>
> Currently it removes record for given database and given table without 
> checking for the catalog, as a result it can end up removing records when it 
> shouldn't. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog

2019-09-18 Thread Vineet Garg (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg updated HIVE-22213:
---
Attachment: HIVE-22213.2.patch

> TxnHander cleanupRecords should only clean records belonging to default 
> catalog
> ---
>
> Key: HIVE-22213
> URL: https://issues.apache.org/jira/browse/HIVE-22213
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22213.1.patch, HIVE-22213.2.patch
>
>
> Currently it removes record for given database and given table without 
> checking for the catalog, as a result it can end up removing records when it 
> shouldn't. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22217) Better Logging for Hive JAR Reload

2019-09-18 Thread David Mollitor (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Mollitor updated HIVE-22217:
--
Status: Patch Available  (was: Open)

> Better Logging for Hive JAR Reload
> --
>
> Key: HIVE-22217
> URL: https://issues.apache.org/jira/browse/HIVE-22217
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 2.3.6, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22217.1.patch
>
>
> Troubleshooting Hive Reloadable Auxiliary JARs has always been difficult.
> Add logging to at least confirm which JAR files are being loaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22217) Better Logging for Hive JAR Reload

2019-09-18 Thread David Mollitor (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Mollitor updated HIVE-22217:
--
Attachment: HIVE-22217.1.patch

> Better Logging for Hive JAR Reload
> --
>
> Key: HIVE-22217
> URL: https://issues.apache.org/jira/browse/HIVE-22217
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 3.2.0, 2.3.6
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
> Attachments: HIVE-22217.1.patch
>
>
> Troubleshooting Hive Reloadable Auxiliary JARs has always been difficult.
> Add logging to at least confirm which JAR files are being loaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Assigned] (HIVE-22217) Better Logging for Hive JAR Reload

2019-09-18 Thread David Mollitor (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Mollitor reassigned HIVE-22217:
-


> Better Logging for Hive JAR Reload
> --
>
> Key: HIVE-22217
> URL: https://issues.apache.org/jira/browse/HIVE-22217
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Affects Versions: 2.3.6, 3.2.0
>Reporter: David Mollitor
>Assignee: David Mollitor
>Priority: Minor
>
> Troubleshooting Hive Reloadable Auxiliary JARs has always been difficult.
> Add logging to at least confirm which JAR files are being loaded.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932628#comment-16932628
 ] 

Hive QA commented on HIVE-22195:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980589/HIVE-22195.8.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 16836 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18637/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18637/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18637/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980589 - PreCommit-HIVE-Build

> Configure authentication type for Zookeeper when different from the default 
> cluster wide
> 
>
> Key: HIVE-22195
> URL: https://issues.apache.org/jira/browse/HIVE-22195
> Project: Hive
>  Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch, 
> HIVE-22195.4.patch, HIVE-22195.5.patch, HIVE-22195.6.patch, 
> HIVE-22195.7.patch, HIVE-22195.8.patch
>
>
> This could be useful in case when cluster is kerberized, but Zookeeper is not.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932575#comment-16932575
 ] 

Hive QA commented on HIVE-22195:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
 8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
21s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
33s{color} | {color:blue} common in master has 65 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
24s{color} | {color:blue} llap-client in master has 26 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  1m 
11s{color} | {color:blue} standalone-metastore/metastore-server in master has 
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
39s{color} | {color:blue} service in master has 49 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
22s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
22s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 22s{color} 
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} llap-client: The patch generated 1 new + 48 unchanged 
- 2 fixed = 49 total (was 50) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  0m 
21s{color} | {color:red} service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m  5s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18637/dev-support/hive-personality.sh
 |
| git revision | master / c05a32c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| mvninstall | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18637/yetus/patch-mvninstall-service.txt
 |
| compile | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18637/yetus/patch-compile-service.txt
 |
| javac | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18637/yetus/patch-compile-service.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18637/yetus/diff-checkstyle-llap-client.txt
 |
| findbugs | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18637/yetus/patch-findbugs-service.txt
 |
| modules | C: common llap-client standalone-metastore/metastore-server service 
U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18637/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Configure authentication type for Zookeeper when different from the default 
> cluster wide
> 
>
> Key: HIVE-22195
>   

[jira] [Updated] (HIVE-22084) Implement exchange partitions related methods on temporary tables

2019-09-18 Thread Laszlo Pinter (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laszlo Pinter updated HIVE-22084:
-
Attachment: HIVE-22084.01.patch

> Implement exchange partitions related methods on temporary tables
> -
>
> Key: HIVE-22084
> URL: https://issues.apache.org/jira/browse/HIVE-22084
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22084.01.patch
>
>
> IMetaStoreClient exposes the following methods related to exchanging 
> partitions:
> {code:java}
> Partition exchange_partition(Map partitionSpecs, String 
> sourceDb, String sourceTable, String destdb, String destTableName);
> Partition exchange_partition(Map partitionSpecs, String 
> sourceCat, String sourceDb, String sourceTable, String destCat, String 
> destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs, 
> String sourceDb, String sourceTable, String destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs, 
> String sourceCat, String sourceDb, String sourceTable, String destCat, String 
> destdb, String destTableName);{code}
> In order to support partitions on temporary tables, these methods must be 
> implemented. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22084) Implement exchange partitions related methods on temporary tables

2019-09-18 Thread Laszlo Pinter (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laszlo Pinter updated HIVE-22084:
-
Status: Patch Available  (was: Open)

> Implement exchange partitions related methods on temporary tables
> -
>
> Key: HIVE-22084
> URL: https://issues.apache.org/jira/browse/HIVE-22084
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22084.01.patch
>
>
> IMetaStoreClient exposes the following methods related to exchanging 
> partitions:
> {code:java}
> Partition exchange_partition(Map partitionSpecs, String 
> sourceDb, String sourceTable, String destdb, String destTableName);
> Partition exchange_partition(Map partitionSpecs, String 
> sourceCat, String sourceDb, String sourceTable, String destCat, String 
> destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs, 
> String sourceDb, String sourceTable, String destdb, String destTableName);
> List exchange_partitions(Map partitionSpecs, 
> String sourceCat, String sourceDb, String sourceTable, String destCat, String 
> destdb, String destTableName);{code}
> In order to support partitions on temporary tables, these methods must be 
> implemented. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected

2019-09-18 Thread Ashutosh Chauhan (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932477#comment-16932477
 ] 

Ashutosh Chauhan commented on HIVE-20983:
-

+1

> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Mustafa Iman
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, 
> HIVE-20983.3.patch, HIVE-20983.4.patch, HIVE-20983.5.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in 
> place, but an accurate estimate does not always result in a low number of 
> collisions.
> The hashtables which contain a very small number of items tend to lose their 
> O(1) lookup performance where there are collisions. Since collisions are easy 
> to detect within the fast hashtable implementation, a rehashing to a higher 
> size will help these small hashtables avoid collisions and go back to O(1) 
> perf.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22211) Change maven phase to generate test sources

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932471#comment-16932471
 ] 

Hive QA commented on HIVE-22211:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980577/HIVE-22211.01.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16833 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18636/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18636/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18636/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980577 - PreCommit-HIVE-Build

> Change maven phase to generate test sources
> ---
>
> Key: HIVE-22211
> URL: https://issues.apache.org/jira/browse/HIVE-22211
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22211.01.patch, HIVE-22211.01.patch
>
>
> Some protobuf files are generated in the wrong phase; so I get compile errors 
> because they are not there for eclipse...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22211) Change maven phase to generate test sources

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932405#comment-16932405
 ] 

Hive QA commented on HIVE-22211:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 53s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  xml  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18636/dev-support/hive-personality.sh
 |
| git revision | master / c05a32c |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18636/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Change maven phase to generate test sources
> ---
>
> Key: HIVE-22211
> URL: https://issues.apache.org/jira/browse/HIVE-22211
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22211.01.patch, HIVE-22211.01.patch
>
>
> Some protobuf files are generated in the wrong phase; so I get compile errors 
> because they are not there for eclipse...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932380#comment-16932380
 ] 

Hive QA commented on HIVE-22038:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980576/HIVE-22038.02.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 16762 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
 (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
 (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
 (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics 
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps 
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse 
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError 
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty 
(batchId=233)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.org.apache.hadoop.hive.ql.TestWarehouseExternalDir
 (batchId=274)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testExternalDefaultPaths 
(batchId=274)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testQueueingWithThreads 
(batchId=319)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18635/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18635/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18635/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980576 - PreCommit-HIVE-Build

> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch, 
> HIVE-22038.02.patch, HIVE-22038.02.patch, HIVE-22038.02.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-21875) Implement drop partition related methods on temporary tables

2019-09-18 Thread Peter Vary (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Vary updated HIVE-21875:
--
Fix Version/s: 4.0.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

Pushed to master.
Thanks for the patch [~lpinter]!

> Implement drop partition related methods on temporary tables
> 
>
> Key: HIVE-21875
> URL: https://issues.apache.org/jira/browse/HIVE-21875
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21875.01.patch, HIVE-21875.02.patch, 
> HIVE-21875.03.patch, HIVE-21875.04.patch, HIVE-21875.05.patch, 
> HIVE-21875.06.patch
>
>
> IMetaStoreClient exposes the following methods related to dropping partitions:
> {code:java}
> boolean dropPartition(String db_name, String tbl_name, List 
> part_vals, boolean deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name, 
> List part_vals, boolean deleteData);
> boolean dropPartition(String db_name, String tbl_name, List 
> part_vals, PartitionDropOptions options);
> boolean dropPartition(String catName, String db_name, String tbl_name, 
> List part_vals, PartitionDropOptions options);
> List dropPartitions(String dbName, String tblName, 
> List> partExprs, boolean deleteData, boolean 
> ifExists);
> List dropPartitions(String catName, String dbName, String tblName, 
> List> partExprs, boolean deleteData, boolean 
> ifExists);
> List dropPartitions(String dbName, String tblName, 
> List> partExprs, boolean deleteData, boolean 
> ifExists, boolean needResults);
> List dropPartitions(String catName, String dbName, String tblName, 
> List> partExprs, boolean deleteData, boolean 
> ifExists, boolean needResults);
> List dropPartitions(String dbName, String tblName, 
> List> partExprs, PartitionDropOptions options);
> List dropPartitions(String catName, String dbName, String tblName, 
> List> partExprs, PartitionDropOptions options);
> boolean dropPartition(String db_name, String tbl_name, String name, boolean 
> deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name, String 
> name, boolean deleteData){code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932328#comment-16932328
 ] 

Hive QA commented on HIVE-22038:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
5s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m  1s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18635/dev-support/hive-personality.sh
 |
| git revision | master / 79e7d2c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18635/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch, 
> HIVE-22038.02.patch, HIVE-22038.02.patch, HIVE-22038.02.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932307#comment-16932307
 ] 

Hive QA commented on HIVE-22213:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980565/HIVE-22213.1.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18634/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18634/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18634/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL 
https://issues.apache.org/jira/secure/attachment/12980565/HIVE-22213.1.patch 
was found in seen patch url's cache and a test was probably run already on it. 
Aborting...
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980565 - PreCommit-HIVE-Build

> TxnHander cleanupRecords should only clean records belonging to default 
> catalog
> ---
>
> Key: HIVE-22213
> URL: https://issues.apache.org/jira/browse/HIVE-22213
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22213.1.patch
>
>
> Currently it removes record for given database and given table without 
> checking for the catalog, as a result it can end up removing records when it 
> shouldn't. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932305#comment-16932305
 ] 

Hive QA commented on HIVE-22213:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980565/HIVE-22213.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 985 failed/errored test(s), 16763 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
 (batchId=297)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_bloom_filter_orc_file_dump]
 (batchId=94)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_insert_overwrite] 
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_join] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin] 
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_nullscan] 
(batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats2] (batchId=52)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats3] (batchId=16)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats4] (batchId=67)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats5] (batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_stats] (batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_subquery] 
(batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_directories_test]
 (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats] 
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_vectorization] 
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_vectorization_partition]
 (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_vectorization_project]
 (batchId=21)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_view_delete] 
(batchId=37)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_table_column_stats]
 (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_tbl_part] 
(batchId=54)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[annotate_stats_part] 
(batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_delete] 
(batchId=92)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_delete_own_table]
 (batchId=73)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_update] 
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[authorization_update_own_table]
 (batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[autoColumnStats_4] 
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join15] (batchId=17)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[avro_compression_enabled_native]
 (batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_transactional_full_acid]
 (batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_transactional_insert_only]
 (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas] (batchId=7)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_compact1] 
(batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_compact2] 
(batchId=45)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_compact3] 
(batchId=56)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks] 
(batchId=87)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_all_non_partitioned]
 (batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_all_partitioned] 
(batchId=31)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_orig_table] 
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_where_no_match] 
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_where_non_partitioned]
 (batchId=43)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_where_partitioned]
 (batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[delete_whole_partition] 
(batchId=11)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_opt_bucketing]
 (batchId=95)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[escape_clusterby1] 
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_locks] 
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[filter_join_breaktask] 
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[groupby7_map_skew] 
(batchId=48)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_acid_dynamic_partition]
 (batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_nonacid_from_acid]
 (batchId=82)

[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected

2019-09-18 Thread Himanshu Mishra (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Himanshu Mishra updated HIVE-22201:
---
Attachment: HIVE-22201.03.patch
Status: Patch Available  (was: Open)

> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws 
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch, 
> HIVE-22201.03.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end 
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1, 
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected

2019-09-18 Thread Himanshu Mishra (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Himanshu Mishra updated HIVE-22201:
---
Status: Open  (was: Patch Available)

> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws 
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end 
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1, 
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22213) TxnHander cleanupRecords should only clean records belonging to default catalog

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932271#comment-16932271
 ] 

Hive QA commented on HIVE-22213:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  1m 
11s{color} | {color:blue} standalone-metastore/metastore-server in master has 
178 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
27s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
23s{color} | {color:red} standalone-metastore/metastore-server: The patch 
generated 1 new + 533 unchanged - 0 fixed = 534 total (was 533) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m 
20s{color} | {color:red} standalone-metastore/metastore-server generated 2 new 
+ 178 unchanged - 0 fixed = 180 total (was 178) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 30s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
|  |  Null passed for non-null parameter of 
org.apache.hadoop.hive.metastore.txn.TxnStore.cleanupRecords(HiveObjectType, 
Database, Table, Iterator) in 
org.apache.hadoop.hive.metastore.AcidEventListener.onDropPartition(DropPartitionEvent)
  At AcidEventListener.java:of 
org.apache.hadoop.hive.metastore.txn.TxnStore.cleanupRecords(HiveObjectType, 
Database, Table, Iterator) in 
org.apache.hadoop.hive.metastore.AcidEventListener.onDropPartition(DropPartitionEvent)
  At AcidEventListener.java:[line 73] |
|  |  Null passed for non-null parameter of 
org.apache.hadoop.hive.metastore.txn.TxnStore.cleanupRecords(HiveObjectType, 
Database, Table, Iterator) in 
org.apache.hadoop.hive.metastore.AcidEventListener.onDropTable(DropTableEvent)  
At AcidEventListener.java:of 
org.apache.hadoop.hive.metastore.txn.TxnStore.cleanupRecords(HiveObjectType, 
Database, Table, Iterator) in 
org.apache.hadoop.hive.metastore.AcidEventListener.onDropTable(DropTableEvent)  
At AcidEventListener.java:[line 65] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18633/dev-support/hive-personality.sh
 |
| git revision | master / 79e7d2c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18633/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
 |
| findbugs | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18633/yetus/new-findbugs-standalone-metastore_metastore-server.html
 |
| modules | C: standalone-metastore/metastore-server U: 
standalone-metastore/metastore-server |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18633/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> TxnHander cleanupRecords should only clean records belonging to default 
> catalog
> 

[jira] [Updated] (HIVE-22214) Explain vectorization should disable user level explain

2019-09-18 Thread Rajesh Balamohan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajesh Balamohan updated HIVE-22214:

Attachment: HIVE-22214.1.patch

> Explain vectorization should disable user level explain
> ---
>
> Key: HIVE-22214
> URL: https://issues.apache.org/jira/browse/HIVE-22214
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22214.1.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22214) Explain vectorization should disable user level explain

2019-09-18 Thread Rajesh Balamohan (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rajesh Balamohan updated HIVE-22214:

Assignee: Rajesh Balamohan
  Status: Patch Available  (was: Open)

> Explain vectorization should disable user level explain
> ---
>
> Key: HIVE-22214
> URL: https://issues.apache.org/jira/browse/HIVE-22214
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive
>Reporter: Rajesh Balamohan
>Assignee: Rajesh Balamohan
>Priority: Trivial
> Attachments: HIVE-22214.1.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22201) ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws ArrayIndexOutOfBoundsException if no big table is selected

2019-09-18 Thread Ashutosh Chauhan (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932258#comment-16932258
 ] 

Ashutosh Chauhan commented on HIVE-22201:
-

+1
 [~himanshum] we require a green run. Can you please re-upload your patch one 
more time to get a new run.

> ConvertJoinMapJoin#checkShuffleSizeForLargeTable throws 
> ArrayIndexOutOfBoundsException if no big table is selected
> --
>
> Key: HIVE-22201
> URL: https://issues.apache.org/jira/browse/HIVE-22201
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Himanshu Mishra
>Assignee: Himanshu Mishra
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-22201.01.patch, HIVE-22201.02.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In case when {{bigTableCandidateSet}} is empty e.g. full outer join, we end 
> up calling {{checkShuffleSizeForLargeTable}} with {{bigTablePosition}} as -1, 
> resulting in {{ArrayIndexOutOfBoundsException}}.
> Also, should we return as soon as we see {{bigTableCandidateSet}} is empty ?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22202) Remove unnecessary code from ObjectStore.java

2019-09-18 Thread Jira


[ 
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932254#comment-16932254
 ] 

Hankó Gergely commented on HIVE-22202:
--

[~abstractdog]: type is always null at this point so actually pm.retrieve(null) 
was called. I've checked the implementations and the only thing that 
pm.retrieve(null) does is that it asserts that the persistence manager is open. 
But that's still done later when the query is executed.

> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
>  Issue Type: Task
>  Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch, 
> HIVE-22202.3.patch
>
>
> Remove unnecessary code
>  * remove unnecessary/unused variables
>  * remove unused private methods
>  * remove unnecessary throws declarations
>  * remove unnecessary conditionals that are guaranteed to always give the 
> same result
>  * simplify collection handling
>  ** use member sort() instead of Collections.sort()
>  ** use addAll() instead of adding one-by-one in a loop
>  * remove Javadoc that doesn't give additional information
>  * use try with resource where possible
>  * remove unnecessary explicit type arguments
>  * remove redundant warning suppression



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22202) Remove unnecessary code from ObjectStore.java

2019-09-18 Thread Laszlo Bodor (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932255#comment-16932255
 ] 

Laszlo Bodor commented on HIVE-22202:
-

thanks for the clarification
+1


> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
>  Issue Type: Task
>  Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch, 
> HIVE-22202.3.patch
>
>
> Remove unnecessary code
>  * remove unnecessary/unused variables
>  * remove unused private methods
>  * remove unnecessary throws declarations
>  * remove unnecessary conditionals that are guaranteed to always give the 
> same result
>  * simplify collection handling
>  ** use member sort() instead of Collections.sort()
>  ** use addAll() instead of adding one-by-one in a loop
>  * remove Javadoc that doesn't give additional information
>  * use try with resource where possible
>  * remove unnecessary explicit type arguments
>  * remove redundant warning suppression



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Comment Edited] (HIVE-22202) Remove unnecessary code from ObjectStore.java

2019-09-18 Thread Laszlo Bodor (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932244#comment-16932244
 ] 

Laszlo Bodor edited comment on HIVE-22202 at 9/18/19 9:42 AM:
--

[~ghanko]: thanks for this refactor
I can't see why this line is removed: 
https://github.com/abstractdog/hive/commit/3c92843f042ecdff89aa473f0f3b6bf2f7dd61a8#diff-33821c5e69c8a323e566c485d0b3db7bL948
other than that, it looks good to me


was (Author: abstractdog):
[~ghanko]: thanks for this refactor
I can't why this line is removed: 
https://github.com/abstractdog/hive/commit/3c92843f042ecdff89aa473f0f3b6bf2f7dd61a8#diff-33821c5e69c8a323e566c485d0b3db7bL948
other than that, it looks good to me

> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
>  Issue Type: Task
>  Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch, 
> HIVE-22202.3.patch
>
>
> Remove unnecessary code
>  * remove unnecessary/unused variables
>  * remove unused private methods
>  * remove unnecessary throws declarations
>  * remove unnecessary conditionals that are guaranteed to always give the 
> same result
>  * simplify collection handling
>  ** use member sort() instead of Collections.sort()
>  ** use addAll() instead of adding one-by-one in a loop
>  * remove Javadoc that doesn't give additional information
>  * use try with resource where possible
>  * remove unnecessary explicit type arguments
>  * remove redundant warning suppression



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932248#comment-16932248
 ] 

Hive QA commented on HIVE-20113:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980540/HIVE-20113.9.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16761 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning]
 (batchId=193)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18632/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18632/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18632/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980540 - PreCommit-HIVE-Build

> Shuffle avoidance: Disable 1-1 edges for sorted shuffle 
> 
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
>  Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, 
> HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, 
> HIVE-20113.5.patch, HIVE-20113.6.patch, HIVE-20113.7.patch, 
> HIVE-20113.8.patch, HIVE-20113.9.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets 
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to 
> have a final merge at all, it should open a single compressed file and write 
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to 
> disable the optimization for sorted shuffle edges and stop rewriting the 
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22202) Remove unnecessary code from ObjectStore.java

2019-09-18 Thread Laszlo Bodor (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932244#comment-16932244
 ] 

Laszlo Bodor commented on HIVE-22202:
-

[~ghanko]: thanks for this refactor
I can't why this line is removed: 
https://github.com/abstractdog/hive/commit/3c92843f042ecdff89aa473f0f3b6bf2f7dd61a8#diff-33821c5e69c8a323e566c485d0b3db7bL948
other than that, it looks good to me

> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
>  Issue Type: Task
>  Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch, 
> HIVE-22202.3.patch
>
>
> Remove unnecessary code
>  * remove unnecessary/unused variables
>  * remove unused private methods
>  * remove unnecessary throws declarations
>  * remove unnecessary conditionals that are guaranteed to always give the 
> same result
>  * simplify collection handling
>  ** use member sort() instead of Collections.sort()
>  ** use addAll() instead of adding one-by-one in a loop
>  * remove Javadoc that doesn't give additional information
>  * use try with resource where possible
>  * remove unnecessary explicit type arguments
>  * remove redundant warning suppression



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22163:
--
Status: Open  (was: Patch Available)

> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats: 
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567 
> Basic stats: COMPLETE Column stats: NONE'
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available  (was: Open)

> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats: 
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567 
> Basic stats: COMPLETE Column stats: NONE'
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Krisztian Kasa (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.5.patch

> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats: 
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567 
> Basic stats: COMPLETE Column stats: NONE'
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932214#comment-16932214
 ] 

Hive QA commented on HIVE-20113:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
 6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
9s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
56s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
1s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
41s{color} | {color:red} ql: The patch generated 1 new + 347 unchanged - 1 
fixed = 348 total (was 348) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m  9s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18632/dev-support/hive-personality.sh
 |
| git revision | master / 79e7d2c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18632/yetus/diff-checkstyle-ql.txt
 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18632/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Shuffle avoidance: Disable 1-1 edges for sorted shuffle 
> 
>
> Key: HIVE-20113
> URL: https://issues.apache.org/jira/browse/HIVE-20113
> Project: Hive
>  Issue Type: Bug
>  Components: Tez
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
>  Labels: Branch3Candidate
> Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, 
> HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, 
> HIVE-20113.5.patch, HIVE-20113.6.patch, HIVE-20113.7.patch, 
> HIVE-20113.8.patch, HIVE-20113.9.patch
>
>
> The sorted shuffle avoidance can have some issues when the shuffle data gets 
> broken up into multiple chunks on disk.
> The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to 
> have a final merge at all, it should open a single compressed file and write 
> a single index entry.
> Until the shuffle issue is resolved & a lot more testing, it is prudent to 
> disable the optimization for sorted shuffle edges and stop rewriting the 
> RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD).



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide

2019-09-18 Thread Denys Kuzmenko (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Denys Kuzmenko updated HIVE-22195:
--
Attachment: HIVE-22195.8.patch

> Configure authentication type for Zookeeper when different from the default 
> cluster wide
> 
>
> Key: HIVE-22195
> URL: https://issues.apache.org/jira/browse/HIVE-22195
> Project: Hive
>  Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch, 
> HIVE-22195.4.patch, HIVE-22195.5.patch, HIVE-22195.6.patch, 
> HIVE-22195.7.patch, HIVE-22195.8.patch
>
>
> This could be useful in case when cluster is kerberized, but Zookeeper is not.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932193#comment-16932193
 ] 

Hive QA commented on HIVE-22163:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980530/HIVE-22163.5.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16733 tests 
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=180)

[vector_interval_2.q,schema_evol_orc_acid_table_update.q,stats_date.q,metadataonly1.q,materialized_view_rewrite_10.q,vectorization_div0.q,auto_join_nulls.q,insert1_overwrite_partitions.q,schema_evol_text_nonvec_part_all_complex.q,orc_schema_evol_quoted.q,alter_merge_orc.q,vector_between_columns.q,vector_groupby_grouping_sets6.q,udaf_collect_set_2.q,update_after_multiple_inserts.q,offset_limit_ppd_optimizer.q,materialized_view_describe.q,orc_merge_incompat1.q,vectorized_parquet_types.q,vector_windowing_gby2.q,empty_array.q,results_cache_1.q,vector_identity_reuse.q,schema_evol_text_vecrow_part_all_complex_llap_io.q,external_jdbc_table4.q,schema_evol_orc_acidvec_part.q,materialized_view_create_rewrite_multi_db.q,acid_no_buckets.q,auto_sortmerge_join_9.q,vector_groupby_grouping_id2.q]
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/18631/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18631/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18631/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12980530 - PreCommit-HIVE-Build

> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats: 
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567 
> Basic stats: COMPLETE Column stats: NONE'
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932162#comment-16932162
 ] 

Hive QA commented on HIVE-22163:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m 
49s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  6m  
1s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 22s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18631/dev-support/hive-personality.sh
 |
| git revision | master / 79e7d2c |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-18631/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> CBO: Enabling CBO turns on stats estimation, even when the estimation is 
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch, 
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch, 
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch, 
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set 
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> '  alias: claims'
> '  filterExpr: (typ_c = 3) (type: boolean)'
> '  Statistics: Num rows: 1154941534 Data size: 4388777832 
> Basic stats: COMPLETE Column stats: NONE'
> '  GatherStats: false'
> '  Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> '

[jira] [Commented] (HIVE-21884) Scheduled query support

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932157#comment-16932157
 ] 

Hive QA commented on HIVE-21884:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
49s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  5m 
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  2m 
20s{color} | {color:blue} standalone-metastore/metastore-common in master has 
32 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
33s{color} | {color:blue} common in master has 65 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  1m  
7s{color} | {color:blue} standalone-metastore/metastore-server in master has 
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
56s{color} | {color:blue} ql in master has 1571 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
37s{color} | {color:blue} service in master has 49 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
39s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
45s{color} | {color:blue} itests/util in master has 51 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m 
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m 
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
38s{color} | {color:red} standalone-metastore: The patch generated 19 new + 
1743 unchanged - 15 fixed = 1762 total (was 1758) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
16s{color} | {color:red} standalone-metastore/metastore-common: The patch 
generated 1 new + 493 unchanged - 0 fixed = 494 total (was 493) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
16s{color} | {color:red} common: The patch generated 2 new + 367 unchanged - 0 
fixed = 369 total (was 367) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
 9s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
28s{color} | {color:red} standalone-metastore/metastore-server: The patch 
generated 18 new + 1250 unchanged - 15 fixed = 1268 total (was 1265) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
45s{color} | {color:red} ql: The patch generated 59 new + 710 unchanged - 0 
fixed = 769 total (was 710) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
15s{color} | {color:green} service: The patch generated 0 new + 38 unchanged - 
1 fixed = 38 total (was 39) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  3m 
13s{color} | {color:red} root: The patch generated 97 new + 2905 unchanged - 16 
fixed = 3002 total (was 2921) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
19s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
27s{color} | {color:red} itests/hive-unit: The patch generated 17 new + 0 
unchanged - 0 fixed = 17 total 

[jira] [Commented] (HIVE-21884) Scheduled query support

2019-09-18 Thread Hive QA (Jira)


[ 
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932137#comment-16932137
 ] 

Hive QA commented on HIVE-21884:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980521/HIVE-21884.12.patch

{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 46 failed/errored test(s), 16772 tests 
executed
*Failed tests:*
{noformat}
TestCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=60)

[timestamp_lazy.q,cbo_windowing.q,special_character_in_tabnames_2.q,udf_aes_decrypt.q,masking_3.q,subquery_corr.q,union13.q,groupby5_map.q,dynamic_rdd_cache.q,groupby6.q,vector_and_or_scalar_col.q,materialized_view_rewrite_ssb_2.q,infer_bucket_sort_convert_join.q,exim_01_nonpart.q,semijoin2.q,parallel_orderby.q,udf_translate.q,rand_partitionpruner2.q,spark_dynamic_partition_pruning_recursive_mapjoin.q,alter_table_stats_status.q,exim_14_managed_location_over_existing.q,auto_sortmerge_join_16.q,union_remove_3.q,input11.q,avro_native.q,bucket_num_reducers_acid.q,windowing_order_null.q,schema_evol_undecorated.q,ptf_streaming.q,loadpart1.q]
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] 
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq] 
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join_hash]
 (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_reduce_side]
 (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_case_when_2]
 (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_expressions]
 (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_distinct_2]
 (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_udf_string_to_boolean]
 (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet]
 (batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing] 
(batchId=176)
org.apache.hadoop.hive.metastore.tools.schematool.TestSchemaToolForMetastore.testMetastoreDbPropertiesAfterUpgrade
 (batchId=227)
org.apache.hadoop.hive.metastore.tools.schematool.TestSchemaToolForMetastore.testValidateSchemaTables
 (batchId=227)
org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.testScheduledQueryExecutionImpersonation
 (batchId=277)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testCancelRenewTokenFlow 
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testConnection 
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValid (batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValidNeg 
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeProxyAuth 
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeTokenAuth 
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNoKrbSASLTokenAuthNeg 
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLAuth 
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLFullNameAuth
 (batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testProxyAuth 
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testRenewDelegationToken 
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testTokenAuth 
(batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testCancelRenewTokenFlow 
(batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValid (batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValidNeg (batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeProxyAuth 
(batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth 
(batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testRenewDelegationToken 
(batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testTokenAuth (batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testCancelRenewTokenFlow 
(batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testConnection 
(batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValid (batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValidNeg 
(batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeProxyAuth 
(batchId=298)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeTokenAuth 
(batchId=298)

[jira] [Updated] (HIVE-22210) Vectorization may reuse computation output columns involved in filtering

2019-09-18 Thread Zoltan Haindrich (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-22210:

Fix Version/s: 4.0.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

pushed to master. Thank you [~abstractdog] for reviewing the changes!

> Vectorization may reuse computation output columns involved in filtering
> 
>
> Key: HIVE-22210
> URL: https://issues.apache.org/jira/browse/HIVE-22210
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22210.01.patch, HIVE-22210.02.patch
>
>
> running the following test with TestMiniLlapLocalCliDriver leads to an 
> unexpected results; the coalesce calculated inside the subquery has a value 
> of 1 instead of the correct(922) value.
> {code}
> drop table if exists  u_table_4;
> create table u_table_4(smallint_col_22 smallint, int_col_5 int);
> insert into u_table_4 values(238,922);
> drop table u_table_7;
> create table u_table_7 ( bigint_col_3 bigint, int_col_10 int);
> insert into u_table_7 values (571,198);
> drop table u_table_19;
> create table u_table_19 (bigint_col_18 bigint ,int_col_19 int, STRING_COL_7 
> string);
> insert into u_table_19 values (922,5,'500');
> set hive.mapjoin.full.outer=true;
> set hive.auto.convert.join=true;
> set hive.query.results.cache.enabled=false;
> set hive.merge.nway.joins=true;
> set hive.vectorized.execution.enabled=true;
> --explain analyze
>  SELECT
> a5.int_col,
>   922 as expected,
>   COALESCE(a5.int_col, a5.aa) as expected2,
>   a5.int_col_3 as reality
> FROMu_table_19 a1 
> FULL OUTER JOIN 
> ( 
>SELECT a2.int_col_5 AS int_col, 
>   a2.smallint_col_22 as aa,
>   COALESCE(a2.int_col_5, a2.smallint_col_22) AS 
> int_col_3 
>FROM   u_table_4 a2
> ) a5 
> ON  ( 
> a1.bigint_col_18) = (a5.int_col_3) 
> INNER JOIN 
> ( 
>  SELECT   a3.bigint_col_3 
>   AS int_col,
>   Cast (COALESCE(a3.bigint_col_3, 
> a3.bigint_col_3, a3.int_col_10) AS BIGINT) * Cast (a3.bigint_col_3 AS BIGINT) 
> AS int_col_3
>  FROM u_table_7 a3 
>  WHEREbigint_col_3=571 
> ) a4
> ON  (a1.int_col_19=5) 
> OR  ((a5.int_col_3) IN (a4.int_col, 10)) 
> where
>   a1.STRING_COL_7='500'
> ORDER BYint_col DESC nulls last limit 100
> ;
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22211) Change maven phase to generate test sources

2019-09-18 Thread Zoltan Haindrich (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-22211:

Attachment: HIVE-22211.01.patch

> Change maven phase to generate test sources
> ---
>
> Key: HIVE-22211
> URL: https://issues.apache.org/jira/browse/HIVE-22211
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22211.01.patch, HIVE-22211.01.patch
>
>
> Some protobuf files are generated in the wrong phase; so I get compile errors 
> because they are not there for eclipse...



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions

2019-09-18 Thread Zoltan Haindrich (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-22038:

Attachment: HIVE-22038.02.patch

> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch, HIVE-22038.02.patch, 
> HIVE-22038.02.patch, HIVE-22038.02.patch, HIVE-22038.02.patch
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)