[jira] [Commented] (HIVE-19656) Upgrade Hive to PARQUET 1.10.0

[Igor Kryvenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484185#comment-16484185
 ] 

Igor Kryvenko commented on HIVE-19656:
--

Is it the clone of HIVE-19464 ?

> Upgrade Hive to PARQUET 1.10.0
> --
>
> Key: HIVE-19656
> URL: https://issues.apache.org/jira/browse/HIVE-19656
> Project: Hive
>  Issue Type: Improvement
>  Components: Serializers/Deserializers
>Affects Versions: 3.1.0
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
>
> In the future, the new Parquet logical types for the timestamp type should be 
> introduced to Hive. The implementation of these logical types is planned to 
> be released in the next Parquet version. Before this we should upgrade to 
> Parquet version 1.10.0 which is already released.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18513) Query results caching

[Thai Bui (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484190#comment-16484190
 ] 

Thai Bui commented on HIVE-18513:
-

Thanks for the quick reply. That’s a good news . I can just enable that
flag for all the external tables. Our tables are immutable and the changing
ones are handled with new partitions so this should work.

However, we have a special case where we would drop the existing table and
recreate it with a new external location sometimes. Will that invalidate
the existing cache?


-- 
Thai


> Query results caching
> -
>
> Key: HIVE-18513
> URL: https://issues.apache.org/jira/browse/HIVE-18513
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18513.1.patch, HIVE-18513.2.patch, 
> HIVE-18513.3.patch, HIVE-18513.4.patch, HIVE-18513.5.patch, HIVE-18513.6.patch
>
>
> Add a query results cache that can save the results of an executed Hive query 
> for reuse on subsequent queries. This may be useful in cases where the same 
> query is issued many times, since Hive can return back the results of a 
> cached query rather than having to execute the full query on the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets

[Zhang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Li updated HIVE-19653:

Description: 
Consider the following query:
{code:java}
CREATE TABLE T1(a STRING, b STRING, s BIGINT);
INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);

SELECT * FROM (
SELECT a, b, sum(s)
FROM T1
GROUP BY a, b GROUPING SETS ((), (a), (b), (a, b))
) t WHERE a IS NOT NULL;
{code}
When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will 
output:
{code:java}
NULLNULL123456
NULL123456
NULL123456
123456
{code}
We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.

When performing PPD optimization for a GBY operator, we should make sure all 
grouping sets contains the processing expr before pushdown. otherwise the expr 
value after GBY is changed and the result is wrong.

  was:
Consider the following query:
{code:java}
CREATE TABLE T1(a STRING, b STRING, s BIGINT);
INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);

SELECT * FROM (
SELECT a, b, sum(s)
FROM T1
GROUP BY a, b GROUPING SETS ((a), (a, b))
) t WHERE a IS NOT NULL;
{code}
When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will 
output:
{code:java}
NULLNULL123456
NULL123456
NULL123456
123456
{code}
We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.

When performing PPD optimization for a GBY operator, we should make sure all 
grouping sets contains the processing expr before pushdown. otherwise the expr 
value after GBY is changed and the result is wrong.


> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
>  Issue Type: Bug
>  Components: Logical Optimizer
>Affects Versions: 3.1.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19653.patch
>
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((), (a), (b), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will 
> output:
> {code:java}
> NULL  NULL123456
> NULL  123456
>   NULL123456
>   123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all 
> grouping sets contains the processing expr before pushdown. otherwise the 
> expr value after GBY is changed and the result is wrong.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19656) Upgrade Hive to PARQUET 1.10.0

[Marta Kuczora (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marta Kuczora updated HIVE-19656:
-
Description: 
In the future, the new Parquet logical types for the timestamp type should be 
introduced to Hive. The implementation of these logical types is planned to be 
released in the next Parquet version. Before this we should upgrade to Parquet 
version 1.10.0 which is already released.

 

  was:
In the future, the new Parquet logical types for the timestamp type should be 
introduced to Hive. The implementation of these logical types is planned to be 
released in the next Parquet version. Before this we should upgrade to the 
Parquet version 1.10.0 which is already released.

 


> Upgrade Hive to PARQUET 1.10.0
> --
>
> Key: HIVE-19656
> URL: https://issues.apache.org/jira/browse/HIVE-19656
> Project: Hive
>  Issue Type: Improvement
>  Components: Serializers/Deserializers
>Affects Versions: 3.1.0
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
>
> In the future, the new Parquet logical types for the timestamp type should be 
> introduced to Hive. The implementation of these logical types is planned to 
> be released in the next Parquet version. Before this we should upgrade to 
> Parquet version 1.10.0 which is already released.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19638) Configuration not passed to ORC Reader.Options

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484179#comment-16484179
 ] 

Hive QA commented on HIVE-19638:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
48s{color} | {color:blue} ql in master has 2322 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
38s{color} | {color:red} ql: The patch generated 2 new + 341 unchanged - 0 
fixed = 343 total (was 341) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m  3s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11137/dev-support/hive-personality.sh
 |
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11137/yetus/diff-checkstyle-ql.txt
 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11137/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Configuration not passed to ORC Reader.Options
> --
>
> Key: HIVE-19638
> URL: https://issues.apache.org/jira/browse/HIVE-19638
> Project: Hive
>  Issue Type: Bug
>  Components: Configuration, File Formats, ORC
>Affects Versions: 2.3.0, 2.3.1, 2.3.2
>Reporter: Rentao Wu
>Assignee: Rentao Wu
>Priority: Major
> Attachments: HIVE-19638.patch
>
>
> Configuration is not passed to ORC's Reader.Option in OrcFileInputFormat 
> which causes some [ORC 
> configurations|https://github.com/apache/orc/blob/master/java/core/src/java/org/apache/orc/Reader.java#L170-L176]
>  to not be able to be picked up.
> Related issues:
> For example, the ORC upgrade in Hive 2.3.x changed schema evolution from 
> positional to column name matching. A backwards compatibility configuration 
> "orc.force.positional.evolution" could be set in ORC Reader.Options by 
> [ORC-120|https://issues.apache.org/jira/browse/ORC-120] however it could not 
> be picked up resulting in null values when querying ORC tables where the 
> column names do not match.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19656) Upgrade Hive to PARQUET 1.10.0

[Marta Kuczora (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marta Kuczora updated HIVE-19656:
-
Component/s: Serializers/Deserializers

> Upgrade Hive to PARQUET 1.10.0
> --
>
> Key: HIVE-19656
> URL: https://issues.apache.org/jira/browse/HIVE-19656
> Project: Hive
>  Issue Type: Improvement
>  Components: Serializers/Deserializers
>Affects Versions: 3.1.0
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
>
> In the future, the new Parquet logical types for the timestamp type should be 
> introduced to Hive. The implementation of these logical types is planned to 
> be released in the next Parquet version. Before this we should upgrade to the 
> Parquet version 1.10.0 which is already released.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19656) Upgrade Hive to PARQUET 1.10.0

[Marta Kuczora (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marta Kuczora reassigned HIVE-19656:



> Upgrade Hive to PARQUET 1.10.0
> --
>
> Key: HIVE-19656
> URL: https://issues.apache.org/jira/browse/HIVE-19656
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 3.1.0
>Reporter: Marta Kuczora
>Assignee: Marta Kuczora
>Priority: Major
>
> In the future, the new Parquet logical types for the timestamp type should be 
> introduced to Hive. The implementation of these logical types is planned to 
> be released in the next Parquet version. Before this we should upgrade to the 
> Parquet version 1.10.0 which is already released.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19655) Mask stats for TestMiniLlapLocalCliDriver#smb_mapjoin_15

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez reassigned HIVE-19655:
--


> Mask stats for TestMiniLlapLocalCliDriver#smb_mapjoin_15
> 
>
> Key: HIVE-19655
> URL: https://issues.apache.org/jira/browse/HIVE-19655
> Project: Hive
>  Issue Type: Test
>  Components: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Flaky sometimes. We care about algorithm selection, it should be OK to mask.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19654) Change tmp staging mapred directory for TestBlobstoreCliDriver

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484166#comment-16484166
 ] 

Jesus Camacho Rodriguez commented on HIVE-19654:


[~prasanth_j], could you review? This is similar to HIVE-19626, apparently 
TestBlobstoreCliDriver has its own {{hive-site.xml}} file.

> Change tmp staging mapred directory for TestBlobstoreCliDriver
> --
>
> Key: HIVE-19654
> URL: https://issues.apache.org/jira/browse/HIVE-19654
> Project: Hive
>  Issue Type: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19654.patch
>
>
> Similar to HIVE-19626.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (HIVE-19654) Change tmp staging mapred directory for TestBlobstoreCliDriver

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484166#comment-16484166
 ] 

Jesus Camacho Rodriguez edited comment on HIVE-19654 at 5/22/18 3:48 PM:
-

[~prasanth_j], could you review? This is similar to HIVE-19626, apparently 
TestBlobstoreCliDriver has its own {{hive-site.xml}} file.

https://builds.apache.org/job/PreCommit-HIVE-Build/11131/testReport/junit/org.apache.hadoop.hive.cli/TestBlobstoreCliDriver/testCliDriver_insert_into_dynamic_partitions_/


was (Author: jcamachorodriguez):
[~prasanth_j], could you review? This is similar to HIVE-19626, apparently 
TestBlobstoreCliDriver has its own {{hive-site.xml}} file.

> Change tmp staging mapred directory for TestBlobstoreCliDriver
> --
>
> Key: HIVE-19654
> URL: https://issues.apache.org/jira/browse/HIVE-19654
> Project: Hive
>  Issue Type: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19654.patch
>
>
> Similar to HIVE-19626.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19654) Change tmp staging mapred directory for TestBlobstoreCliDriver

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-19654:
---
Attachment: HIVE-19654.patch

> Change tmp staging mapred directory for TestBlobstoreCliDriver
> --
>
> Key: HIVE-19654
> URL: https://issues.apache.org/jira/browse/HIVE-19654
> Project: Hive
>  Issue Type: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19654.patch
>
>
> Similar to HIVE-19626.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work started] (HIVE-19654) Change tmp staging mapred directory for TestBlobstoreCliDriver

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-19654 started by Jesus Camacho Rodriguez.
--
> Change tmp staging mapred directory for TestBlobstoreCliDriver
> --
>
> Key: HIVE-19654
> URL: https://issues.apache.org/jira/browse/HIVE-19654
> Project: Hive
>  Issue Type: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19654.patch
>
>
> Similar to HIVE-19626.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19654) Change tmp staging mapred directory for TestBlobstoreCliDriver

[Jesus Camacho Rodriguez (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez reassigned HIVE-19654:
--


> Change tmp staging mapred directory for TestBlobstoreCliDriver
> --
>
> Key: HIVE-19654
> URL: https://issues.apache.org/jira/browse/HIVE-19654
> Project: Hive
>  Issue Type: Test
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> Similar to HIVE-19626.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19652) Incorrect predicate pushdown for groupby with grouping sets

[Zhang Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484161#comment-16484161
 ] 

Zhang Li commented on HIVE-19652:
-

[~jcamachorodriguez] I'm so sorry I made a duplicated issue, please goto 
https://issues.apache.org/jira/browse/HIVE-19653

> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19652
> URL: https://issues.apache.org/jira/browse/HIVE-19652
> Project: Hive
>  Issue Type: Bug
>  Components: Logical Optimizer
>Affects Versions: 3.1.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Fix For: 3.1.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets

[Zhang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Li updated HIVE-19653:

Attachment: (was: HIVE-19653.patch)

> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
>  Issue Type: Bug
>  Components: Logical Optimizer
>Affects Versions: 3.1.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19653.patch
>
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((a), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will 
> output:
> {code:java}
> NULL  NULL123456
> NULL  123456
>   NULL123456
>   123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all 
> grouping sets contains the processing expr before pushdown. otherwise the 
> expr value after GBY is changed and the result is wrong.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19617) Rename test tables to avoid collisions during execution in batches

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484157#comment-16484157
 ] 

Jesus Camacho Rodriguez commented on HIVE-19617:


[~kgyrtkirk], I cannot repro HIVE-18700 in latest master, I guess that's a good 
sign :)

> Rename test tables to avoid collisions during execution in batches
> --
>
> Key: HIVE-19617
> URL: https://issues.apache.org/jira/browse/HIVE-19617
> Project: Hive
>  Issue Type: Test
>  Components: Testing Infrastructure
>Affects Versions: 3.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Fix For: 3.1.0
>
> Attachments: HIVE-19617.01.patch, HIVE-19617.02.patch, 
> HIVE-19617.03.patch, HIVE-19617.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets

[Zhang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Li updated HIVE-19653:

Attachment: HIVE-19653.patch
Status: Patch Available  (was: Open)

> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
>  Issue Type: Bug
>  Components: Logical Optimizer
>Affects Versions: 3.1.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19653.patch, HIVE-19653.patch
>
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((a), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will 
> output:
> {code:java}
> NULL  NULL123456
> NULL  123456
>   NULL123456
>   123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all 
> grouping sets contains the processing expr before pushdown. otherwise the 
> expr value after GBY is changed and the result is wrong.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets

[Zhang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Li updated HIVE-19653:

Attachment: HIVE-19653.patch

> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
>  Issue Type: Bug
>  Components: Logical Optimizer
>Affects Versions: 3.1.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19653.patch
>
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((a), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will 
> output:
> {code:java}
> NULL  NULL123456
> NULL  123456
>   NULL123456
>   123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all 
> grouping sets contains the processing expr before pushdown. otherwise the 
> expr value after GBY is changed and the result is wrong.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HIVE-19652) Incorrect predicate pushdown for groupby with grouping sets

[Zhang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Li resolved HIVE-19652.
-
Resolution: Duplicate

> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19652
> URL: https://issues.apache.org/jira/browse/HIVE-19652
> Project: Hive
>  Issue Type: Bug
>  Components: Logical Optimizer
>Affects Versions: 3.1.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Fix For: 3.1.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19653) Incorrect predicate pushdown for groupby with grouping sets

[Zhang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Li reassigned HIVE-19653:
---


> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19653
> URL: https://issues.apache.org/jira/browse/HIVE-19653
> Project: Hive
>  Issue Type: Bug
>  Components: Logical Optimizer
>Affects Versions: 3.1.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Fix For: 3.1.0
>
>
> Consider the following query:
> {code:java}
> CREATE TABLE T1(a STRING, b STRING, s BIGINT);
> INSERT OVERWRITE TABLE T1 VALUES ('', '', 123456);
> SELECT * FROM (
> SELECT a, b, sum(s)
> FROM T1
> GROUP BY a, b GROUPING SETS ((a), (a, b))
> ) t WHERE a IS NOT NULL;
> {code}
> When hive.optimize.ppd is enabled (and hive.cbo.enable=false), the query will 
> output:
> {code:java}
> NULL  NULL123456
> NULL  123456
>   NULL123456
>   123456
> {code}
> We can see the predicate "a IS NOT NULL" takes no effect, which is incorrect.
> When performing PPD optimization for a GBY operator, we should make sure all 
> grouping sets contains the processing expr before pushdown. otherwise the 
> expr value after GBY is changed and the result is wrong.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HIVE-17370) Some tests are failing with java.io.FileNotFoundException: File file:/tmp/hadoop/mapred/...

[Peter Vary (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-17370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Vary resolved HIVE-17370.
---
   Resolution: Duplicate
Fix Version/s: 3.1.0

Seems like a duplicate

> Some tests are failing with java.io.FileNotFoundException: File 
> file:/tmp/hadoop/mapred/...
> ---
>
> Key: HIVE-17370
> URL: https://issues.apache.org/jira/browse/HIVE-17370
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Fix For: 3.1.0
>
>
> Mostly independently of the given tests, I found this flakiness in several 
> tests:
> {code}
> 2017-08-21T06:34:26,916  WARN [Thread-7815] mapred.LocalJobRunner: 
> job_local245886038_0060
> java.io.FileNotFoundException: File 
> file:/tmp/hadoop/mapred/staging/hiveptest245886038/.staging/job_local245886038_0060/job.splitmetainfo
>  does not exist
>   at 
> org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:635)
>  ~[hadoop-common-2.8.0.jar:?]
>   at 
> org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:861)
>  ~[hadoop-common-2.8.0.jar:?]
>   at 
> org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:625)
>  ~[hadoop-common-2.8.0.jar:?]
>   at 
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:435)
>  ~[hadoop-common-2.8.0.jar:?]
>   at 
> org.apache.hadoop.mapreduce.split.SplitMetaInfoReader.readSplitMetaInfo(SplitMetaInfoReader.java:51)
>  ~[hadoop-mapreduce-client-core-2.8.0.jar:?]
>   at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:534) 
> [hadoop-mapreduce-client-common-2.8.0.jar:?]
> {code}
> One example is:
> https://builds.apache.org/job/PreCommit-HIVE-Build/6469/testReport/org.apache.hadoop.hive.cli/TestAccumuloCliDriver/testCliDriver_accumulo_queries_/
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18513) Query results caching

[Jason Dere (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484147#comment-16484147
 ] 

Jason Dere commented on HIVE-18513:
---

The cache is disabled for queries involving external tables.
It is disabled by default for non-transactional tables as well, but can be 
enabled for those with hive.query.results.cache.nontransactional.tables.enabled.

> Query results caching
> -
>
> Key: HIVE-18513
> URL: https://issues.apache.org/jira/browse/HIVE-18513
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18513.1.patch, HIVE-18513.2.patch, 
> HIVE-18513.3.patch, HIVE-18513.4.patch, HIVE-18513.5.patch, HIVE-18513.6.patch
>
>
> Add a query results cache that can save the results of an executed Hive query 
> for reuse on subsequent queries. This may be useful in cases where the same 
> query is issued many times, since Hive can return back the results of a 
> cached query rather than having to execute the full query on the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19558) HiveAuthorizationProviderBase gets catalog name from config rather than db object

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-19558:
--
Attachment: HIVE-19558.1take4.patch

> HiveAuthorizationProviderBase gets catalog name from config rather than db 
> object
> -
>
> Key: HIVE-19558
> URL: https://issues.apache.org/jira/browse/HIVE-19558
> Project: Hive
>  Issue Type: Bug
>  Components: Authorization
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.1
>
> Attachments: HIVE-19558.1take2.patch, HIVE-19558.1take3.patch, 
> HIVE-19558.1take4.patch, HIVE-19558.patch
>
>
> HiveAuthorizationProviderBase.getDatabase uses just the database name to 
> fetch the database, relying on getDefaultCatalog() to fetch the catalog name 
> from the conf file.  This does not work when the client has passed in an 
> object for a different catalog.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19576) IHMSHandler.getTable not always fetching the right catalog

[Alan Gates (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alan Gates updated HIVE-19576:
--
Attachment: HIVE-19576.1take4.patch

> IHMSHandler.getTable not always fetching the right catalog
> --
>
> Key: HIVE-19576
> URL: https://issues.apache.org/jira/browse/HIVE-19576
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.1
>
> Attachments: HIVE-19576.1take2.patch, HIVE-19576.1take3.patch, 
> HIVE-19576.1take4.patch, HIVE-19576.patch
>
>
> {{IHMSHandler.get_table_core(String dbName, String tableName)}} fetches the 
> catalog name from the conf.  This causes issues when doing an operation where 
> the catalog is known and does not match the default provided in the 
> configuration file (e.g. adding a partition).  This method should be removed 
> and callers forced to use {{IHMSHandler.get_table_core(String catName, String 
> dbName, String tableName)}} instead since callers will know whether they have 
> the catalog name or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19617) Rename test tables to avoid collisions during execution in batches

[Zoltan Haindrich (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484145#comment-16484145
 ] 

Zoltan Haindrich commented on HIVE-19617:
-

actually...I've picked up some experties in looking into these...I've a qtest 
bisecter which narrows down to 2 qtests which just don't like each-other...it 
takes time and effort to identify each of them...
not sure if this is related to table names; but there are problems like: 
HIVE-18700
the table protector is named: EnforceReadOnlyTables and the schema 
scanner/table dropper is inside qtestutil

will see if this actually helps or not..I hope so



> Rename test tables to avoid collisions during execution in batches
> --
>
> Key: HIVE-19617
> URL: https://issues.apache.org/jira/browse/HIVE-19617
> Project: Hive
>  Issue Type: Test
>  Components: Testing Infrastructure
>Affects Versions: 3.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Fix For: 3.1.0
>
> Attachments: HIVE-19617.01.patch, HIVE-19617.02.patch, 
> HIVE-19617.03.patch, HIVE-19617.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19652) Incorrect predicate pushdown for groupby with grouping sets

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484141#comment-16484141
 ] 

Jesus Camacho Rodriguez commented on HIVE-19652:


[~richox], can you add more information to the JIRA, e.g., a repro case, so we 
can look into it? Thanks

> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19652
> URL: https://issues.apache.org/jira/browse/HIVE-19652
> Project: Hive
>  Issue Type: Bug
>  Components: Logical Optimizer
>Affects Versions: 3.1.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Fix For: 3.1.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19639) a transactional Hive table cannot be imported as an external table

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484143#comment-16484143
 ] 

Hive QA commented on HIVE-19639:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924419/HIVE-19639.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14390 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/11136/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11136/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11136/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12924419 - PreCommit-HIVE-Build

> a transactional Hive table cannot be imported as an external table
> --
>
> Key: HIVE-19639
> URL: https://issues.apache.org/jira/browse/HIVE-19639
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19639.patch
>
>
> When transactional table is imported to a external table, the table should be 
> imported as a non transactional table, as external tables cannot be 
> transactional.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19454) Test failure : org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails with java.lang.AssertionError

[Eugene Koifman (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484134#comment-16484134
 ] 

Eugene Koifman commented on HIVE-19454:
---

[~djaiswal], we just voted to have green runs before commits.  Is there one?

> Test failure : 
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1 fails 
> with java.lang.AssertionError
> --
>
> Key: HIVE-19454
> URL: https://issues.apache.org/jira/browse/HIVE-19454
> Project: Hive
>  Issue Type: Bug
> Environment: uname -a
> Linux pts00433-vm40 3.10.0-862.el7.ppc64le #1 SMP Wed Mar 21 22:20:19 UTC 
> 2018 ppc64le ppc64le ppc64le GNU/Linux
>Reporter: Alisha Prabhu
>Priority: Major
> Attachments: HIVE-19454.1.patch, HIVE-19454.2.patch
>
>
> org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion1
>  refers to the same test case.
> Maven command used is : mvn -Dtest=TestTxnCommands2 test
> Error:
> {code:java}
> [INFO] Running org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3, Time elapsed: 
> 618.215 s <<< FAILURE! - in org.apache.hadoop.hive.ql.TestTxnCommands2
> [ERROR] 
> testNonAcidToAcidConversion1(org.apache.hadoop.hive.ql.TestTxnCommands2)  
> Time elapsed: 17.557 s  <<< FAILURE!
> java.lang.AssertionError
>     at org.junit.Assert.fail(Assert.java:86)
>     at org.junit.Assert.assertTrue(Assert.java:41)
>     at org.junit.Assert.assertTrue(Assert.java:52)
>     at 
> org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion1(TestTxnCommands2.java:499)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:498)
>     at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
>     at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>     at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
>     at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>     at 
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
>     at 
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>     at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
>     at 
> org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:168)
>     at org.junit.rules.RunRules.evaluate(RunRules.java:20)
>     at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
>     at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
>     at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
>     at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
>     at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
>     at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
>     at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
>     at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
>     at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
>     at 
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
>     at 
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
>     at 
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
>     at 
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
>     at 
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
>     at 
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
>     at 
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
>     at 
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> [INFO]
> [INFO] Results:
> [INFO]
> [ERROR] Failures:
> [ERROR]   TestTxnCommands2.testNonAcidToAcidConversion1:499
> [INFO]
> [ERROR] Tests run: 44, Failures: 1, Errors: 0, Skipped: 3
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19617) Rename test tables to avoid collisions during execution in batches

[Jesus Camacho Rodriguez (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484136#comment-16484136
 ] 

Jesus Camacho Rodriguez commented on HIVE-19617:


[~kgyrtkirk], thanks for the feedback. I do not think we ever had a mechanism 
to reset Hive to its initial state between tests when we chain their execution:
{code}
mvn test -Dtest=TestCliDriver -Dqfile=test1.q,test2.q 
-Dtest.output.overwrite=true
{code}
Devs have been seeing this kind of interaction among tests before, e.g., I am 
sure you have seen that many q file tests start with DROP TABLE {{t}} if {{t}} 
is going to be created just afterwards, but {{t}} should not exist if the test 
is executed in isolation.
Making the names unique is certainly a workaround, and this patch seems to help 
to get rid of the flakiness. I think the correct way to go is what you propose, 
but that needs some work to identify those changes that have happened during 
Hive execution; I do not think you want to tear down and initialize the 
metastore for each individual test execution since that will impact test 
execution time.

> Rename test tables to avoid collisions during execution in batches
> --
>
> Key: HIVE-19617
> URL: https://issues.apache.org/jira/browse/HIVE-19617
> Project: Hive
>  Issue Type: Test
>  Components: Testing Infrastructure
>Affects Versions: 3.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Fix For: 3.1.0
>
> Attachments: HIVE-19617.01.patch, HIVE-19617.02.patch, 
> HIVE-19617.03.patch, HIVE-19617.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19652) Incorrect predicate pushdown for groupby with grouping sets

[Zhang Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhang Li reassigned HIVE-19652:
---


> Incorrect predicate pushdown for groupby with grouping sets
> ---
>
> Key: HIVE-19652
> URL: https://issues.apache.org/jira/browse/HIVE-19652
> Project: Hive
>  Issue Type: Bug
>  Components: Logical Optimizer
>Affects Versions: 3.1.0
>Reporter: Zhang Li
>Assignee: Zhang Li
>Priority: Major
> Fix For: 3.1.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19651) only add arrow dependency once

[Igor Kryvenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Kryvenko updated HIVE-19651:
-
Status: Patch Available  (was: Open)

> only add arrow dependency once
> --
>
> Key: HIVE-19651
> URL: https://issues.apache.org/jira/browse/HIVE-19651
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-19651.01.patch
>
>
> in serde/pom.xml
> {code}
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> {code}
> maven is complaining:
> {code}
> [WARNING] 
> [WARNING] Some problems were encountered while building the effective model 
> for org.apache.hive:hive-serde:jar:3.1.0-SNAPSHOT
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
> be unique: org.apache.arrow:arrow-vector:jar -> duplicate declaration of 
> version ${arrow.version} @ org.apache.hive:hive-serde:[unknown-version], 
> /home/kirk/projects/hive/HIVE-19237-oid-once/serde/pom.xml, line 73, column 17
> [WARNING] 
> [WARNING] It is highly recommended to fix these problems because they 
> threaten the stability of your build.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19651) only add arrow dependency once

[Igor Kryvenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Kryvenko updated HIVE-19651:
-
Attachment: HIVE-19651.01.patch

> only add arrow dependency once
> --
>
> Key: HIVE-19651
> URL: https://issues.apache.org/jira/browse/HIVE-19651
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-19651.01.patch
>
>
> in serde/pom.xml
> {code}
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> {code}
> maven is complaining:
> {code}
> [WARNING] 
> [WARNING] Some problems were encountered while building the effective model 
> for org.apache.hive:hive-serde:jar:3.1.0-SNAPSHOT
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
> be unique: org.apache.arrow:arrow-vector:jar -> duplicate declaration of 
> version ${arrow.version} @ org.apache.hive:hive-serde:[unknown-version], 
> /home/kirk/projects/hive/HIVE-19237-oid-once/serde/pom.xml, line 73, column 17
> [WARNING] 
> [WARNING] It is highly recommended to fix these problems because they 
> threaten the stability of your build.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19651) only add arrow dependency once

[Igor Kryvenko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Kryvenko reassigned HIVE-19651:


Assignee: Igor Kryvenko

> only add arrow dependency once
> --
>
> Key: HIVE-19651
> URL: https://issues.apache.org/jira/browse/HIVE-19651
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Igor Kryvenko
>Priority: Major
>
> in serde/pom.xml
> {code}
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> {code}
> maven is complaining:
> {code}
> [WARNING] 
> [WARNING] Some problems were encountered while building the effective model 
> for org.apache.hive:hive-serde:jar:3.1.0-SNAPSHOT
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
> be unique: org.apache.arrow:arrow-vector:jar -> duplicate declaration of 
> version ${arrow.version} @ org.apache.hive:hive-serde:[unknown-version], 
> /home/kirk/projects/hive/HIVE-19237-oid-once/serde/pom.xml, line 73, column 17
> [WARNING] 
> [WARNING] It is highly recommended to fix these problems because they 
> threaten the stability of your build.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19460) Improve stats estimations for NOT IN operator

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19460:

Attachment: HIVE-19460.05.patch

> Improve stats estimations for NOT IN operator
> -
>
> Key: HIVE-19460
> URL: https://issues.apache.org/jira/browse/HIVE-19460
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19460.01wip01.patch, HIVE-19460.01wip02.patch, 
> HIVE-19460.01wip03.patch, HIVE-19460.02.patch, HIVE-19460.03.patch, 
> HIVE-19460.04.patch, HIVE-19460.05.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19651) only add arrow dependency once

[Zoltan Haindrich (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484087#comment-16484087
 ] 

Zoltan Haindrich commented on HIVE-19651:
-

[~ikryvenko] sure! it's unassigned :)

> only add arrow dependency once
> --
>
> Key: HIVE-19651
> URL: https://issues.apache.org/jira/browse/HIVE-19651
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
>
> in serde/pom.xml
> {code}
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> {code}
> maven is complaining:
> {code}
> [WARNING] 
> [WARNING] Some problems were encountered while building the effective model 
> for org.apache.hive:hive-serde:jar:3.1.0-SNAPSHOT
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
> be unique: org.apache.arrow:arrow-vector:jar -> duplicate declaration of 
> version ${arrow.version} @ org.apache.hive:hive-serde:[unknown-version], 
> /home/kirk/projects/hive/HIVE-19237-oid-once/serde/pom.xml, line 73, column 17
> [WARNING] 
> [WARNING] It is highly recommended to fix these problems because they 
> threaten the stability of your build.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19651) only add arrow dependency once

[Igor Kryvenko (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484083#comment-16484083
 ] 

Igor Kryvenko commented on HIVE-19651:
--

[~kgyrtkirk] Hi, Can I take it?

> only add arrow dependency once
> --
>
> Key: HIVE-19651
> URL: https://issues.apache.org/jira/browse/HIVE-19651
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
>
> in serde/pom.xml
> {code}
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> 
>   org.apache.arrow
>   arrow-vector
>   ${arrow.version}
> 
> {code}
> maven is complaining:
> {code}
> [WARNING] 
> [WARNING] Some problems were encountered while building the effective model 
> for org.apache.hive:hive-serde:jar:3.1.0-SNAPSHOT
> [WARNING] 'dependencies.dependency.(groupId:artifactId:type:classifier)' must 
> be unique: org.apache.arrow:arrow-vector:jar -> duplicate declaration of 
> version ${arrow.version} @ org.apache.hive:hive-serde:[unknown-version], 
> /home/kirk/projects/hive/HIVE-19237-oid-once/serde/pom.xml, line 73, column 17
> [WARNING] 
> [WARNING] It is highly recommended to fix these problems because they 
> threaten the stability of your build.
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18513) Query results caching

[Thai Bui (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484084#comment-16484084
 ] 

Thai Bui commented on HIVE-18513:
-

Hi [~jdere], this new feature is awesome. I'm glad we are implementing it in 
the next generation of Hive. I'm currently using Hive 3.1.0-SNAPSHOT but 
haven't seen this working yet, perhaps there's a couple of configurations 
parameters that I am missing. Also, does it work with EXTERNAL tables or just 
managed / ACID tables?

> Query results caching
> -
>
> Key: HIVE-18513
> URL: https://issues.apache.org/jira/browse/HIVE-18513
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18513.1.patch, HIVE-18513.2.patch, 
> HIVE-18513.3.patch, HIVE-18513.4.patch, HIVE-18513.5.patch, HIVE-18513.6.patch
>
>
> Add a query results cache that can save the results of an executed Hive query 
> for reuse on subsequent queries. This may be useful in cases where the same 
> query is issued many times, since Hive can return back the results of a 
> cached query rather than having to execute the full query on the cluster.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19617) Rename test tables to avoid collisions during execution in batches

[Zoltan Haindrich (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484078#comment-16484078
 ] 

Zoltan Haindrich commented on HIVE-19617:
-

[~jcamachorodriguez] I'm not sure; but there is a mechanism which cleans up 
garbage between tests; it used to work

I think if there are some problems because a table with the same name existed 
before - that could possibly be simptome of that the system somehow "remembers" 
the previous table ...and it might cause trouble even in production 
environments...

> Rename test tables to avoid collisions during execution in batches
> --
>
> Key: HIVE-19617
> URL: https://issues.apache.org/jira/browse/HIVE-19617
> Project: Hive
>  Issue Type: Test
>  Components: Testing Infrastructure
>Affects Versions: 3.1.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Fix For: 3.1.0
>
> Attachments: HIVE-19617.01.patch, HIVE-19617.02.patch, 
> HIVE-19617.03.patch, HIVE-19617.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19639) a transactional Hive table cannot be imported as an external table

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484077#comment-16484077
 ] 

Hive QA commented on HIVE-19639:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
51s{color} | {color:blue} ql in master has 2322 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} ql: The patch generated 0 new + 175 unchanged - 1 
fixed = 175 total (was 176) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 10s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11136/dev-support/hive-personality.sh
 |
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11136/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> a transactional Hive table cannot be imported as an external table
> --
>
> Key: HIVE-19639
> URL: https://issues.apache.org/jira/browse/HIVE-19639
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19639.patch
>
>
> When transactional table is imported to a external table, the table should be 
> imported as a non transactional table, as external tables cannot be 
> transactional.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work started] (HIVE-19649) Clean up inputs in JDBC PreparedStatement. Add unit tests.

[Mykhailo Kysliuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-19649 started by Mykhailo Kysliuk.
---
> Clean up inputs in JDBC PreparedStatement. Add unit tests.
> --
>
> Key: HIVE-19649
> URL: https://issues.apache.org/jira/browse/HIVE-19649
> Project: Hive
>  Issue Type: Test
>Reporter: Mykhailo Kysliuk
>Assignee: Mykhailo Kysliuk
>Priority: Minor
> Attachments: HIVE-19649.patch
>
>
> Add unit tests for feature that was implemented in 
> [HIVE-18788|https://issues.apache.org/jira/browse/HIVE-18788].
> The integration tests are present, but it will be useful to catch errors 
> during module build.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19632) Remove webapps directory from standalone jar

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16484024#comment-16484024
 ] 

Hive QA commented on HIVE-19632:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924407/HIVE-19632.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14390 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_format_nonpart]
 (batchId=257)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/11135/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11135/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11135/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12924407 - PreCommit-HIVE-Build

> Remove webapps directory from standalone jar
> 
>
> Key: HIVE-19632
> URL: https://issues.apache.org/jira/browse/HIVE-19632
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19632.1.patch, HIVE-19632.2.patch
>
>
> JDBC standalone jar contains webapps static files which just adds to the jar 
> size and are not required by the clients. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19649) Clean up inputs in JDBC PreparedStatement. Add unit tests.

[Mykhailo Kysliuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mykhailo Kysliuk updated HIVE-19649:

Attachment: HIVE-19649.patch

> Clean up inputs in JDBC PreparedStatement. Add unit tests.
> --
>
> Key: HIVE-19649
> URL: https://issues.apache.org/jira/browse/HIVE-19649
> Project: Hive
>  Issue Type: Test
>Reporter: Mykhailo Kysliuk
>Assignee: Mykhailo Kysliuk
>Priority: Minor
> Attachments: HIVE-19649.patch
>
>
> Add unit tests for feature that was implemented in 
> [HIVE-18788|https://issues.apache.org/jira/browse/HIVE-18788].
> The integration tests are present, but it will be useful to catch errors 
> during module build.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19557) stats: filters for dates are not taking advantage of min/max values

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19557:

Attachment: HIVE-19557.03.patch

> stats: filters for dates are not taking advantage of min/max values
> ---
>
> Key: HIVE-19557
> URL: https://issues.apache.org/jira/browse/HIVE-19557
> Project: Hive
>  Issue Type: Bug
>  Components: Query Planning
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19557.01.patch, HIVE-19557.02.patch, 
> HIVE-19557.03.patch
>
>
> in StatsRulesProcFactory 
> [https://github.com/apache/hive/blob/ab189f54047bbf6beeeaf8d0dcfd5fbe92e465fb/ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java#L754|dates
>  are assumed to be an integer]; however this is currently not true - and the 
> resulting exception is handled as a default case... for N/3
> {code}
> set hive.explain.user=true;
> create table d1(d date);
> --  tblproperties('transactional'='false');
> insert into d1 values
> ('2010-10-01'),
> ('2010-10-02'),
> ('2010-10-03'),
> ('2010-10-04'),
> ('2010-10-05'),
> ('2010-10-06'),
> ('2010-10-07'),
> ('2010-10-08'),
> ('2010-10-09'),
> ('2010-10-10');
> analyze table d1 compute statistics for columns;
> desc formatted d1;
> desc formatted d1 d;
> explain
> select 'stats: FIL ~0 read',count(1) from d1 where d < '2010-03-01';
> explain
> select 'stats: FIL estimate some read',count(1) from d1 where d < 
> '2010-10-03';
> explain
> select 'stats: FIL estimate all read',count(1) from d1 where d < '2010-11-03';
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19103) Nested structure Projection Push Down in Hive with ORC

[Ashish Sharma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Sharma updated HIVE-19103:
-
Component/s: (was: ORC)
 Hive

> Nested structure Projection Push Down in Hive with ORC
> --
>
> Key: HIVE-19103
> URL: https://issues.apache.org/jira/browse/HIVE-19103
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive, ORC
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Critical
>  Labels: pull-request-available
> Fix For: 2.3.2
>
> Attachments: HIVE-19103-0.patch, HIVE-19103-1.patch, 
> HIVE-19103-2.patch
>
>
> Reading required columns only in nested structure schema
> Example - 
> *Current state* - 
> Schema  -  struct>
> Query - select c.e.f from t where c.e.f > 10;
> Current state - read entire c struct from the file and then filter because 
> "hive.io.file.readcolumn.ids" is referred due to which all the children 
> column are select to read from the file.
> Conf -
>  _hive.io.file.readcolumn.ids  = "2"
>  hive.io.file.readNestedColumn.paths = "c.e.f"_
> Result -   
> boolean[ ] include  = [true,false,false,true,true,true,true,true]
> *Expected state* -
> Schema  -  struct>
> Query - select c.e.f from t where c.e.f > 10;
> Expected state - instead of reading entire c struct from the file just read 
> only the f column by referring the  " hive.io.file.readNestedColumn.paths".
> Conf -
>  _hive.io.file.readcolumn.ids  = "2"
>  hive.io.file.readNestedColumn.paths = "c.e.f"_
> Result -   
> boolean[ ] include  = [true,false,false,true,false,true,true,false]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19103) Nested structure Projection Push Down in Hive with ORC

[Ashish Sharma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Sharma updated HIVE-19103:
-
Affects Version/s: (was: 2.3.2)

> Nested structure Projection Push Down in Hive with ORC
> --
>
> Key: HIVE-19103
> URL: https://issues.apache.org/jira/browse/HIVE-19103
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive, ORC
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Major
>  Labels: pull-request-available
> Fix For: 2.3.2
>
> Attachments: HIVE-19103-0.patch, HIVE-19103-1.patch, 
> HIVE-19103-2.patch
>
>
> Reading required columns only in nested structure schema
> Example - 
> *Current state* - 
> Schema  -  struct>
> Query - select c.e.f from t where c.e.f > 10;
> Current state - read entire c struct from the file and then filter because 
> "hive.io.file.readcolumn.ids" is referred due to which all the children 
> column are select to read from the file.
> Conf -
>  _hive.io.file.readcolumn.ids  = "2"
>  hive.io.file.readNestedColumn.paths = "c.e.f"_
> Result -   
> boolean[ ] include  = [true,false,false,true,true,true,true,true]
> *Expected state* -
> Schema  -  struct>
> Query - select c.e.f from t where c.e.f > 10;
> Expected state - instead of reading entire c struct from the file just read 
> only the f column by referring the  " hive.io.file.readNestedColumn.paths".
> Conf -
>  _hive.io.file.readcolumn.ids  = "2"
>  hive.io.file.readNestedColumn.paths = "c.e.f"_
> Result -   
> boolean[ ] include  = [true,false,false,true,false,true,true,false]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19103) Nested structure Projection Push Down in Hive with ORC

[Ashish Sharma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Sharma updated HIVE-19103:
-
Priority: Critical  (was: Major)

> Nested structure Projection Push Down in Hive with ORC
> --
>
> Key: HIVE-19103
> URL: https://issues.apache.org/jira/browse/HIVE-19103
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive, ORC
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Critical
>  Labels: pull-request-available
> Fix For: 2.3.2
>
> Attachments: HIVE-19103-0.patch, HIVE-19103-1.patch, 
> HIVE-19103-2.patch
>
>
> Reading required columns only in nested structure schema
> Example - 
> *Current state* - 
> Schema  -  struct>
> Query - select c.e.f from t where c.e.f > 10;
> Current state - read entire c struct from the file and then filter because 
> "hive.io.file.readcolumn.ids" is referred due to which all the children 
> column are select to read from the file.
> Conf -
>  _hive.io.file.readcolumn.ids  = "2"
>  hive.io.file.readNestedColumn.paths = "c.e.f"_
> Result -   
> boolean[ ] include  = [true,false,false,true,true,true,true,true]
> *Expected state* -
> Schema  -  struct>
> Query - select c.e.f from t where c.e.f > 10;
> Expected state - instead of reading entire c struct from the file just read 
> only the f column by referring the  " hive.io.file.readNestedColumn.paths".
> Conf -
>  _hive.io.file.readcolumn.ids  = "2"
>  hive.io.file.readNestedColumn.paths = "c.e.f"_
> Result -   
> boolean[ ] include  = [true,false,false,true,false,true,true,false]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19103) Nested structure Projection Push Down in Hive with ORC

[Ashish Sharma (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashish Sharma updated HIVE-19103:
-
Component/s: ORC

> Nested structure Projection Push Down in Hive with ORC
> --
>
> Key: HIVE-19103
> URL: https://issues.apache.org/jira/browse/HIVE-19103
> Project: Hive
>  Issue Type: Improvement
>  Components: Hive, ORC
>Reporter: Ashish Sharma
>Assignee: Ashish Sharma
>Priority: Critical
>  Labels: pull-request-available
> Fix For: 2.3.2
>
> Attachments: HIVE-19103-0.patch, HIVE-19103-1.patch, 
> HIVE-19103-2.patch
>
>
> Reading required columns only in nested structure schema
> Example - 
> *Current state* - 
> Schema  -  struct>
> Query - select c.e.f from t where c.e.f > 10;
> Current state - read entire c struct from the file and then filter because 
> "hive.io.file.readcolumn.ids" is referred due to which all the children 
> column are select to read from the file.
> Conf -
>  _hive.io.file.readcolumn.ids  = "2"
>  hive.io.file.readNestedColumn.paths = "c.e.f"_
> Result -   
> boolean[ ] include  = [true,false,false,true,true,true,true,true]
> *Expected state* -
> Schema  -  struct>
> Query - select c.e.f from t where c.e.f > 10;
> Expected state - instead of reading entire c struct from the file just read 
> only the f column by referring the  " hive.io.file.readNestedColumn.paths".
> Conf -
>  _hive.io.file.readcolumn.ids  = "2"
>  hive.io.file.readNestedColumn.paths = "c.e.f"_
> Result -   
> boolean[ ] include  = [true,false,false,true,false,true,true,false]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19650) Hive query fails with ArrayIndexOutOfBoundsException with multi-insert query when execution engine is MapReduce

[Aditya Pant (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aditya Pant updated HIVE-19650:
---
Description: 
When a Hive query of the form:

FROM ...

INSERT OVERWRITE TABLE ...

INSERT OVERWRITE TABLE ...

INSERT OVERWRITE TABLE ;

is run from the Hive shell, it fails with the following exception:

{{Caused by: java.lang.ArrayIndexOutOfBoundsException: 70077}}
{{ \{{ at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.readVInt(LazyBinaryUtils.java:304)
{{ \{{ at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.checkObjectByteInfo(LazyBinaryUtils.java:183)
{{ \{{ at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.parse(LazyBinaryStruct.java:142)
{{ \{{ at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getField(LazyBinaryStruct.java:199)
{{ \{{ at 
org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:64)
{{ \{{ at 
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:94)
{{ \{{ at 
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)
{{ \{{ at 
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)
{{ \{{ at 
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.populateCachedDistributionKeys(ReduceSinkOperator.java:438)
{{ \{{ at 
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:339)
{{ \{{ ... 13 more

The full stack trace (*error_stack_trace.log*) is attached along with a zip 
file (*ebf11671_repro_script.zip*) that contains a script to reproduce this 
issue from the Hive shell.

  was:
When a Hive query of the form:

FROM ...

INSERT OVERWRITE TABLE ...

INSERT OVERWRITE TABLE ...

INSERT OVERWRITE TABLE ;

is run from the Hive shell, it fails with the following exception:

{{Caused by: java.lang.ArrayIndexOutOfBoundsException: 70077}}
{{ at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.readVInt(LazyBinaryUtils.java:304)}}
{{ at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.checkObjectByteInfo(LazyBinaryUtils.java:183)}}
{{ at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.parse(LazyBinaryStruct.java:142)}}
{{ at 
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getField(LazyBinaryStruct.java:199)}}
{{ at 
org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:64)}}
{{ at 
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:94)}}
{{ at 
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:77)}}
{{ at 
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluator.evaluate(ExprNodeEvaluator.java:65)}}
{{ at 
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.populateCachedDistributionKeys(ReduceSinkOperator.java:438)}}
{{ at 
org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:339)}}
{{ ... 13 more}}

The full stack trace (*error_stack_trace.log*) is attached along with a zip 
file (*ebf11671_repro_script.zip*) that contains a script to reproduce this 
issue from the Hive shell.


> Hive query fails with ArrayIndexOutOfBoundsException with multi-insert query 
> when execution engine is MapReduce
> ---
>
> Key: HIVE-19650
> URL: https://issues.apache.org/jira/browse/HIVE-19650
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.2.1
>Reporter: Aditya Pant
>Priority: Critical
> Attachments: ebf11671_repro_script.zip, error_stack_trace.log
>
>
> When a Hive query of the form:
> FROM ...
> INSERT OVERWRITE TABLE ...
> INSERT OVERWRITE TABLE ...
> INSERT OVERWRITE TABLE ;
> is run from the Hive shell, it fails with the following exception:
> {{Caused by: java.lang.ArrayIndexOutOfBoundsException: 70077}}
> {{ \{{ at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.readVInt(LazyBinaryUtils.java:304)
> {{ \{{ at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils.checkObjectByteInfo(LazyBinaryUtils.java:183)
> {{ \{{ at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.parse(LazyBinaryStruct.java:142)
> {{ \{{ at 
> org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct.getField(LazyBinaryStruct.java:199)
> {{ \{{ at 
> org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:64)
> {{ \{{ at 
> org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate(ExprNodeColumnEvaluator.java:94)
> {{ \{{ at 
> 

[jira] [Commented] (HIVE-19632) Remove webapps directory from standalone jar

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483936#comment-16483936
 ] 

Hive QA commented on HIVE-19632:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
34s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
23s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 5 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 13s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  xml  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11135/dev-support/hive-personality.sh
 |
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11135/yetus/whitespace-tabs.txt
 |
| modules | C: druid-handler jdbc U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11135/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Remove webapps directory from standalone jar
> 
>
> Key: HIVE-19632
> URL: https://issues.apache.org/jira/browse/HIVE-19632
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19632.1.patch, HIVE-19632.2.patch
>
>
> JDBC standalone jar contains webapps static files which just adds to the jar 
> size and are not required by the clients. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19649) Clean up inputs in JDBC PreparedStatement. Add unit tests.

[Mykhailo Kysliuk (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mykhailo Kysliuk reassigned HIVE-19649:
---


> Clean up inputs in JDBC PreparedStatement. Add unit tests.
> --
>
> Key: HIVE-19649
> URL: https://issues.apache.org/jira/browse/HIVE-19649
> Project: Hive
>  Issue Type: Test
>Reporter: Mykhailo Kysliuk
>Assignee: Mykhailo Kysliuk
>Priority: Minor
>
> Add unit tests for feature that was implemented in 
> [HIVE-18788|https://issues.apache.org/jira/browse/HIVE-18788].
> The integration tests are present, but it will be useful to catch errors 
> during module build.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-6998) Select query can only support maximum 128 distinct expressions

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483911#comment-16483911
 ] 

Hive QA commented on HIVE-6998:
---



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924428/HIVE-6998.03.patch

{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 14391 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.exec.vector.TestVectorSerDeRow.testVectorLazyBinaryDeserializeRow
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.TestVectorSerDeRow.testVectorLazyBinarySerializeRow
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRowsClipped
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRowsClippedExact
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testBigIntRowsExact
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testIntRowsClipped
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testIntRowsExact
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClipped1
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClipped2
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClippedExact1
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClippedExact2
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsClippedExact3
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsExact1
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsExact2
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testMultiKeyRowsExact3
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRowsClipped
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRowsClippedExact
 (batchId=293)
org.apache.hadoop.hive.ql.exec.vector.mapjoin.fast.TestVectorMapJoinFastRowHashMap.testStringRowsExact
 (batchId=293)
org.apache.hadoop.hive.serde2.lazybinary.TestLazyBinaryFast.testLazyBinaryFastComplexDepthFour
 (batchId=307)
org.apache.hadoop.hive.serde2.lazybinary.TestLazyBinaryFast.testLazyBinaryFastComplexDepthOne
 (batchId=307)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/11134/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11134/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11134/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12924428 - PreCommit-HIVE-Build

> Select query can only support maximum 128 distinct expressions
> --
>
> Key: HIVE-6998
> URL: https://issues.apache.org/jira/browse/HIVE-6998
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor, Serializers/Deserializers
>Affects Versions: 0.14.0
>Reporter: Chaoyu Tang
>Assignee: Eugene Koifman
>Priority: Major
> Attachments: HIVE-6998.01.patch, HIVE-6998.02.patch, 
> HIVE-6998.03.patch
>
>
> Select query can only support maximum 128 distinct expressions. Otherwise, 
> you will be thrown ArrayIndexOutOfBoundsException. For a query like:
> select count(distinct c1),  count(distinct c2),  count(distinct c3),  
> count(distinct c4),  count(distinct c5),  count(distinct c6), , 
> count(distinct c128),  count(distinct c129) from tbl_129columns;
> you will get error like:
> {code}
> java.lang.Exception: java.lang.RuntimeException: Hive Runtime Error while 
> closing operators
> at 
> org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:354)
> Caused by: java.lang.RuntimeException: Hive Runtime Error while closing 
> operators
> at 
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:260)
> at 

[jira] [Commented] (HIVE-6998) Select query can only support maximum 128 distinct expressions

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-6998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483874#comment-16483874
 ] 

Hive QA commented on HIVE-6998:
---

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
31s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
37s{color} | {color:blue} serde in master has 190 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
17s{color} | {color:blue} itests/custom-serde in master has 1 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
53s{color} | {color:blue} ql in master has 2322 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
12s{color} | {color:red} custom-serde in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
30s{color} | {color:red} serde: The patch generated 9 new + 1957 unchanged - 23 
fixed = 1966 total (was 1980) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
53s{color} | {color:red} ql: The patch generated 14 new + 2192 unchanged - 11 
fixed = 2206 total (was 2203) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
48s{color} | {color:green} serde generated 0 new + 189 unchanged - 1 fixed = 
189 total (was 190) {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
25s{color} | {color:green} custom-serde in the patch passed. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
56s{color} | {color:green} ql in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
12s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 52s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11134/dev-support/hive-personality.sh
 |
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11134/yetus/patch-mvninstall-itests_custom-serde.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11134/yetus/diff-checkstyle-serde.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11134/yetus/diff-checkstyle-ql.txt
 |
| asflicense | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11134/yetus/patch-asflicense-problems.txt
 |
| modules | C: serde itests/custom-serde ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11134/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Select query can only support maximum 

[jira] [Updated] (HIVE-19625) Archive partition can not be dropped

[Wang XL (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wang XL updated HIVE-19625:
---
Affects Version/s: (was: 1.2.1)
   Attachment: HIVE-19625-trunk.001.patch
   Status: Patch Available  (was: Open)

> Archive partition can not be dropped
> 
>
> Key: HIVE-19625
> URL: https://issues.apache.org/jira/browse/HIVE-19625
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Reporter: Wang XL
>Priority: Major
> Attachments: HIVE-19625-trunk.001.patch
>
>
> In our environment, we use hive Archive Partition command {{ALTER TABLE 
> table_name ARCHIVE PARTITION partition_spec;}}. But when I try to delete a 
> partition by using {{ALTER TABLE table_name DROP [IF EXISTS] PARTITION 
> partition_spec}}, I get error and the stack as follow:
> {code:java}
> 2018-01-15 22:08:36,921 ERROR [fe67c601-9bd7-4d5b-8e6e-8aea50a1167e]: 
> exec.DDLTask (DDLTask.java:failed(526)) - 
> org.apache.hadoop.hive.ql.metadata.HiveException: Table partition not deleted
> since 
> har:/nn01/warehouse/test.db/xiaolong_test/dt=20170826/hour=16/ctime=2017082616
>  is not writable by hadoop-data
>   at 
> org.apache.hadoop.hive.ql.metadata.Hive.dropPartitions(Hive.java:1990)
>   at 
> org.apache.hadoop.hive.ql.metadata.Hive.dropPartitions(Hive.java:1971)
>   at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropPartitions(DDLTask.java:3718)
>   at 
> org.apache.hadoop.hive.ql.exec.DDLTask.dropTableOrPartitions(DDLTask.java:3679)
>   at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:337)
>   at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
>   at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:92)
>   at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1704)
>   at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1446)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1087)
>   at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1223)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1087)
>   at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:216)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:386)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:321)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:725)
>   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:698)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:634)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:606)
>   at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:168)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:386)
>   at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:321)
>   at 
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:725)
>   at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:698)
>   at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:634)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1690)
>   at 
> org.apache.hadoop.security.SecurityUtil.doAsConfigUser(SecurityUtil.java:649)
>   at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: MetaException(message:Table partition not deleted since 
> har:/nn01/warehouse/test.db/xiaolong_test/dt=20170826/hour=16/ctime=2017082616
>  is not writable by hadoop-data)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_partitions_req_result$drop_partitions_req_resultStandardScheme.read(ThriftHiveMetastore.java)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_partitions_req_result$drop_partitions_req_resultStandardScheme.read(ThriftHiveMetastore.java)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$drop_partitions_req_result.read(ThriftHiveMetastore.java:65522)
>   at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:86)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_drop_partitions_req(ThriftHiveMetastore.java:1833)
>   at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.drop_partitions_req(ThriftHiveMetastore.java:1820)
>   at 
> 

[jira] [Updated] (HIVE-19326) stats auto gather: incorrect aggregation during UNION queries (may lead to incorrect results)

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19326:

Status: Open  (was: Patch Available)

I agree; the opId inconsistency might probably cause trouble elsewhere to...

> stats auto gather: incorrect aggregation during UNION queries (may lead to 
> incorrect results)
> -
>
> Key: HIVE-19326
> URL: https://issues.apache.org/jira/browse/HIVE-19326
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Sergey Shelukhin
>Assignee: Zoltan Haindrich
>Priority: Critical
> Attachments: HIVE-19326.01wip01.patch, HIVE-19326.02.patch, 
> HIVE-19326.03.patch, HIVE-19326.04.patch, HIVE-19326.05.patch
>
>
> Found when investigating the results change after converting tables to MM, 
> turns out the MM result is correct but the current one is not.
> The test ends like so:
> {noformat}
> desc formatted small_alltypesorc_a;
> ANALYZE TABLE small_alltypesorc_a COMPUTE STATISTICS;
> desc formatted small_alltypesorc_a;
> insert into table small_alltypesorc_a select * from small_alltypesorc1a;
> desc formatted small_alltypesorc_a;
> {noformat}
> The results from the descs in the golden file are:
> {noformat}
>   COLUMN_STATS_ACCURATE   {\"BASIC_STATS\":\"true\"}
>   numFiles1   
>   numRows 5   
> ...
>   COLUMN_STATS_ACCURATE   {\"BASIC_STATS\":\"true\"}
>   numFiles1   
>   numRows 15
> ...
>   COLUMN_STATS_ACCURATE   {\"BASIC_STATS\":\"true\"}
>   numFiles2   
>   numRows 20  
> {noformat}
> Note the result change after analyze - the original nomRows is inaccurate, 
> but  BASIC_STATS is set to true.
> I am assuming with metadata only optimization this can produce incorrect 
> results.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19237) Only use an operatorId once in a plan

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich reassigned HIVE-19237:
---

Assignee: Zoltan Haindrich

> Only use an operatorId once in a plan
> -
>
> Key: HIVE-19237
> URL: https://issues.apache.org/jira/browse/HIVE-19237
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>
> Column stats autogather plan part is added from a plan compiled by the driver 
> itself; however that driver starts to use operatorIds from 1 ; so it's 
> possible that 2 SEL_1 operators end up in the same plan...



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-6980) Drop table by using direct sql

[Peter Vary (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-6980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Peter Vary updated HIVE-6980:
-
Attachment: HIVE-6980.6.patch

> Drop table by using direct sql
> --
>
> Key: HIVE-6980
> URL: https://issues.apache.org/jira/browse/HIVE-6980
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 0.12.0
>Reporter: Selina Zhang
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-6980.2.patch, HIVE-6980.3.patch, HIVE-6980.4.patch, 
> HIVE-6980.5.patch, HIVE-6980.6.patch, HIVE-6980.patch, drop_table_after.png, 
> drop_table_before.png
>
>
> Dropping table which has lots of partitions is slow. Even after applying the 
> patch of HIVE-6265, the drop table still takes hours (100K+ partitions). 
> The fixes come with two parts:
> 1. use directSQL to query the partitions protect mode;
> the current implementation needs to transfer the Partition object to client 
> and check the protect mode for each partition. I'd like to move this part of 
> logic to metastore. The check will be done by direct sql (if direct sql is 
> disabled, execute the same logic in the ObjectStore);
> 2. use directSQL to drop partitions for table;
> there maybe two solutions here:
> 1. add "DELETE CASCADE" in the schema. In this way we only need to delete 
> entries from partitions table use direct sql. May need to change 
> datanucleus.deletionPolicy = DataNucleus. 
> 2. clean up the dependent tables by issue DELETE statement. This also needs 
> to turn on datanucleus.query.sql.allowAll
> Both of above solutions should be able to fix the problem. The DELETE CASCADE 
> has to change schemas and prepare upgrade scripts. The second solutions added 
> maintenance cost if new tables added in the future releases.
> Please advice. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19631) reduce epic locking in AbstractService

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483834#comment-16483834
 ] 

Hive QA commented on HIVE-19631:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924393/HIVE-19631.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14390 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/11133/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11133/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11133/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12924393 - PreCommit-HIVE-Build

> reduce epic locking in AbstractService
> --
>
> Key: HIVE-19631
> URL: https://issues.apache.org/jira/browse/HIVE-19631
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19631.patch
>
>
> Some services have synchronized method that do lengthy remote calls, and 
> these block everyone trying to e.g. getHiveConf. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19631) reduce epic locking in AbstractService

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483790#comment-16483790
 ] 

Hive QA commented on HIVE-19631:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
36s{color} | {color:blue} service in master has 50 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
13s{color} | {color:red} service: The patch generated 2 new + 54 unchanged - 0 
fixed = 56 total (was 54) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
44s{color} | {color:green} service generated 0 new + 49 unchanged - 1 fixed = 
49 total (was 50) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 11m 12s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11133/dev-support/hive-personality.sh
 |
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11133/yetus/diff-checkstyle-service.txt
 |
| modules | C: service U: service |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11133/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> reduce epic locking in AbstractService
> --
>
> Key: HIVE-19631
> URL: https://issues.apache.org/jira/browse/HIVE-19631
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19631.patch
>
>
> Some services have synchronized method that do lengthy remote calls, and 
> these block everyone trying to e.g. getHiveConf. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19531) TransactionalValidationListener is getting catalog name from conf instead of table object.

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483781#comment-16483781
 ] 

Hive QA commented on HIVE-19531:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924392/HIVE-19531.1take4.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14390 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/11132/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11132/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11132/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12924392 - PreCommit-HIVE-Build

> TransactionalValidationListener is getting catalog name from conf instead of 
> table object.
> --
>
> Key: HIVE-19531
> URL: https://issues.apache.org/jira/browse/HIVE-19531
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Blocker
> Fix For: 3.0.1
>
> Attachments: HIVE-19531.1again.patch, HIVE-19531.1take3.patch, 
> HIVE-19531.1take4.patch, HIVE-19531.patch
>
>
> TransactionalValidationListener.validateTableStructure get the catalog from 
> the conf file rather than taking it from the passed in table structure.  This 
> causes createTable operations to fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19647) use bitvectors in IN operators

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19647:

Attachment: partial.patch

> use bitvectors in IN operators
> --
>
> Key: HIVE-19647
> URL: https://issues.apache.org/jira/browse/HIVE-19647
> Project: Hive
>  Issue Type: Improvement
>  Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: partial.patch
>
>
> for strings there is no easy way to filter things out ; even min/max is null;
> but the bitvectors could be used if they are available



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19647) use bitvectors in IN operators

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich reassigned HIVE-19647:
---


> use bitvectors in IN operators
> --
>
> Key: HIVE-19647
> URL: https://issues.apache.org/jira/browse/HIVE-19647
> Project: Hive
>  Issue Type: Improvement
>  Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
>
> for strings there is no easy way to filter things out ; even min/max is null;
> but the bitvectors could be used if they are available



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19460) Improve stats estimations for NOT IN operator

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19460:

Component/s: Statistics

> Improve stats estimations for NOT IN operator
> -
>
> Key: HIVE-19460
> URL: https://issues.apache.org/jira/browse/HIVE-19460
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19460.01wip01.patch, HIVE-19460.01wip02.patch, 
> HIVE-19460.01wip03.patch, HIVE-19460.02.patch, HIVE-19460.03.patch, 
> HIVE-19460.04.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19460) Improve stats estimations for NOT IN operator

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19460:

Attachment: HIVE-19460.04.patch

> Improve stats estimations for NOT IN operator
> -
>
> Key: HIVE-19460
> URL: https://issues.apache.org/jira/browse/HIVE-19460
> Project: Hive
>  Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19460.01wip01.patch, HIVE-19460.01wip02.patch, 
> HIVE-19460.01wip03.patch, HIVE-19460.02.patch, HIVE-19460.03.patch, 
> HIVE-19460.04.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (HIVE-19646) Filesystem closed error in HiveProtoLoggingHook

[Harish Jaiprakash (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483741#comment-16483741
 ] 

Harish Jaiprakash edited comment on HIVE-19646 at 5/22/18 10:09 AM:


Create patch with following changes:
 * Always get filesystem object from the path, instead of storing it.
 * Retry logic when logging fails.
 * Moved the code copied from tez to the tez folders, this is to make sure we 
can remove this code in a backwards compatible way, when hive moves to next 
release tez version. SequenceFile writes the key and value class in the file 
and prevents being read if the class package name is different.

[~anishek], Please review.


was (Author: harishjp):
Create patch with following changes:
 * Always get filesystem object from the path, instead of storing it.
 * Retry logic when logging fails.
 * Moved the code copied from tez to the tez folders, this is to make sure we 
can remove this code in a backwards compatible way, when hive moves to next 
release tez version. SequenceFile writes the key and value class in the file 
and prevents being read if the class package name is different.

> Filesystem closed error in HiveProtoLoggingHook
> ---
>
> Key: HIVE-19646
> URL: https://issues.apache.org/jira/browse/HIVE-19646
> Project: Hive
>  Issue Type: Bug
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
>Priority: Major
> Attachments: HIVE-19646.01.patch
>
>
> Exception in proto logging hook on secure cluster.
> {code}
> 2018-05-18T04:48:01,136 ERROR [Hive Hook Proto Log Writer 0]: 
> hooks.HiveProtoLoggingHook (:()) - Error writing proto message for query 
> hive_20180518043717_ca3ab4df-6cab-4920-aa44-2340ae246ad2, eventType: 
> QUERY_SUBMITTED:
> java.io.IOException: Filesystem closed
>  at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:472) 
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1652) 
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1569)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1566)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>  ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1581)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734) 
> ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getPathForDate(DatePartitionedLogger.java:89)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getWriter(DatePartitionedLogger.java:73)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.writeEvent(HiveProtoLoggingHook.java:283)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.lambda$generateEvent$1(HiveProtoLoggingHook.java:274)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  ~[?:1.8.0_161]
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  ~[?:1.8.0_161]
>  at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19646) Filesystem closed error in HiveProtoLoggingHook

[Harish Jaiprakash (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Jaiprakash updated HIVE-19646:
-
Status: Patch Available  (was: Open)

Create patch with following changes:
 * Always get filesystem object from the path, instead of storing it.
 * Retry logic when logging fails.
 * Moved the code copied from tez to the tez folders, this is to make sure we 
can remove this code in a backwards compatible way, when hive moves to next 
release tez version. SequenceFile writes the key and value class in the file 
and prevents being read if the class package name is different.

> Filesystem closed error in HiveProtoLoggingHook
> ---
>
> Key: HIVE-19646
> URL: https://issues.apache.org/jira/browse/HIVE-19646
> Project: Hive
>  Issue Type: Bug
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
>Priority: Major
> Attachments: HIVE-19646.01.patch
>
>
> Exception in proto logging hook on secure cluster.
> {code}
> 2018-05-18T04:48:01,136 ERROR [Hive Hook Proto Log Writer 0]: 
> hooks.HiveProtoLoggingHook (:()) - Error writing proto message for query 
> hive_20180518043717_ca3ab4df-6cab-4920-aa44-2340ae246ad2, eventType: 
> QUERY_SUBMITTED:
> java.io.IOException: Filesystem closed
>  at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:472) 
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1652) 
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1569)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1566)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>  ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1581)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734) 
> ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getPathForDate(DatePartitionedLogger.java:89)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getWriter(DatePartitionedLogger.java:73)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.writeEvent(HiveProtoLoggingHook.java:283)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.lambda$generateEvent$1(HiveProtoLoggingHook.java:274)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  ~[?:1.8.0_161]
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  ~[?:1.8.0_161]
>  at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19646) Filesystem closed error in HiveProtoLoggingHook

[Harish Jaiprakash (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Jaiprakash reassigned HIVE-19646:



> Filesystem closed error in HiveProtoLoggingHook
> ---
>
> Key: HIVE-19646
> URL: https://issues.apache.org/jira/browse/HIVE-19646
> Project: Hive
>  Issue Type: Bug
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
>Priority: Major
> Attachments: HIVE-19646.01.patch
>
>
> Exception in proto logging hook on secure cluster.
> {code}
> 2018-05-18T04:48:01,136 ERROR [Hive Hook Proto Log Writer 0]: 
> hooks.HiveProtoLoggingHook (:()) - Error writing proto message for query 
> hive_20180518043717_ca3ab4df-6cab-4920-aa44-2340ae246ad2, eventType: 
> QUERY_SUBMITTED:
> java.io.IOException: Filesystem closed
>  at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:472) 
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1652) 
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1569)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1566)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>  ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1581)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734) 
> ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getPathForDate(DatePartitionedLogger.java:89)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getWriter(DatePartitionedLogger.java:73)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.writeEvent(HiveProtoLoggingHook.java:283)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.lambda$generateEvent$1(HiveProtoLoggingHook.java:274)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  ~[?:1.8.0_161]
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  ~[?:1.8.0_161]
>  at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19646) Filesystem closed error in HiveProtoLoggingHook

[Harish Jaiprakash (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Jaiprakash updated HIVE-19646:
-
Attachment: HIVE-19646.01.patch

> Filesystem closed error in HiveProtoLoggingHook
> ---
>
> Key: HIVE-19646
> URL: https://issues.apache.org/jira/browse/HIVE-19646
> Project: Hive
>  Issue Type: Bug
>Reporter: Harish Jaiprakash
>Assignee: Harish Jaiprakash
>Priority: Major
> Attachments: HIVE-19646.01.patch
>
>
> Exception in proto logging hook on secure cluster.
> {code}
> 2018-05-18T04:48:01,136 ERROR [Hive Hook Proto Log Writer 0]: 
> hooks.HiveProtoLoggingHook (:()) - Error writing proto message for query 
> hive_20180518043717_ca3ab4df-6cab-4920-aa44-2340ae246ad2, eventType: 
> QUERY_SUBMITTED:
> java.io.IOException: Filesystem closed
>  at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:472) 
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1652) 
> ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1569)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1566)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>  ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1581)
>  ~[hadoop-hdfs-client-3.0.0.3.0.0.0-1298.jar:?]
>  at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734) 
> ~[hadoop-common-3.0.0.3.0.0.0-1298.jar:?]
>  at 
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getPathForDate(DatePartitionedLogger.java:89)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.DatePartitionedLogger.getWriter(DatePartitionedLogger.java:73)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.writeEvent(HiveProtoLoggingHook.java:283)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> org.apache.hadoop.hive.ql.hooks.HiveProtoLoggingHook$EventLogger.lambda$generateEvent$1(HiveProtoLoggingHook.java:274)
>  ~[hive-exec-3.0.0.3.0.0.0-1298.jar:3.0.0.3.0.0.0-1298]
>  at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>  ~[?:1.8.0_161]
>  at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>  ~[?:1.8.0_161]
>  at java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19531) TransactionalValidationListener is getting catalog name from conf instead of table object.

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483736#comment-16483736
 ] 

Hive QA commented on HIVE-19531:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
16s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  2m 
49s{color} | {color:blue} standalone-metastore in master has 216 extant 
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
17s{color} | {color:red} standalone-metastore: The patch generated 1 new + 19 
unchanged - 0 fixed = 20 total (was 19) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
10s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 41s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11132/dev-support/hive-personality.sh
 |
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11132/yetus/diff-checkstyle-standalone-metastore.txt
 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11132/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> TransactionalValidationListener is getting catalog name from conf instead of 
> table object.
> --
>
> Key: HIVE-19531
> URL: https://issues.apache.org/jira/browse/HIVE-19531
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Blocker
> Fix For: 3.0.1
>
> Attachments: HIVE-19531.1again.patch, HIVE-19531.1take3.patch, 
> HIVE-19531.1take4.patch, HIVE-19531.patch
>
>
> TransactionalValidationListener.validateTableStructure get the catalog from 
> the conf file rather than taking it from the passed in table structure.  This 
> causes createTable operations to fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19558) HiveAuthorizationProviderBase gets catalog name from config rather than db object

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483712#comment-16483712
 ] 

Hive QA commented on HIVE-19558:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924390/HIVE-19558.1take3.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14390 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions]
 (batchId=257)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_mapjoin_15]
 (batchId=174)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/11131/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11131/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11131/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12924390 - PreCommit-HIVE-Build

> HiveAuthorizationProviderBase gets catalog name from config rather than db 
> object
> -
>
> Key: HIVE-19558
> URL: https://issues.apache.org/jira/browse/HIVE-19558
> Project: Hive
>  Issue Type: Bug
>  Components: Authorization
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.1
>
> Attachments: HIVE-19558.1take2.patch, HIVE-19558.1take3.patch, 
> HIVE-19558.patch
>
>
> HiveAuthorizationProviderBase.getDatabase uses just the database name to 
> fetch the database, relying on getDefaultCatalog() to fetch the catalog name 
> from the conf file.  This does not work when the client has passed in an 
> object for a different catalog.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19166) TestMiniLlapLocalCliDriver sysdb failure

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19166:

Attachment: HIVE-19166.10.patch

> TestMiniLlapLocalCliDriver sysdb failure
> 
>
> Key: HIVE-19166
> URL: https://issues.apache.org/jira/browse/HIVE-19166
> Project: Hive
>  Issue Type: Sub-task
>  Components: Test
>Reporter: Vineet Garg
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19166.04.patch, HIVE-19166.05.patch, 
> HIVE-19166.06.patch, HIVE-19166.09.patch, HIVE-19166.1.patch, 
> HIVE-19166.10.patch, HIVE-19166.2.patch, HIVE-19166.3.patch
>
>
> Broken by HIVE-18715



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19485) dump directory for non native tables should not be created

[anishek (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anishek updated HIVE-19485:
---
Attachment: HIVE-19485.02-branch-3.patch

> dump directory for non native tables should not be created
> --
>
> Key: HIVE-19485
> URL: https://issues.apache.org/jira/browse/HIVE-19485
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19485.0.patch, HIVE-19485.02-branch-3.patch, 
> HIVE-19485.1.patch, HIVE-19485.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19485) dump directory for non native tables should not be created

[anishek (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anishek updated HIVE-19485:
---
Attachment: (was: HIVE-19485.02.branch-3.patch)

> dump directory for non native tables should not be created
> --
>
> Key: HIVE-19485
> URL: https://issues.apache.org/jira/browse/HIVE-19485
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19485.0.patch, HIVE-19485.02-branch-3.patch, 
> HIVE-19485.1.patch, HIVE-19485.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19485) dump directory for non native tables should not be created

[anishek (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anishek updated HIVE-19485:
---
Attachment: HIVE-19485.02.branch-3.patch

> dump directory for non native tables should not be created
> --
>
> Key: HIVE-19485
> URL: https://issues.apache.org/jira/browse/HIVE-19485
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19485.0.patch, HIVE-19485.02.branch-3.patch, 
> HIVE-19485.1.patch, HIVE-19485.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19485) dump directory for non native tables should not be created

[anishek (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anishek updated HIVE-19485:
---
Attachment: (was: HIVE-19485.1.branch-3.patch)

> dump directory for non native tables should not be created
> --
>
> Key: HIVE-19485
> URL: https://issues.apache.org/jira/browse/HIVE-19485
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19485.0.patch, HIVE-19485.1.patch, 
> HIVE-19485.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Reopened] (HIVE-19485) dump directory for non native tables should not be created

[anishek (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anishek reopened HIVE-19485:


get the patch for 3.0.1 release as well

> dump directory for non native tables should not be created
> --
>
> Key: HIVE-19485
> URL: https://issues.apache.org/jira/browse/HIVE-19485
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19485.0.patch, HIVE-19485.1.patch, 
> HIVE-19485.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19485) dump directory for non native tables should not be created

[anishek (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anishek updated HIVE-19485:
---
Attachment: HIVE-19485.1.branch-3.patch

> dump directory for non native tables should not be created
> --
>
> Key: HIVE-19485
> URL: https://issues.apache.org/jira/browse/HIVE-19485
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19485.0.patch, HIVE-19485.1.patch, 
> HIVE-19485.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19485) dump directory for non native tables should not be created

[anishek (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

anishek updated HIVE-19485:
---
Fix Version/s: 3.0.1

> dump directory for non native tables should not be created
> --
>
> Key: HIVE-19485
> URL: https://issues.apache.org/jira/browse/HIVE-19485
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: anishek
>Assignee: anishek
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 3.0.1
>
> Attachments: HIVE-19485.0.patch, HIVE-19485.1.patch, 
> HIVE-19485.2.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18767) Some alterPartitions invocations throw 'NumberFormatException: null'

[Peter Vary (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483674#comment-16483674
 ] 

Peter Vary commented on HIVE-18767:
---

[~massdos...@gmail.com]: Will do. Thanks!

> Some alterPartitions invocations throw 'NumberFormatException: null'
> 
>
> Key: HIVE-18767
> URL: https://issues.apache.org/jira/browse/HIVE-18767
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 3.0.0, 2.3.2
>Reporter: Yuming Wang
>Assignee: Mass Dosage
>Priority: Major
> Fix For: 2.3.3, 3.1.0
>
> Attachments: HIVE-18767-branch-2.3.patch, HIVE-18767.1.patch, 
> HIVE-18767.2.patch, HIVE-18767.3.patch, HIVE-18767.4.patch
>
>
> Error messages:
> {noformat}
> [info] Cause: java.lang.NumberFormatException: null
> [info] at java.lang.Long.parseLong(Long.java:552)
> [info] at java.lang.Long.parseLong(Long.java:631)
> [info] at 
> org.apache.hadoop.hive.metastore.MetaStoreUtils.isFastStatsSame(MetaStoreUtils.java:315)
> [info] at 
> org.apache.hadoop.hive.metastore.HiveAlterHandler.alterPartitions(HiveAlterHandler.java:605)
> [info] at 
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.alter_partitions_with_environment_context(HiveMetaStore.java:3837)
> [info] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [info] at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> [info] at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [info] at java.lang.reflect.Method.invoke(Method.java:498)
> [info] at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> [info] at 
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> [info] at 
> com.sun.proxy.$Proxy23.alter_partitions_with_environment_context(Unknown 
> Source)
> [info] at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.alter_partitions(HiveMetaStoreClient.java:1527)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19558) HiveAuthorizationProviderBase gets catalog name from config rather than db object

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483669#comment-16483669
 ] 

Hive QA commented on HIVE-19558:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
51s{color} | {color:blue} ql in master has 2322 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
38s{color} | {color:red} ql: The patch generated 1 new + 300 unchanged - 0 
fixed = 301 total (was 300) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 18s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11131/dev-support/hive-personality.sh
 |
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11131/yetus/diff-checkstyle-ql.txt
 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11131/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> HiveAuthorizationProviderBase gets catalog name from config rather than db 
> object
> -
>
> Key: HIVE-19558
> URL: https://issues.apache.org/jira/browse/HIVE-19558
> Project: Hive
>  Issue Type: Bug
>  Components: Authorization
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.1
>
> Attachments: HIVE-19558.1take2.patch, HIVE-19558.1take3.patch, 
> HIVE-19558.patch
>
>
> HiveAuthorizationProviderBase.getDatabase uses just the database name to 
> fetch the database, relying on getDefaultCatalog() to fetch the catalog name 
> from the conf file.  This does not work when the client has passed in an 
> object for a different catalog.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19500) Prevent multiple selectivity estimations for the same variable in conjuctions

[Zoltan Haindrich (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zoltan Haindrich updated HIVE-19500:

Attachment: HIVE-19500.03.patch

> Prevent multiple selectivity estimations for the same variable in conjuctions
> -
>
> Key: HIVE-19500
> URL: https://issues.apache.org/jira/browse/HIVE-19500
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19500.01.patch, HIVE-19500.02.patch, 
> HIVE-19500.03.patch
>
>
> see HIVE-19097 for problem description
> for filters like: {{(d_year in (2001,2002) and d_year = 2001)}} the current 
> estimation is around {{(1/NDV)**2}} (iff column stats are available) 
> actually the source of the problem was a small typo in HIVE-17465 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19578) HLL merges tempList on every add

[Prasanth Jayachandran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19578:
-
Attachment: HIVE-19578.3.patch

> HLL merges tempList on every add
> 
>
> Key: HIVE-19578
> URL: https://issues.apache.org/jira/browse/HIVE-19578
> Project: Hive
>  Issue Type: Bug
>Reporter: Sergey Shelukhin
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19578.1.patch, HIVE-19578.2.patch, 
> HIVE-19578.3.patch, Screen Shot 2018-05-16 at 15.29.12 .png
>
>
>  See comments on HIVE-18866; this has significant perf overhead after the 
> even bigger overhead from hashing is removed.  !Screen Shot 2018-05-16 at 
> 15.29.12 .png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19613) GenericUDTFGetSplits should handle fetch task with temp table rewrite

[Prasanth Jayachandran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19613?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19613:
-
Attachment: HIVE-19613.3.patch

> GenericUDTFGetSplits should handle fetch task with temp table rewrite
> -
>
> Key: HIVE-19613
> URL: https://issues.apache.org/jira/browse/HIVE-19613
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Wohlstadter
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19613.1.patch, HIVE-19613.2.patch, 
> HIVE-19613.3.patch
>
>
> GenericUDTFGetSplits fails for fetch task only queries. Fetch task only 
> queries can be handled same way as >1 task queries using temp tables. 
> {code:java}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Was expecting a 
> single TezTask.
> at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.createPlanFragment(GenericUDTFGetSplits.java:262)
> at 
> org.apache.hadoop.hive.ql.udf.generic.GenericUDTFGetSplits.process(GenericUDTFGetSplits.java:201)
> at org.apache.hadoop.hive.ql.exec.UDTFOperator.process(UDTFOperator.java:116)
> at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:984)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:930)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:917)
> at 
> org.apache.hadoop.hive.ql.exec.SelectOperator.process(SelectOperator.java:95)
> at org.apache.hadoop.hive.ql.exec.Operator.baseForward(Operator.java:984)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:930)
> at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
> at 
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:492)
> at 
> org.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:484)
> at org.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:145)
> ... 16 more{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19614) GenericUDTFGetSplits does not honor ORDER BY

[Prasanth Jayachandran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19614:
-
Attachment: HIVE-19614.2.patch

> GenericUDTFGetSplits does not honor ORDER BY
> 
>
> Key: HIVE-19614
> URL: https://issues.apache.org/jira/browse/HIVE-19614
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0
>Reporter: Eric Wohlstadter
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19614.1.patch, HIVE-19614.2.patch
>
>
> GenericUDTFGetSplits handles ORDER BY by writing the results to temp table. 
> However running select * on that temp table may create >1 splits which will 
> lose the original ordering. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19588) Several invocation of file listing when creating VectorizedOrcAcidRowBatchReader

[Prasanth Jayachandran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19588:
-
Attachment: HIVE-19588.3.patch

> Several invocation of file listing when creating 
> VectorizedOrcAcidRowBatchReader
> 
>
> Key: HIVE-19588
> URL: https://issues.apache.org/jira/browse/HIVE-19588
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Nita Dembla
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19588.1.patch, HIVE-19588.2.patch, 
> HIVE-19588.3.patch, Screen Shot 2018-05-16 at 2.23.25 PM.png
>
>
> Looks like we are doing file listing several times when creating one instance 
> of VectorizedOrcAcidRowBatchReader
>  AcidUtils.parseBaseOrDeltaBucketFilename() does full file listing (when 
> there are files with bucket_* prefix) just to get a single file out of a path 
> to figure out if it has ACID schema (as part of HIVE-18190).
>  There is full file listing where we populate
>  1) ColumnizedDeleteEventRegistry
>  2) SortMergedDeleteEventRegistry
>  3) Twice in computeOffsetAndBucket()
>  
> Attaching profiles which [~gopalv] took while debugging. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19633) Remove/Migrate Minimr tests

[Prasanth Jayachandran (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483645#comment-16483645
 ] 

Prasanth Jayachandran commented on HIVE-19633:
--

Following tests have been moved to TestCliDriver
infer_bucket_sort_map_operators.q,infer_bucket_sort_dyn_part.q,infer_bucket_sort_merge.q,infer_bucket_sort_reducers_power_two.q,infer_bucket_sort_num_buckets.q
root_dir_external_table.q
parallel_orderby.q
udf_using.q
scriptfile1.q

bucket_num_reducers_acid.q is migrated to Mini LLAP

root_dir_external_table.q is removed. This test moves the file to root hdfs 
directory and is creating a external table on root hdfs location which is a 
recipe for disaster as any number of files can exists in the root hdfs 
location. I tried to move this to MiniLlap/tez but root location has tez jars 
and misc files causing incorrect results. So removed this test altogether from 
MR and tez (spark has still coverage).

index_bitmap3.q,index_bitmap_auto.q,scriptfile1_win.q query files did not even 
exist. 

> Remove/Migrate Minimr tests
> ---
>
> Key: HIVE-19633
> URL: https://issues.apache.org/jira/browse/HIVE-19633
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19633.1.patch
>
>
> MR has been deprecated for a long time. Minimr tests are incredibly slow. We 
> should remove the tests or migrate to faster options for coverage. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19488) enable CM root based on db parameter, identifying a db as source of replication.

[mahesh kumar behera (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19488:
---
Attachment: HIVE-19488.06.patch

> enable CM root based on db parameter, identifying a db as source of 
> replication.
> 
>
> Key: HIVE-19488
> URL: https://issues.apache.org/jira/browse/HIVE-19488
> Project: Hive
>  Issue Type: Task
>  Components: Hive, HiveServer2, repl
>Affects Versions: 3.1.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0
>
> Attachments: HIVE-19488.01.patch, HIVE-19488.02.patch, 
> HIVE-19488.03.patch, HIVE-19488.04.patch, HIVE-19488.05.patch, 
> HIVE-19488.06.patch
>
>
> * add a parameter at db level to identify if its a source of replication. 
> beacon will set this.
>  * Enable CM root only for databases that are a source of a replication 
> policy, for other db's skip the CM root functionality.
>  * prevent database drop if the parameter indicating its source of a 
> replication, is set.
>  * as an upgrade to this version, beacon should set the property on all 
> existing database policies, in affect.
>  * the parameter should be of the form . –  repl.source.for : List < policy 
> ids >



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19633) Remove/Migrate Minimr tests

[Prasanth Jayachandran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19633:
-
Status: Patch Available  (was: Open)

> Remove/Migrate Minimr tests
> ---
>
> Key: HIVE-19633
> URL: https://issues.apache.org/jira/browse/HIVE-19633
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19633.1.patch
>
>
> MR has been deprecated for a long time. Minimr tests are incredibly slow. We 
> should remove the tests or migrate to faster options for coverage. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19633) Remove/Migrate Minimr tests

[Prasanth Jayachandran (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483637#comment-16483637
 ] 

Prasanth Jayachandran commented on HIVE-19633:
--

[~sershe]/[~jcamachorodriguez] can someone please review?

> Remove/Migrate Minimr tests
> ---
>
> Key: HIVE-19633
> URL: https://issues.apache.org/jira/browse/HIVE-19633
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19633.1.patch
>
>
> MR has been deprecated for a long time. Minimr tests are incredibly slow. We 
> should remove the tests or migrate to faster options for coverage. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-19633) Remove/Migrate Minimr tests

[Prasanth Jayachandran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19633:
-
Attachment: HIVE-19633.1.patch

> Remove/Migrate Minimr tests
> ---
>
> Key: HIVE-19633
> URL: https://issues.apache.org/jira/browse/HIVE-19633
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.1.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19633.1.patch
>
>
> MR has been deprecated for a long time. Minimr tests are incredibly slow. We 
> should remove the tests or migrate to faster options for coverage. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19576) IHMSHandler.getTable not always fetching the right catalog

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483633#comment-16483633
 ] 

Hive QA commented on HIVE-19576:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924391/HIVE-19576.1take3.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14390 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges
 (batchId=247)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/11130/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11130/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11130/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12924391 - PreCommit-HIVE-Build

> IHMSHandler.getTable not always fetching the right catalog
> --
>
> Key: HIVE-19576
> URL: https://issues.apache.org/jira/browse/HIVE-19576
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.1
>
> Attachments: HIVE-19576.1take2.patch, HIVE-19576.1take3.patch, 
> HIVE-19576.patch
>
>
> {{IHMSHandler.get_table_core(String dbName, String tableName)}} fetches the 
> catalog name from the conf.  This causes issues when doing an operation where 
> the catalog is known and does not match the default provided in the 
> configuration file (e.g. adding a partition).  This method should be removed 
> and callers forced to use {{IHMSHandler.get_table_core(String catName, String 
> dbName, String tableName)}} instead since callers will know whether they have 
> the catalog name or not.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-19645) Console info/error messages should be accessible by clients using hive server as well

[Amruth S (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-19645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Amruth S reassigned HIVE-19645:
---


> Console info/error messages should be accessible by clients using hive server 
> as well
> -
>
> Key: HIVE-19645
> URL: https://issues.apache.org/jira/browse/HIVE-19645
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 2.3.2
>Reporter: Amruth S
>Assignee: Amruth S
>Priority: Minor
>
> Messages printed in console and console err (mostly mapred/tez logs) etc 
> should be available to the clients using hive server 2 as well.
> This property seem to solve it to some extent 
> (hive.server2.logging.operation.enabled) but its not complete. 
> Wherever logs are happening, we need to tee it to the operation logs as well 
> so that the clients can discover them.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-19576) IHMSHandler.getTable not always fetching the right catalog

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483614#comment-16483614
 ] 

Hive QA commented on HIVE-19576:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
58s{color} | {color:blue} ql in master has 2322 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  2m 
51s{color} | {color:blue} standalone-metastore in master has 216 extant 
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m  
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
16s{color} | {color:red} standalone-metastore: The patch generated 1 new + 2 
unchanged - 1 fixed = 3 total (was 3) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  7m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m  
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 33m 56s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11130/dev-support/hive-personality.sh
 |
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11130/yetus/diff-checkstyle-standalone-metastore.txt
 |
| modules | C: ql standalone-metastore U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11130/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> IHMSHandler.getTable not always fetching the right catalog
> --
>
> Key: HIVE-19576
> URL: https://issues.apache.org/jira/browse/HIVE-19576
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
> Fix For: 3.0.1
>
> Attachments: HIVE-19576.1take2.patch, HIVE-19576.1take3.patch, 
> HIVE-19576.patch
>
>
> {{IHMSHandler.get_table_core(String dbName, String tableName)}} fetches the 
> catalog name from the conf.  This causes issues when doing an operation where 
> the catalog is known and does not match the default provided in the 
> configuration file (e.g. adding a partition).  This method should be removed 
> and callers forced to use {{IHMSHandler.get_table_core(String catName, String 
> dbName, String tableName)}} instead since callers will know whether they have 
> the 

[jira] [Commented] (HIVE-19500) Prevent multiple selectivity estimations for the same variable in conjuctions

[Zoltan Haindrich (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-19500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483609#comment-16483609
 ] 

Zoltan Haindrich commented on HIVE-19500:
-

that's interesting...because this ticket fixes a serious underestimation - I 
think it shouldn't be causing others to get OOMs
I think PerfCliDriver should be fixed; because currently in reality it doesn't 
use column stats correctly...
[~ndembla] could you tell me on which queries you get OOMs with this patch?

> Prevent multiple selectivity estimations for the same variable in conjuctions
> -
>
> Key: HIVE-19500
> URL: https://issues.apache.org/jira/browse/HIVE-19500
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-19500.01.patch, HIVE-19500.02.patch
>
>
> see HIVE-19097 for problem description
> for filters like: {{(d_year in (2001,2002) and d_year = 2001)}} the current 
> estimation is around {{(1/NDV)**2}} (iff column stats are available) 
> actually the source of the problem was a small typo in HIVE-17465 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18866) Semijoin: Implement a Long -> Hash64 vector fast-path

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483577#comment-16483577
 ] 

Hive QA commented on HIVE-18866:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924385/HIVE-18866.04.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14391 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/11129/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11129/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11129/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12924385 - PreCommit-HIVE-Build

> Semijoin: Implement a Long -> Hash64 vector fast-path
> -
>
> Key: HIVE-18866
> URL: https://issues.apache.org/jira/browse/HIVE-18866
> Project: Hive
>  Issue Type: Improvement
>  Components: Vectorization
>Reporter: Gopal V
>Assignee: Sergey Shelukhin
>Priority: Major
>  Labels: performance
> Attachments: 0001-hash64-WIP.patch, HIVE-18866.01.patch, 
> HIVE-18866.02.patch, HIVE-18866.03.patch, HIVE-18866.04.patch, 
> HIVE-18866.patch, perf-hash64-long.png
>
>
> A significant amount of CPU is wasted with JMM restrictions on byte[] arrays.
> To transform from one Long -> another Long, this goes into a byte[] array, 
> which shows up as a hotspot.
> !perf-hash64-long.png!



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18866) Semijoin: Implement a Long -> Hash64 vector fast-path

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-18866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483559#comment-16483559
 ] 

Hive QA commented on HIVE-18866:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
26s{color} | {color:blue} storage-api in master has 48 extant Findbugs 
warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 13m 
42s{color} | {color:red} branch/itests/hive-jmh cannot run convertXmlToText 
from findbugs {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  2m 
51s{color} | {color:blue} standalone-metastore in master has 216 extant 
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
11s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
11s{color} | {color:red} storage-api: The patch generated 4 new + 109 unchanged 
- 0 fixed = 113 total (was 109) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
10s{color} | {color:red} itests/hive-jmh: The patch generated 8 new + 0 
unchanged - 0 fixed = 8 total (was 0) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
15s{color} | {color:red} standalone-metastore: The patch generated 1 new + 13 
unchanged - 3 fixed = 14 total (was 16) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch has 5 line(s) that end in whitespace. Use git 
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply 
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 10m  
0s{color} | {color:red} patch/itests/hive-jmh cannot run convertXmlToText from 
findbugs {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
30s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 59s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  xml  compile  findbugs  
checkstyle  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-11129/dev-support/hive-personality.sh
 |
| git revision | master / 7b61c76 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11129/yetus/branch-findbugs-itests_hive-jmh.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11129/yetus/diff-checkstyle-storage-api.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11129/yetus/diff-checkstyle-itests_hive-jmh.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-11129/yetus/diff-checkstyle-standalone-metastore.txt
 |
| whitespace |