[jira] [Created] (HIVE-19043) Vectorization: LazySimpleDeserializeRead fewer fields handling broken for Complex Types

2018-03-23 Thread Matt McCline (JIRA) 
Matt McCline created HIVE-19043:
---

 Summary: Vectorization: LazySimpleDeserializeRead fewer fields 
handling broken for Complex Types
 Key: HIVE-19043
 URL: https://issues.apache.org/jira/browse/HIVE-19043
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 3.0.0
Reporter: Matt McCline
Assignee: Matt McCline


Issues were revealed by vectorizing create_struct_table.q



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 66266: HIVE-18994: Handle client connections on failover

2018-03-23 Thread j . prasanth . j 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66266/
---

Review request for hive and Sergey Shelukhin.


Bugs: HIVE-18994
https://issues.apache.org/jira/browse/HIVE-18994


Repository: hive-git


Description
---

HIVE-18994: Handle client connections on failover


Diffs
-

  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestActivePassiveHA.java 
e53826df1e65edbb7f84ea9987208e3018858bce 
  itests/util/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java 
fa5edec1a660d0eedca7f822a8e2d79358b2bc17 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 
90ba752bff6b493c4d0010617bea0a7a9da29876 


Diff: https://reviews.apache.org/r/66266/diff/1/


Testing
---


Thanks,

Prasanth_J

Re: Review Request 66237: HIVE-18971 add HS2 WM metrics for use in Grafana and such

2018-03-23 Thread j . prasanth . j 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66237/#review199922
---




common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/CodahaleMetrics.java
Lines 304 (patched)


why do we need to remove the gauge names? why not add a guage and forget 
about it.



ql/src/java/org/apache/hadoop/hive/ql/exec/tez/GuaranteedTasksAllocator.java
Lines 103 (patched)


nit: whitespace in many places.



ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WmPoolMetrics.java
Lines 159 (patched)


why isn't this tag alone be sufficient? With this tag we can get all 
metrics associated/registered under pool. right?

instead of emitting metrics like
WM__

why not build a wrapper around this getMetrics() which gets all pool names 
and set the tag. So we will have something like

{
"tag.SessionId": "6020e225-f36e-470b-a170-b18e69af6fc8",
"tag.Poolname": "llap",
"NumExecutors": 2,
"NumSessions": 2
}

If you try to run 2 LLAP daemons on the same host, you would get 2 
different metrics with different SessionId. This looks similar to that except 
that only thing that changes here is poolName. Am I missing something?


- Prasanth_J


On March 23, 2018, 8:14 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66237/
> ---
> 
> (Updated March 23, 2018, 8:14 p.m.)
> 
> 
> Review request for hive and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> .
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/metrics/LegacyMetrics.java 
> effe26b6b6 
>   common/src/java/org/apache/hadoop/hive/common/metrics/common/Metrics.java 
> 88c513b8cd 
>   
> common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/CodahaleMetrics.java
>  a43b09db8c 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8d9b5a3194 
>   
> llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java
>  3a2c19a3e6 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/tez/GuaranteedTasksAllocator.java 
> a52928cc7a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/QueryAllocationManager.java 
> 9885ce7221 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WmPoolMetrics.java 
> PRE-CREATION 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java 
> f0e620c684 
>   
> ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezProgressMonitor.java
>  a14cdb609a 
>   ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java 
> 20a5947291 
> 
> 
> Diff: https://reviews.apache.org/r/66237/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

[jira] [Created] (HIVE-19042) set MALLOC_ARENA_MAX for LLAP

2018-03-23 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-19042:
---

 Summary: set MALLOC_ARENA_MAX for LLAP
 Key: HIVE-19042
 URL: https://issues.apache.org/jira/browse/HIVE-19042
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-19041) Thrift deserialization of Partition objects should intern fields

2018-03-23 Thread Vihang Karajgaonkar (JIRA) 
Vihang Karajgaonkar created HIVE-19041:
--

 Summary: Thrift deserialization of Partition objects should intern 
fields
 Key: HIVE-19041
 URL: https://issues.apache.org/jira/browse/HIVE-19041
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Affects Versions: 2.3.2, 3.0.0
Reporter: Vihang Karajgaonkar
Assignee: Vihang Karajgaonkar


When a client is creating large number of partitions, the thrift objects are 
deserialized into Partition objects. The read method of these objects does not 
intern the inputformat, location, outputformat which cause large number of 
duplicate Strings in the HMS memory. We should intern these objects while 
deserialization to reduce memory pressure. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-19040) get_partitions_by_expr() implementation in HiveMetaStore causes backward incompatibility easily

2018-03-23 Thread Aihua Xu (JIRA) 
Aihua Xu created HIVE-19040:
---

 Summary: get_partitions_by_expr() implementation  in HiveMetaStore 
causes backward incompatibility easily
 Key: HIVE-19040
 URL: https://issues.apache.org/jira/browse/HIVE-19040
 Project: Hive
  Issue Type: Improvement
  Components: Standalone Metastore
Affects Versions: 2.0.0
Reporter: Aihua Xu


In the HiveMetaStore implementation of {{public PartitionsByExprResult 
get_partitions_by_expr(PartitionsByExprRequest req) throws TException}} , an 
expression is serialized into byte array from the client side and passed 
through  PartitionsByExprRequest. Then HMS will deserialize back into the 
expression and filter the partitions by it.

Such partition filtering expression can contain various UDFs. If there are some 
changes to one of the UDFs between different Hive versions, HS2 on the older 
version will serialize the expression in old format which won't be able to be 
deserialized by HMS on the newer version.  One example of that is, GenericUDFIn 
class adds {{transient}}  to the field constantInSet which will cause such 
incompatibility.

One approach I'm thinking is, instead of converting the expression object to 
byte array, we can pass the expression string directly. 

 

 

  

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66264: HIVE-19027

2018-03-23 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66264/
---

(Updated March 23, 2018, 10:47 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
---

Moving conf to minutes instead of seconds.


Bugs: HIVE-19027
https://issues.apache.org/jira/browse/HIVE-19027


Repository: hive-git


Description
---

HIVE-19027

Includes HIVE-18825 and HIVE-19017, which are blockers for this patch.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
8d9b5a3194708ffacabfdb69d6af7d6193dcf156 
  metastore/scripts/upgrade/derby/048-HIVE-14498.derby.sql 
e28d002ad654ce0c7c3cf4a7d77601f97c87f088 
  metastore/scripts/upgrade/derby/hive-schema-3.0.0.derby.sql 
2b1dd5bf7ff3ea0a0293eb5d1fb5e3832f70b820 
  metastore/scripts/upgrade/hive/hive-schema-3.0.0.hive.sql 
339ce4f7ee1dfd32897df8ab87b3c9cbbe5c 
  metastore/scripts/upgrade/mssql/033-HIVE-14498.mssql.sql 
cb41b99abdbedae06311629590d8cd9f0e210d97 
  metastore/scripts/upgrade/mysql/048-HIVE-14498.mysql.sql 
1a791a6d00ffcec5c55e77c36fc71776e9809f1f 
  metastore/scripts/upgrade/oracle/048-HIVE-14498.oracle.sql 
9c3e3cc6f6a892ea492e55d3c6922df598d513a9 
  metastore/scripts/upgrade/postgres/047-HIVE-14498.postgres.sql 
8d1c5b2d7cf4eed4904902f10ff133b2363c109d 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 
75f928b69d3d7b206564216d24be450848a1fe8a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
4eafcde7fb397a1dbf731aa4a9c48f835688ec27 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MaterializedViewTask.java 
de120afbbcd02dc23dff163c14fd7c8fba25db0c 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
da690b4aa45fcd64ae2e3bfb573de5af55f57e16 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java 
ba0d56ae37761091876db52984302d2f7fd34d6f 
  ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDbTxnManager2.java 
efc0cfe94089fe1987e5c1a9740596d5af2a2993 
  
ql/src/test/queries/clientpositive/materialized_view_create_rewrite_time_window.q
 0755d39f1970876369cb5b7cc5e15c8e02f95698 
  
ql/src/test/results/clientpositive/materialized_view_create_rewrite_time_window.q.out
 bfbac3f3492fdd15b55475cd423cf262793cff84 
  standalone-metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 
7206e296fdde1a100ebfbd6fbd097b8b18c3a5e3 
  standalone-metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
b7a3b929bee0e969a261cc5d2b6c0e4b43cb852c 
  
standalone-metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp
 8d9ad254a333fc037baef89f4ed1cd075868e876 
  standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
05a7a29fcc46b26786a90ee41944009db9bc1b0b 
  standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 
620d6ef6bac988101927f1abc59a4b87df3c3f87 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CreationMetadata.java
 ed89b2e9386ee96d55cac005500d147fc437be13 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FindSchemasByColsResp.java
 01bc32d70b373f23c88a749d4ad35020f2cf91ce 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Materialization.java
 dc6dc0d560b56e9847c2ed10009b7d0a7f2cab1c 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SchemaVersion.java
 fb0be40fba434435be7ce2ac183f4d75cbbf4011 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
 8c5ceafb251997e102ee3e828e8adf8e245de9b2 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/WMFullResourcePlan.java
 2fc0e00d6c45706bd25511ad92fae714e25527ea 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/WMGetAllResourcePlanResponse.java
 0ddb2b2dff5bebec1891bc957916f1179dbad693 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/WMGetTriggersForResourePlanResponse.java
 93fa2b7ee92dfafa44b5f521a00f51b8a9e355fd 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/WMValidateResourcePlanResponse.java
 97d33c1da4a20eb409e623c1fabfb68c175a502d 
  standalone-metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 
efe693a65e0eac03bf5cb6e33510ce7e58a4a3ce 
  standalone-metastore/src/gen/thrift/gen-php/metastore/Types.php 
6e3ec622ccc089b2f00f836c5bed109b6f28b66b 
  
standalone-metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote
 a8e83863f7065dac53d430d248fba8c2ebc41ede 
  
standalone-metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
 30214d8df82a0a5bf3d3f9131b7a9c85e8c866f5 
  standalone-metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 
486f0612b9d778445826405201813dfd89cbf924

Review Request 66264: HIVE-19027

2018-03-23 Thread Jesús Camacho Rodríguez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66264/
---

Review request for hive and Ashutosh Chauhan.


Bugs: HIVE-19027
https://issues.apache.org/jira/browse/HIVE-19027


Repository: hive-git


Description
---

HIVE-19027

Includes HIVE-18825 and HIVE-19017, which are blockers for this patch.


Diffs
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
8d9b5a3194708ffacabfdb69d6af7d6193dcf156 
  metastore/scripts/upgrade/derby/048-HIVE-14498.derby.sql 
e28d002ad654ce0c7c3cf4a7d77601f97c87f088 
  metastore/scripts/upgrade/derby/hive-schema-3.0.0.derby.sql 
2b1dd5bf7ff3ea0a0293eb5d1fb5e3832f70b820 
  metastore/scripts/upgrade/hive/hive-schema-3.0.0.hive.sql 
339ce4f7ee1dfd32897df8ab87b3c9cbbe5c 
  metastore/scripts/upgrade/mssql/033-HIVE-14498.mssql.sql 
cb41b99abdbedae06311629590d8cd9f0e210d97 
  metastore/scripts/upgrade/mysql/048-HIVE-14498.mysql.sql 
1a791a6d00ffcec5c55e77c36fc71776e9809f1f 
  metastore/scripts/upgrade/oracle/048-HIVE-14498.oracle.sql 
9c3e3cc6f6a892ea492e55d3c6922df598d513a9 
  metastore/scripts/upgrade/postgres/047-HIVE-14498.postgres.sql 
8d1c5b2d7cf4eed4904902f10ff133b2363c109d 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 
75f928b69d3d7b206564216d24be450848a1fe8a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 
4eafcde7fb397a1dbf731aa4a9c48f835688ec27 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MaterializedViewTask.java 
de120afbbcd02dc23dff163c14fd7c8fba25db0c 
  ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 
da690b4aa45fcd64ae2e3bfb573de5af55f57e16 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDTFGetSplits.java 
ba0d56ae37761091876db52984302d2f7fd34d6f 
  ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDbTxnManager2.java 
efc0cfe94089fe1987e5c1a9740596d5af2a2993 
  standalone-metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.h 
7206e296fdde1a100ebfbd6fbd097b8b18c3a5e3 
  standalone-metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore.cpp 
b7a3b929bee0e969a261cc5d2b6c0e4b43cb852c 
  
standalone-metastore/src/gen/thrift/gen-cpp/ThriftHiveMetastore_server.skeleton.cpp
 8d9ad254a333fc037baef89f4ed1cd075868e876 
  standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 
05a7a29fcc46b26786a90ee41944009db9bc1b0b 
  standalone-metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 
620d6ef6bac988101927f1abc59a4b87df3c3f87 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/CreationMetadata.java
 ed89b2e9386ee96d55cac005500d147fc437be13 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/FindSchemasByColsResp.java
 01bc32d70b373f23c88a749d4ad35020f2cf91ce 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Materialization.java
 dc6dc0d560b56e9847c2ed10009b7d0a7f2cab1c 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SchemaVersion.java
 fb0be40fba434435be7ce2ac183f4d75cbbf4011 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
 8c5ceafb251997e102ee3e828e8adf8e245de9b2 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/WMFullResourcePlan.java
 2fc0e00d6c45706bd25511ad92fae714e25527ea 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/WMGetAllResourcePlanResponse.java
 0ddb2b2dff5bebec1891bc957916f1179dbad693 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/WMGetTriggersForResourePlanResponse.java
 93fa2b7ee92dfafa44b5f521a00f51b8a9e355fd 
  
standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/WMValidateResourcePlanResponse.java
 97d33c1da4a20eb409e623c1fabfb68c175a502d 
  standalone-metastore/src/gen/thrift/gen-php/metastore/ThriftHiveMetastore.php 
efe693a65e0eac03bf5cb6e33510ce7e58a4a3ce 
  standalone-metastore/src/gen/thrift/gen-php/metastore/Types.php 
6e3ec622ccc089b2f00f836c5bed109b6f28b66b 
  
standalone-metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore-remote
 a8e83863f7065dac53d430d248fba8c2ebc41ede 
  
standalone-metastore/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py
 30214d8df82a0a5bf3d3f9131b7a9c85e8c866f5 
  standalone-metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 
486f0612b9d778445826405201813dfd89cbf924 
  standalone-metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 
dd7467c50316203e346bf763ed5633a294d3cb24 
  standalone-metastore/src/gen/thrift/gen-rb/thrift_hive_metastore.rb 
e6787c1e0249a91a5dc30784d46035af77a75ea7 
  
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
 84fac2dfa4f29c1cd67d6a06e1f31e6aad016c75

Re: Review Request 66201: HIVE-19014 utilize YARN-8028 (queue ACL check) in Hive Tez session pool

2018-03-23 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66201/
---

(Updated March 23, 2018, 9:44 p.m.)


Review request for hive and Thejas Nair.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8d9b5a3194 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
a051f90195 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/YarnQueueHelper.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/66201/diff/3/

Changes: https://reviews.apache.org/r/66201/diff/2-3/


Testing
---


Thanks,

Sergey Shelukhin

Re: Review Request 66201: HIVE-19014 utilize YARN-8028 (queue ACL check) in Hive Tez session pool

2018-03-23 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66201/#review199913
---




ql/src/java/org/apache/hadoop/hive/ql/exec/tez/YarnQueueHelper.java
Lines 42 (patched)


whitespace


- Sergey Shelukhin


On March 23, 2018, 9:40 p.m., Sergey Shelukhin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66201/
> ---
> 
> (Updated March 23, 2018, 9:40 p.m.)
> 
> 
> Review request for hive and Thejas Nair.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> see jira
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8d9b5a3194 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
> a051f90195 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/tez/YarnQueueHelper.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/66201/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergey Shelukhin
> 
>

Re: Review Request 66201: HIVE-19014 utilize YARN-8028 (queue ACL check) in Hive Tez session pool

2018-03-23 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66201/
---

(Updated March 23, 2018, 9:40 p.m.)


Review request for hive and Thejas Nair.


Repository: hive-git


Description
---

see jira


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8d9b5a3194 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java 
a051f90195 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/YarnQueueHelper.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/66201/diff/2/

Changes: https://reviews.apache.org/r/66201/diff/1-2/


Testing
---


Thanks,

Sergey Shelukhin

[jira] [Created] (HIVE-19039) Allow DDL change using 'Alter Table' for non-native tables

2018-03-23 Thread Tauseef Hussain (JIRA) 
Tauseef Hussain created HIVE-19039:
--

 Summary: Allow DDL change using 'Alter Table' for non-native tables
 Key: HIVE-19039
 URL: https://issues.apache.org/jira/browse/HIVE-19039
 Project: Hive
  Issue Type: New Feature
  Components: Hive
Affects Versions: 2.3.2
Reporter: Tauseef Hussain


Currently schema changes for non-native tables is limited to adding/deleting 
table properties.

For example, attempting to add column to non-native table results in below:
{code:java}
[Code: 10134, SQL State: 42000] Error while compiling statement: FAILED: 
SemanticException [Error 10134]: ALTER TABLE can only be used for [ADDPROPS, 
DROPPROPS] to a non-native table{code}
Request is to allow all schema/DDL changes which are permitted for Hive native 
tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: PTest stuck

2018-03-23 Thread Vihang Karajgaonkar 
It doesn't seem to have executors to run the job. Check #9785

On Fri, Mar 23, 2018 at 1:23 PM, Deepak Jaiswal 
wrote:

> The tests still seem to be stuck. Does it need a restart?
>
> Regards,
> Deepak
>
> On 3/23/18, 6:41 AM, "Peter Vary"  wrote:
>
> Pushed the change to master.
>
> > On Mar 23, 2018, at 2:30 PM, Adam Szita  wrote:
> >
> > This is caused by a whitespace error
> > in itests/src/test/resources/testconfiguration.properties, fixing
> > by HIVE-19036
> >
> > On 22 March 2018 at 21:55, Jesus Camacho Rodriguez <
> > jcamachorodrig...@hortonworks.com> wrote:
> >
> >> There seems to be something wrong still, there are +330 failures,
> it seems
> >> like test config file is not being picked up correctly. What I have
> seen so
> >> far is that TestCliDriver is picking up q files that it should not,
> while
> >> TestMiniLlapLocalCliDriver does not seem to be running at all.
> >>
> >> -Jesús
> >>
> >>
> >> On 3/22/18, 8:06 AM, "Peter Vary"  wrote:
> >>
> >>The disks was full.
> >>Removed log files, and test runs.
> >>
> >>Restarted ptest
> >>
> >>Please reschedule your tests please.
> >>
> >>Thanks,
> >>Peter
> >>
> >>> On Mar 22, 2018, at 9:29 AM, Prasanth Jayachandran <
> >> pjayachand...@hortonworks.com> wrote:
> >>>
> >>> The tests aren’t running anymore.
> >>>
> >>> Thanks
> >>> Prasanth
> >>>
>  On Mar 21, 2018, at 6:52 PM, Vihang Karajgaonkar <
> >> vih...@cloudera.com> wrote:
> 
>  Yeah. Looks like there was something wrong since the job was
> >> running over 5
>  hrs. I aborted the job and updated that JIRA informing the author
> >> about the
>  same. Lets monitor if the issue happens with other jobs as well.
> 
>  On Wed, Mar 21, 2018 at 5:46 PM, Prasanth Jayachandran <
>  pjayachand...@hortonworks.com> wrote:
> 
> > https://builds.apache.org/view/H-L/view/Hive/job/
> > PreCommit-HIVE-Build/9735/ seems to be stuck in git gc loop.
> >> Anyone know
> > what might be happening? Should we kill it and see if it happens
> >> again?
> >
> > Thanks
> > Prasanth
> >
> >>>
> >>
> >>
> >>
> >>
>
>
>
>

Re: PTest stuck

2018-03-23 Thread Deepak Jaiswal 
The tests still seem to be stuck. Does it need a restart?

Regards,
Deepak

On 3/23/18, 6:41 AM, "Peter Vary"  wrote:

Pushed the change to master.

> On Mar 23, 2018, at 2:30 PM, Adam Szita  wrote:
> 
> This is caused by a whitespace error
> in itests/src/test/resources/testconfiguration.properties, fixing
> by HIVE-19036
> 
> On 22 March 2018 at 21:55, Jesus Camacho Rodriguez <
> jcamachorodrig...@hortonworks.com> wrote:
> 
>> There seems to be something wrong still, there are +330 failures, it 
seems
>> like test config file is not being picked up correctly. What I have seen 
so
>> far is that TestCliDriver is picking up q files that it should not, while
>> TestMiniLlapLocalCliDriver does not seem to be running at all.
>> 
>> -Jesús
>> 
>> 
>> On 3/22/18, 8:06 AM, "Peter Vary"  wrote:
>> 
>>The disks was full.
>>Removed log files, and test runs.
>> 
>>Restarted ptest
>> 
>>Please reschedule your tests please.
>> 
>>Thanks,
>>Peter
>> 
>>> On Mar 22, 2018, at 9:29 AM, Prasanth Jayachandran <
>> pjayachand...@hortonworks.com> wrote:
>>> 
>>> The tests aren’t running anymore.
>>> 
>>> Thanks
>>> Prasanth
>>> 
 On Mar 21, 2018, at 6:52 PM, Vihang Karajgaonkar <
>> vih...@cloudera.com> wrote:
 
 Yeah. Looks like there was something wrong since the job was
>> running over 5
 hrs. I aborted the job and updated that JIRA informing the author
>> about the
 same. Lets monitor if the issue happens with other jobs as well.
 
 On Wed, Mar 21, 2018 at 5:46 PM, Prasanth Jayachandran <
 pjayachand...@hortonworks.com> wrote:
 
> https://builds.apache.org/view/H-L/view/Hive/job/
> PreCommit-HIVE-Build/9735/ seems to be stuck in git gc loop.
>> Anyone know
> what might be happening? Should we kill it and see if it happens
>> again?
> 
> Thanks
> Prasanth
> 
>>> 
>> 
>> 
>> 
>>

Re: Review Request 66237: HIVE-18971 add HS2 WM metrics for use in Grafana and such

2018-03-23 Thread Sergey Shelukhin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66237/
---

(Updated March 23, 2018, 8:14 p.m.)


Review request for hive and Prasanth_J.


Repository: hive-git


Description
---

.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/metrics/LegacyMetrics.java 
effe26b6b6 
  common/src/java/org/apache/hadoop/hive/common/metrics/common/Metrics.java 
88c513b8cd 
  
common/src/java/org/apache/hadoop/hive/common/metrics/metrics2/CodahaleMetrics.java
 a43b09db8c 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8d9b5a3194 
  
llap-server/src/java/org/apache/hadoop/hive/llap/io/api/impl/LlapRecordReader.java
 3a2c19a3e6 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/GuaranteedTasksAllocator.java 
a52928cc7a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/QueryAllocationManager.java 
9885ce7221 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WmPoolMetrics.java 
PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java 
f0e620c684 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/TezProgressMonitor.java
 a14cdb609a 
  ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestWorkloadManager.java 
20a5947291 


Diff: https://reviews.apache.org/r/66237/diff/2/

Changes: https://reviews.apache.org/r/66237/diff/1-2/


Testing
---


Thanks,

Sergey Shelukhin

[GitHub] hive pull request #327: HIVE-19038: Fixed inner class format for TokenRenewe...

2018-03-23 Thread arunmahadevan 
GitHub user arunmahadevan opened a pull request:

https://github.com/apache/hive/pull/327

HIVE-19038: Fixed inner class format for TokenRenewer in META-INF/services



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/arunmahadevan/hive HIVE-19038

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/327.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #327


commit 306841bc21f65cdaf7845615095f37eb861ccde9
Author: Arun Mahadevan 
Date:   2018-03-23T18:28:34Z

HIVE-19038: Fixed inner class format for TokenRenewer in META-INF/services




---

[jira] [Created] (HIVE-19038) LLAP: Service loader throws "Provider not found" exception if hive-llap-server is in class path while loading tokens

2018-03-23 Thread Arun Mahadevan (JIRA) 
Arun Mahadevan created HIVE-19038:
-

 Summary: LLAP: Service loader throws "Provider not found" 
exception if hive-llap-server is in class path while loading tokens
 Key: HIVE-19038
 URL: https://issues.apache.org/jira/browse/HIVE-19038
 Project: Hive
  Issue Type: Bug
Reporter: Arun Mahadevan
Assignee: Arun Mahadevan


While testing storm in secure mode, the hive-llap-server jar file was included 
in the class path and resulted in the below exception while trying to renew 
credentials when invoking "org.apache.hadoop.security.token.Token.getRenewer"

 

 
{noformat}
java.util.ServiceConfigurationError: 
org.apache.hadoop.security.token.TokenRenewer: Provider 
org.apache.hadoop.hive.llap.security.LlapTokenIdentifier.Renewer not found at 
java.util.ServiceLoader.fail(ServiceLoader.java:239) ~[?:1.8.0_161] at 
java.util.ServiceLoader.access$300(ServiceLoader.java:185) ~[?:1.8.0_161] at 
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:372) 
~[?:1.8.0_161] at 
java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) 
~[?:1.8.0_161] at java.util.ServiceLoader$1.next(ServiceLoader.java:480) 
~[?:1.8.0_161] at 
org.apache.hadoop.security.token.Token.getRenewer(Token.java:463) 
~[hadoop-common-3.0.0.3.0.0.0-1064.jar:?] at 
org.apache.hadoop.security.token.Token.renew(Token.java:490) 
~[hadoop-common-3.0.0.3.0.0.0-1064.jar:?] at 
org.apache.storm.hdfs.security.AutoHDFS.doRenew(AutoHDFS.java:159) 
~[storm-autocreds-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
org.apache.storm.common.AbstractAutoCreds.renew(AbstractAutoCreds.java:104) 
~[storm-autocreds-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_161] at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 
~[?:1.8.0_161] at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_161] at java.lang.reflect.Method.invoke(Method.java:498) 
~[?:1.8.0_161] at 
clojure.lang.Reflector.invokeMatchingMethod(Reflector.java:93) 
~[clojure-1.7.0.jar:?] at 
clojure.lang.Reflector.invokeInstanceMethod(Reflector.java:28) 
~[clojure-1.7.0.jar:?] at 
org.apache.storm.daemon.nimbus$renew_credentials$fn__9121$fn__9126.invoke(nimbus.clj:1450)
 ~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
org.apache.storm.daemon.nimbus$renew_credentials$fn__9121.invoke(nimbus.clj:1449)
 ~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
org.apache.storm.daemon.nimbus$renew_credentials.invoke(nimbus.clj:1439) 
~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
org.apache.storm.daemon.nimbus$fn__9547$exec_fn__3301__auto9548$fn__9567.invoke(nimbus.clj:2521)
 ~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
org.apache.storm.timer$schedule_recurring$this__1656.invoke(timer.clj:105) 
~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
org.apache.storm.timer$mk_timer$fn__1639$fn__1640.invoke(timer.clj:50) 
~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
org.apache.storm.timer$mk_timer$fn__1639.invoke(timer.clj:42) 
~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
clojure.lang.AFn.run(AFn.java:22) ~[clojure-1.7.0.jar:?] at 
java.lang.Thread.run(Thread.java:748) [?:1.8.0_161] 2018-03-22 22:08:59.088 
o.a.s.util timer [ERROR] Halting process: ("Error when processing an event") 
java.lang.RuntimeException: ("Error when processing an event") at 
org.apache.storm.util$exit_process_BANG_.doInvoke(util.clj:341) 
~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
clojure.lang.RestFn.invoke(RestFn.java:423) ~[clojure-1.7.0.jar:?] at 
org.apache.storm.daemon.nimbus$nimbus_data$fn__8334.invoke(nimbus.clj:221) 
~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
org.apache.storm.timer$mk_timer$fn__1639$fn__1640.invoke(timer.clj:71) 
~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
org.apache.storm.timer$mk_timer$fn__1639.invoke(timer.clj:42) 
~[storm-core-1.2.1.3.0.0.0-1064.jar:1.2.1.3.0.0.0-1064] at 
clojure.lang.AFn.run(AFn.java:22) ~[clojure-1.7.0.jar:?] at 
java.lang.Thread.run(Thread.java:748) [?:1.8.0_161]{noformat}
 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66204: HIVE-19017: Add util function to determine if 2 ValidWriteIdLists are at the same committed ID

2018-03-23 Thread Jesús Camacho Rodríguez 


> On March 22, 2018, 12:26 a.m., Jesús Camacho Rodríguez wrote:
> > storage-api/src/java/org/apache/hive/common/util/TxnIdUtils.java
> > Lines 41 (patched)
> > 
> >
> > Can minOpenId be null for both lists? What's the semantics then?
> 
> Jason Dere wrote:
> Ok, looking at TxnUtils.createValidReaderWriteIdList(), it looks like it 
> could potentially be null in the case that there is no min open ID (they are 
> all committed?). Eugene does that sound right?
> In which case, if both are in that condition then I suspect they would be 
> at the same commit point only if they have the same highwater mark (if they 
> have different highwater marks then one of them has more committed IDs than 
> the other).
> I'll update the logic and tests.
> 
> Jason Dere wrote:
> Spoke to Eugene and looked at this again, actually I don't think the 
> minOpenID is needed at all for this check, just the highwater mark and the 
> invalid IDs list. I've removed the minOpenID checking, and the rest of the 
> checks are unchanged. Also added some additional test cases.

LGTM.


- Jesús


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66204/#review199731
---


On March 23, 2018, 5:25 p.m., Jason Dere wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66204/
> ---
> 
> (Updated March 23, 2018, 5:25 p.m.)
> 
> 
> Review request for hive, Eugene Koifman, Gopal V, and Jesús Camacho Rodríguez.
> 
> 
> Bugs: HIVE-19017
> https://issues.apache.org/jira/browse/HIVE-19017
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Create utility comparison function and some simple tests.
> 
> 
> Diffs
> -
> 
>   storage-api/src/java/org/apache/hive/common/util/TxnIdUtils.java 
> PRE-CREATION 
>   storage-api/src/test/org/apache/hive/common/util/TestTxnIdUtils.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/66204/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jason Dere
> 
>

Re: Review Request 66204: HIVE-19017: Add util function to determine if 2 ValidWriteIdLists are at the same committed ID

2018-03-23 Thread Jason Dere 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66204/
---

(Updated March 23, 2018, 5:25 p.m.)


Review request for hive, Eugene Koifman, Gopal V, and Jesús Camacho Rodríguez.


Changes
---

Removed MinOpenID checking, not necessary.


Bugs: HIVE-19017
https://issues.apache.org/jira/browse/HIVE-19017


Repository: hive-git


Description
---

Create utility comparison function and some simple tests.


Diffs (updated)
-

  storage-api/src/java/org/apache/hive/common/util/TxnIdUtils.java PRE-CREATION 
  storage-api/src/test/org/apache/hive/common/util/TestTxnIdUtils.java 
PRE-CREATION 


Diff: https://reviews.apache.org/r/66204/diff/2/

Changes: https://reviews.apache.org/r/66204/diff/1-2/


Testing
---


Thanks,

Jason Dere

Re: Review Request 66204: HIVE-19017: Add util function to determine if 2 ValidWriteIdLists are at the same committed ID

2018-03-23 Thread Jason Dere 


> On March 22, 2018, 12:26 a.m., Jesús Camacho Rodríguez wrote:
> > storage-api/src/java/org/apache/hive/common/util/TxnIdUtils.java
> > Lines 41 (patched)
> > 
> >
> > Can minOpenId be null for both lists? What's the semantics then?
> 
> Jason Dere wrote:
> Ok, looking at TxnUtils.createValidReaderWriteIdList(), it looks like it 
> could potentially be null in the case that there is no min open ID (they are 
> all committed?). Eugene does that sound right?
> In which case, if both are in that condition then I suspect they would be 
> at the same commit point only if they have the same highwater mark (if they 
> have different highwater marks then one of them has more committed IDs than 
> the other).
> I'll update the logic and tests.

Spoke to Eugene and looked at this again, actually I don't think the minOpenID 
is needed at all for this check, just the highwater mark and the invalid IDs 
list. I've removed the minOpenID checking, and the rest of the checks are 
unchanged. Also added some additional test cases.


- Jason


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66204/#review199731
---


On March 22, 2018, 12:16 a.m., Jason Dere wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66204/
> ---
> 
> (Updated March 22, 2018, 12:16 a.m.)
> 
> 
> Review request for hive, Eugene Koifman, Gopal V, and Jesús Camacho Rodríguez.
> 
> 
> Bugs: HIVE-19017
> https://issues.apache.org/jira/browse/HIVE-19017
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Create utility comparison function and some simple tests.
> 
> 
> Diffs
> -
> 
>   storage-api/src/java/org/apache/hive/common/util/TxnIdUtils.java 
> PRE-CREATION 
>   storage-api/src/test/org/apache/hive/common/util/TestTxnIdUtils.java 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/66204/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Jason Dere
> 
>

[jira] [Created] (HIVE-19037) Vectorization: Miscellaneous cleanup

2018-03-23 Thread Matt McCline (JIRA) 
Matt McCline created HIVE-19037:
---

 Summary: Vectorization: Miscellaneous cleanup
 Key: HIVE-19037
 URL: https://issues.apache.org/jira/browse/HIVE-19037
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 3.0.0
Reporter: Matt McCline
Assignee: Matt McCline



# Extraneous INFO logging in VectorReduceSinkCommonOperator
# NPE in EXPLAIN for some SelectColumnIsTrue vector expressions



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: PTest stuck

2018-03-23 Thread Peter Vary 
Pushed the change to master.

> On Mar 23, 2018, at 2:30 PM, Adam Szita  wrote:
> 
> This is caused by a whitespace error
> in itests/src/test/resources/testconfiguration.properties, fixing
> by HIVE-19036
> 
> On 22 March 2018 at 21:55, Jesus Camacho Rodriguez <
> jcamachorodrig...@hortonworks.com> wrote:
> 
>> There seems to be something wrong still, there are +330 failures, it seems
>> like test config file is not being picked up correctly. What I have seen so
>> far is that TestCliDriver is picking up q files that it should not, while
>> TestMiniLlapLocalCliDriver does not seem to be running at all.
>> 
>> -Jesús
>> 
>> 
>> On 3/22/18, 8:06 AM, "Peter Vary"  wrote:
>> 
>>The disks was full.
>>Removed log files, and test runs.
>> 
>>Restarted ptest
>> 
>>Please reschedule your tests please.
>> 
>>Thanks,
>>Peter
>> 
>>> On Mar 22, 2018, at 9:29 AM, Prasanth Jayachandran <
>> pjayachand...@hortonworks.com> wrote:
>>> 
>>> The tests aren’t running anymore.
>>> 
>>> Thanks
>>> Prasanth
>>> 
 On Mar 21, 2018, at 6:52 PM, Vihang Karajgaonkar <
>> vih...@cloudera.com> wrote:
 
 Yeah. Looks like there was something wrong since the job was
>> running over 5
 hrs. I aborted the job and updated that JIRA informing the author
>> about the
 same. Lets monitor if the issue happens with other jobs as well.
 
 On Wed, Mar 21, 2018 at 5:46 PM, Prasanth Jayachandran <
 pjayachand...@hortonworks.com> wrote:
 
> https://builds.apache.org/view/H-L/view/Hive/job/
> PreCommit-HIVE-Build/9735/ seems to be stuck in git gc loop.
>> Anyone know
> what might be happening? Should we kill it and see if it happens
>> again?
> 
> Thanks
> Prasanth
> 
>>> 
>> 
>> 
>> 
>>

Re: PTest stuck

2018-03-23 Thread Adam Szita 
This is caused by a whitespace error
in itests/src/test/resources/testconfiguration.properties, fixing
by HIVE-19036

On 22 March 2018 at 21:55, Jesus Camacho Rodriguez <
jcamachorodrig...@hortonworks.com> wrote:

> There seems to be something wrong still, there are +330 failures, it seems
> like test config file is not being picked up correctly. What I have seen so
> far is that TestCliDriver is picking up q files that it should not, while
> TestMiniLlapLocalCliDriver does not seem to be running at all.
>
> -Jesús
>
>
> On 3/22/18, 8:06 AM, "Peter Vary"  wrote:
>
> The disks was full.
> Removed log files, and test runs.
>
> Restarted ptest
>
> Please reschedule your tests please.
>
> Thanks,
> Peter
>
> > On Mar 22, 2018, at 9:29 AM, Prasanth Jayachandran <
> pjayachand...@hortonworks.com> wrote:
> >
> > The tests aren’t running anymore.
> >
> > Thanks
> > Prasanth
> >
> >> On Mar 21, 2018, at 6:52 PM, Vihang Karajgaonkar <
> vih...@cloudera.com> wrote:
> >>
> >> Yeah. Looks like there was something wrong since the job was
> running over 5
> >> hrs. I aborted the job and updated that JIRA informing the author
> about the
> >> same. Lets monitor if the issue happens with other jobs as well.
> >>
> >> On Wed, Mar 21, 2018 at 5:46 PM, Prasanth Jayachandran <
> >> pjayachand...@hortonworks.com> wrote:
> >>
> >>> https://builds.apache.org/view/H-L/view/Hive/job/
> >>> PreCommit-HIVE-Build/9735/ seems to be stuck in git gc loop.
> Anyone know
> >>> what might be happening? Should we kill it and see if it happens
> again?
> >>>
> >>> Thanks
> >>> Prasanth
> >>>
> >
>
>
>
>

[jira] [Created] (HIVE-19036) Fix whitespace error in testconfiguration.properties after HIVE-14032

2018-03-23 Thread Adam Szita (JIRA) 
Adam Szita created HIVE-19036:
-

 Summary: Fix whitespace error in testconfiguration.properties 
after HIVE-14032
 Key: HIVE-19036
 URL: https://issues.apache.org/jira/browse/HIVE-19036
 Project: Hive
  Issue Type: Bug
Reporter: Adam Szita
Assignee: Adam Szita


HIVE-14032 has introduced a whitespace error {{in 
itests/src/test/resources/testconfiguration.properties}} (left a space after \) 
which caused 300+ test failures in ptest..

The reason ptest didn't pick this error up originally is that commit of that 
Jira had some extra changes compared to the patch uploaded to HIVE-14032.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66209: HIVE-17098 Race condition in Hbase tables

2018-03-23 Thread Oleksiy Sayankin 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66209/
---

(Updated March 23, 2018, 3:06 p.m.)


Review request for hive, Ashutosh Chauhan, Zoltan Haindrich, and Zoltan 
Haindrich.


Changes
---

Fixed code style issues and rebased for new code


Repository: hive-git


Description
---

Initial commit


Diffs (updated)
-

  
hbase-handler/src/java/org/apache/hadoop/hive/hbase/HiveHBaseTableInputFormat.java
 80c6485d7e 


Diff: https://reviews.apache.org/r/66209/diff/2/

Changes: https://reviews.apache.org/r/66209/diff/1-2/


Testing
---


Thanks,

Oleksiy Sayankin

[jira] [Created] (HIVE-19035) Vectorization: Disable exotic field reference form

2018-03-23 Thread Matt McCline (JIRA) 
Matt McCline created HIVE-19035:
---

 Summary: Vectorization: Disable exotic field reference form
 Key: HIVE-19035
 URL: https://issues.apache.org/jira/browse/HIVE-19035
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 3.0.0
Reporter: Matt McCline
Assignee: Matt McCline


We currently don't support exotic field references like get a struct field from 
array returns a type array.  Attempt causes 
ClassCastException in VectorizationContext that kills query planning.

The Q file is input_testxpath3.q



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-19034) hadoop fs test can check srcipt ok, but beeline -f report no such file

2018-03-23 Thread fengxianghui (JIRA) 
fengxianghui created HIVE-19034:
---

 Summary: hadoop fs test can check srcipt ok, but beeline -f report 
no such file
 Key: HIVE-19034
 URL: https://issues.apache.org/jira/browse/HIVE-19034
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 1.3.0, beeline-cli-branch
 Environment: java version: 1.8.0_112-b15

hadoop version: 2.7.2

hive version:1.3.0

hive JDBS version: 1.3.0

beeline version: 1.3.0
Reporter: fengxianghui
 Fix For: 1.3.0


I test like this

HQL=hdfs://hacluster/tmp/ff.hql

if hadoop fs -test -f ${HQL}

then

   beeline -f ${HQL}

fi

test ${HQL} ok, but beeline report ${HQL} no such file or directory



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)