[jira] [Created] (HIVE-16549) Fix an incompatible change in PredicateLeafImpl from HIVE-15269

2017-04-26 Thread Owen O'Malley (JIRA) 
Owen O'Malley created HIVE-16549:


 Summary: Fix an incompatible change in PredicateLeafImpl from 
HIVE-15269
 Key: HIVE-16549
 URL: https://issues.apache.org/jira/browse/HIVE-16549
 Project: Hive
  Issue Type: Bug
Reporter: Owen O'Malley
Assignee: Owen O'Malley


HIVE-15269 added a parameter to the constructor for PredicateLeafImpl for a 
configuration object. The configuration object is only used for the new 
LiteralDelegates.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16548) LLAP: EncodedReaderImpl.addOneCompressionBuffer throws NPE

2017-04-26 Thread Rajesh Balamohan (JIRA) 
Rajesh Balamohan created HIVE-16548:
---

 Summary: LLAP: EncodedReaderImpl.addOneCompressionBuffer throws NPE
 Key: HIVE-16548
 URL: https://issues.apache.org/jira/browse/HIVE-16548
 Project: Hive
  Issue Type: Bug
  Components: llap
Reporter: Rajesh Balamohan


Env: Based on apr-25 apache master codebase.

{noformat}
Caused by: java.io.IOException: java.lang.IllegalArgumentException: Buffer size 
too small. size = 65536 needed = 3762509
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedStream(EncodedReaderImpl.java:695)
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:454)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:420)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:242)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:239)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:239)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:93)
... 6 more
Caused by: java.lang.IllegalArgumentException: Buffer size too small. size = 
65536 needed = 3762509
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.addOneCompressionBuffer(EncodedReaderImpl.java:1223)
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.prepareRangesForCompressedRead(EncodedReaderImpl.java:813)
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedStream(EncodedReaderImpl.java:685)
... 15 more

Caused by: java.io.IOException: java.io.IOException: 
java.lang.NullPointerException
at 
org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
at 
org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
at 
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
at 
org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:79)
at 
org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:33)
at 
org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
at 
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:151)
at 
org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
at 
org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:62)
... 17 more
Caused by: java.io.IOException: java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedStream(EncodedReaderImpl.java:695)
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:454)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:420)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:242)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader$4.run(OrcEncodedDataReader.java:239)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:239)
at 
org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.callInternal(OrcEncodedDataReader.java:93)
... 6 more
Caused by: java.lang.NullPointerException
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.addOneCompressionBuffer(EncodedReaderImpl.java:1282)
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.prepareRangesForCompressedRead(EncodedReaderImpl.java:813)
at 
org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedStream(EncodedReaderImpl.java:685)
... 15 more  
{noformat}





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16547) LLAP: may not unlock buffers in some cases

2017-04-26 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-16547:
---

 Summary: LLAP: may not unlock buffers in some cases
 Key: HIVE-16547
 URL: https://issues.apache.org/jira/browse/HIVE-16547
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin


Actually this is a pretty major bug, no idea how it slipped before.
If last RG is not selected, dictionary buffers will not be unlocked because of 
bad assumptions about what isLastRg means (last in processing vs last in the 
stripe).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 50787: Add a timezone-aware timestamp

2017-04-26 Thread Rui Li 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50787/
---

(Updated April 27, 2017, 2:32 a.m.)


Review request for hive.


Bugs: HIVE-14412
https://issues.apache.org/jira/browse/HIVE-14412


Repository: hive-git


Description
---

The 1st patch to add timezone-aware timestamp.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/type/TimestampTZ.java 
PRE-CREATION 
  common/src/test/org/apache/hadoop/hive/common/type/TestTimestampTZ.java 
PRE-CREATION 
  contrib/src/test/queries/clientnegative/serde_regex.q a676338 
  contrib/src/test/queries/clientpositive/serde_regex.q d75d607 
  contrib/src/test/results/clientnegative/serde_regex.q.out 58b1c02 
  contrib/src/test/results/clientpositive/serde_regex.q.out 2984293 
  hbase-handler/src/test/queries/positive/hbase_timestamp.q 0350afe 
  hbase-handler/src/test/results/positive/hbase_timestamp.q.out 3918121 
  itests/hive-blobstore/src/test/queries/clientpositive/orc_format_part.q 
358eccd 
  
itests/hive-blobstore/src/test/queries/clientpositive/orc_nonstd_partitions_loc.q
 c462538 
  itests/hive-blobstore/src/test/queries/clientpositive/rcfile_format_part.q 
c563d3a 
  
itests/hive-blobstore/src/test/queries/clientpositive/rcfile_nonstd_partitions_loc.q
 d17c281 
  itests/hive-blobstore/src/test/results/clientpositive/orc_format_part.q.out 
5d1319f 
  
itests/hive-blobstore/src/test/results/clientpositive/orc_nonstd_partitions_loc.q.out
 70e72f7 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_format_part.q.out 
bed10ab 
  
itests/hive-blobstore/src/test/results/clientpositive/rcfile_nonstd_partitions_loc.q.out
 c6442f9 
  jdbc/src/java/org/apache/hive/jdbc/HiveBaseResultSet.java ade1900 
  jdbc/src/java/org/apache/hive/jdbc/JdbcColumn.java 38918f0 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 8dc5f2e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java f8b55da 
  ql/src/java/org/apache/hadoop/hive/ql/exec/SerializationUtilities.java 
247d589 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/TypeConverter.java
 38308c9 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0cf9205 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 0721b92 
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g d98a663 
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 8598fae 
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 8f8eab0 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java bda2050 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToBoolean.java 7cdf2c3 
  ql/src/java/org/apache/hadoop/hive/ql/udf/UDFToString.java 5cacd59 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDF.java 68d98f5 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFDate.java 5a31e61 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFToTimestampTZ.java 
PRE-CREATION 
  
ql/src/test/org/apache/hadoop/hive/ql/parse/TestSQL11ReservedKeyWordsNegative.java
 0dc6b19 
  ql/src/test/queries/clientnegative/serde_regex.q c9cfc7d 
  ql/src/test/queries/clientnegative/serde_regex2.q a29bb9c 
  ql/src/test/queries/clientnegative/serde_regex3.q 4e91f06 
  ql/src/test/queries/clientpositive/create_like.q bd39731 
  ql/src/test/queries/clientpositive/join43.q 12c45a6 
  ql/src/test/queries/clientpositive/serde_regex.q e21c6e1 
  ql/src/test/queries/clientpositive/timestamptz.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_1.q PRE-CREATION 
  ql/src/test/queries/clientpositive/timestamptz_2.q PRE-CREATION 
  ql/src/test/results/clientnegative/serde_regex.q.out a1ec5ca 
  ql/src/test/results/clientnegative/serde_regex2.q.out 374675d 
  ql/src/test/results/clientnegative/serde_regex3.q.out dc0a0e2 
  ql/src/test/results/clientpositive/create_like.q.out ff2e752 
  ql/src/test/results/clientpositive/join43.q.out e8c7278 
  ql/src/test/results/clientpositive/serde_regex.q.out 7bebb0c 
  ql/src/test/results/clientpositive/timestamptz.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_1.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/timestamptz_2.q.out PRE-CREATION 
  serde/if/serde.thrift 1d40d5a 
  serde/src/gen/thrift/gen-cpp/serde_constants.h 8785bd2 
  serde/src/gen/thrift/gen-cpp/serde_constants.cpp 907acf2 
  
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/serdeConstants.java
 2578d3e 
  serde/src/gen/thrift/gen-php/org/apache/hadoop/hive/serde/Types.php ea2dbbe 
  serde/src/gen/thrift/gen-py/org_apache_hadoop_hive_serde/constants.py e3b24eb 
  serde/src/gen/thrift/gen-rb/serde_constants.rb 15efaea 
  serde/src/java/org/apache/hadoop/hive/serde2/SerDeUtils.java 5ecfbca 
  
serde/src/java/org/apache/hadoop/hive/serde2/binarysortable/BinarySortableSerDe.java
 89e15c3

How to set Hive Cli memory through templeton

2017-04-26 Thread Ranjan Banerjee 
Hi,
   I am trying to set the hive client memory through templeton. I can achieve 
this through Ambari by overwriting the HADOOP_CLIENT_OPTS in the hive-env 
template. How do I overwrite the environment variables of hive through 
templeton? Any response will be much appreciated

Thanks
Ranjan

Review Request 58777: HIVE-16546: LLAP: Fail map join tasks if hash table memory exceeds threshold

2017-04-26 Thread j . prasanth . j 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/58777/
---

Review request for hive, Gunther Hagleitner, Sergey Shelukhin, and Siddharth 
Seth.


Bugs: HIVE-16546
https://issues.apache.org/jira/browse/HIVE-16546


Repository: hive-git


Description
---

HIVE-16546: LLAP: Fail map join tasks if hash table memory exceeds threshold


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/MemoryEstimate.java 
PRE-CREATION 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 
d3ea824c21f2fbf98177cb12a18019416f36a3f9 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/BytesBytesMultiHashMap.java
 04e24bd1a556a25f9cc68dc3ba00e315bf16cd53 
  ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HashMapWrapper.java 
a3bccc6b4f34bc05ba854c6df1bf117bcc23bc25 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/HybridHashTableContainer.java
 04e89e8e7494e85ee057bd7886b89da028c6cc20 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinBytesTableContainer.java
 c86e5f541e4113fbc8e4dd69338115f81126ce8c 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/persistence/MapJoinTableContainer.java
 6d71fef65c801ee3e9f4b9e11565a40e0db9cc4e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HashTableLoader.java 
7b13e90125343669cb7429740e3722901b0f78db 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastBytesHashMap.java
 6242dafb002721624b7c882967f7b5347cccef97 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastBytesHashMultiSet.java
 1a41961ebaf90f386cb736a101e251ea7f0e162c 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastBytesHashSet.java
 331867c54af867d5a18af913d25409c02c1b01fb 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastBytesHashTable.java
 b93e9779e6ae030855105cdd7f76dc5ba8665c94 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastHashTable.java
 b6db3bcc1df9020a19d483925a4c68e925540369 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastHashTableLoader.java
 49ecdd1580bb762b757a73c1121935bf28d9d8ef 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastKeyStore.java
 be51693c557382bc55bbf611a3b05a51603126fd 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastLongHashMap.java
 6fe98f9aed81c7fec5396a38104f78902d33562f 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastLongHashMultiSet.java
 9140aee1cb2d9b4ba5e480720ca787b19c751193 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastLongHashSet.java
 d3efb118edda493362f79b690ecc30540b07d86a 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastLongHashTable.java
 8bfa07c78d046227e10fc6ebd237fd6c17386a30 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastMultiKeyHashMap.java
 add4788b61a355e9971b27687da06b9438e6ddf2 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastMultiKeyHashMultiSet.java
 faefdbb4365e296cf83ee29d51df381ba8831b15 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastMultiKeyHashSet.java
 532891079542ee5c8c1fe3c6b4923a525bd8a730 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastStringHashMap.java
 f13034ff6bef9003abf81ff36240baa34b9a718b 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastStringHashMultiSet.java
 53ad7b4b65f22c35c459e6dba94ad0d83912caea 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastStringHashSet.java
 723c729bcdb47aad47f372f6d384046929d5858f 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastTableContainer.java
 05f1cf1b3ff6a099fed9b8d419811edd5edc4860 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/fast/VectorMapJoinFastValueStore.java
 f9c5b340d5d254ec703dfb5f31d3b662876e4ea3 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/hashtable/VectorMapJoinHashTable.java
 c7e585c29ba27450072c8eeeb00416fd0efee107 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedHashSet.java
 93a89d7716e7200a4a09cf228108d4f2d78f8b37 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedHashTable.java
 5fe7861cb118e092f4635488dea3435129e74fb8 
  
ql/src/java/org/apache/hadoop/hive/ql/exec/vector/mapjoin/optimized/VectorMapJoinOptimizedStringHashSet.java
 f921b9ca5464958f1c7f351000b6ec17df8fab39 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ConvertJoinMapJoin.java 
ad77e87a05b6d00a370a8ca225778300141bb025

[jira] [Created] (HIVE-16546) LLAP: Fail map join tasks if hash table memory exceeds threshold

2017-04-26 Thread Prasanth Jayachandran (JIRA) 
Prasanth Jayachandran created HIVE-16546:


 Summary: LLAP: Fail map join tasks if hash table memory exceeds 
threshold
 Key: HIVE-16546
 URL: https://issues.apache.org/jira/browse/HIVE-16546
 Project: Hive
  Issue Type: Bug
  Components: llap
Affects Versions: 3.0.0
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran


When map join task is running in llap, it can potentially use lot more memory 
than its limit which could be memory per executor or no conditional task size. 
If it uses more memory, it can adversely affect other query performance or it 
can even bring down the daemon. In such cases, it is better to fail the query 
than to bring down the daemon. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16545) LLAP: bug in arena size determination logic

2017-04-26 Thread Sergey Shelukhin (JIRA) 
Sergey Shelukhin created HIVE-16545:
---

 Summary: LLAP: bug in arena size determination logic
 Key: HIVE-16545
 URL: https://issues.apache.org/jira/browse/HIVE-16545
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16544) NPE during IMPORT when database does not exist

2017-04-26 Thread Prasanth Jayachandran (JIRA) 
Prasanth Jayachandran created HIVE-16544:


 Summary: NPE during IMPORT when database does not exist
 Key: HIVE-16544
 URL: https://issues.apache.org/jira/browse/HIVE-16544
 Project: Hive
  Issue Type: Bug
  Components: repl
Affects Versions: 3.0.0
Reporter: Prasanth Jayachandran


There was some old database from which I exported a table. Recently I dropped 
this database. When I tried to import the table (for which corresponding 
database was dropped), NPE is thrown

{code}
2017-04-26T17:22:24,374 ERROR [d581f7d4-dba2-4a4f-ad02-1ce089d467d7 main]: 
ql.Driver (SessionState.java:printError(1126)) - FAILED: SemanticException 
[Error 10324]: Import Semantic Analyzer Error
org.apache.hadoop.hive.ql.parse.SemanticException: Import Semantic Analyzer 
Error
at 
org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:147)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:258)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:544)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1345)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1477)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1261)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1251)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
at 
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.hooks.Entity.doComputeName(Entity.java:336)
at org.apache.hadoop.hive.ql.hooks.Entity.computeName(Entity.java:330)
at org.apache.hadoop.hive.ql.hooks.Entity.(Entity.java:175)
at 
org.apache.hadoop.hive.ql.hooks.WriteEntity.(WriteEntity.java:65)
at 
org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.createRegularImportTasks(ImportSemanticAnalyzer.java:753)
at 
org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.prepareImport(ImportSemanticAnalyzer.java:297)
at 
org.apache.hadoop.hive.ql.parse.ImportSemanticAnalyzer.analyzeInternal(ImportSemanticAnalyzer.java:138)
... 18 more
{code} 




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16543) Deprecate HIVESTATSAUTOGATHER

2017-04-26 Thread Pengcheng Xiong (JIRA) 
Pengcheng Xiong created HIVE-16543:
--

 Summary: Deprecate HIVESTATSAUTOGATHER
 Key: HIVE-16543
 URL: https://issues.apache.org/jira/browse/HIVE-16543
 Project: Hive
  Issue Type: Sub-task
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong


The overhead for auto collecting table stats is quite low. We can deprecate 
this configuration and treat it always as true.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: ptest not running?

2017-04-26 Thread Vihang Karajgaonkar 
Does anyone have access to https://builds.apache.org/job/PreCommit-Admin/
job? It is supposed to run every 10 min. It shows at the last build ran 10
hours ago.

On Wed, Apr 26, 2017 at 9:28 AM, Chaoyu Tang  wrote:

> It looks the ptest is still having the problem. Either submitting patch in
> JIRA or manual build could not trigger the tests.
>
> On Tue, Apr 25, 2017 at 11:31 PM, Eugene Koifman  >
> wrote:
>
> > I see HIVE-16445 in it – it was picked up automatically
> >
> > On 4/25/17, 5:44 PM, "Vihang Karajgaonkar"  wrote:
> >
> > I see some builds queued up on the pre-commit job. Were these
> submitted
> > manually?
> >
> > On Tue, Apr 25, 2017 at 2:00 PM, Thejas Nair 
> > wrote:
> >
> > > Thanks for the update Vihang!
> > >
> > >
> > > On Tue, Apr 25, 2017 at 12:54 PM, Vihang Karajgaonkar <
> > vih...@cloudera.com
> > > >
> > > wrote:
> > >
> > > > This seems to be related to a larger infrastructure issue. Other
> > projects
> > > > (hadoop, sentry, hbase etc) are facing similar issues as well.
> > > >
> > > > https://issues.apache.org/jira/browse/INFRA-13985
> > > >
> > > > On Tue, Apr 25, 2017 at 11:54 AM, Thejas Nair <
> > thejas.n...@gmail.com>
> > > > wrote:
> > > >
> > > > > Looks like nothing is getting scheduled since yesterday
> evening,
> > even
> > > > with
> > > > > the manual launch
> > > > >
> > > > > https://builds.apache.org/job/PreCommit-HIVE-Build/
> > > > >
> > > > > Can someone with access please take a look ?
> > > > >
> > > > >
> > > > > On Mon, Apr 24, 2017 at 5:26 PM, Eugene Koifman <
> > > > ekoif...@hortonworks.com>
> > > > > wrote:
> > > > >
> > > > > > Something is not working with the part of the flow that
> > triggers the
> > > > > build
> > > > > > automatically.
> > > > > > In the meantime, if you log in with apache credentials, and
> go
> > to
> > > > “Build
> > > > > > with Parameters”
> > > > > > you can manually launch the test by entering the bug number:
> > just the
> > > > > > digits, skip the “HIVE-“ part
> > > > > >
> > > > > >
> > > > > > On 4/24/17, 5:01 PM, "Vihang Karajgaonkar" <
> > vih...@cloudera.com>
> > > > wrote:
> > > > > >
> > > > > > I see that builds are getting picked up at
> > > > > https://builds.apache.org/
> > > > > > job/PreCommit-HIVE-Build/ The PTest server was restarted
> a
> > couple
> > > > of
> > > > > > days
> > > > > > back. If you had a patch in the queue around that time,
> > may be it
> > > > was
> > > > > > lost
> > > > > > due to restart. You may have to re-attach the patch on
> the
> > > upstream
> > > > > > JIRA if
> > > > > > that is the case.
> > > > > >
> > > > > >
> > > > > > On Mon, Apr 24, 2017 at 2:44 PM, Eugene Koifman <
> > > > > > ekoif...@hortonworks.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > > It is not picking up new patches and there are none
> > queued up.
> > > > > Could
> > > > > > > someone check please?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Eugene
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> >
> >
>

[jira] [Created] (HIVE-16542) make merge that targets acid 2.0 table fail-fast

2017-04-26 Thread Eugene Koifman (JIRA) 
Eugene Koifman created HIVE-16542:
-

 Summary: make merge that targets acid 2.0 table fail-fast 
 Key: HIVE-16542
 URL: https://issues.apache.org/jira/browse/HIVE-16542
 Project: Hive
  Issue Type: Sub-task
  Components: Transactions
Affects Versions: 2.2.0
Reporter: Eugene Koifman
Assignee: Eugene Koifman
Priority: Critical


Until HIVE-14947 is fixed, need to add a check so that acid 2.0 tables are not 
written to by Merge stmt that has both Insert and Update clauses



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16541) PTF: Avoid shuffling constant keys for empty OVER()

2017-04-26 Thread Gopal V (JIRA) 
Gopal V created HIVE-16541:
--

 Summary: PTF: Avoid shuffling constant keys for empty OVER()
 Key: HIVE-16541
 URL: https://issues.apache.org/jira/browse/HIVE-16541
 Project: Hive
  Issue Type: Bug
  Components: PTF-Windowing
Affects Versions: 3.0.0
Reporter: Gopal V
Assignee: Gopal V


Generating surrogate keys with 

{code}
select row_number() over() as p_key, * from table; 
{code}

uses a sorted edge with "0 ASC NULLS FIRST" as the sort order.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16540) dynamic_semijoin_user_level is failing on MiniLlap

2017-04-26 Thread Pengcheng Xiong (JIRA) 
Pengcheng Xiong created HIVE-16540:
--

 Summary: dynamic_semijoin_user_level is failing on MiniLlap
 Key: HIVE-16540
 URL: https://issues.apache.org/jira/browse/HIVE-16540
 Project: Hive
  Issue Type: Test
Reporter: Pengcheng Xiong
Assignee: Pengcheng Xiong
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (HIVE-16539) Add PTF tests for blobstores

2017-04-26 Thread JIRA 
Juan Rodríguez Hortalá created HIVE-16539:
-

 Summary: Add PTF tests for blobstores
 Key: HIVE-16539
 URL: https://issues.apache.org/jira/browse/HIVE-16539
 Project: Hive
  Issue Type: Bug
Reporter: Juan Rodríguez Hortalá


This patch covers 6 tests that check the integration of several Partitioned 
Table Functions with blobstores



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: ptest not running?

2017-04-26 Thread Chaoyu Tang 
It looks the ptest is still having the problem. Either submitting patch in
JIRA or manual build could not trigger the tests.

On Tue, Apr 25, 2017 at 11:31 PM, Eugene Koifman 
wrote:

> I see HIVE-16445 in it – it was picked up automatically
>
> On 4/25/17, 5:44 PM, "Vihang Karajgaonkar"  wrote:
>
> I see some builds queued up on the pre-commit job. Were these submitted
> manually?
>
> On Tue, Apr 25, 2017 at 2:00 PM, Thejas Nair 
> wrote:
>
> > Thanks for the update Vihang!
> >
> >
> > On Tue, Apr 25, 2017 at 12:54 PM, Vihang Karajgaonkar <
> vih...@cloudera.com
> > >
> > wrote:
> >
> > > This seems to be related to a larger infrastructure issue. Other
> projects
> > > (hadoop, sentry, hbase etc) are facing similar issues as well.
> > >
> > > https://issues.apache.org/jira/browse/INFRA-13985
> > >
> > > On Tue, Apr 25, 2017 at 11:54 AM, Thejas Nair <
> thejas.n...@gmail.com>
> > > wrote:
> > >
> > > > Looks like nothing is getting scheduled since yesterday evening,
> even
> > > with
> > > > the manual launch
> > > >
> > > > https://builds.apache.org/job/PreCommit-HIVE-Build/
> > > >
> > > > Can someone with access please take a look ?
> > > >
> > > >
> > > > On Mon, Apr 24, 2017 at 5:26 PM, Eugene Koifman <
> > > ekoif...@hortonworks.com>
> > > > wrote:
> > > >
> > > > > Something is not working with the part of the flow that
> triggers the
> > > > build
> > > > > automatically.
> > > > > In the meantime, if you log in with apache credentials, and go
> to
> > > “Build
> > > > > with Parameters”
> > > > > you can manually launch the test by entering the bug number:
> just the
> > > > > digits, skip the “HIVE-“ part
> > > > >
> > > > >
> > > > > On 4/24/17, 5:01 PM, "Vihang Karajgaonkar" <
> vih...@cloudera.com>
> > > wrote:
> > > > >
> > > > > I see that builds are getting picked up at
> > > > https://builds.apache.org/
> > > > > job/PreCommit-HIVE-Build/ The PTest server was restarted a
> couple
> > > of
> > > > > days
> > > > > back. If you had a patch in the queue around that time,
> may be it
> > > was
> > > > > lost
> > > > > due to restart. You may have to re-attach the patch on the
> > upstream
> > > > > JIRA if
> > > > > that is the case.
> > > > >
> > > > >
> > > > > On Mon, Apr 24, 2017 at 2:44 PM, Eugene Koifman <
> > > > > ekoif...@hortonworks.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > > It is not picking up new patches and there are none
> queued up.
> > > > Could
> > > > > > someone check please?
> > > > > >
> > > > > > Thanks,
> > > > > > Eugene
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
>
>

[jira] [Created] (HIVE-16538) TestExecDriver fails if run after TestOperators#testScriptOperator

2017-04-26 Thread Yussuf Shaikh (JIRA) 
Yussuf Shaikh created HIVE-16538:


 Summary: TestExecDriver fails if run after 
TestOperators#testScriptOperator
 Key: HIVE-16538
 URL: https://issues.apache.org/jira/browse/HIVE-16538
 Project: Hive
  Issue Type: Bug
  Components: Tests
Affects Versions: 3.0.0
 Environment: 
# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.5 LTS"
# uname -a
Linux 3b9700711ca1 3.19.0-37-generic #42-Ubuntu SMP Fri Nov 20 18:22:05 UTC 
2015 x86_64 x86_64 x86_64 GNU/Linux

Reporter: Yussuf Shaikh
Assignee: Yussuf Shaikh
Priority: Minor


Failed tests:
  TestExecDriver.testMapPlan1:498->fileDiff:182 expected: but was:
  TestExecDriver.testMapPlan2:506->fileDiff:182 expected: but was:
  TestExecDriver.testMapRedPlan1:515->fileDiff:182 expected: but 
was:
  TestExecDriver.testMapRedPlan2:524->fileDiff:182 expected: but 
was:
  TestExecDriver.testMapRedPlan3:533->fileDiff:182 expected: but 
was:
  TestExecDriver.testMapRedPlan4:542->fileDiff:182 expected: but 
was:
  TestExecDriver.testMapRedPlan5:551->fileDiff:182 expected: but 
was:
  TestExecDriver.testMapRedPlan6:560->fileDiff:182 expected: but 
was:




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)