[jira] [Created] (HIVE-14223) beeline should look for jdbc standalone jar in dist/jdbc dir instead of dist/lib

[Prasanth Jayachandran (JIRA)]

Prasanth Jayachandran created HIVE-14223:


 Summary: beeline should look for jdbc standalone jar in dist/jdbc 
dir instead of dist/lib
 Key: HIVE-14223
 URL: https://issues.apache.org/jira/browse/HIVE-14223
 Project: Hive
  Issue Type: Bug
  Components: Beeline
Affects Versions: 2.0.1, 2.2.0, 2.1.1
Reporter: Prasanth Jayachandran
Assignee: Prasanth Jayachandran


HIVE-13134 changed the jdbc-standalone jar path to dist/jdbc instead of 
dist/lib. beeline.sh still looks for the jar in dist/lib which throws the 
following error

{code}
ls: cannot access /work/hive2/lib/hive-jdbc-*-standalone.jar: No such file or 
directory
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 49952: HIVE-14205: Hive doesn't support union type with AVRO file format

[Chaoyu Tang]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49952/#review141997
---



Thanks [~yshi] for patch. It looks good. But I have a couple of questions:
It seems to me that the union in existing code is only used to support Nullable 
type in Avro, and has not been fully supported as a data type in general. This 
patch actually extends (or adds) this type support. 
So with the patch, how can we be able to distinguish an Avro union between 
nullable and non-nullable, for example, for following field schema, both might 
end with type uniontype
{code}
  "fields":[
   {
 "name":"value",
 "type":[
"null",
 "int",
 "long"
  ],
  "default":null
]
---
  "fields":[
   {
 "name":"value",
 "type":[
 "int",
 "long"
  ],
  "default": 0
]
{code}
Will there be any problem? Also could we add some qtests using Avro union data 
(with or without null)?

- Chaoyu Tang


On July 12, 2016, 11:07 a.m., Yibing Shi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49952/
> ---
> 
> (Updated July 12, 2016, 11:07 a.m.)
> 
> 
> Review request for hive and Chaoyu Tang.
> 
> 
> Bugs: HIVE-14205
> https://issues.apache.org/jira/browse/HIVE-14205
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-14205: Hive doesn't support union type with AVRO file format
> 
> 
> Diffs
> -
> 
>   serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java 
> 6165138 
>   serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 
> 08ee62b 
>   serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroDeserializer.java 
> 986b803 
>   serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 
> 0013b78 
> 
> Diff: https://reviews.apache.org/r/49952/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Yibing Shi
> 
>

[jira] [Created] (HIVE-14222) PTF: Operator initialization does not clean state

[Gopal V (JIRA)]

Gopal V created HIVE-14222:
--

 Summary: PTF: Operator initialization does not clean state
 Key: HIVE-14222
 URL: https://issues.apache.org/jira/browse/HIVE-14222
 Project: Hive
  Issue Type: Bug
  Components: PTF-Windowing
Affects Versions: 1.2.1, 2.2.0
Reporter: Gopal V
Assignee: Wei Zheng


PTFOperator::initializeOp() does not reset currentKeys to null.

{code}
  if (currentKeys != null && !keysAreEqual) {
ptfInvocation.finishPartition();
  }

  if (currentKeys == null) {
  currentKeys = newKeys.copyKey();
} else {
  currentKeys.copyKey(newKeys);
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14221) set SQLStdHiveAuthorizerFactoryForTest as default HIVE_AUTHORIZATION_MANAGER

[Pengcheng Xiong (JIRA)]

Pengcheng Xiong created HIVE-14221:
--

 Summary: set SQLStdHiveAuthorizerFactoryForTest as default 
HIVE_AUTHORIZATION_MANAGER
 Key: HIVE-14221
 URL: https://issues.apache.org/jira/browse/HIVE-14221
 Project: Hive
  Issue Type: Sub-task
Reporter: Pengcheng Xiong






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14220) Protected users from Reader.rows(Options) modifying the Options object

[Owen O'Malley (JIRA)]

Owen O'Malley created HIVE-14220:


 Summary: Protected users from Reader.rows(Options) modifying the 
Options object
 Key: HIVE-14220
 URL: https://issues.apache.org/jira/browse/HIVE-14220
 Project: Hive
  Issue Type: Bug
Reporter: Owen O'Malley
Assignee: Owen O'Malley


This is a matching fix to HIVE-14004 where ACID was getting in to trouble 
because it was reusing the Reader.Options argument between files and 
Reader.rows was modifying it. HIVE-14004 just fixed the Hive case, but we need 
a corresponding fix over here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14219) LLAP external client on secure cluster: Protocol interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol is not known

[Jason Dere (JIRA)]

Jason Dere created HIVE-14219:
-

 Summary: LLAP external client on secure cluster: Protocol 
interface org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol is not 
known
 Key: HIVE-14219
 URL: https://issues.apache.org/jira/browse/HIVE-14219
 Project: Hive
  Issue Type: Sub-task
  Components: llap
Reporter: Jason Dere
Assignee: Jason Dere


{noformat}
2016-07-07T23:10:35,249 INFO  [TaskHeartbeatThread[]]: task.TezTaskRunner2 
(:()) - TaskReporter reporter error which will cause the task to fail
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException):
 Protocol interface 
org.apache.hadoop.hive.llap.protocol.LlapTaskUmbilicalProtocol is not known.
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1551)
at org.apache.hadoop.ipc.Client.call(Client.java:1495)
at org.apache.hadoop.ipc.Client.call(Client.java:1395)
at 
org.apache.hadoop.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:241)
at com.sun.proxy.$Proxy39.heartbeat(Unknown Source)
at 
org.apache.hadoop.hive.llap.daemon.impl.LlapTaskReporter$HeartbeatCallable.heartbeat(LlapTaskReporter.java:280)
at 
org.apache.hadoop.hive.llap.daemon.impl.LlapTaskReporter$HeartbeatCallable.call(LlapTaskReporter.java:202)
at 
org.apache.hadoop.hive.llap.daemon.impl.LlapTaskReporter$HeartbeatCallable.call(LlapTaskReporter.java:139)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 49965: HIVE-13995 Hive generates inefficient metastore queries for TPCDS tables with 1800+ partitions leading to higher compile time

[Hari Sankar Sivarama Subramaniyan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49965/
---

Review request for hive and Ashutosh Chauhan.


Repository: hive-git


Description
---

Hive generates inefficient metastore queries for TPCDS tables with 1800+ 
partitions leading to higher compile time


Diffs
-

  metastore/if/hive_metastore.thrift 4d92b73 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AggregatePartitionsStatsRequest.java
 PRE-CREATION 
  
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/AggregatePartitionsStatsResponse.java
 PRE-CREATION 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
c945bf7 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 
44d73d4 
  metastore/src/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java 
b6fe502 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
9c900af 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 5adfa02 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java bbd47b8 
  metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseStore.java 
c65c7a4 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 1ea72a0 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 3e6acc7 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/RelOptHiveTable.java 
73ca9bf 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ppr/PartitionPruner.java 
26e936e 
  
ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
 5625091 
  ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java d8acf94 

Diff: https://reviews.apache.org/r/49965/diff/


Testing
---


Thanks,

Hari Sankar Sivarama Subramaniyan

[jira] [Created] (HIVE-14218) LLAP: ACL validation fails if the user name is different from principal user name

[Sergey Shelukhin (JIRA)]

Sergey Shelukhin created HIVE-14218:
---

 Summary: LLAP: ACL validation fails if the user name is different 
from principal user name
 Key: HIVE-14218
 URL: https://issues.apache.org/jira/browse/HIVE-14218
 Project: Hive
  Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14217) Druid integration

[Julian Hyde (JIRA)]

Julian Hyde created HIVE-14217:
--

 Summary: Druid integration
 Key: HIVE-14217
 URL: https://issues.apache.org/jira/browse/HIVE-14217
 Project: Hive
  Issue Type: Bug
Reporter: Julian Hyde


Allow Hive to query data in Druid



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 49919: HIVE-14135 : beeline output not formatted correctly for large column widths

[Vihang Karajgaonkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49919/
---

(Updated July 12, 2016, 5:51 p.m.)


Review request for hive, Mohit Sabharwal, Sergio Pena, Sahil Takiar, and Thejas 
Nair.


Changes
---

Added Sahil's suggestions.


Repository: hive-git


Description
---

HIVE-14135 : beeline output not formatted correctly for large column widths


Diffs (updated)
-

  beeline/pom.xml a720d0835314221ec3bd9e8d354d148498ff794c 
  beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 
5aaa38527734d46de037352ff51e54e0ae1cede0 
  beeline/src/java/org/apache/hive/beeline/BufferedRows.java 
962c5319bb7e6e448979e1cef80a086cadd2ecc6 
  beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java PRE-CREATION 

Diff: https://reviews.apache.org/r/49919/diff/


Testing
---


Thanks,

Vihang Karajgaonkar

Re: Review Request 49919: HIVE-14135 : beeline output not formatted correctly for large column widths

[Sahil Takiar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49919/#review141920
---



I took a look at the unit test and the use of Mockito, the approach there looks 
good, just two minor comments.


beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java (lines 37 - 39)


You can delete this, right? Since it looks like the method is empty.



beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java (line 42)


Could you add a more descriptive name to this method? Also some comments 
about the test is trying to test would be helpful


- Sahil Takiar


On July 11, 2016, 8:28 p.m., Vihang Karajgaonkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49919/
> ---
> 
> (Updated July 11, 2016, 8:28 p.m.)
> 
> 
> Review request for hive, Mohit Sabharwal, Sergio Pena, Sahil Takiar, and 
> Thejas Nair.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> HIVE-14135 : beeline output not formatted correctly for large column widths
> 
> 
> Diffs
> -
> 
>   beeline/pom.xml a720d0835314221ec3bd9e8d354d148498ff794c 
>   beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 
> 5aaa38527734d46de037352ff51e54e0ae1cede0 
>   beeline/src/java/org/apache/hive/beeline/BufferedRows.java 
> 962c5319bb7e6e448979e1cef80a086cadd2ecc6 
>   beeline/src/test/org/apache/hive/beeline/TestBufferedRows.java PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/49919/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Vihang Karajgaonkar
> 
>

[jira] [Created] (HIVE-14216) CREATE TABLE LIKE doesn't copy some attributes

[Ruslan Dautkhanov (JIRA)]

Ruslan Dautkhanov created HIVE-14216:


 Summary: CREATE TABLE LIKE doesn't copy some attributes
 Key: HIVE-14216
 URL: https://issues.apache.org/jira/browse/HIVE-14216
 Project: Hive
  Issue Type: Bug
Affects Versions: 1.1.0
 Environment: Hive 1.1
Hadoop 2.6

Reporter: Ruslan Dautkhanov
Priority: Critical


CREATE TABLE LIKE doesn't copy some attributes, like skip.header.line.count 

We use CREATE TABLE LIKE  to create tables from a template table.

We have to do re-apply skip.header.line.count=1 every time to the new table, 
although template table has skip.header.line.count set to 1, CREATE TABLE LIKE 
does not carry it over to the new table.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-14215) Displaying inconsistent CPU usage data with MR execution engine

[Peter Vary (JIRA)]

Peter Vary created HIVE-14215:
-

 Summary: Displaying inconsistent CPU usage data with MR execution 
engine
 Key: HIVE-14215
 URL: https://issues.apache.org/jira/browse/HIVE-14215
 Project: Hive
  Issue Type: Bug
Affects Versions: 2.2.0
Reporter: Peter Vary
Assignee: Peter Vary
Priority: Minor


If the MR task is finished after printing the cumulative CPU time then there is 
the possibility to print inconsistent CPU usage information.

Correct one:
{noformat}
2016-07-12 11:31:42,961 Stage-3 map = 0%,  reduce = 0%
2016-07-12 11:31:48,237 Stage-3 map = 100%,  reduce = 0%, Cumulative CPU 2.5 sec
MapReduce Total cumulative CPU time: 2 seconds 500 msec
Ended Job = job_1468321038188_0003
MapReduce Jobs Launched: 
Stage-Stage-3: Map: 1   Cumulative CPU: 2.5 sec   HDFS Read: 5864 HDFS Write: 
103 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 500 msec
{noformat}

One type of inconsistent data (easily reproducible one):
{noformat}
2016-07-12 11:39:00,540 Stage-3 map = 0%,  reduce = 0%
Ended Job = job_1468321038188_0004
MapReduce Jobs Launched: 
Stage-Stage-3: Map: 1   Cumulative CPU: 2.51 sec   HDFS Read: 5864 HDFS Write: 
103 SUCCESS
Total MapReduce CPU Time Spent: 2 seconds 510 msec
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 49952: HIVE-14205: Hive doesn't support union type with AVRO file format

[Yibing Shi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49952/
---

Review request for hive.


Bugs: HIVE-14205
https://issues.apache.org/jira/browse/HIVE-14205


Repository: hive-git


Description
---

HIVE-14205: Hive doesn't support union type with AVRO file format


Diffs
-

  serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java 
6165138 
  serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 08ee62b 
  serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroDeserializer.java 
986b803 
  serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerdeUtils.java 
0013b78 

Diff: https://reviews.apache.org/r/49952/diff/


Testing
---


Thanks,

Yibing Shi

Review Request 49948: HIVE-14195: HiveMetaStoreClient getFunction() does not throw NoSuchObjectException

[Peter Vary]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49948/
---

Review request for hive, Sergey Shelukhin and Sergio Pena.


Bugs: HIVE-14195
https://issues.apache.org/jira/browse/HIVE-14195


Repository: hive-git


Description
---

Catch, and rethrow the NoSuchObjectException


Diffs
-

  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
 641abab 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
3cbcc8f 

Diff: https://reviews.apache.org/r/49948/diff/


Testing
---

Manually, and modified the apropriate unit tests


Thanks,

Peter Vary

[jira] [Created] (HIVE-14214) ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)

[Matt McCline (JIRA)]

Matt McCline created HIVE-14214:
---

 Summary: ORC Schema Evolution and Predicate Push Down do not work 
together (no rows returned)
 Key: HIVE-14214
 URL: https://issues.apache.org/jira/browse/HIVE-14214
 Project: Hive
  Issue Type: Bug
  Components: Hive
Reporter: Matt McCline
Priority: Critical


In Schema Evolution, the reader schema is different than the file schema which 
is used to evaluate predicate push down.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)