review board

2019-01-18 Thread Malcolm Taylor 
Hi,
I want to log on to the review board, but can't remember my password. Is
there a way to reset it ?
thanks,
Malcolm

[jira] [Created] (HIVE-21138) Fix some of the alerts raised by lgtm.com

2019-01-18 Thread Malcolm Taylor (JIRA) 
Malcolm Taylor created HIVE-21138:
-

 Summary: Fix some of the alerts raised by lgtm.com
 Key: HIVE-21138
 URL: https://issues.apache.org/jira/browse/HIVE-21138
 Project: Hive
  Issue Type: Improvement
Reporter: Malcolm Taylor
Assignee: Malcolm Taylor


lgtm.com has identified a number of issues with the hive code base 
([https://lgtm.com/projects/g/apache/hive/alerts/?mode=tree).] This issue is to 
address some of the more straightforward ones. Specifically,
 * Array index out of bounds inOpProcFactory
 * Dereferenced variable is always null in ObjectStore
 * Self assignment in CheckConstraint and Vectorizer
 * several cases of Contradictory type checks
 * several cases of Missing format argument



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (HIVE-21137) JDBC: HiveDatabaseMetaData.getTables does not adhere to jdbc spec

2019-01-18 Thread Vaibhav Gumashta (JIRA) 
Vaibhav Gumashta created HIVE-21137:
---

 Summary: JDBC: HiveDatabaseMetaData.getTables does not adhere to 
jdbc spec
 Key: HIVE-21137
 URL: https://issues.apache.org/jira/browse/HIVE-21137
 Project: Hive
  Issue Type: Bug
  Components: JDBC
Affects Versions: 2.3.4, 3.1.1
Reporter: Vaibhav Gumashta
 Attachments: HiveJdbcClient.java

The {{types}} parameter in {{HiveDatabaseMetaData.getTable(String catalog, 
String schemaPattern, String tableNamePattern, String[] types)}} is supposed to 
honor only the return values from  {{HiveDatabaseMetaData.getTableTypes}}. 
However following is the output from the attached test jdbc programs: 

{code}
*** Using dbMetadata.getTables ***


*** With only EXTERNAL TABLE ***
Table: test1
Table: test_2
Table: names_text
Table: names_text_1


*** With only TABLE ***
Table: test1
Table: test_2
Table: names_text
Table: names_text_1


*** With only EXTERNAL_TABLE ***
Table: test1
Table: test_2
Table: names_text
Table: names_text_1


*** With empty array ***
Table: test1
Table: test_2
Table: names_text
Table: names_text_1


*** With VIEW ***


*** With INDEX_TABLE ***
Table: test1
Table: test_2
Table: names_text
Table: names_text_1


*** With VIEW, INDEX_TABLE ***


*** With EXTERNAL_TABLE, VIEW, INDEX_TABLE ***


*** With TABLE, VIEW, INDEX_TABLE ***
Table: test1
Table: test_2
Table: names_text
Table: names_text_1


*** With a random string ***
Table: test1
Table: test_2
Table: names_text
Table: names_text_1


*** getTableTypes ***
Table: TABLE
Table: TABLE
Table: VIEW
Table: MATERIALIZED_VIEW
{code}

We should fix the api so that clients can see expected behaviour.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69367: Query based compactor for full CRUD Acid tables

2019-01-18 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69367/
---

(Updated Jan. 18, 2019, 10:41 p.m.)


Review request for hive and Eugene Koifman.


Changes
---

Rebased on master


Bugs: HIVE-20699
https://issues.apache.org/jira/browse/HIVE-20699


Repository: hive-git


Description
---

https://jira.apache.org/jira/browse/HIVE-20699


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b213609f39 
  itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestAcidOnTez.java 
d6a41919bf 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCrudCompactorOnTez.java
 PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bbe7fb0697 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java 
15c14c9be5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/SplitGrouper.java 7f8bd229a6 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRawRecordMerger.java 
fbb931cbcd 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java 6d4578e7a0 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java 4d55592b63 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
0e5b3e5473 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 
dc05e1990e 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java a0df82cb20 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFValidateAcidSortOrder.java
 PRE-CREATION 
  ql/src/test/results/clientpositive/show_functions.q.out 0fdcbda66f 


Diff: https://reviews.apache.org/r/69367/diff/6/

Changes: https://reviews.apache.org/r/69367/diff/5-6/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 69367: Query based compactor for full CRUD Acid tables

2019-01-18 Thread Vaibhav Gumashta 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69367/
---

(Updated Jan. 18, 2019, 9:40 p.m.)


Review request for hive and Eugene Koifman.


Changes
---

Rebased on master


Bugs: HIVE-20699
https://issues.apache.org/jira/browse/HIVE-20699


Repository: hive-git


Description
---

https://jira.apache.org/jira/browse/HIVE-20699


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java b213609f39 
  itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/TestAcidOnTez.java 
d6a41919bf 
  
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/txn/compactor/TestCrudCompactorOnTez.java
 PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java bbe7fb0697 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java 
15c14c9be5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/tez/SplitGrouper.java 7f8bd229a6 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRawRecordMerger.java 
fbb931cbcd 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcRecordUpdater.java 6d4578e7a0 
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcSplit.java 4d55592b63 
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 
0e5b3e5473 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/CompactorMR.java 
dc05e1990e 
  ql/src/java/org/apache/hadoop/hive/ql/txn/compactor/Initiator.java a0df82cb20 
  
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFValidateAcidSortOrder.java
 PRE-CREATION 
  ql/src/test/results/clientpositive/show_functions.q.out 0fdcbda66f 


Diff: https://reviews.apache.org/r/69367/diff/5/

Changes: https://reviews.apache.org/r/69367/diff/4-5/


Testing
---


Thanks,

Vaibhav Gumashta

Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on server-side in addition to client-side

2019-01-18 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/#review212157
---




standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java
Line 1290 (original), 1290 (patched)


I change the expected exception type to be consistent with 
testListPartitionsAllNoDbName().

Same for test cases for NoTblName


- Na Li


On Jan. 18, 2019, 9:30 p.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69585/
> ---
> 
> (Updated Jan. 18, 2019, 9:30 p.m.)
> 
> 
> Review request for hive, Adam Holley, Karthick Sankarachary, Morio 
> Ramdenbourg, Peter Vary, Sergio Pena, and Vihang Karajgaonkar.
> 
> 
> Bugs: HIVE-20776
> https://issues.apache.org/jira/browse/HIVE-20776
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> add filtering to read result at HMS server, so user cannot see metadata 
> he/she has no privileges. Filtering is enabled/disabled based on 
> configuration.
> 
> 
> Diffs
> -
> 
>   
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
>  19bd9bac84c20f94ac819a80e3cc89e0dc66396d 
>   
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
>  be1f8c78497fe3d0816ad3935ba07cd5ad379b08 
>   
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/FilterUtils.java
>  PRE-CREATION 
>   
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  a9398ae1e79404a15894aa42f451df5d18ed3e4c 
>   
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestFilterHooks.java
>  7dc69bc4e92875c8962dcd313b16f0f90ea8b057 
>   
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java
>  a338bd4032eb746cd541a562ead0fa2caf9e3ed4 
> 
> 
> Diff: https://reviews.apache.org/r/69585/diff/11/
> 
> 
> Testing
> ---
> 
> Existing unit tests passed. 
> add new unit tests for filtering at HMS server and HMS client
> add code to enabled/disable filtering at HMS client based on configuration
> add code to enabled/disable filtering at HMS server based on configuration
> 
> 
> Thanks,
> 
> Na Li
> 
>

Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on server-side in addition to client-side

2019-01-18 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/
---

(Updated Jan. 18, 2019, 9:30 p.m.)


Review request for hive, Adam Holley, Karthick Sankarachary, Morio Ramdenbourg, 
Peter Vary, Sergio Pena, and Vihang Karajgaonkar.


Bugs: HIVE-20776
https://issues.apache.org/jira/browse/HIVE-20776


Repository: hive-git


Description
---

add filtering to read result at HMS server, so user cannot see metadata he/she 
has no privileges. Filtering is enabled/disabled based on configuration.


Diffs (updated)
-

  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
 19bd9bac84c20f94ac819a80e3cc89e0dc66396d 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
 be1f8c78497fe3d0816ad3935ba07cd5ad379b08 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/FilterUtils.java
 PRE-CREATION 
  
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
 a9398ae1e79404a15894aa42f451df5d18ed3e4c 
  
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestFilterHooks.java
 7dc69bc4e92875c8962dcd313b16f0f90ea8b057 
  
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/client/TestListPartitions.java
 a338bd4032eb746cd541a562ead0fa2caf9e3ed4 


Diff: https://reviews.apache.org/r/69585/diff/11/

Changes: https://reviews.apache.org/r/69585/diff/10-11/


Testing
---

Existing unit tests passed. 
add new unit tests for filtering at HMS server and HMS client
add code to enabled/disable filtering at HMS client based on configuration
add code to enabled/disable filtering at HMS server based on configuration


Thanks,

Na Li

[jira] [Created] (HIVE-21136) Kryo exception : Unable to create serializer for class AtomicReference

2019-01-18 Thread Rajesh Balamohan (JIRA) 
Rajesh Balamohan created HIVE-21136:
---

 Summary: Kryo exception : Unable to create serializer  for class 
AtomicReference
 Key: HIVE-21136
 URL: https://issues.apache.org/jira/browse/HIVE-21136
 Project: Hive
  Issue Type: Bug
  Components: Serializers/Deserializers
Reporter: Rajesh Balamohan
Assignee: Rajesh Balamohan


{noformat}

Caused by: org.apache.hive.com.esotericsoftware.kryo.KryoException: 
java.lang.IllegalArgumentException: Unable to create serializer 
"org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer" for 
class: java.util.concurrent.atomic.AtomicReference
Serialization trace:
_tableInfo (org.codehaus.jackson.sym.BytesToNameCanonicalizer)
_rootByteSymbols (org.codehaus.jackson.JsonFactory)
jsonFactory (brickhouse.udf.json.ToJsonUDF)
genericUDF (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
chidren (org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc)
colExprMap (org.apache.hadoop.hive.ql.exec.GroupByOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.LateralViewJoinOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator)
childOperators (org.apache.hadoop.hive.ql.exec.LateralViewJoinOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator)
childOperators (org.apache.hadoop.hive.ql.exec.LateralViewJoinOperator)
childOperators (org.apache.hadoop.hive.ql.exec.SelectOperator)
childOperators (org.apache.hadoop.hive.ql.exec.LateralViewForwardOperator)
childOperators (org.apache.hadoop.hive.ql.exec.GroupByOperator)
reducer (org.apache.hadoop.hive.ql.plan.ReduceWork)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:144)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
    at 
org.apache.hive.com.esotericsoftware.kryo.Kryo.readObjectOrNull(Kryo.java:759)
    at 
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObjectOrNull(SerializationUtilities.java:199)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:132)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
    at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
    at 
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:214)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
    at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
    at 
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:214)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
    at 
org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)
    at 
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClassAndObject(SerializationUtilities.java:176)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:134)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:40)
    at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
    at 
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:214)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:551)
    at 
org.apache.hive.com.esotericsoftware.kryo.Kryo.readClassAndObject(Kryo.java:790)
    at 
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readClassAndObject(SerializationUtilities.java:176)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:161)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.MapSerializer.read(MapSerializer.java:39)
    at org.apache.hive.com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:708)
    at 
org.apache.hadoop.hive.ql.exec.SerializationUtilities$KryoWithHooks.readObject(SerializationUtilities.java:214)
    at 
org.apache.hive.com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
    at

Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on server-side in addition to client-side

2019-01-18 Thread Vihang Karajgaonkar via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/#review212147
---


Fix it, then Ship it!




Overall the patch looks good. I have some minor suggestions below. RLGTM.


standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/FilterUtils.java
Lines 36 (patched)


please add documentation for each method



standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/FilterUtils.java
Lines 236 (patched)


more descriptive message would be dbName is null. Same for line 251 tblName 
is null.



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 628 (patched)


nit, could be simplified as 
filterHook = isServerFilterEnabled ? loadFilterHooks() : null;



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 668-671 (patched)


Isn't this redundant since you already checked for a valid configuration in 
getIfServerFilterenabled() method? A easier way would be to add 

Preconditions.checkState(!isBlank(MetastoreConf.getVar(conf, 
ConfVars.FILTER_HOOK)));



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 4731 (patched)


nit, remove "For improved performance". I am not very convinced that this 
is helping the performance. its okay to say "we'll check if the said db and 
table are to be filtered out, if so, then we won't proceed with querying the 
partitions."



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 4873 (patched)


nit, remove "For improved performance". I am not very convinced that this 
is helping the performance. its okay to say "we'll check if the said db and 
table are to be filtered out, if so, then we won't proceed with querying the 
partitions."



standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
Lines 5682 (patched)


nit, remove "For improved performance". I am not very convinced that this 
is helping the performance. its okay to say "we'll check if the said db and 
table are to be filtered out, if so, then we won't proceed with querying the 
partitions."

Same comment for other places with this comment.



standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetastoreFilterHook.java
Lines 52 (patched)


If this test class added more test coverage to TestFilterHooks test, I 
would suggest to move new tests in TestFilterHooks instead of removing 
TestFilterHooks and adding a new test class which copies all the code from 
TestFilterHooks. That way you don't lose the git history of TestFilterHooks 
unnecessarily.


- Vihang Karajgaonkar


On Jan. 18, 2019, 3:45 p.m., Na Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69585/
> ---
> 
> (Updated Jan. 18, 2019, 3:45 p.m.)
> 
> 
> Review request for hive, Adam Holley, Karthick Sankarachary, Morio 
> Ramdenbourg, Peter Vary, Sergio Pena, and Vihang Karajgaonkar.
> 
> 
> Bugs: HIVE-20776
> https://issues.apache.org/jira/browse/HIVE-20776
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> add filtering to read result at HMS server, so user cannot see metadata 
> he/she has no privileges. Filtering is enabled/disabled based on 
> configuration.
> 
> 
> Diffs
> -
> 
>   
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
>  19bd9bac84c20f94ac819a80e3cc89e0dc66396d 
>   
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
>  be1f8c78497fe3d0816ad3935ba07cd5ad379b08 
>   
> standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/FilterUtils.java
>  PRE-CREATION 
>   
> standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
>  a9398ae1e79404a15894aa42f451df5d18ed3e4c 
>   
> standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestFilterHooks.java
>  7dc69bc4e92875c8962dcd313b16f0f90ea8b057 
>   
>

[GitHub] hive pull request #516: Update to Apache Druid 0.13.0-incubating

2019-01-18 Thread nishantmonu51 
GitHub user nishantmonu51 opened a pull request:

https://github.com/apache/hive/pull/516

Update to Apache Druid 0.13.0-incubating

This PR updates to apache druid 0.13.0-incubating. 


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/nishantmonu51/hive druid-0.13-rc3

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/516.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #516


commit ad5d7616134a7e97afccab4d60f7534417494689
Author: Nishant 
Date:   2019-01-18T18:16:47Z

Update to Druid 0.13.0

remove unneeded files

fix checkstyle

fix test




---

Re: Review Request 69585: HIVE-20776: Run HMS filterHooks on server-side in addition to client-side

2019-01-18 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69585/
---

(Updated Jan. 18, 2019, 3:45 p.m.)


Review request for hive, Adam Holley, Karthick Sankarachary, Morio Ramdenbourg, 
Peter Vary, Sergio Pena, and Vihang Karajgaonkar.


Bugs: HIVE-20776
https://issues.apache.org/jira/browse/HIVE-20776


Repository: hive-git


Description
---

add filtering to read result at HMS server, so user cannot see metadata he/she 
has no privileges. Filtering is enabled/disabled based on configuration.


Diffs (updated)
-

  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
 19bd9bac84c20f94ac819a80e3cc89e0dc66396d 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/conf/MetastoreConf.java
 be1f8c78497fe3d0816ad3935ba07cd5ad379b08 
  
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/utils/FilterUtils.java
 PRE-CREATION 
  
standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
 a9398ae1e79404a15894aa42f451df5d18ed3e4c 
  
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestFilterHooks.java
 7dc69bc4e92875c8962dcd313b16f0f90ea8b057 
  
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetastoreFilterHook.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/69585/diff/10/

Changes: https://reviews.apache.org/r/69585/diff/9-10/


Testing
---

Existing unit tests passed. 
add new unit tests for filtering at HMS server and HMS client
add code to enabled/disable filtering at HMS client based on configuration
add code to enabled/disable filtering at HMS server based on configuration


Thanks,

Na Li

[jira] [Created] (HIVE-21135) Grouping sets size can not be greater than 64 compile error

2019-01-18 Thread Nagaraju Anthati (JIRA) 
Nagaraju Anthati created HIVE-21135:
---

 Summary: Grouping sets size can not be greater than 64 compile 
error
 Key: HIVE-21135
 URL: https://issues.apache.org/jira/browse/HIVE-21135
 Project: Hive
  Issue Type: Bug
  Components: Hive
Affects Versions: 3.1.0
 Environment: HDP 3.1 ,Ubuntu environment ,mysql metastore
Reporter: Nagaraju Anthati


I am trying to get distinct records  from an alias dataset , then the hive is 
throwing compile error .The alias dataset has 8k columns , could not proceed 
further  , not sure if it is config related.

query : select distinct data.* from ( . some join+8kcolumns in the 
result set...)data

 

 

error :compile error : Grouping sets size can not be grater than 64:27:x 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69683: [HIVE-21071] Improve getInputSummary

2019-01-18 Thread Peter Vary via Review Board 


> On jan. 8, 2019, 10:04 de, Peter Vary wrote:
> > Thanks for the patch!
> > Two nits below.
> > Also a bit concerned about the size calculation - seems ok, but it would be 
> > good to have a few test case which validates the contentsummary 
> > calculations (when every path is cached/only few patch is cached/no path is 
> > cached), so we can be sure that further changes will not break the 
> > functionality.
> > 
> > What do you think?
> > 
> > Peter
> 
> David Mollitor wrote:
> Thank you Peter for the review. This functionality is unit tested 
> already. Do you have suggestions for additional unit tests?
> 
> 
> 
> https://github.com/apache/hive/blob/ae008b79b5d52ed6a38875b73025a505725828eb/ql/src/test/org/apache/hadoop/hive/ql/exec/TestUtilities.java

I missed the ones which are testing the full getInputSummary method, and found 
only the ones (testGetInputSummaryPool, testGetInputSummaryPoolAndFailure) 
using the getInputSummaryWithPool.
Seeing those tests I feel much better. Maybe it is an edge case, but it might 
be good to add a test where we call the getInputSummary twice. First with one 
set of path (p1, p2) and then again with another set of path (p1, p2, p3, p4) 
so we can check that the merge of the cached results and the newly fetched ones 
is working.


- Peter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69683/#review211758
---


On jan. 14, 2019, 3:36 du, David Mollitor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69683/
> ---
> 
> (Updated jan. 14, 2019, 3:36 du)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Improve performance of method getInputSummary by changing data structures and 
> allowing multiple threads to do calculations.
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 2ff9ad3 
> 
> 
> Diff: https://reviews.apache.org/r/69683/diff/2/
> 
> 
> Testing
> ---
> 
> Unit
> 
> 
> Thanks,
> 
> David Mollitor
> 
>

[jira] [Created] (HIVE-21134) Hive Build Version as UDF

2019-01-18 Thread Mani M (JIRA) 
Mani M created HIVE-21134:
-

 Summary: Hive Build Version as UDF
 Key: HIVE-21134
 URL: https://issues.apache.org/jira/browse/HIVE-21134
 Project: Hive
  Issue Type: Improvement
Reporter: Mani M
Assignee: Mani M
 Fix For: 4.0.0


This Jira is to get the Hive Build Version as UDF.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)