[jira] [Assigned] (HIVE-22961) Drop function in Hive should not send request for drop database to Ranger plugin.
[
https://issues.apache.org/jira/browse/HIVE-22961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sai Hemanth Gantasala reassigned HIVE-22961:
Assignee: Sai Hemanth Gantasala (was: Sam An)
> Drop function in Hive should not send request for drop database to Ranger
> plugin.
> --
>
> Key: HIVE-22961
> URL: https://issues.apache.org/jira/browse/HIVE-22961
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Sam An
>Assignee: Sai Hemanth Gantasala
>Priority: Major
>
> Issue here is how HIVE sends the "HivePrivilegeObjects" to Ranger when DROP
> fUNTION is done. This is different from how DROP TABLE is done.
> DROP TABLE the following is the request:
> {code:java}
> 'checkPrivileges':{'hiveOpType':DROPTABLE,
> 'inputHObjs':['HivePrivilegeObject':{'type':TABLE_OR_VIEW, 'dbName':testdemo,
> 'objectType':TABLE_OR_VIEW, 'objectName':t1, 'columns':[], 'partKeys':[],
> 'commandParams':[], 'actionType':OTHER, 'owner':systest}],
> 'outputHObjs':['HivePrivilegeObject':{'type':TABLE_OR_VIEW,
> 'dbName':testdemo, 'objectType':TABLE_OR_VIEW, 'objectName':t1, 'columns':[],
> 'partKeys':[], 'commandParams':[], 'actionType':OTHER, 'owner':systest}],
> 'context':{'clientType':HIVESERVER2, 'commandString':drop table t1,
> 'ipAddress':10.65.42.125, 'forwardedAddresses':null,
> 'sessionString':58f89a16-2df5-4124-af0e-913aabbefe06}, 'user':systest,
> 'groups':[systest, wheel]}{code}
> Where as in DROP FUNCTION:
> {code:java}
> {'hiveOpType':DROPFUNCTION,
> 'inputHObjs':['HivePrivilegeObject':{'type':FUNCTION, 'dbName':udfdemo,
> 'objectType':FUNCTION, 'objectName':aes1, 'columns':[], 'partKeys':[],
> 'commandParams':[], 'actionType':OTHER, 'owner':null}],
> 'outputHObjs':['HivePrivilegeObject':{'type':DATABASE, 'dbName':udfdemo,
> 'objectType':DATABASE, 'objectName':null, 'columns':[], 'partKeys':[],
> 'commandParams':[], 'actionType':OTHER,
> 'owner':systest},'HivePrivilegeObject':{'type':FUNCTION, 'dbName':udfdemo,
> 'objectType':FUNCTION, 'objectName':aes1, 'columns':[], 'partKeys':[],
> 'commandParams':[], 'actionType':OTHER, 'owner':null}],
> 'context':{'clientType':HIVESERVER2, 'commandString':drop function
> udfdemo.aes1, 'ipAddress':10.65.42.125, 'forwardedAddresses':null,
> 'sessionString':442ca4d3-f34a-470c-878a-18542b99016c}, 'user':systest,
> 'groups':[systest, wheel]}
> {code}
> in DROP function in outputHObjs, there is this addition, DATABASE object
> which should not there and this causes the Ranger requested to be generated
> differently.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-27522) Iceberg: Bucket partition transformation date type support
[
https://issues.apache.org/jira/browse/HIVE-27522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko resolved HIVE-27522.
---
Fix Version/s: 4.0.0-beta-1
Assignee: Denys Kuzmenko
Resolution: Fixed
> Iceberg: Bucket partition transformation date type support
> --
>
> Key: HIVE-27522
> URL: https://issues.apache.org/jira/browse/HIVE-27522
> Project: Hive
> Issue Type: Task
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-beta-1
>
>
> {code}
> Caused by: org.apache.hadoop.hive.ql.exec.UDFArgumentException:
> ICEBERG_BUCKET() only takes
> STRING/CHAR/VARCHAR/BINARY/INT/LONG/DECIMAL/FLOAT/DOUBLE types as first
> argument, got DATE
> at
> org.apache.iceberg.mr.hive.GenericUDFIcebergBucket.initialize(GenericUDFIcebergBucket.java:162)
> ~[hive-iceberg-handler-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:149)
> ~[hive-exec-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:235)
> ~[hive-exec-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.iceberg.mr.hive.HiveIcebergStorageHandler.lambda$null$0(HiveIcebergStorageHandler.java:142)
> ~[hive-iceberg-handler-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.hadoop.hive.ql.optimizer.SortedDynPartitionOptimizer$SortedDynamicPartitionProc.allStaticPartitions(SortedDynPartitionOptimizer.java:420)
> ~[hive-exec-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.hadoop.hive.ql.optimizer.SortedDynPartitionOptimizer$SortedDynamicPartitionProc.process(SortedDynPartitionOptimizer.java:195)
> ~[hive-exec-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27522) Iceberg: Bucket partition transformation date type support
[
https://issues.apache.org/jira/browse/HIVE-27522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747058#comment-17747058
]
Denys Kuzmenko commented on HIVE-27522:
---
Merged to master.
Thanks [~ayushsaxena], [~sbadhya], [~zhangbutao] for the review!
> Iceberg: Bucket partition transformation date type support
> --
>
> Key: HIVE-27522
> URL: https://issues.apache.org/jira/browse/HIVE-27522
> Project: Hive
> Issue Type: Task
>Reporter: Denys Kuzmenko
>Priority: Major
> Labels: pull-request-available
>
> {code}
> Caused by: org.apache.hadoop.hive.ql.exec.UDFArgumentException:
> ICEBERG_BUCKET() only takes
> STRING/CHAR/VARCHAR/BINARY/INT/LONG/DECIMAL/FLOAT/DOUBLE types as first
> argument, got DATE
> at
> org.apache.iceberg.mr.hive.GenericUDFIcebergBucket.initialize(GenericUDFIcebergBucket.java:162)
> ~[hive-iceberg-handler-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:149)
> ~[hive-exec-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.hadoop.hive.ql.plan.ExprNodeGenericFuncDesc.newInstance(ExprNodeGenericFuncDesc.java:235)
> ~[hive-exec-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.iceberg.mr.hive.HiveIcebergStorageHandler.lambda$null$0(HiveIcebergStorageHandler.java:142)
> ~[hive-iceberg-handler-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.hadoop.hive.ql.optimizer.SortedDynPartitionOptimizer$SortedDynamicPartitionProc.allStaticPartitions(SortedDynPartitionOptimizer.java:420)
> ~[hive-exec-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> at
> org.apache.hadoop.hive.ql.optimizer.SortedDynPartitionOptimizer$SortedDynamicPartitionProc.process(SortedDynPartitionOptimizer.java:195)
> ~[hive-exec-3.1.3000.7.2.17.0-335.jar:3.1.3000.7.2.17.0-335]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HIVE-23127) Replace listPartitionsByExpr with GetPartitionsWithSpecs in Partition pruner
[ https://issues.apache.org/jira/browse/HIVE-23127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramesh Kumar Thangarajan reassigned HIVE-23127: --- Assignee: Ramesh Kumar Thangarajan (was: Vineet Garg) > Replace listPartitionsByExpr with GetPartitionsWithSpecs in Partition pruner > > > Key: HIVE-23127 > URL: https://issues.apache.org/jira/browse/HIVE-23127 > Project: Hive > Issue Type: Task > Components: HiveServer2 >Reporter: Vineet Garg >Assignee: Ramesh Kumar Thangarajan >Priority: Major > Attachments: HIVE-23127.1.patch, HIVE-23127.2.patch > > > GetPartitionsWithSpecs reduces data transfer by deduplicating storage > descriptor -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27528) Hive iceberg: Alter table command should not call the metastore to update column stats
[ https://issues.apache.org/jira/browse/HIVE-27528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simhadri Govindappa updated HIVE-27528: --- Description: l. The bit vector that contains the ndv values is being overwritten by updatecolstats which is called during alter table command. was:It overwrites the previously calculated bit vectors for ndv as well. > Hive iceberg: Alter table command should not call the metastore to update > column stats > --- > > Key: HIVE-27528 > URL: https://issues.apache.org/jira/browse/HIVE-27528 > Project: Hive > Issue Type: Improvement >Reporter: Simhadri Govindappa >Assignee: Simhadri Govindappa >Priority: Major > > l. > The bit vector that contains the ndv values is being overwritten by > updatecolstats which is called during alter table command. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27528) Hive iceberg: Alter table command should not call the metastore to update column stats
[ https://issues.apache.org/jira/browse/HIVE-27528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simhadri Govindappa updated HIVE-27528: --- Description: The bit vector that contains the ndv values is being overwritten by updatecolstats which is called during alter table command. was: l. The bit vector that contains the ndv values is being overwritten by updatecolstats which is called during alter table command. > Hive iceberg: Alter table command should not call the metastore to update > column stats > --- > > Key: HIVE-27528 > URL: https://issues.apache.org/jira/browse/HIVE-27528 > Project: Hive > Issue Type: Improvement >Reporter: Simhadri Govindappa >Assignee: Simhadri Govindappa >Priority: Major > > > > The bit vector that contains the ndv values is being overwritten by > updatecolstats which is called during alter table command. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27528) Hive iceberg: Alter table command should not call the metastore to update column stats
[ https://issues.apache.org/jira/browse/HIVE-27528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simhadri Govindappa updated HIVE-27528: --- Description: It overwrites the previously calculated bit vectors for ndv as well. > Hive iceberg: Alter table command should not call the metastore to update > column stats > --- > > Key: HIVE-27528 > URL: https://issues.apache.org/jira/browse/HIVE-27528 > Project: Hive > Issue Type: Improvement >Reporter: Simhadri Govindappa >Assignee: Simhadri Govindappa >Priority: Major > > It overwrites the previously calculated bit vectors for ndv as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HIVE-27528) Hive iceberg: Alter table command should not call the metastore to update column stats
Simhadri Govindappa created HIVE-27528: -- Summary: Hive iceberg: Alter table command should not call the metastore to update column stats Key: HIVE-27528 URL: https://issues.apache.org/jira/browse/HIVE-27528 Project: Hive Issue Type: Improvement Reporter: Simhadri Govindappa Assignee: Simhadri Govindappa -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HIVE-27513) Iceberg: Fetch task returns wrong results for Timestamp with local time zone datatype for Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-27513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746853#comment-17746853
]
Sourabh Badhya commented on HIVE-27513:
---
Thanks [~ayushtkn] and [~dkuzmenko] for the reviews.
> Iceberg: Fetch task returns wrong results for Timestamp with local time zone
> datatype for Iceberg tables
>
>
> Key: HIVE-27513
> URL: https://issues.apache.org/jira/browse/HIVE-27513
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Sourabh Badhya
>Assignee: Sourabh Badhya
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-beta-1
>
>
> Fetch task returns wrong results for Timestamp with local time zone datatype
> for Iceberg tables
> Repro queries -
> {code:java}
> create external table ice_ts_4(a int, ts timestamp with local time zone)
> stored by iceberg;
> insert into ice_ts_4 values (1, current_timestamp());
> set hive.fetch.task.conversion=none;
> select * from ice_ts_4;
> +-+-+
> | ice_ts_4.a | ice_ts_4.ts |
> +-+-+
> | 1 | 2021-08-16 06:37:30.425 US/Pacific |
> +-+-+
> set hive.fetch.task.conversion=more;
> select * from ice_ts_4;
> +-++
> | ice_ts_4.a |ice_ts_4.ts |
> +-++
> | 1 | 2021-08-16 13:37:30.425 Z |
> +-++ {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HIVE-27513) Iceberg: Fetch task returns wrong results for Timestamp with local time zone datatype for Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-27513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko resolved HIVE-27513.
---
Fix Version/s: 4.0.0-beta-1
Resolution: Fixed
> Iceberg: Fetch task returns wrong results for Timestamp with local time zone
> datatype for Iceberg tables
>
>
> Key: HIVE-27513
> URL: https://issues.apache.org/jira/browse/HIVE-27513
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Sourabh Badhya
>Assignee: Sourabh Badhya
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-beta-1
>
>
> Fetch task returns wrong results for Timestamp with local time zone datatype
> for Iceberg tables
> Repro queries -
> {code:java}
> create external table ice_ts_4(a int, ts timestamp with local time zone)
> stored by iceberg;
> insert into ice_ts_4 values (1, current_timestamp());
> set hive.fetch.task.conversion=none;
> select * from ice_ts_4;
> +-+-+
> | ice_ts_4.a | ice_ts_4.ts |
> +-+-+
> | 1 | 2021-08-16 06:37:30.425 US/Pacific |
> +-+-+
> set hive.fetch.task.conversion=more;
> select * from ice_ts_4;
> +-++
> | ice_ts_4.a |ice_ts_4.ts |
> +-++
> | 1 | 2021-08-16 13:37:30.425 Z |
> +-++ {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HIVE-27513) Iceberg: Fetch task returns wrong results for Timestamp with local time zone datatype for Iceberg tables
[
https://issues.apache.org/jira/browse/HIVE-27513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17746845#comment-17746845
]
Denys Kuzmenko commented on HIVE-27513:
---
Merged to master.
[~sbadhya] thanks for the patch and [~ayushsaxena] for the review!
> Iceberg: Fetch task returns wrong results for Timestamp with local time zone
> datatype for Iceberg tables
>
>
> Key: HIVE-27513
> URL: https://issues.apache.org/jira/browse/HIVE-27513
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
>Reporter: Sourabh Badhya
>Assignee: Sourabh Badhya
>Priority: Major
> Labels: pull-request-available
>
> Fetch task returns wrong results for Timestamp with local time zone datatype
> for Iceberg tables
> Repro queries -
> {code:java}
> create external table ice_ts_4(a int, ts timestamp with local time zone)
> stored by iceberg;
> insert into ice_ts_4 values (1, current_timestamp());
> set hive.fetch.task.conversion=none;
> select * from ice_ts_4;
> +-+-+
> | ice_ts_4.a | ice_ts_4.ts |
> +-+-+
> | 1 | 2021-08-16 06:37:30.425 US/Pacific |
> +-+-+
> set hive.fetch.task.conversion=more;
> select * from ice_ts_4;
> +-++
> | ice_ts_4.a |ice_ts_4.ts |
> +-++
> | 1 | 2021-08-16 13:37:30.425 Z |
> +-++ {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
