[
https://issues.apache.org/jira/browse/HIVE-25463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chinmay Kulkarni updated HIVE-25463:
------------------------------------
Description:
Today for setups that use standalone metastore, we set the following:
{noformat}
<property>
<name>metastore.expression.proxy</name>
<value>org.apache.hadoop.hive.metastore.DefaultPartitionExpressionProxy</value>
</property>
{noformat}
However,
[DefaultPartitionExpressionProxy|https://github.com/apache/hive/blob/d61c9160ffa5afbd729887c3db690eccd7ef8238/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/DefaultPartitionExpressionProxy.java]
throws UnsupportedOperationException throughout, thus preventing any partition
pruning to be successful.
The effect of this is, other projects that rely on just standalone Hive
Metastore (for example Trino) are unable to do things like invoke bulk
partition drops using say,
[HiveMetaStore.drop_partitions_req|https://github.com/apache/hive/blob/d61c9160ffa5afbd729887c3db690eccd7ef8238/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L5008]
via a Thrift client.
We should implement a separate PartitionExpressionProxy implementation for such
clients for use with standalone HMS. Note that an implementation like
PartitionExpressionForMetastore cannot be used in such cases since standalone
HMS installs will/should not have hive-exec dependencies.
Please refer to [Trino issue#7249|https://github.com/trinodb/trino/issues/7249]
for more context
was:
Today for setups that use standalone metastore, we set the following:
{noformat}
<property>
<name>metastore.expression.proxy</name>
<value>org.apache.hadoop.hive.metastore.DefaultPartitionExpressionProxy</value>
</property>
{noformat}
However,
[DefaultPartitionExpressionProxy|https://github.com/apache/hive/blob/d61c9160ffa5afbd729887c3db690eccd7ef8238/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/DefaultPartitionExpressionProxy.java]
throws UnsupportedOperationException throughout, thus preventing any partition
pruning to be successful.
The effect of this is, other projects that rely on just standalone Hive
Metastore (for example Trino) are unable to do things like invoke bulk
partition drops using say,
[HiveMetaStore.drop_partitions_req|https://github.com/apache/hive/blob/d61c9160ffa5afbd729887c3db690eccd7ef8238/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L5008]
via a Thrift client.
We should implement a separate PartitionExpressionProxy implementation for such
clients for use with standalone HMS. Note that an implementation like
PartitionExpressionForMetastore cannot be used in such cases since standalone
HMS installs will/should not have hive-exec dependencies.
> Add support to standalone Hive metastore for bulk dropping partitions out of
> the box
> ------------------------------------------------------------------------------------
>
> Key: HIVE-25463
> URL: https://issues.apache.org/jira/browse/HIVE-25463
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
> Affects Versions: 3.1.2
> Reporter: Chinmay Kulkarni
> Priority: Major
>
> Today for setups that use standalone metastore, we set the following:
> {noformat}
> <property>
> <name>metastore.expression.proxy</name>
>
> <value>org.apache.hadoop.hive.metastore.DefaultPartitionExpressionProxy</value>
> </property>
> {noformat}
> However,
> [DefaultPartitionExpressionProxy|https://github.com/apache/hive/blob/d61c9160ffa5afbd729887c3db690eccd7ef8238/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/DefaultPartitionExpressionProxy.java]
> throws UnsupportedOperationException throughout, thus preventing any
> partition pruning to be successful.
> The effect of this is, other projects that rely on just standalone Hive
> Metastore (for example Trino) are unable to do things like invoke bulk
> partition drops using say,
> [HiveMetaStore.drop_partitions_req|https://github.com/apache/hive/blob/d61c9160ffa5afbd729887c3db690eccd7ef8238/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java#L5008]
> via a Thrift client.
> We should implement a separate PartitionExpressionProxy implementation for
> such clients for use with standalone HMS. Note that an implementation like
> PartitionExpressionForMetastore cannot be used in such cases since standalone
> HMS installs will/should not have hive-exec dependencies.
> Please refer to [Trino
> issue#7249|https://github.com/trinodb/trino/issues/7249] for more context
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
