[
https://issues.apache.org/jira/browse/HIVE-27692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhihua Deng updated HIVE-27692:
-------------------------------
Description:
The always tasks are running in the leader HMS now, the properties for
configuring the leader should only belong to HMS, other engines such as
Spark/Impala doesn't need to know these properties. For most cases, the engine
only cares about the hive.metastore.uris.
Every time when a new apps uses an embedded Metastore, it will start the HMS
always tasks by default. Imaging we have hundreds of apps, then hundreds of
pieces of tasks are running, this will put extra burden to the underlying
databases, such as the flooding queries, connection limit.
I think we can remove always tasks from the embeded Metastore, the always task
will be taken care of by the standalone Metastore and a standalone Metastore
should be here in production environment.
was:
The always tasks are running in the leader HMS now, the properties for
configuring the leader should only belong to HMS, other engines such as
Spark/Impala don't need to know these properties. For most cases, the engine
only cares about the hive.metastore.uris.
Every time when a new apps uses an embedded Metastore, it will start the HMS
always tasks by default. Imaging we have hundreds of apps, then hundreds of
pieces of tasks are running, this will put extra burden to the underlying
databases, such as the flooding queries, connection limit.
I think we can remove always tasks from the embeded Metastore, the always task
will be taken care of by the standalone Metastore and a standalone Metastore
should be here in production environment.
> Explore removing the always task from embedded HMS
> --------------------------------------------------
>
> Key: HIVE-27692
> URL: https://issues.apache.org/jira/browse/HIVE-27692
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
> Reporter: Zhihua Deng
> Assignee: Zhihua Deng
> Priority: Major
>
> The always tasks are running in the leader HMS now, the properties for
> configuring the leader should only belong to HMS, other engines such as
> Spark/Impala doesn't need to know these properties. For most cases, the
> engine only cares about the hive.metastore.uris.
> Every time when a new apps uses an embedded Metastore, it will start the HMS
> always tasks by default. Imaging we have hundreds of apps, then hundreds of
> pieces of tasks are running, this will put extra burden to the underlying
> databases, such as the flooding queries, connection limit.
> I think we can remove always tasks from the embeded Metastore, the always
> task will be taken care of by the standalone Metastore and a standalone
> Metastore should be here in production environment.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
