Henri Biestro created HIVE-27186:
------------------------------------
Summary: A persistent property store
Key: HIVE-27186
URL: https://issues.apache.org/jira/browse/HIVE-27186
Project: Hive
Issue Type: Improvement
Components: Metastore
Affects Versions: 4.0.0-alpha-2
Reporter: Henri Biestro
WHAT
A persistent property store usable as a support facility for any metadata
augmentation feature.
WHY
When adding new meta-data oriented features, we usually need to persist
information linking the feature data and the HiveMetaStore objects it applies
to. Any information related to a database, a table or the cluster - like
statistics for example or any operational data state or data (think rolling
backup) - fall in this use-case.
Typically, accommodating such a feature requires modifying the Metastore
database schema by adding or altering a table. It also usually implies
modifying the thrift APIs to expose such meta-data to consumers.
The proposed feature wants to solve the persistence and query/transport for
these types of use-cases by exposing a 'key/(meta)value' store exposed as a
property system.
HOW
A property-value model is the simple and generic exposed API.
To provision for several usage scenarios, the model entry point is a
'namespace' that qualifies the feature-component property manager. For example,
'stats' could be the namespace for all properties related to the 'statistics'
feature.
The namespace identifies a manager that handles property-groups persisted as
property-maps. For instance, all statistics pertaining to a given table would
be collocated in the same property-group. As such, all properties (say number
of 'unique_values' per columns) for a given HMS table 'relation0' would all be
stored and persisted in the same property-map instance.
Property-maps may be decorated by an (optional) schema that may declare the
name and value-type of allowed properties (and their optional default value).
Each property is addressed by a name, a path uniquely identifying the property
in a given property map.
The manager also handles transforming property-map names to the property-map
keys used to persist them in the DB.
The API provides inserting/updating properties in bulk transactionally. It also
provides selection/projection to help reduce the volume of exchange between
client/server; selection can use (JEXL expression) predicates to filter maps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
