[
https://issues.apache.org/jira/browse/HIVE-27186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17833943#comment-17833943
]
Denys Kuzmenko edited comment on HIVE-27186 at 4/4/24 2:29 PM:
---------------------------------------------------------------
PR was merged in Jun 8, 2023 and was part of beta-1 release and now 4.0
There are no usages of introduced IMetaStoreClient[setProperties,getProperties]
public APIs. Still, at the same time, we have PropertyServlet with a bunch of
hardcode + [commented-out
code|https://github.com/apache/hive/blob/b0d3503eea03015d5be9f66e439bee016a176754/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PropertyServlet.java#L245-L256]
that doesn't even use it.
*SonarCloud Quality Gate*
Vulnerability B 5 Vulnerabilities
Security Hotspot E 1 Security Hotspot
Code Smell A 90 Code Smells
Since [~henrib] can't explain the purpose of this abandoned feature, [~ngangam]
could you please shed some light and share what is the intent of it and how it
would be used? Wouldn't Redis be a natural fit for "'key/(meta)value' store"
was (Author: dkuzmenko):
There are no usages of introduced IMetaStoreClient[setProperties,getProperties]
public APIs. Still, at the same time, we have PropertyServlet with a bunch of
hardcode + [commented-out
code|https://github.com/apache/hive/blob/b0d3503eea03015d5be9f66e439bee016a176754/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/PropertyServlet.java#L245-L256]
that doesn't even use it.
*SonarCloud Quality Gate*
Vulnerability B 5 Vulnerabilities
Security Hotspot E 1 Security Hotspot
Code Smell A 90 Code Smells
Since [~henrib] can't explain the purpose of this abandoned feature, [~ngangam]
could you please shed some light and share what is the intent of it and how it
would be used? Wouldn't Redis be a natural fit for "'key/(meta)value' store"
> A persistent property store
> ----------------------------
>
> Key: HIVE-27186
> URL: https://issues.apache.org/jira/browse/HIVE-27186
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 4.0.0-alpha-2
> Reporter: Henri Biestro
> Assignee: Henri Biestro
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-beta-1
>
> Time Spent: 19h 10m
> Remaining Estimate: 0h
>
> WHAT
> A persistent property store usable as a support facility for any metadata
> augmentation feature.
> WHY
> When adding new meta-data oriented features, we usually need to persist
> information linking the feature data and the HiveMetaStore objects it applies
> to. Any information related to a database, a table or the cluster - like
> statistics for example or any operational data state or data (think rolling
> backup) - fall in this use-case.
> Typically, accommodating such a feature requires modifying the Metastore
> database schema by adding or altering a table. It also usually implies
> modifying the thrift APIs to expose such meta-data to consumers.
> The proposed feature wants to solve the persistence and query/transport for
> these types of use-cases by exposing a 'key/(meta)value' store exposed as a
> property system.
> HOW
> A property-value model is the simple and generic exposed API.
> To provision for several usage scenarios, the model entry point is a
> 'namespace' that qualifies the feature-component property manager. For
> example, 'stats' could be the namespace for all properties related to the
> 'statistics' feature.
> The namespace identifies a manager that handles property-groups persisted as
> property-maps. For instance, all statistics pertaining to a given table would
> be collocated in the same property-group. As such, all properties (say number
> of 'unique_values' per columns) for a given HMS table 'relation0' would all
> be stored and persisted in the same property-map instance.
> Property-maps may be decorated by an (optional) schema that may declare the
> name and value-type of allowed properties (and their optional default value).
> Each property is addressed by a name, a path uniquely identifying the
> property in a given property map.
> The manager also handles transforming property-map names to the property-map
> keys used to persist them in the DB.
> The API provides inserting/updating properties in bulk transactionally. It
> also provides selection/projection to help reduce the volume of exchange
> between client/server; selection can use (JEXL expression) predicates to
> filter maps.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
