[
https://issues.apache.org/jira/browse/HDDS-2466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDDS-2466:
--------------------------------
Target Version/s: 0.7.0
> Split OM Key into a Prefix Part and a Name Part
> -----------------------------------------------
>
> Key: HDDS-2466
> URL: https://issues.apache.org/jira/browse/HDDS-2466
> Project: Hadoop Distributed Data Store
> Issue Type: Improvement
> Components: Ozone Manager
> Reporter: Supratim Deka
> Assignee: Supratim Deka
> Priority: Major
>
> This jira will track all changes required for introducing a separate prefix
> for keys in the OM.
> In the current implementation, OM stores every key in a Key table, which maps
> the full key to a KeyInfo.
> FS semantics require unique names. Specifically - inside any given directory
> (the prefix of any OM object key), each object must have a unique name. This
> means that a sub-directory and a file which share the same prefix, cannot
> have the same name.
> if there exists an object(file) "/a/b/c/d/e/file" in OM, we cannot allow a
> new object(file/dir) "/a/b/c" to be created. it is implicit as per FS
> semantics that /a/b/c is the name of an existing directory.
> The problem is, OM stores full keys in the key table. So the file create
> handler must iterate through the entire key table to determine uniqueness of
> the name. This makes create operations very resource intensive.
> To improve this behaviour, we propose to store every valid prefix in a
> separate table in the OM, alongside the Key Table. The File create request
> handler can iterate this KeyPrefix table to check if the name already exists.
> Specific optimisations will be covered as separate jira items.
>
> The following 2 sets of tasks are required to complete the key prefix
> infrastructure - base infra followed by extension to achieve prefix
> compression(in-memory + on-disk) for the OM Key table.
> Base Infra:
> 1. Create prefix entry in KeyPrefix OM Table for every new key inserted in
> the OM Key Table - include file create, directory create and rename. Change
> File create request handler to check the KeyPrefix table, instead of
> iterating the entire Key Table.
> 2. Handle stale prefix entries in the KeyPrefix table - handle the situation
> where the last object associated with a prefix gets deleted. This task
> involves defining the semantics in this situation and subsequently the design
> for detecting stale entries.
> Extension to support prefix compression in Key Table:
> 3. Assign unique ID to each prefix in the KeyPrefix table - include
> synchronisation.
> 4. Node restart - ensure the unique ID generation scheme handles restart.
> 5. Change the key in OM Key Table to <Prefix ID>/<name> instead of the full
> name.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: ozone-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: ozone-issues-h...@hadoop.apache.org
