[
https://issues.apache.org/jira/browse/HDDS-2939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17166945#comment-17166945
]
Rakesh Radhakrishnan commented on HDDS-2939:
--------------------------------------------
Thanks [~linyiqun] for the comments.
{quote}It reuses the PrefixTable to store like directory info I have a
question: this will not break original key lookup behavior?
{quote}
The {{lookup}} logic at OzoneManager server side will change but there won't be
any OzoneFS API contract changes to the users. Today, OM seeks {{dirs, files}}
directly from the 'KeyTable' but with my proposal it requires to do
incremental(please refer below example case) seek 'PrefixTable' for building
the prefix part and then seek 'KeyTable' for the file part. I have a chain of
patches to implement the logic of 'createFiles', 'createKeys', 'getFileStatus',
'listStatus', 'lookupKey', 'rename', 'delete' functions etc..
For example, getFileStatus(new Path("/a/b/c/d/file1"))
*PrefixTable:*
get "512/a" -> 1025
get "1025/b" -> 1026
get "1026/c" -> 1027
get "1027/d" -> 1028
Then for the 'file1', it requires another seek on *KeyTable:*
get "1028/file1" -> 1030
Like I said in my previous comment, there is a plan to build 'in-memory dir/key
cache' to avoid the additional look up overheads compares to the existing code.
{quote}We don't have the object id assigned for keys in prefix table, but
current logic assumes that each entry key has its own objectId in table. Or
maybe we will have a step to generate this object id firstly when users start
to enable this feature from their existed Ozone system.
{quote}
Very good point. I am using the existing Object Id generation logic -
[OMFileRequest.getObjIDFromTxId|https://github.com/apache/hadoop-ozone/blob/master/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/request/key/OMKeyRequest.java#L250]
, which is a unique key generated from the 'txnId'. Along with this feature,
one task is planned to provide a {{migration/conversion tool}} to migrate
existing 'KeyTable' data content into the new data format, ofcourse we need to
add a key generation logic here. Will brainstorm more about this on the
conversion tool implementation Jira task. Hope this is fine for you?
> Ozone FS namespace
> ------------------
>
> Key: HDDS-2939
> URL: https://issues.apache.org/jira/browse/HDDS-2939
> Project: Hadoop Distributed Data Store
> Issue Type: New Feature
> Components: Ozone Manager
> Reporter: Supratim Deka
> Assignee: Rakesh Radhakrishnan
> Priority: Major
> Labels: Triaged
> Attachments: Ozone FS Namespace Proposal v1.0.docx
>
>
> Create the structures and metadata layout required to support efficient FS
> namespace operations in Ozone - operations involving folders/directories
> required to support the Hadoop compatible Filesystem interface.
> The details are described in the attached document. The work is divided up
> into sub-tasks as per the task list in the document.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
