This is an automated email from the ASF dual-hosted git repository.
msingh pushed a commit to branch HDDS-2939
in repository https://gitbox.apache.org/repos/asf/ozone.git
The following commit(s) were added to refs/heads/HDDS-2939 by this push:
new 45c62e8 HDDS-5067. [FSO] Add documentation (#2117)
45c62e8 is described below
commit 45c62e8725516cd33bd1bd5d0408bdc8bf08f0a6
Author: Rakesh Radhakrishnan <[email protected]>
AuthorDate: Thu Apr 8 14:02:34 2021 +0530
HDDS-5067. [FSO] Add documentation (#2117)
---
.../docs/content/feature/PrefixFSO-Delete.png | Bin 0 -> 253792 bytes
.../docs/content/feature/PrefixFSO-Format.png | Bin 0 -> 270550 bytes
.../docs/content/feature/PrefixFSO-Rename.png | Bin 0 -> 227851 bytes
hadoop-hdds/docs/content/feature/PrefixFSO.md | 75 +++++++++++++++++++++
.../hadoop/ozone/om/DirectoryDeletingService.java | 2 +-
5 files changed, 76 insertions(+), 1 deletion(-)
diff --git a/hadoop-hdds/docs/content/feature/PrefixFSO-Delete.png
b/hadoop-hdds/docs/content/feature/PrefixFSO-Delete.png
new file mode 100644
index 0000000..ec697ce
Binary files /dev/null and
b/hadoop-hdds/docs/content/feature/PrefixFSO-Delete.png differ
diff --git a/hadoop-hdds/docs/content/feature/PrefixFSO-Format.png
b/hadoop-hdds/docs/content/feature/PrefixFSO-Format.png
new file mode 100644
index 0000000..37707f4
Binary files /dev/null and
b/hadoop-hdds/docs/content/feature/PrefixFSO-Format.png differ
diff --git a/hadoop-hdds/docs/content/feature/PrefixFSO-Rename.png
b/hadoop-hdds/docs/content/feature/PrefixFSO-Rename.png
new file mode 100644
index 0000000..c401c10
Binary files /dev/null and
b/hadoop-hdds/docs/content/feature/PrefixFSO-Rename.png differ
diff --git a/hadoop-hdds/docs/content/feature/PrefixFSO.md
b/hadoop-hdds/docs/content/feature/PrefixFSO.md
new file mode 100644
index 0000000..55b7134
--- /dev/null
+++ b/hadoop-hdds/docs/content/feature/PrefixFSO.md
@@ -0,0 +1,75 @@
+---
+title: "Prefix based FileSystem Optimization"
+weight: 2
+menu:
+ main:
+ parent: Features
+summary: Supports atomic rename and delete operation.
+---
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+The prefix based FileSystem optimization feature supports atomic rename and
+ delete of any directory at any level in the namespace. Also, it will perform
+ rename and delete of any directory in a deterministic/constant time.
+
+Note: This feature works only when `ozone.om.enable.filesystem.paths` is
+ enabled which means that Hadoop Compatible File System compatibility is
+ favored instead of S3 compatibility. Some irregular S3 key names may be
+ rejected or normalized.
+
+This feature is strongly recommended to be turned ON when Ozone buckets are
+ mainly used via Hadoop compatible interfaces, especially with high number of
+ files in deep directory hierarchy.
+
+## OzoneManager Metadata layout format
+OzoneManager supports two metadata layout formats - legacy and prefix.
+
+Legacy is the existing OM metadata format, which stores key entry with full
path
+ name. In Prefix based optimization, OM metadata format stores intermediate
+ directories into `DirectoryTable` and files into `FileTable` as shown in the
+ below picture. The key to the table is the name of a directory or a file
+ prefixed by the unique identifier of its parent directory, `<parent
+ unique-id>/<filename>`.
+
+
+
+
+### Directory delete operation with prefix layout: ###
+Following picture describes the OM metadata changes while performing a delete
+ operation on a directory.
+
+
+### Directory rename operation with prefix layout: ###
+Following picture describes the OM metadata changes while performing a rename
+ operation on a directory.
+
+
+## Configuration
+By default the feature is disabled. It can be enabled with the following
+ settings in `ozone-site.xml`:
+
+```XML
+<property>
+ <name>ozone.om.enable.filesystem.paths</name>
+ <value>true</value>
+</property>
+<property>
+ <name>ozone.om.metadata.layout</name>
+ <value>PREFIX</value>
+</property>
+```
\ No newline at end of file
diff --git
a/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DirectoryDeletingService.java
b/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DirectoryDeletingService.java
index ec5c3a9..bb33722 100644
---
a/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DirectoryDeletingService.java
+++
b/hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/DirectoryDeletingService.java
@@ -47,7 +47,7 @@ import static
org.apache.hadoop.ozone.om.OMConfigKeys.OZONE_PATH_DELETING_LIMIT_
* <p>
* This will scan the metadata of om periodically to get the orphan dirs from
* DeletedDirectoryTable and find its sub paths. It will fetch all sub-files
- * from KeyTable and move those to DeletedTable so that OM's
+ * from FileTable and move those to DeletedTable so that OM's
* KeyDeletingService will cleanup those files later. It will fetch all
* sub-directories from the DirectoryTable and move those to
* DeletedDirectoryTable so that these will be visited in next iterations.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
