echonesis commented on code in PR #213: URL: https://github.com/apache/ozone-site/pull/213#discussion_r2660193725
########## docs/03-core-concepts/03-namespace/02-ozone-manager.md: ########## @@ -0,0 +1,99 @@ +--- +sidebar_label: Ozone Manager +--- + +# Ozone Manager + + + +Ozone Manager (OM) is the namespace manager for Ozone. + +This means that when you want to write some data, you ask Ozone Manager for a block and Ozone Manager gives you a block and remembers that information. When you want to read that file back, you need to find the address of the block and the Ozone Manager returns it to you. + +Ozone Manager also allows users to organize keys under a volume and bucket. Volumes and buckets are part of the namespace and managed by Ozone Manager. + +Each Ozone volume is the root of an independent namespace under OM. This is very different from HDFS which provides a single rooted file system. + +Ozone's namespace is a collection of volumes or is a forest instead of a single rooted tree as in HDFS. This property makes it easy to deploy multiple OMs for scaling(pending future development). + +## Ozone Manager Metadata + +OM maintains a list of volumes, buckets, and keys. For each user, it maintains a list of volumes. For each volume, the list of buckets and for each bucket the list of keys. + +Ozone Manager will use [Apache Ratis](https://ratis.apache.org/) (A Raft protocol implementation) to replicate Ozone Manager state. <!-- TODO: Link to OM HA page when created --> This will ensure High Availability for Ozone Manager. + +## Ozone Manager and Storage Container Manager + +<!-- TODO: Link to SCM page when created --> The relationship between Ozone Manager and Storage Container Manager (SCM) is best understood if we trace what happens during a key write and key read. + +### Key Write + + + +- To write a key to Ozone, a client tells Ozone Manager that it would like to write a key into a bucket that lives inside a specific volume. Once Ozone Manager determines that you are allowed to write a key to the specified bucket, OM needs to allocate a block for the client to write data. + +- To allocate a block, Ozone Manager sends a request to Storage Container Manager (SCM); <!-- TODO: Link to Datanodes page when created --> SCM is the manager of data nodes. SCM picks three data nodes into which client can write data. SCM allocates the block and returns the block ID to Ozone Manager. + +- Ozone Manager records this block information in its metadata and returns the block and a <!-- TODO: Link to block token/security page when created --> block token (a security permission to write data to the block) to the client. + +- The client uses the block token to prove that it is allowed to write data to the block and writes data to the data node. + +- Once the write is complete on the data node, the client will update the block information on Ozone Manager. + +### Key Reads + + + +- Key reads are simpler, the client requests the block list from the Ozone Manager +- Ozone Manager will return the block list and block tokens which allows the client to read the data from data nodes. +- Client connects to the data node and presents the block token and reads the data from the data node. + +## Main Components of the Ozone Manager + +For a detailed view of Ozone Manager this section gives a quick overview about the provided network services and the stored persisted data. + +### Network Services Provided by Ozone Manager + +Ozone provides a network service for the client and for administration commands. The main service calls: + +- Key, Bucket, Volume / CRUD +- Multipart upload (Initiate, Complete…) + - Supports upload of huge files in multiple steps +- FS related calls (optimized for hierarchical queries instead of a flat ObjectStore namespace) + - GetFileStatus, CreateDirectory, CreateFile, LookupFile +- ACL related + - <!-- TODO: Link to internal ACLs and Ranger pages when created --> Managing ACLs if internal ACLs are used instead of Ranger Review Comment: Please refer to this issue: https://issues.apache.org/jira/browse/HDDS-14338 for `internal ACLs`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
