[
https://issues.apache.org/jira/browse/HDDS-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17711467#comment-17711467
]
Neil Joshi commented on HDDS-8410:
----------------------------------
Hi [~Myskov] , we have a similar ticket open. A doc is mostly prepared for
this. I can make it available see:
https://issues.apache.org/jira/browse/HDDS-7711
> Implement storage policies
> --------------------------
>
> Key: HDDS-8410
> URL: https://issues.apache.org/jira/browse/HDDS-8410
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Maksim Myskov
> Priority: Major
>
> In order to optimize read performance, it is required to implement HDFS-like
> storage policies: HOT, WARM, COLD.
> {*}HOT{*}: all container’s replicas are stored on fast storage
> {*}WARM{*}: one container replica is stored on fast storage, others are
> stored on regular storage
> {*}COLD{*}: all container’s replicas are stored on slow storage
>
> *Use case example 1:*
> There is a cluster of K datanodes. Each datanode has N regular disks and M
> fast disks.
>
> *Use case example 2:*
> There is a cluster of N regular datanodes and M fast datanodes. Regular
> datanodes have regular (HDD) disks and fast datanodes have fast (SSD) disks.
>
> Requirements:
> * If a container’s heat state (hot-warm-cold) changes, container can be
> moved to a different disk on the same node or to a different node, depending
> on the cluster configuration.
> * container’s heat is automatically calculated based on data request
> statistics.
> * A file/directory’s heat state can be set manually. Example: if one set
> “key1/subkey/*” to be HOT data, all chunks related to this prefix must be
> considered HOT despite heatmap.
> Open questions:
> * How to calculate data heat map? A possible solution is that statistics are
> collected by datanodes and then sent to SCM in reports. SCM analyses these
> reports to build an actual data heat map.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
