[
https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wei Zhou updated HDFS-7343:
---------------------------
Attachment: HDFSSmartStorageManagement-Phase1-20170315.pdf
HDFSSmartStorageManagement-General-20170315.pdf
Based on the discussion and feedbacks collected, we updated the design. The
design is separated into two: one for SSM general discussion and the other one
focusing on Phase1 only.
In Phase 1, We focus on 3 use cases and go further to the details:
# Defines the metrics to collect. No modification to DN and Client, all metrics
are got through NN. The modification and overhead for NN are under control.
# Defines the objects/attributes/events/commands to support in rule
# Defines the execution flow of rules
# Defines the data to be persistent for SSM and NN
Thanks [~andrew.wang], [~anu], [~umamaheswararao], [~rakeshr], [~eddyxu] for
reviewing the design and the great comments. Hopefully this update has
incorporated your thoughts and addressed your questions well. Thanks!
> HDFS smart storage management
> -----------------------------
>
> Key: HDFS-7343
> URL: https://issues.apache.org/jira/browse/HDFS-7343
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: Kai Zheng
> Assignee: Wei Zhou
> Attachments: HDFSSmartStorageManagement-General-20170315.pdf,
> HDFS-Smart-Storage-Management.pdf,
> HDFSSmartStorageManagement-Phase1-20170315.pdf,
> HDFS-Smart-Storage-Management-update.pdf, move.jpg
>
>
> As discussed in HDFS-7285, it would be better to have a comprehensive and
> flexible storage policy engine considering file attributes, metadata, data
> temperature, storage type, EC codec, available hardware capabilities,
> user/application preference and etc.
> Modified the title for re-purpose.
> We'd extend this effort some bit and aim to work on a comprehensive solution
> to provide smart storage management service in order for convenient,
> intelligent and effective utilizing of erasure coding or replicas, HDFS cache
> facility, HSM offering, and all kinds of tools (balancer, mover, disk
> balancer and so on) in a large cluster.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
