[
https://issues.apache.org/jira/browse/HDFS-10285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16273744#comment-16273744
]
Uma Maheswara Rao G commented on HDFS-10285:
--------------------------------------------
Discussion summary:
[~rakeshr], [~anu] and Me had discussion on the SPS.
Anu proposed couple of options to start SPS as separate service in HDFS.
1. Run SPS as tool like Balancer
Discussion conclusion: We have this option already. For the java
applications it is difficult to control the movement via tool. For different
Java applications, they need to run different tool instances and there will not
be any co-ordination on load etc. Conclusion is, we will not adopt this
approach.
2. Run SPS at client side itself.
Discussion Conclusion: This also has similar effect like lacking of
co-ordination. It is difficult to maintain the requests from users without
persisting. When process restarts, every such application needs to scan
complete namespace for knowing SPS xattr and also difficult to distinguish the
Xattrs of different application. Conclusion is, we will not adopt this approach.
3. Run SPS as independent Service.
Discussion Conclusion: This is more like start SPS in different JVM
instead of a thread in same JVM of Namnode. This option seems like reasonable
on compromising the extra process maintenance.
The plan is to start SPS daemon as independent service. It will have its won
RPC server. SPS sends the move requests directly to Das instead of via NN-DN
heartbeats.
NN will expose required information to SPS, then SPS will pull the
information and analyze.
Make sure to keep SPS as sateless. SPS state should be able to rebuild by
querying the Xattrs Inodes from Namenode on restarts. This make the SPS HA
implementation simple.
Advantages: 1. This will avoid Namenode burden due to SPS work
2. This could be a first step for bringing other similar
service to this process in future.
Disadvantage: one extra process in cluster. I think should be a reasonable
compromise
If any concerns on this changes, please comment early, so that we could
in-corporate. Other wise it will be painful to rework after finishing the work.
We will update in design doc soon about the new changes to SPS.
Thanks to Anu for your help on reviews and also offering his help on making SPS
as separate process work. Appreciate your contributions. Thank you.
[~rakeshr] [~anu] please update if I miss any points from discussion.
> Storage Policy Satisfier in Namenode
> ------------------------------------
>
> Key: HDFS-10285
> URL: https://issues.apache.org/jira/browse/HDFS-10285
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, namenode
> Affects Versions: HDFS-10285
> Reporter: Uma Maheswara Rao G
> Assignee: Uma Maheswara Rao G
> Attachments: HDFS-10285-consolidated-merge-patch-00.patch,
> HDFS-10285-consolidated-merge-patch-01.patch,
> HDFS-10285-consolidated-merge-patch-02.patch,
> HDFS-10285-consolidated-merge-patch-03.patch,
> HDFS-SPS-TestReport-20170708.pdf,
> Storage-Policy-Satisfier-in-HDFS-June-20-2017.pdf,
> Storage-Policy-Satisfier-in-HDFS-May10.pdf,
> Storage-Policy-Satisfier-in-HDFS-Oct-26-2017.pdf
>
>
> Heterogeneous storage in HDFS introduced the concept of storage policy. These
> policies can be set on directory/file to specify the user preference, where
> to store the physical block. When user set the storage policy before writing
> data, then the blocks could take advantage of storage policy preferences and
> stores physical block accordingly.
> If user set the storage policy after writing and completing the file, then
> the blocks would have been written with default storage policy (nothing but
> DISK). User has to run the ‘Mover tool’ explicitly by specifying all such
> file names as a list. In some distributed system scenarios (ex: HBase) it
> would be difficult to collect all the files and run the tool as different
> nodes can write files separately and file can have different paths.
> Another scenarios is, when user rename the files from one effected storage
> policy file (inherited policy from parent directory) to another storage
> policy effected directory, it will not copy inherited storage policy from
> source. So it will take effect from destination file/dir parent storage
> policy. This rename operation is just a metadata change in Namenode. The
> physical blocks still remain with source storage policy.
> So, Tracking all such business logic based file names could be difficult for
> admins from distributed nodes(ex: region servers) and running the Mover tool.
> Here the proposal is to provide an API from Namenode itself for trigger the
> storage policy satisfaction. A Daemon thread inside Namenode should track
> such calls and process to DN as movement commands.
> Will post the detailed design thoughts document soon.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
