[
https://issues.apache.org/jira/browse/HDFS-10885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15643727#comment-15643727
]
Rakesh R commented on HDFS-10885:
---------------------------------
Thanks [~drankye], [~zhouwei] for the useful thoughts. Please let me know your
feedback on below points.
bq. It looks to me more natural that we have StoragePolicySatisfier as a first
citizen class (not just under BM) under NameNode since it needs to access both
BM and the name system. I'm thinking that in future we may extend SPS to do
more NN wide things.
Yes, presently the layering is causing difficulties in handling the
simultaneous actions between Mover tool and SPS. Last day myself and
[~umamaheswararao] had an internal discussion about the same and discussed in
similar lines. Probably we could move SPS to NN and add the FSNamesystem
reference to access its APIs, that will reduce lot of complexities.
bq. I'm not sure it's a good idea to create the MOVER_ID file here in SPS,
because it may be the duty of mover to create the flag file. And also, not sure
how easy it is to do the creation in NN without inviting race or tricky
condition.
There are few ideas came up in the discussion to handle the race condition.
Below is the draft idea which we([~umamaheswararao] & me) discussed. Uma,
please feel free to update if I missed anything. Thanks!
*SPS:*
Should introduce a mechanism to get the running status of SPS so that Mover can
use this and understand the status. It could be a new RPC call.
During SPS startup, it should do following checks:
1) Check the lease of MOVER_ID (INodesInPath) if not exists then continue with
SPS startup
2) After the startup, should do a double check ensuring that there is no lease
exists. If lease exists, then stop SPS.
*Mover:*
During Mover startup, it should do following checks:
1) Ensure SPS is not running by using the new RPC call or function. If SPS is
not running then continue with Mover run by creating the MOVER_ID.
2) After MOVER_ID creation, should do a double check ensuring that SPS is not
running. If SPS running, then do Mover exists by deleting the MOVER_ID.
> [SPS]: Mover tool should not be allowed to run when Storage Policy Satisfier
> is on
> ----------------------------------------------------------------------------------
>
> Key: HDFS-10885
> URL: https://issues.apache.org/jira/browse/HDFS-10885
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Affects Versions: HDFS-10285
> Reporter: Wei Zhou
> Assignee: Wei Zhou
> Fix For: HDFS-10285
>
> Attachments: HDFS-10800-HDFS-10885-00.patch,
> HDFS-10800-HDFS-10885-01.patch, HDFS-10800-HDFS-10885-02.patch,
> HDFS-10885-HDFS-10285.03.patch, HDFS-10885-HDFS-10285.04.patch,
> HDFS-10885-HDFS-10285.05.patch
>
>
> These two can not work at the same time to avoid conflicts and fight with
> each other.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
