[
https://issues.apache.org/jira/browse/HDFS-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175522#comment-16175522
]
Xiao Chen commented on HDFS-12291:
----------------------------------
Thank you for the ping [~umamaheswararao], thanks [~surendrasingh] and all for
the work!
A few comments:
- {{checkINodeReady}} for SPS should still check the basics of the NN (standby,
safemode), since it's running within a readlock.
- From existing code, I think we'd better assert no lock is held at the
beginning of {{throttle}} method.
- Not sure if we worry about create/delete inside the base dir during traversal
here. This was important for re-encryption, where a bunch of race tests are
added in TestReencryption.
- Heads up: there's a recent bug/improvement HDFS-12518 for HDFS-10899, will
likely create some conflicts here. Issue itself is mostly for re-encryption
though.
Otherwise LGTM
> [SPS]: Provide a mechanism to recursively iterate and satisfy storage policy
> of all the files under the given dir
> -----------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-12291
> URL: https://issues.apache.org/jira/browse/HDFS-12291
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Reporter: Rakesh R
> Assignee: Surendra Singh Lilhore
> Attachments: HDFS-12291-HDFS-10285-01.patch,
> HDFS-12291-HDFS-10285-02.patch, HDFS-12291-HDFS-10285-03.patch,
> HDFS-12291-HDFS-10285-04.patch
>
>
> For the given source path directory, presently SPS consider only the files
> immediately under the directory(only one level of scanning) for satisfying
> the policy. It WON’T do recursive directory scanning and then schedules SPS
> tasks to satisfy the storage policy of all the files till the leaf node.
> The idea of this jira is to discuss & implement an efficient recursive
> directory iteration mechanism and satisfies storage policy for all the files
> under the given directory.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
