[
https://issues.apache.org/jira/browse/HDFS-12897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308695#comment-16308695
]
Xiao Chen commented on HDFS-12897:
----------------------------------
Thanks a lot [~GeLiXin] for revving!
Sorry didn't find this earlier. But poking at it, I found another issue that's
not correct with snapshots: snapshots are supposed to be immutable, so in the
following sequence of events for the same dir happens:
# set ec policy to RS(6,3)
# take snapshot s1
# set ec policy to RS(3,2)
# take snapshot s2
{{getPolicy}} on s1 should say RS(6,3), and on s2 should say RS(3,2).
I think this 1-liner should be able to fix EC policy:
{code:title=FSDirErasureCodingOp#getErasureCodingPolicyForPath}
- final XAttrFeature xaf = inode.getXAttrFeature();
+ final XAttrFeature xaf =
inode.getXAttrFeature(iip.getPathSnapshotId());
{code}
This is not exactly related to the 'path not found' error, but I suggest we fix
that while we're at it. (StoragePolicy isn't correct on this either, can do
that as a follow-on)
Some minor comments:
- Suggest we use {{src}} instead of {{iip.getPath()}} as the
{{FileNotFoundException}} message, as throwing the original input would be more
obvious to the caller. This also saves the possible path construction.
- In {{TestErasureCodingPolicies}}, can just use {{fs.mkdirs(ecDir);}} instead
of {{mkdir}} with the default perm.
- Please add a test of the new scenario mentioned above.
- Feels to me we can merge {{testErasureCodingPolicyOnReservedDir}} and
{{testErasureCodingPolicyOnReservedRawDir}}, to just verify /.reserved at the
beginning.
- Change the comments {{verify the EC policy is null other than an exception}}
to {{verify the EC policy is null, not an exception}} (['other
than'|https://www.merriam-webster.com/dictionary/other%20than] isn't right here)
- {{FSDirErasureCodingOp#getErasureCodingPolicy}}, javadocs change could be
updated. 'if it could not be set' is confusing on a get api. I'm fine if we
either 1) explaining that null could be due to no policy, or replication, or 2)
leave it as-is.
> Path not found when we get the ec policy for a .snapshot dir
> ------------------------------------------------------------
>
> Key: HDFS-12897
> URL: https://issues.apache.org/jira/browse/HDFS-12897
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: erasure-coding, hdfs, snapshots
> Affects Versions: 3.0.0-alpha1, 3.1.0
> Reporter: Harshakiran Reddy
> Assignee: LiXin Ge
> Attachments: HDFS-12897.001.patch, HDFS-12897.002.patch,
> HDFS-12897.003.patch
>
>
> Scenario:-
> ---------------
> Operation on snapshot dir.
> *EC policy*
> bin> ./hdfs ec -getPolicy -path /dir/
> RS-3-2-1024k
> bin> ./hdfs ec -getPolicy -path /dir/.snapshot/
> {{FileNotFoundException: Path not found: /dir/.snapshot}}
> bin> ./hdfs dfs -ls /dir/.snapshot/
> Found 2 items
> drwxr-xr-x - user group 0 2017-12-05 12:27 /dir/.snapshot/s1
> drwxr-xr-x - user group 0 2017-12-05 12:28 /dir/.snapshot/s2
> *Storagepolicies*
> bin> ./hdfs storagepolicies -getStoragePolicy -path /dir/.snapshot/
> {{The storage policy of /dir/.snapshot/ is unspecified}}
> bin> ./hdfs storagepolicies -getStoragePolicy -path /dir/
> The storage policy of /dir/:
> BlockStoragePolicy{COLD:2, storageTypes=[ARCHIVE], creationFallbacks=[],
> replicationFallbacks=[]}
> *Which is the correct behavior ?*
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
