[ https://issues.apache.org/jira/browse/HDFS-11197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wellington Chevreuil updated HDFS-11197: ---------------------------------------- Attachment: HDFS-11197-7.patch Hi [~jojochuang], am proposing a new patch, adding end-to-end tests for this specific issue on {{hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testCryptoConf.xml and TestCryptoAdminCLI}}. Few changes performed on {{TestCryptoAdminCLI}} for these tests: 1) Since the issue involved snapshots, I had to make {{TestConfigFileParserCryptoAdmin}} to extend {{CLITestHelperDFS.TestConfigFileParserDFS}}, instead of {{CLITestHelper.TestConfigFileParser}} directly, so that it could parse *dfs-admin-command* commands from {{testCryptoConf.xml}}. This was needed because the test had to run a *dfsadmin -allowSnapshot* command; 2) The issue also involved files being moved to trash, prior to permanent deletion. For that purpose, I had to enable "move to Trash" behaviour on {{TestCryptoAdminCLI.setup}} method, by adding {{conf.setLong(CommonConfigurationKeysPublic.FS_TRASH_INTERVAL_KEY, 10);}} The additional tests check for listZones output under the following cases: 1) Creates a single EZ then call listZones and check if only the created EZ was returned; 2) Creates two EZs, then delete one of those, the call listZones and check if the two are listed. The deleted EZ should be listed "/user/*/.Trash/Current" directory, since Trash is enabled; 3) Creates two EZs, then remove one of the EZ, then remove the deleted one from Trash, then call listZones and checks if only the non-deleted EZ was returned; 4) Creates two EZs under a snapshottable directory, takes a snapshot for the directory, remove one of the EZs, then remove this EZ from Trash, calls listZones and check if only the non-deleted EZ was returned. Please let me know on your thoughts. > Listing encryption zones fails when deleting a EZ that is on a snapshotted > directory > ------------------------------------------------------------------------------------ > > Key: HDFS-11197 > URL: https://issues.apache.org/jira/browse/HDFS-11197 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs > Affects Versions: 2.6.0 > Reporter: Wellington Chevreuil > Assignee: Wellington Chevreuil > Priority: Minor > Attachments: HDFS-11197-1.patch, HDFS-11197-2.patch, > HDFS-11197-3.patch, HDFS-11197-4.patch, HDFS-11197-5.patch, > HDFS-11197-6.patch, HDFS-11197-7.patch > > > If a EZ directory is under a snapshotable directory, and a snapshot has been > taking, then if this EZ is permanently deleted, it causes *hdfs crypto > listZones* command to fail without showing any of the still available zones. > This happens only after the EZ is removed from Trash folder. For example, > considering */test-snap* folder is snapshotable and there is already an > snapshot for it: > {noformat} > $ hdfs crypto -listZones > /user/systest my-key > /test-snap/EZ-1 my-key > $ hdfs dfs -rmr /test-snap/EZ-1 > INFO fs.TrashPolicyDefault: Moved: 'hdfs://ns1/test-snap/EZ-1' to trash at: > hdfs://ns1/user/hdfs/.Trash/Current/test-snap/EZ-1 > $ hdfs crypto -listZones > /user/systest my-key > /user/hdfs/.Trash/Current/test-snap/EZ-1 my-key > $ hdfs dfs -rmr /user/hdfs/.Trash/Current/test-snap/EZ-1 > Deleted /user/hdfs/.Trash/Current/test-snap/EZ-1 > $ hdfs crypto -listZones > RemoteException: Absolute path required > {noformat} > Once this error happens, *hdfs crypto -listZones* only works again if we > remove the snapshot: > {noformat} > $ hdfs dfs -deleteSnapshot /test-snap snap1 > $ hdfs crypto -listZones > /user/systest my-key > {noformat} > If we instead delete the EZ using *skipTrash* option, *hdfs crypto > -listZones* does not break: > {noformat} > $ hdfs crypto -listZones > /user/systest my-key > /test-snap/EZ-2 my-key > $ hdfs dfs -rmr -skipTrash /test-snap/EZ-2 > Deleted /test-snap/EZ-2 > $ hdfs crypto -listZones > /user/systest my-key > {noformat} > The different behaviour seems to be because when removing the EZ trash > folder, it's related INode is left with no parent INode. This causes > *EncryptionZoneManager.listEncryptionZones* to throw the seen error, when > trying to resolve the inodes in the given path. > Am proposing a patch that fixes this issue by simply performing an additional > check on *EncryptionZoneManager.listEncryptionZones* for the case an inode > has no parent, so that it would be skipped on the list without trying to > resolve it. Feedback on the proposal is appreciated. > -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org