[
https://issues.apache.org/jira/browse/HDFS-17045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz-wo Sze updated HDFS-17045:
------------------------------
Description:
HDFS-16972 added a
[shouldDestroy|https://github.com/szetszwo/hadoop/blob/331e075115b4a35574622318b26f6d4731658d57/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeReference.java#L834-L845]
method which caused the following bug.
h3. Background:
- When {{FileSystem.rename(src, dst)}} from a snapshottable dir (src) to a
snapshottable dir (dst), dstSnapshotId is set to the latest snapshot at dst. As
a result, dst is NOT in dstSnapshotId because dstSnapshotId was already taken
before rename.
- snapshotToBeDeleted is the snapshot id of the current operation if the
operation is {{{}FileSystem.deleteSnapshot{}}}. Otherwise, snapshotToBeDeleted
is set to CURRENT_STATE_ID.
- If (snapshotToBeDeleted > dstSnapshotId), dst is in snapshotToBeDeleted. The
shouldDestroy method returns true to continue deletion.
- If (snapshotToBeDeleted <= dstSnapshotId), dst must not be in
snapshotToBeDeleted. The shouldDestroy method returns false to stop deletion.
All the above are correct for renaming within snapshottable directories.
h3. Bug:
- If rename(src, dst) from a snapshottable dir (src) to a non-snapshottable
dir (dst), dstSnapshotId becomes CURRENT_STATE_ID.
- When {{FileSystem.delete(dst)}} happens, snapshotToBeDeleted is also set to
CURRENT_STATE_ID.
- In this case, snapshotToBeDeleted == dstSnapshotId, the shouldDestroy method
will return false and it incorrectly stops the deletion.
Not that this bug may cause fsimage corruption and quota miscalculation since
some files can be partially deleted. Fortunately, this bug won't cause data
loss.
was:
HDFS-16972 added a
[shouldDestroy|https://github.com/szetszwo/hadoop/blob/331e075115b4a35574622318b26f6d4731658d57/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeReference.java#L834-L845]
method which caused the following bug.
h3. Background:
- When {{FileSystem.rename(src, dst)}} from a snapshottable dir (src),
dstSnapshotId is set to the latest snapshot at dst. As a result, dst is NOT in
dstSnapshotId because dstSnapshotId was already taken before rename.
- snapshotToBeDeleted is the snapshot id of the current operation if the
operation is {{{}FileSystem.deleteSnapshot{}}}. Otherwise, snapshotToBeDeleted
is set to CURRENT_STATE_ID.
- If (snapshotToBeDeleted > dstSnapshotId), dst is in snapshotToBeDeleted. The
shouldDestroy method returns true to continue deletion.
- If (snapshotToBeDeleted <= dstSnapshotId), dst must not be in
snapshotToBeDeleted. The shouldDestroy method returns false to stop deletion.
h3. Bug
- If rename(src, dst) from a snapshottable dir (src) to a non-snapshottable
dir (dst), dstSnapshotId becomes CURRENT_STATE_ID.
- When {{FileSystem.delete(dst)}} happens, snapshotToBeDeleted is also set to
CURRENT_STATE_ID.
- In this case, snapshotToBeDeleted == dstSnapshotId, the shouldDestroy method
will return false and it incorrectly stops the deletion.
> File renamed from a snapshottable dir to a non-snapshottable dir cannot be
> deleted.
> -----------------------------------------------------------------------------------
>
> Key: HDFS-17045
> URL: https://issues.apache.org/jira/browse/HDFS-17045
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode, snapshots
> Reporter: Tsz-wo Sze
> Assignee: Tsz-wo Sze
> Priority: Major
> Labels: pull-request-available
>
> HDFS-16972 added a
> [shouldDestroy|https://github.com/szetszwo/hadoop/blob/331e075115b4a35574622318b26f6d4731658d57/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeReference.java#L834-L845]
> method which caused the following bug.
> h3. Background:
> - When {{FileSystem.rename(src, dst)}} from a snapshottable dir (src) to a
> snapshottable dir (dst), dstSnapshotId is set to the latest snapshot at dst.
> As a result, dst is NOT in dstSnapshotId because dstSnapshotId was already
> taken before rename.
> - snapshotToBeDeleted is the snapshot id of the current operation if the
> operation is {{{}FileSystem.deleteSnapshot{}}}. Otherwise,
> snapshotToBeDeleted is set to CURRENT_STATE_ID.
> - If (snapshotToBeDeleted > dstSnapshotId), dst is in snapshotToBeDeleted.
> The shouldDestroy method returns true to continue deletion.
> - If (snapshotToBeDeleted <= dstSnapshotId), dst must not be in
> snapshotToBeDeleted. The shouldDestroy method returns false to stop deletion.
> All the above are correct for renaming within snapshottable directories.
> h3. Bug:
> - If rename(src, dst) from a snapshottable dir (src) to a non-snapshottable
> dir (dst), dstSnapshotId becomes CURRENT_STATE_ID.
> - When {{FileSystem.delete(dst)}} happens, snapshotToBeDeleted is also set
> to CURRENT_STATE_ID.
> - In this case, snapshotToBeDeleted == dstSnapshotId, the shouldDestroy
> method will return false and it incorrectly stops the deletion.
> Not that this bug may cause fsimage corruption and quota miscalculation since
> some files can be partially deleted. Fortunately, this bug won't cause data
> loss.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
