[
https://issues.apache.org/jira/browse/HDFS-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arpit Agarwal updated HDFS-6978:
--------------------------------
Description:
It used to be very unlikely that the directory scanner encountered two replicas
of the same block on different volumes.
With memory storage, it is very likely to hit this with the following sequence
of events:
# Block is written to RAM disk
# Lazy writer saves a copy on persistent volume
# DN attempts to evict the original replica from RAM disk, file deletion fails
as the replica is in use.
# Directory scanner finds a replica on both RAM disk and persistent storage.
The directory scanner should never delete the block on persistent storage.
was:
Currently it would be a very unlikely situation for the directory scanner to
encounter two replicas of the same block on different volumes. Hence the
scanner does not handle this situation predictable.
With memory storage, it is common to save blocks to persistent storage and
eviction of the original RAM disk block could fail since the block is in use.
In this situation, the directory scanner should never delete the block on
persistent storage.
> Directory scanner should correctly reconcile blocks on RAM disk
> ---------------------------------------------------------------
>
> Key: HDFS-6978
> URL: https://issues.apache.org/jira/browse/HDFS-6978
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: HDFS-6581
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
>
> It used to be very unlikely that the directory scanner encountered two
> replicas of the same block on different volumes.
> With memory storage, it is very likely to hit this with the following
> sequence of events:
> # Block is written to RAM disk
> # Lazy writer saves a copy on persistent volume
> # DN attempts to evict the original replica from RAM disk, file deletion
> fails as the replica is in use.
> # Directory scanner finds a replica on both RAM disk and persistent storage.
> The directory scanner should never delete the block on persistent storage.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
