[
https://issues.apache.org/jira/browse/HDFS-11154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15684862#comment-15684862
]
Anu Engineer edited comment on HDFS-11154 at 11/21/16 10:05 PM:
----------------------------------------------------------------
bq. The operations calling into StorageManager are already syncd on
StorageManager's own lock. The next patch will include locks for those
operations on the levelDB instance.
Perhaps I am mistaken, but let me walk you through an extreme scenario. This
is more of an exercise to make sure that my understanding is correct.
Client Thread 1: Calls into DeleteVolume , executes storageManager.deleteVolume
; thread gets suspended.
Client Thread 2: Calls into createVolume, with the same username and volume
name -- Since thread1 has executed storageManager.deleteVolume,
storageManager.createVolume is successful. Thread 2 goes to execute
writeToPresistentStore, we return success to the caller.
Thread 1 wakes up, continues the deleteCall by removeFromPersistentStore, but
in this case the Database Key is from createVolume call from the Thread2 and
not from the createVolume call that you had when you called deleteVolume.
So the client gets two success calls, but in some strange cases the volume will
not exist. In other words, there seems to be a race condition.
bq. Probably overriding equals is making more sense but it requires
implementing hashcode also. Since this is only a simple method for testing, I
decided not to go with a potentially confusing way. What do you think?
Probably we should do equals .. Hashcode is generally not too much code.
bq. Simply because I assumed the use of levelDB is only temporary and at the
end of the day we will switch to RAFT (which I suppose is the same for Ozone).
Even with RAFT, we will still need a proper store that backs the data that we
read from the log. Feel free to move files around as it suits you.
was (Author: anu):
bq. The operations calling into StorageManager are already syncd on
StorageManager's own lock. The next patch will include locks for those
operations on the levelDB instance.
Perhaps I am mistaken, but let me walk you through an extreme scenario. This
is more of an exercise to make sure that my understanding is correct.
Client Thread 1: Calls into DeleteVolume , executes storageManager.deleteVolume
; thread gets suspended.
Client Thread 2: Calls into createVolume, with the same username and volume
name -- Since thread1 has executed storageManager.deleteVolume,
storageManager.createVolume is successful. Thread 2 goes to execute
writeToPresistentStore, we return success to the caller.
Thread 1 wakes up, continues the deleteCall by removeFromPersistentStore, but
in this case the Database Key is from createVolume call from the Thread2 and
not from the createVolume call that you had when you called deleteVolume.
So the client gets two success calls, but in some strange cases the volume will
not exist. In other words, there seems to be a race condition.
bq. Probably overriding equals is making more sense but it requires
implementing hashcode also. Since this is only a simple method for testing, I
decided not to go with a potentially confusing way. What do you think?
Probably we should do equals .. Hashcode is generally not too much code.
> Block Storage : store server state to persistent storage
> --------------------------------------------------------
>
> Key: HDFS-11154
> URL: https://issues.apache.org/jira/browse/HDFS-11154
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs
> Reporter: Chen Liang
> Assignee: Chen Liang
> Attachments: HDFS-11154-HDFS-7240.001.patch,
> HDFS-11154-HDFS-7240.002.patch, HDFS-11154-HDFS-7240.003.patch
>
>
> Currently, all the storage state are kept in server memory. If server
> crashes, we would lose all the volume information. This JIRA stores server
> internal state into its local disk. Such that on server failure, we can
> simply restart server and restore volume information from disk.
> More specifically, the internal state written to disk is mainly the mapping
> from volume to its underlying containers, plus some meta information such as
> volume size, block size, etc.
> Note that this is only a simple, minimum set mechanism for persistence. It is
> more like a counterpart of fsimage in HDFS, but without edit logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
