[
https://issues.apache.org/jira/browse/HDFS-12387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16157609#comment-16157609
]
Anu Engineer commented on HDFS-12387:
-------------------------------------
[~nandakumar131] Thanks for your valuable comments.
bq. Case 1) If another client does an allocateBlock call in between step 1 and
2, we will return AllocatedBlock with create container flag set to true. This
will cause two clients to trigger create container call to same pipeline and
one will result in CONTAINER_EXISTS exception.
Completely agree, we will need to add the code in the client that says if we
get this exception we should continue or another option that we have considered
is to add a lease manager which keeps a lease on the container that is being
given out to a client for creation. But I completely agree with your point this
needs to be fixed and it is skipped in this patch for time being. Thanks for
flagging it.
bq. Case 2) If container creation call (step 3) fails with an exception, we
will never revert the container state from CREATING to ALLOCATED.
You are right again, the idea -- again missing code -- this patch is already
153 KB, is that if we get a timeout, we will move to the state of delete this
container. We do need to define the time out and the container delete path.We
will rely on the delete path that [~cheersyang] is building to do that. Hence
it is missing in this patch. Just to make sure we are on the same page, once a
container is given out to a client for CREATING state, the state machine will
wait for some time, say 30 mins, if the creation does happen in that duration
then that container will get deleted both from SCM and Datandoes. The code for
the time keeper will part of the lease manager that we discussed above.
So case 2, case 3, all are handled by this catch all meachnism. When we have
the lease manager ( I might just update the patch, since you have already
finished reading it once, adding a bit more code would not hurt) then the time
based state can be handled. So if we time out, we will delete the container
both from SCM and datanodes.
bq. No check is done before setting the pipeline replication type as
ReplicationType.STAND_ALONE
Thanks for catching that, i will fix it in the next patch.
> Ozone: Support Ratis as a first class replication mechanism
> -----------------------------------------------------------
>
> Key: HDFS-12387
> URL: https://issues.apache.org/jira/browse/HDFS-12387
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Affects Versions: HDFS-7240
> Reporter: Anu Engineer
> Assignee: Anu Engineer
> Priority: Critical
> Labels: ozoneMerge
> Attachments: HDFS-12387-HDFS-7240.001.patch
>
>
> Ozone container layer supports pluggable replication policies. This JIRA
> brings Apache Ratis based replication to Ozone. Apache Ratis is a java
> implementation of Raft protocol.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
