[
https://issues.apache.org/jira/browse/HDFS-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16030899#comment-16030899
]
Weiwei Yang commented on HDFS-11886:
------------------------------------
Hi [~anu]
Thank you for the quick response. Keep uncommitted keys in memory is less
reliable but faster and easier. If you are going to store state in DB, then we
need to deal with rollback problems and more error handling code (update that
DB has chance to fail too). And we need to think how to recover state from that
when KSM restarted, is it necessary or not to recover uncommitted keys in this
DB? It will be hard because we don't know which stage this key has been
processed to.
> Ozone : improve error handling for putkey operation
> ---------------------------------------------------
>
> Key: HDFS-11886
> URL: https://issues.apache.org/jira/browse/HDFS-11886
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Chen Liang
> Attachments: design-notes-putkey.pdf
>
>
> Ozone's putKey operations involve a couple steps:
> 1. KSM calls allocateBlock to SCM, writes this info to KSM's local metastore
> 2. allocatedBlock gets returned to client, client checks to see if container
> needs to be created on datanode, if yes, create the container
> 3. writes the data to container.
> it is possible that 1 succeeded, but 2 or 3 failed, in this case there will
> be an entry in KSM's local metastore, but the key is actually nowhere to be
> found. We need to revert 1 is 2 or 3 failed in this case.
> To resolve this, we need at least two things to be implemented first.
> 1. We need deleteKey() to be added KSM first.
> 2. We also need container reports to be implemented first such that SCM can
> track whether the container is actually added.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
