[
https://issues.apache.org/jira/browse/HDFS-11886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Liang updated HDFS-11886:
------------------------------
Description:
Ozone's putKey operations involve a couple steps:
1. KSM calls allocateBlock to SCM, writes this info to KSM's local metastore
2. allocatedBlock gets returned to client, client checks to see if container
needs to be created on datanode, if yes, create the container
3. writes the data to container.
it is possible that 1 succeeded, but 2 or 3 failed, in this case there will be
an entry in KSM's local metastore, but the key is actually nowhere to be found.
We need to revert 1 is 2 or 3 failed in this case.
To resolve this, we need at least two things to be implemented first.
1. We need deleteKey() to be added KSM first.
2. We also need container reports to be implemented first such that SCM can
track whether the container is actually added.
was:
Ozone's putKey operations involve a couple steps:
1. KSM calls allocateBlock to SCM, writes this info to KSM's local metastore
2. allocatedBlock gets returned to client, client checks to see if container
needs to be created on datanode, if yes, create the container
3. writes the data to container.
it is possible that 1 succeeded, but 2 or 3 failed, in this case there will be
an entry in KSM's local metastore, but the key is actually nowhere to be found.
We need to revert 1 is 2 or 3 failed in this case. This can be done with a
deleteKey() call to KSM.
> Ozone : improving error handling for putkey operation
> -----------------------------------------------------
>
> Key: HDFS-11886
> URL: https://issues.apache.org/jira/browse/HDFS-11886
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ozone
> Reporter: Chen Liang
>
> Ozone's putKey operations involve a couple steps:
> 1. KSM calls allocateBlock to SCM, writes this info to KSM's local metastore
> 2. allocatedBlock gets returned to client, client checks to see if container
> needs to be created on datanode, if yes, create the container
> 3. writes the data to container.
> it is possible that 1 succeeded, but 2 or 3 failed, in this case there will
> be an entry in KSM's local metastore, but the key is actually nowhere to be
> found. We need to revert 1 is 2 or 3 failed in this case.
> To resolve this, we need at least two things to be implemented first.
> 1. We need deleteKey() to be added KSM first.
> 2. We also need container reports to be implemented first such that SCM can
> track whether the container is actually added.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
