[
https://issues.apache.org/jira/browse/HADOOP-1762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12526648
]
shv edited comment on HADOOP-1762 at 9/11/07 6:15 PM:
----------------------------------------------------------------------
Mostly cosmetic comments:
# Datanode.setNewStorageID():
{code} } catch (UnknownHostException ignored) {} {code}
Should not be ignored but rather logged.
# We should deprecate FSEdits.OP_DATANODE_ADD and FSEdits.OP_DATANODE_REMOVE
so that we could eventually remove them sometimes.
# FSNamesystem.registerDatanode():
{code}
NameNode.stateChangeLog.info( "BLOCK* NameSystem.registerDatanode: "
+ "node " + nodeS.getName()
+ " is replaced by " + nodeReg.getName() +
" since they have the same storageID " +
nodeReg.getStorageID() +
". If this message repeats, both nodes " +
"might have same storageID by random " +
"chance. You need to restart one of the "
+
"nodes with its data cleared. ");
{code}
This message may cause confusion in the regular case.
The possibility of storage id collision is so negligible that we should not
hint on it every time one node *_legally_* replaces another with the same
storage id.
Let's retain the original redaction.
I also reviewed the first variant of the patch. They are very similar. The main
difference is who generates the new storage id
for the new data-node. And I prefer the case when data-nodes are able to
independently generate unique storage ids for themselves.
The ids here are unique since:
- data-nodes started on different nodes have different ips;
- data-nodes started on the same node (but in different storage directories) at
the same time will have different ports;
- data-nodes started on the same node on the same port but at different times
will have different timestamps;
- in the latter case even somebody turns time back before starting the second
node the random number is likely to prevent the collision.
B.t.w. something like that should be in the comments for
Datanode.setNewStorageID().
was (Author: shv):
Mostly cosmetic comments:
# Datanode.setNewStorageID():
{code} } catch (UnknownHostException ignored) {} {code}
Should not be ignored but rather logged.
# We should deprecate FSEdits.OP_DATANODE_ADD and FSEdits.OP_DATANODE_REMOVE
so that we could eventually remove them sometimes.
# FSNamesystem.registerDatanode():
{code}
NameNode.stateChangeLog.info( "BLOCK* NameSystem.registerDatanode: "
+ "node " + nodeS.getName()
+ " is replaced by " + nodeReg.getName() +
" since they have the same storageID " +
nodeReg.getStorageID() +
". If this message repeats, both nodes " +
"might have same storageID by random " +
"chance. You need to restart one of the "
+
"nodes with its data cleared. ");
{code}
This message may cause confusion in the regular case.
The possibility of storage id collision is so negligible that we should not
hint on it every time one node *_legally_* replaces another with the same
storage id.
Let's retain the original redaction.
> Namenode does not need to store storageID and datanodeID persistently
> ---------------------------------------------------------------------
>
> Key: HADOOP-1762
> URL: https://issues.apache.org/jira/browse/HADOOP-1762
> Project: Hadoop
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.14.0
> Reporter: Raghu Angadi
> Assignee: Raghu Angadi
> Attachments: HADOOP-1762-Opt2.patch, HADOOP-1762-Opt2.patch,
> HADOOP-1762-Opt2.patch, HADOOP-1762-Opt2.patch, HADOOP-1762.patch
>
>
> Currently Namenode stores all the storage-ids it generates since the
> beginning (since last format). It allocates a new storageID everytime a new
> datanode comes online. It also stores all the known datanode ids since the
> beginning.
> It would be better if Namenode did not have to keep track of these. I will
> describe a proposal in the next comment.
> This has implecations regd how Namenode helps administrators identify 'dead
> datanodes' etc. These issues are addressed in HADOOP-1138.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
