[
https://issues.apache.org/jira/browse/HDFS-5667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Sirianni updated HDFS-5667:
--------------------------------
Description:
The fix for HDFS-5484 was accidentally regressed by the following change made
via HDFS-5542
{code}
+ DatanodeStorageInfo updateStorage(DatanodeStorage s) {
synchronized (storageMap) {
DatanodeStorageInfo storage = storageMap.get(s.getStorageID());
if (storage == null) {
@@ -670,8 +658,6 @@
" for DN " + getXferAddr());
storage = new DatanodeStorageInfo(this, s);
storageMap.put(s.getStorageID(), storage);
- } else {
- storage.setState(s.getState());
}
return storage;
}
{code}
By removing the 'else' and no longer updating the state in the BlockReport
processing path, we effectively get the bogus state & type that is set via the
first heartbeat (see the fix for HDFS-5455):
{code}
+ if (storage == null) {
+ // This is seen during cluster initialization when the heartbeat
+ // is received before the initial block reports from each storage.
+ storage = updateStorage(new DatanodeStorage(report.getStorageID()));
{code}
Even reverting the change and reintroducing the 'else' leaves the state & type
temporarily inaccurate until the first block report.
As discussed with [~arpitagarwal], a better fix would be to simply include the
full DatanodeStorage object in the StorageReport (as opposed to only the
Storage ID). This requires adding the {{DatanodeStorage}} object to
{{StorageReportProto}}. It needs to be a new optional field and we cannot
remove the existing {{StorageUuid}} for protocol compatibility.
was:
The fields in DatanodeStorageInfo are updated from two distinct paths:
# block reports
# storage reports (via heartbeats)
The {{state}} and {{storageType}} fields are updated via the Block Report.
However, as seen in the code blow, these fields are populated from a "dummy"
{{DatanodeStorage}} object constructed in the DataNode:
{code}
BPServiceActor.blockReport() {
//...
// Dummy DatanodeStorage object just for sending the block report.
DatanodeStorage dnStorage = new DatanodeStorage(storageID);
//...
}
{code}
The net effect is that the {{state}} and {{storageType}} fields are always the
default of {{NORMAL}} and {{DISK}} in the NameNode.
The recommended fix is to change {{FsDatasetSpi.getBlockReports()}} from:
{code}
public Map<String, BlockListAsLongs> getBlockReports(String bpid);
{code}
to:
{code}
public Map<DatanodeStorage, BlockListAsLongs> getBlockReports(String bpid);
{code}
thereby allowing {{BPServiceActor}} to send the "real" {{DatanodeStorage}}
object with the block report.
> StorageType and State in DatanodeStorageInfo in NameNode is not accurate
> ------------------------------------------------------------------------
>
> Key: HDFS-5667
> URL: https://issues.apache.org/jira/browse/HDFS-5667
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Affects Versions: Heterogeneous Storage (HDFS-2832)
> Reporter: Eric Sirianni
> Fix For: Heterogeneous Storage (HDFS-2832)
>
>
> The fix for HDFS-5484 was accidentally regressed by the following change made
> via HDFS-5542
> {code}
> + DatanodeStorageInfo updateStorage(DatanodeStorage s) {
> synchronized (storageMap) {
> DatanodeStorageInfo storage = storageMap.get(s.getStorageID());
> if (storage == null) {
> @@ -670,8 +658,6 @@
> " for DN " + getXferAddr());
> storage = new DatanodeStorageInfo(this, s);
> storageMap.put(s.getStorageID(), storage);
> - } else {
> - storage.setState(s.getState());
> }
> return storage;
> }
> {code}
> By removing the 'else' and no longer updating the state in the BlockReport
> processing path, we effectively get the bogus state & type that is set via
> the first heartbeat (see the fix for HDFS-5455):
> {code}
> + if (storage == null) {
> + // This is seen during cluster initialization when the heartbeat
> + // is received before the initial block reports from each storage.
> + storage = updateStorage(new DatanodeStorage(report.getStorageID()));
> {code}
> Even reverting the change and reintroducing the 'else' leaves the state &
> type temporarily inaccurate until the first block report.
> As discussed with [~arpitagarwal], a better fix would be to simply include
> the full DatanodeStorage object in the StorageReport (as opposed to only the
> Storage ID). This requires adding the {{DatanodeStorage}} object to
> {{StorageReportProto}}. It needs to be a new optional field and we cannot
> remove the existing {{StorageUuid}} for protocol compatibility.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)
