[
https://issues.apache.org/jira/browse/HDFS-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13133419#comment-13133419
]
Steve Loughran commented on HDFS-2492:
--------------------------------------
The short term workaround is to document that regardless of how topology is
determined, to make HDFS care about whether or not blocks are all on the same
rack, you need to set this filename key.
The medium term workaround is to set the {{shouldCheckForEnoughRacks}} flag if
the key is set or the switch mapping implementation class is anything other
than an instance of {{ScriptBasedMapping}}. That is somewhat inelegant.
Ideally the {{DNSToSwitchMapping}} interface should be extended so that it can
declare whether there are multiple racks, and if there is more than one rack
then the block mapper should go rack aware. Having knowledge of the expected
number of racks could be of use for making other parts of the system multi-rack
for better availability.
> BlockManager cross-rack replication checks only work for ScriptBasedMapping
> ---------------------------------------------------------------------------
>
> Key: HDFS-2492
> URL: https://issues.apache.org/jira/browse/HDFS-2492
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Steve Loughran
>
> The BlockManager cross-rack replication checks only works if script files are
> used for replication, not if alternate plugins provide the topology
> information.
> This is because the BlockManager sets its rack checking flag if there is a
> filename key
> {code}
> shouldCheckForEnoughRacks =
> conf.get(DFSConfigKeys.NET_TOPOLOGY_SCRIPT_FILE_NAME_KEY) != null;
> {code}
> yet this filename key is only used if the topology mapper defined by
> {code}
> DFSConfigKeys.NET_TOPOLOGY_NODE_SWITCH_MAPPING_IMPL_KEY
> {code}
> is an instance of {{ScriptBasedMapping}}
> If any other mapper is used, the system may be multi rack, but the Block
> Manager will not be aware of this fact unless the filename key is set to
> something non-null
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
