[
https://issues.apache.org/jira/browse/HDFS-8582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lei (Eddy) Xu updated HDFS-8582:
--------------------------------
Attachment: HDFS-8582.003.patch
Thanks a lot for the reviews, [~cmccabe]. I made the following changes based on
your comments:
* {{listAllowed}} to {{properties}} subcommand.
* {{GetReconfigurationAllowedPropertiesResponse}} to
{{ListReconfigurablePropertiesResponse}}. The reason that I did not change it
to {{GetReconfigurablePropertiesResponse}} is that {{ReconfiguableBase}} has a
{{#getReconfigurableProperties()}} function, thus I could not use the same name
in {{ClientProtocol}}. Naming the protobuf to {{List...}} makes it consistent
with {{ClientDatanodeProtocol}}.
bq. So it should use the correct defaults for the datanode.
Currently, the following properties that are different
{noformat}
rpc.engine.org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolPB
rpc.engine.org.apache.hadoop.hdfs.protocolPB.InterDatanodeProtocolPB
rpc.engine.org.apache.hadoop.ipc.ProtocolMetaInfoPB
rpc.engine.org.apache.hadoop.hdfs.protocolPB.ClientDatanodeProtocolPB
rpc.engine.org.apache.hadoop.tracing.TraceAdminProtocolPB
mapreduce.client.genericoptionsparser.used
dfs.datanode.startup
{noformat}
{{rpc.engine.*}} are set in {{DataNode#initIpcServer}}, {{dfs.datanode.setup}}
is set in {{DataNode#parseArguments()}} which is in {{main()}}, and
{{mapreduce.client.genericoptionsparser.used}} is set in
{{GenericOptionsParser}}. It might not clean to simply set default values of
these properties in {{DataNode#getNewConf()}}, because the further code changes
might break the consistency between them. So for now, I let {{ReconfigureBase}}
ignore non-reconfigurable properties and put them in log. User should use
{{-reconfig properties}} to check whether a property is supported first.
[~cmccabe] What would you think?
> Reduce failure messages when running datanode reconfiguration
> -------------------------------------------------------------
>
> Key: HDFS-8582
> URL: https://issues.apache.org/jira/browse/HDFS-8582
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: HDFS
> Affects Versions: 2.7.0
> Reporter: Lei (Eddy) Xu
> Assignee: Lei (Eddy) Xu
> Priority: Minor
> Attachments: HDFS-8582.000.patch, HDFS-8582.001.patch,
> HDFS-8582.002.patch, HDFS-8582.003.patch
>
>
> When running a DN reconfig to hotswap some drives, it spits out this output:
> {noformat}
> $ hdfs dfsadmin -reconfig datanode localhost:9023 status
> 15/06/09 14:58:10 WARN util.NativeCodeLoader: Unable to load native-hadoop
> library for your platform... using builtin-java classes where applicable
> Reconfiguring status for DataNode[localhost:9023]: started at Tue Jun 09
> 14:57:37 PDT 2015 and finished at Tue Jun 09 14:57:56 PDT 2015.
> FAILED: Change property
> rpc.engine.org.apache.hadoop.hdfs.protocolPB.ClientDatanodeProtocolPB
> From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
> To: ""
> Error: Property
> rpc.engine.org.apache.hadoop.hdfs.protocolPB.ClientDatanodeProtocolPB is not
> reconfigurable.
> FAILED: Change property mapreduce.client.genericoptionsparser.used
> From: "true"
> To: ""
> Error: Property mapreduce.client.genericoptionsparser.used is not
> reconfigurable.
> FAILED: Change property rpc.engine.org.apache.hadoop.ipc.ProtocolMetaInfoPB
> From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
> To: ""
> Error: Property rpc.engine.org.apache.hadoop.ipc.ProtocolMetaInfoPB
> is not reconfigurable.
> SUCCESS: Change property dfs.datanode.data.dir
> From: "file:///data/1/user/dfs"
> To: "file:///data/1/user/dfs,file:///data/2/user/dfs"
> FAILED: Change property dfs.datanode.startup
> From: "REGULAR"
> To: ""
> Error: Property dfs.datanode.startup is not reconfigurable.
> FAILED: Change property
> rpc.engine.org.apache.hadoop.hdfs.protocolPB.InterDatanodeProtocolPB
> From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
> To: ""
> Error: Property
> rpc.engine.org.apache.hadoop.hdfs.protocolPB.InterDatanodeProtocolPB is not
> reconfigurable.
> FAILED: Change property
> rpc.engine.org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolPB
> From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
> To: ""
> Error: Property
> rpc.engine.org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolPB is not
> reconfigurable.
> FAILED: Change property
> rpc.engine.org.apache.hadoop.tracing.TraceAdminProtocolPB
> From: "org.apache.hadoop.ipc.ProtobufRpcEngine"
> To: ""
> Error: Property
> rpc.engine.org.apache.hadoop.tracing.TraceAdminProtocolPB is not
> reconfigurable.
> {noformat}
> These failed messages are spurious and should not be shown.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
