[
https://issues.apache.org/jira/browse/HDFS-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14113038#comment-14113038
]
James Thomas commented on HDFS-6800:
------------------------------------
[~arpitagarwal], realized what I wrote above is completely wrong. There is no
{{-upgrade}} option for the DNs -- they automatically check whether the
software layout version is higher than the stored layout version and upgrade if
it is (see the end of {{BlockPoolSliceStorage#doTransition}}). So I think we're
fine on that front.
The deletion of the previous directory is actually quite a hairy issue -- the
NN currently responds to block reports with an upgrade finalization status
based on the isUpgradeFinalized flag in FSImage, and since rolling upgrades do
not modify this flag (leaving it set to true), the NN will actually instruct
DNs to remove their previous dirs before {{hdfs dfsadmin -rollingUpgrade
finalize}} is executed.
I think that we can have rolling upgrades set this flag to false when they are
started and set it again to true when {{hdfs dfsadmin -rollingUpgrade
finalize}} is executed, and we should get the behaviour we want.
Thoughts?
> Determine how Datanode layout changes should interact with rolling upgrade
> --------------------------------------------------------------------------
>
> Key: HDFS-6800
> URL: https://issues.apache.org/jira/browse/HDFS-6800
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.6.0
> Reporter: Colin Patrick McCabe
> Assignee: James Thomas
> Attachments: HDFS-6800.2.patch, HDFS-6800.3.patch, HDFS-6800.4.patch,
> HDFS-6800.5.patch, HDFS-6800.patch
>
>
> We need to handle attempts to rolling-upgrade the DataNode to a new storage
> directory layout.
> One approach is to disallow such upgrades. If we choose this approach, we
> should make sure that the system administrator gets a helpful error message
> and a clean failure when trying to use rolling upgrade to a version that
> doesn't support it. Based on the compatibility guarantees described in
> HDFS-5535, this would mean that *any* future DataNode layout changes would
> require a major version upgrade.
> Another approach would be to support rolling upgrade from an old DN storage
> layout to a new layout. This approach requires us to change our
> documentation to explain to users that they should supply the {{\-rollback}}
> command on the command-line when re-starting the DataNodes during rolling
> rollback. Currently the documentation just says to restart the DataNode
> normally.
> Another issue here is that the DataNode's usage message describes rollback
> options that no longer exist. The help text says that the DN supports
> {{\-rollingupgrade rollback}}, but this option was removed by HDFS-6005.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
