[
https://issues.apache.org/jira/browse/HDFS-2983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13248595#comment-13248595
]
Todd Lipcon commented on HDFS-2983:
-----------------------------------
Here's another proposal which I think makes sense:
1) Ensure that version compatibility is checked both on the NN side and the DN
side. So, when the DN first connects to the NN, the NN verifies the DN's
version. If it is deemed incompatible, it is rejected. Then, then DN verifies
the NN version in the response. If it is deemed incompatible, it does not
proceed with registration.
2) Add a function to compare two version numbers in the straightforward manner:
split the numbers on ".", then componentwise, do comparisons according to
"string numerical value" (like sort -n). Some examples: 2.0.1 > 2.0.0. 10.0 >
2.0.0. 2.0.0a > 2.0.0. 2.0.0b > 2.0.0.a. (this is the comparison mechanism
package managers tend to use)
3) In hdfs-default.xml, add a configuration like
{{cluster.min.supported.version}}. In branch-2, we set this to "2.0.0". So, by
default, any 2.x.x can talk to any other 2.x.x. When we release 3.x.x, if it is
incompatible with 2.x.x, then we just need to bump that config in 3.0's
hdfs-default.xml.
This supports the following use cases/requirements:
- rolling upgrade can be done for most users without having to change any
configs.
- new versions of Hadoop can be marked incompatible with old versions of Hadoop
- cluster admins can still override it if they want to disallow older nodes
from connecting. For example, imagine there is a critical security bug fixed in
2.0.0a - the admin can set the config to 2.0.0.a, and then 2.0.0 nodes may no
longer join the cluster (even though they are protocol-wise compatible)
Thoughts?
> Relax the build version check to permit rolling upgrades within a release
> -------------------------------------------------------------------------
>
> Key: HDFS-2983
> URL: https://issues.apache.org/jira/browse/HDFS-2983
> Project: Hadoop HDFS
> Issue Type: Improvement
> Affects Versions: 2.0.0
> Reporter: Eli Collins
> Assignee: Aaron T. Myers
> Attachments: HDFS-2983.patch
>
>
> Currently the version check for DN/NN communication is strict (it checks the
> exact svn revision or git hash, Storage#getBuildVersion calls
> VersionInfo#getRevision), which prevents rolling upgrades across any
> releases. Once we have the PB-base RPC in place (coming soon to branch-23)
> we'll have the necessary pieces in place to loosen this restriction, though
> perhaps it takes another 23 minor release or so before we're ready to commit
> to making the minor versions compatible.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
