[
https://issues.apache.org/jira/browse/HDFS-5535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13912212#comment-13912212
]
Colin Patrick McCabe commented on HDFS-5535:
--------------------------------------------
I took a look at the design doc and some of the subtasks. It's clever to use
NameNode HA to provide a zero-downtime upgrade. Splitting the NN and DN layout
versions is sensible and will avoid headaches in the future. I always thought
the DN registration version check was a hack. It's good to see it go away,
replaced by a simple check of the LayoutVersion and protocol version.
HDFS-5496 is also a good idea.
It is too bad that the newly introduced "downgrade" functionality is only
available between dot releases, but I think we can live with that (just like
we're living with it now). It's not a regression. I agree that downgrade
between dot releases should be fairly straightforward.
HDFS-5498, caching the DU result, seems reasonable. Similarly, parallelizing
the block scanner seems like an obvious improvement.
One area where I see some complications is in the out-of-band notification sent
to DFSClient instances when a Datanode is about to go down. This is certainly
something that HBase (among others) could use, but it seems like a big new
change to a rarely used codepath. In the interest of getting this into 2.4,
might there be some benefit to splitting out this part? We still have a lot of
unaddressed issues on the DN write pipeline like HDFS-4504, and I'm nervous
about adding too many new features until those bugs are addressed. I think
some small changes like HDFS-6016 would be sufficient to dramatically improve
DN rolling upgrade.
> Umbrella jira for improved HDFS rolling upgrades
> ------------------------------------------------
>
> Key: HDFS-5535
> URL: https://issues.apache.org/jira/browse/HDFS-5535
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: datanode, ha, hdfs-client, namenode
> Affects Versions: 3.0.0, 2.2.0
> Reporter: Nathan Roberts
> Attachments: HDFSRollingUpgradesHighLevelDesign.pdf,
> h5535_20140219.patch, h5535_20140220-1554.patch, h5535_20140220b.patch,
> h5535_20140221-2031.patch, h5535_20140224-1931.patch,
> h5535_20140225-1225.patch
>
>
> In order to roll a new HDFS release through a large cluster quickly and
> safely, a few enhancements are needed in HDFS. An initial High level design
> document will be attached to this jira, and sub-jiras will itemize the
> individual tasks.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
