[
https://issues.apache.org/jira/browse/HDFS-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14550876#comment-14550876
]
Chris Nauroth commented on HDFS-5223:
-------------------------------------
I have deleted the design document and patch and moved it over to HDFS-8432, so
that this patch can remain focused on discussion of the proposal for feature
flags.
> Allow edit log/fsimage format changes without changing layout version
> ---------------------------------------------------------------------
>
> Key: HDFS-5223
> URL: https://issues.apache.org/jira/browse/HDFS-5223
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 2.1.1-beta
> Reporter: Aaron T. Myers
> Assignee: Colin Patrick McCabe
> Attachments: HDFS-5223.004.patch
>
>
> Currently all HDFS on-disk formats are version by the single layout version.
> This means that even for changes which might be backward compatible, like the
> addition of a new edit log op code, we must go through the full `namenode
> -upgrade' process which requires coordination with DNs, etc. HDFS should
> support a lighter weight alternative.
> Copied description from HDFS-8075 which is a duplicate and now closed. (by
> sanjay on APril 7 2015)
> Background
> * HDFS image layout was changed to use Protobufs to allow easier forward and
> backward compatibility.
> * Hdfs has a layout version which is changed on each change (even if it an
> optional protobuf field was added).
> * Hadoop supports two ways of going back during an upgrade:
> ** downgrade: go back to old binary version but use existing image/edits so
> that newly created files are not lost
> ** rollback: go back to "checkpoint" created before upgrade was started -
> hence newly created files are lost.
> Layout needs to be revisited if we want to support downgrade is some
> circumstances which we dont today. Here are use cases:
> * Some changes can support downgrade even though they was a change in layout
> since there is not real data loss but only loss of new functionality. E.g.
> when we added ACLs one could have downgraded - there is no data loss but you
> will lose the newly created ACLs. That is acceptable for a user since one
> does not expect to retain the newly added ACLs in an old version.
> * Some changes may lead to data-loss if the functionality was used. For
> example, the recent truncate will cause data loss if the functionality was
> actually used. Now one can tell admins NOT use such new such new features
> till the upgrade is finalized in which case one could potentially support
> downgrade.
> * A fairly fundamental change to layout where a downgrade is not possible but
> a rollback is. Say we change the layout completely from protobuf to something
> else. Another example is when HDFS moves to support partial namespace in
> memory - they is likely to be a fairly fundamental change in layout.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
