Sanjay Radia created HDFS-8075:
----------------------------------
Summary: Revist layout version
Key: HDFS-8075
URL: https://issues.apache.org/jira/browse/HDFS-8075
Project: Hadoop HDFS
Issue Type: Bug
Components: HDFS
Affects Versions: 2.6.0
Reporter: Sanjay Radia
Background
* HDFS image layout was changed to use Protobufs to allow easier forward and
backward compatibility.
* Hdfs has a layout version which is changed on each change (even if it an
optional protobuf field was added).
* Hadoop supports two ways of going back during an upgrade:
** downgrade: go back to old binary version but use existing image/edits so
that newly created files are not lost
** rollback: go back to "checkpoint" created before upgrade was started - hence
newly created files are lost.
Layout needs to be revisited if we want to support downgrade is some
circumstances which we dont today. Here are use cases:
* Some changes can support downgrade even though they was a change in layout
since there is not real data loss but only loss of new functionality. E.g. when
we added ACLs one could have downgraded - there is no data loss but you will
lose the newly created ACLs. That is acceptable for a user since one does not
expect to retain the newly added ACLs in an old version.
* Some changes may lead to data-loss if the functionality was used. For
example, the recent truncate will cause data loss if the functionality was
actually used. Now one can tell admins NOT use such new such new features till
the upgrade is finalized in which case one could potentially support downgrade.
* A fairly fundamental change to layout where a downgrade is not possible but a
rollback is. Say we change the layout completely from protobuf to something
else. Another example is when HDFS moves to support partial namespace in memory
- they is likely to be a fairly fundamental change in layout.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
