[
https://issues.apache.org/jira/browse/HDFS-11096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15816405#comment-15816405
]
Sean Mackrory edited comment on HDFS-11096 at 1/10/17 10:46 PM:
----------------------------------------------------------------
After looking at where SortedMapWritable is used some more, I'm more convinced
it's only a concern if we care about source compatibility, which is not
required for rolling upgrades. I also had a second look at wire compatibility
and I found a few concerning things I'll look at and possibly fix:
* The message getHdfsBlockLocations has disappeared as well as related types
* The field nonDfsUsed in DatanodeInfoProto changed from index 9 to index 15
For YARN (CC [~kasha]), the field nodeLabels in several structures in
yarn_protos changed from string to a custom type, and memory in ResourceProto
change from int32 to int64 (not sure if it's the case in protobuf, but that may
not be incompatible?)
There's also a lot of messages moving between files, but not otherwise changing
in any incompatible way. That's not a concern is it?
If anyone else wants to see the changes in protobuf, this is what I did (if
anything, you'll want to replace meld with your own diff-tool-of-choice):
{code}
#!/usr/bin/env bash
cd /tmp
OLD=branch-2.7
NEW=trunk
mkdir new
mkdir old
git clone git://git.apache.org/hadoop.git
function gather_protos() {
SOURCE=${1}
TARGET=${2}
for proto in $(cd ${SOURCE} && find . -name \*.proto | sed -e 's|^\./||'); do
#flattened=${proto//\//_} # Trips up on files that moved
flattened=$(basename ${proto})
cp ${SOURCE}/${proto} ${TARGET}/${flattened}
done
}
(cd hadoop; git checkout ${OLD})
gather_protos hadoop old
(cd hadoop; git checkout ${NEW})
gather_protos hadoop new
meld old new
{code}
was (Author: mackrorysd):
After looking at where SortedMapWritable is used some more, I'm more convinced
it's only a concern if we care about source compatibility, which is not
required for rolling upgrades. I also had a second look at wire compatibility
and I found a few concerning things I'll look at and possibly fix:
* The message getHdfsBlockLocations has disappeared as well as related types
* The field nonDfsUsed in DatanodeInfoProto changed from index 9 to index 15
For YARN (CC [~kasha]), the field nodeLabels in several structures in
yarn_protos changed from string to a custom type, and memory in ResourceProto
change from int32 to int64 (not sure if it's the case in protobuf, but that may
not be incompatible?)
There's also a lot of messages moving between files, but not otherwise changing
in any incompatible way. That's not a concern is it?
If anyone else wants to see the changes in protobuf, this is what I did (if
anything, you'll want to replace meld with your own diff-tool-of-choice):
{quote}
#!/usr/bin/env bash
cd /tmp
OLD=branch-2.7
NEW=trunk
mkdir new
mkdir old
git clone git://git.apache.org/hadoop.git
function gather_protos() {
SOURCE=${1}
TARGET=${2}
for proto in $(cd ${SOURCE} && find . -name \*.proto | sed -e 's|^\./||'); do
#flattened=${proto//\//_} # Trips up on files that moved
flattened=$(basename ${proto})
cp ${SOURCE}/${proto} ${TARGET}/${flattened}
done
}
(cd hadoop; git checkout ${OLD})
gather_protos hadoop old
(cd hadoop; git checkout ${NEW})
gather_protos hadoop new
meld old new
{quote}
> Support rolling upgrade between 2.x and 3.x
> -------------------------------------------
>
> Key: HDFS-11096
> URL: https://issues.apache.org/jira/browse/HDFS-11096
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: rolling upgrades
> Affects Versions: 3.0.0-alpha1
> Reporter: Andrew Wang
> Priority: Blocker
>
> trunk has a minimum software version of 3.0.0-alpha1. This means we can't
> rolling upgrade between branch-2 and trunk.
> This is a showstopper for large deployments. Unless there are very compelling
> reasons to break compatibility, let's restore the ability to rolling upgrade
> to 3.x releases.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
