[
https://issues.apache.org/jira/browse/HDFS-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16313599#comment-16313599
]
Xiao Chen commented on HDFS-12990:
----------------------------------
Thanks [~arpitagarwal] and [~anu] for the comments.
We definitely value HADOOP-14876 and all compat policies, so theoretically we
do NOT want incompatible changes in 3.0.x.
This one is a tough call, so I'm raising this jira as a special case and
leaving it here for discussions. We all agree the cons are incompatibility, and
possible 3.0.0 -> 3.0.1 confusion / complications.
The advantage, though, is that people upgrading from 2.x to 3.0.1 will not have
to do the most intrusive switch of the NN RPC port. As shown in the
description, the NN RPC port requires huge downstream efforts. This isn't
something that each downstream can do in a jira, to switch their default ports.
This includes the actual metadata location in hive tables, oozie workflows, and
more of the scripts than I could imagine.
I'm not saying this is the greatest idea, but since we have _just_ released
3.0.0, and 3.0.1 isn't far away, we may have time to correct this so downstream
adoption would be a lot easier. I know the metaphor for Python 2.x \-> 3.0 is
abused, but hopefully this one will make Hadoop 2.x\->3.0 less like that. :)
Until this point, there are probably very few people / cluster have upgraded to
3.0.0., so it's likely that this will not negatively impact anyone. But if we
do not make this change, eventually all existing deployments will be affected.
Let me know what you think. If we agree on the idea, we may make an official
notice later to encourage upgrading to 3.0.1 instead of 3.0.0.
> Change default NameNode RPC port back to 8020
> ---------------------------------------------
>
> Key: HDFS-12990
> URL: https://issues.apache.org/jira/browse/HDFS-12990
> Project: Hadoop HDFS
> Issue Type: Task
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Xiao Chen
> Assignee: Xiao Chen
> Priority: Critical
> Attachments: HDFS-12990.01.patch
>
>
> In HDFS-9427 (HDFS should not default to ephemeral ports), we changed all
> default ports to ephemeral ports, which is very appreciated by admin. As part
> of that change, we also modified the NN RPC port from the famous 8020 to
> 9820, to be closer to other ports changed there.
> With more integration going on, it appears that all the other ephemeral port
> changes are fine, but the NN RPC port change is painful for downstream on
> migrating to Hadoop 3. Some examples include:
> # Hive table locations pointing to hdfs://nn:port/dir
> # Downstream minicluster unit tests that assumed 8020
> # Oozie workflows / downstream scripts that used 8020
> This isn't a problem for HA URLs, since that does not include the port
> number. But considering the downstream impact, instead of requiring all of
> them change their stuff, it would be a way better experience to leave the NN
> port unchanged. This will benefit Hadoop 3 adoption and ease unnecessary
> upgrade burdens.
> It is of course incompatible, but giving 3.0.0 is just out, IMO it worths to
> switch the port back.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
