[
https://issues.apache.org/jira/browse/HDFS-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16325225#comment-16325225
]
Eric Yang commented on HDFS-12990:
----------------------------------
Maybe we should look at NN RPC port as a new feature to ensure the down stream
projects really tested with Hadoop 3 instead of gambling on compatibility. In
Hadoop bylaws, it doesn't mention compatibility between major versions, only
minor version within the same major version. This bylaw is challenged by this
JIRA because major incompatible change have been made. This is greatest
challenge to Hadoop policy. If this policy doesn't hold up when challenged,
then I would suggest to revise the policy first. Most complains are knee jerk
reaction toward uncertainty toward certifying a new product version. I don't
think this issue is worthy of revising Hadoop policy. We should not try to
rationalize this is a bug to bend the policy, otherwise, it would be
disrespectful to Hadoop policy, and encourage others to break the rules.
We do not know if down stream projects depend on other ports that were changed.
Therefore, I would recommend for downstream projects which has hard coded 8020
ports to make new releases that have properly certified with Hadoop 3. I found
other projects like Chukwa and Ambari are both immune to NN rpc port change
problem. Code with good design are most likely immune to this problem. If we
committed this change, future major release might need to break compatibility
again to shake away bad design flaws. We will be unable to do that for sake of
compatibility. While I admire Microsoft's ability to enable upgrade from
Windows 1.0 to Windows 10, but Windows still don't preserve all my theme and
coloring between Window versions, or make my old scanner work. Can Hadoop make
the same kind of commitment without making huge investment? Having seen
multi-generation of programmers retired in IBM. I know that forever
compatibility is not sustainable and it only cost more in the long run. We
should move on and do something better for Hadoop.
Please respect my -1 vote because I believe in the wisdom of current version
scheme, and port change ensures the downstream projects really certified with
Hadoop 3.
> Change default NameNode RPC port back to 8020
> ---------------------------------------------
>
> Key: HDFS-12990
> URL: https://issues.apache.org/jira/browse/HDFS-12990
> Project: Hadoop HDFS
> Issue Type: Task
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Xiao Chen
> Assignee: Xiao Chen
> Priority: Critical
> Attachments: HDFS-12990.01.patch
>
>
> In HDFS-9427 (HDFS should not default to ephemeral ports), we changed all
> default ports to ephemeral ports, which is very appreciated by admin. As part
> of that change, we also modified the NN RPC port from the famous 8020 to
> 9820, to be closer to other ports changed there.
> With more integration going on, it appears that all the other ephemeral port
> changes are fine, but the NN RPC port change is painful for downstream on
> migrating to Hadoop 3. Some examples include:
> # Hive table locations pointing to hdfs://nn:port/dir
> # Downstream minicluster unit tests that assumed 8020
> # Oozie workflows / downstream scripts that used 8020
> This isn't a problem for HA URLs, since that does not include the port
> number. But considering the downstream impact, instead of requiring all of
> them change their stuff, it would be a way better experience to leave the NN
> port unchanged. This will benefit Hadoop 3 adoption and ease unnecessary
> upgrade burdens.
> It is of course incompatible, but giving 3.0.0 is just out, IMO it worths to
> switch the port back.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
