[
https://issues.apache.org/jira/browse/AMBARI-25438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
amarnath reddy pappu updated AMBARI-25438:
------------------------------------------
Description:
Steps to reproduce:
# Install Ambari 2.6.x and HDP 2.6.x
# enable NN HA
# Upgrade Ambari 2.7.x
# Now Start HDP upgrade to 3.x and after HDFS upgrade is done pause the Upgrade
# Now stop both NNs are then attempt to start one from Ambari UI - then
Ambari would attempt to start the NN with "–upgrade flag"
# If so it fails to come up with below exception.
{noformat}
2019-12-02 22:05:58,627 ERROR namenode.NameNode (NameNode.java:main(1715)) -
Failed to start namenode.
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
/hadoop/hdfs/namenode is in an inconsistent state: previous fs state should not
exist during upgrade. Finalize or rollback first. at
org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:411)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:418)
at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:438)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:310)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1090)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:714)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:632)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:694)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:937) at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:910) at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1643)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1710)
2019-12-02 22:05:58,629 INFO util.ExitUtil (ExitUtil.java:terminate(210)) -
Exiting with status 1:
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
/hadoop/hdfs/namenode is in an inconsistent state: previous fs state should not
exist during upgrade. Finalize or rollback first.
{noformat}
Ambari should check if NN upgrade is already done - if it is done already then
it should not check start "upgrade" flag.
was:
Steps to reproduce:
# Install Ambari 2.6.x and HDP 2.6.x
# enable NN HA
# Upgrade Ambari 2.7.x
# Now Start HDP upgrade to 3.x and after HDFS upgrade is done if you attempt
to re-start Namenode outside of Upgrade flow ( Pause Upgrade) then Ambari would
attempt to start the NN with "–upgrade flag"
# If so it fails to come up with below exception.
{noformat}
2019-12-02 22:05:58,627 ERROR namenode.NameNode (NameNode.java:main(1715)) -
Failed to start namenode.
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
/hadoop/hdfs/namenode is in an inconsistent state: previous fs state should not
exist during upgrade. Finalize or rollback first. at
org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:411)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:418)
at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:438)
at
org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:310)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1090)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:714)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:632)
at
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:694)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:937) at
org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:910) at
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1643)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1710)
2019-12-02 22:05:58,629 INFO util.ExitUtil (ExitUtil.java:terminate(210)) -
Exiting with status 1:
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
/hadoop/hdfs/namenode is in an inconsistent state: previous fs state should not
exist during upgrade. Finalize or rollback first.
{noformat}
Ambari should check if NN upgrade is already done - if it is done already then
it should not check start "upgrade" flag.
> Ambari starts NN with incorrect flag during upgrade
> ---------------------------------------------------
>
> Key: AMBARI-25438
> URL: https://issues.apache.org/jira/browse/AMBARI-25438
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.7.3
> Reporter: amarnath reddy pappu
> Priority: Major
>
> Steps to reproduce:
> # Install Ambari 2.6.x and HDP 2.6.x
> # enable NN HA
> # Upgrade Ambari 2.7.x
> # Now Start HDP upgrade to 3.x and after HDFS upgrade is done pause the
> Upgrade
> # Now stop both NNs are then attempt to start one from Ambari UI - then
> Ambari would attempt to start the NN with "–upgrade flag"
> # If so it fails to come up with below exception.
> {noformat}
> 2019-12-02 22:05:58,627 ERROR namenode.NameNode (NameNode.java:main(1715)) -
> Failed to start namenode.
> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
> /hadoop/hdfs/namenode is in an inconsistent state: previous fs state should
> not exist during upgrade. Finalize or rollback first. at
> org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:411)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.checkUpgrade(FSImage.java:418)
> at org.apache.hadoop.hdfs.server.namenode.FSImage.doUpgrade(FSImage.java:438)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:310)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1090)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:714)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:632)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:694)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:937)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:910)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1643)
> at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1710)
> 2019-12-02 22:05:58,629 INFO util.ExitUtil (ExitUtil.java:terminate(210)) -
> Exiting with status 1:
> org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory
> /hadoop/hdfs/namenode is in an inconsistent state: previous fs state should
> not exist during upgrade. Finalize or rollback first.
> {noformat}
> Ambari should check if NN upgrade is already done - if it is done already
> then it should not check start "upgrade" flag.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
