dannytbecker opened a new pull request, #6013: URL: https://github.com/apache/hadoop/pull/6013
### Description of PR NameNode currently uses a "StartupOption" to decide whether to start a node as an Observer NameNode. This causes an issue during a rolling upgrade because "rollingUpgrade" is also a StartupOption and NameNode will only allow 1 startup option, choosing the last startup option in the list. Observer in our environment starts with the following startup options: ["-rollingUpgrade", "started", "-observer"]. This means that the rolling upgrade gets ignored which causes Observer to have an issue when an actual rolling upgrade is ongoing: ``` 2023-08-23T14:59:03.486-0700,WARN,[main],org.apache.hadoop.hdfs.server.namenode.FSNamesystem,"Encountered exception loading fsimage java.io.IOException: File system image contains an old layout version -63. An upgrade to version -66 is required. Please restart NameNode with the \'-rollingUpgrade started\' option if a rolling upgrade is already started; or restart NameNode with the \'-upgrade\' option to start a new upgrade. at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:271) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1116) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:724) at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:681) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:763) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:1013) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:992) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1743) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1811) " ``` ### How was this patch tested? This was tested in an environment where the Observer was not able to start with the "-rollingUpgrade started" startup option because it also had the "-observer" startup option. The Observer needs to start with "-rollingUpgrade started" in the environment while a rolling upgrade is happening or it will fail to load the fsimage with a layout version that is not equal to its `NameNodeLayoutVersion.CURRENT_LAYOUT_VERSION` ### For code changes: - [ x ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
