ArafatKhan2198 commented on code in PR #9436:
URL: https://github.com/apache/ozone/pull/9436#discussion_r2597175553
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java:
##########
@@ -272,6 +272,7 @@ public void start() {
throw new RuntimeException(runtimeException);
}
}
+ reconTaskController.updateOMMetadataManager(omMetadataManager);
Review Comment:
I had actually gotten this error when I was trying to fix this problem with
a different fix -
This is the following code for ReconServer.java class of my earlier
iteration of fix where I kept the `start()` before Upgrade logic.
```
LOG.debug("Now starting all services of Recon...");
this.start();
this.isStarted = true;
LOG.info("Initializing support of Recon Features...");
FeatureProvider.initFeatureSupport(this.configuration);
LOG.info("Finalizing Layout Features.");
ReconSchemaVersionTableManager versionTableManager =
(ReconSchemaVersionTableManager)this.injector.getInstance(ReconSchemaVersionTableManager.class);
DataSource dataSource =
(DataSource)this.injector.getInstance(DataSource.class);
ReconLayoutVersionManager layoutVersionManager = new
ReconLayoutVersionManager(versionTableManager, reconContext, dataSource);
layoutVersionManager.finalizeLayoutFeatures();
LOG.info("Recon schema versioning completed.");
LOG.info("Recon server initialized successfully!");
```
I got the following error in the logs while trying to rebuild the NSSummary
tree -
```
2025-12-02 19:41:57,739 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Feature
versioned 0 finalized successfully.
2025-12-02 19:41:57,751 INFO
[main]-org.apache.hadoop.ozone.recon.ReconSchemaVersionTableManager: Updated
schema version to '1'.
2025-12-02 19:41:57,751 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: MLV
updated to: 1
2025-12-02 19:41:57,753 INFO
[main]-org.apache.ozone.recon.schema.SqlDbUtils: RECON_TASK_STATUS table
already exists, skipping creation.
2025-12-02 19:41:57,754 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconTaskStatusTableUpgradeAction:
Adding 'last_task_run_status' column to task status table
2025-12-02 19:41:57,757 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconTaskStatusTableUpgradeAction:
Adding 'is_current_task_running' column to task status table
2025-12-02 19:41:57,765 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconTaskStatusTableUpgradeAction:
Updated 9 rows with default value for new columns
2025-12-02 19:41:57,779 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Feature
versioned 1 finalized successfully.
2025-12-02 19:41:57,780 INFO
[main]-org.apache.hadoop.ozone.recon.ReconSchemaVersionTableManager: Updated
schema version to '2'.
2025-12-02 19:41:57,780 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: MLV
updated to: 2
2025-12-02 19:41:57,784 INFO
[main]-org.apache.ozone.recon.schema.SqlDbUtils: UNHEALTHY_CONTAINERS table
already exists, skipping creation.
2025-12-02 19:41:57,790 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.UnhealthyContainerReplicaMismatchAction:
Added the updated constraint to the UNHEALTHY_CONTAINERS table for enum state
values: [MISSING, EMPTY_MISSING, UNDER_REPLICATED, OVER_REPLICATED,
MIS_REPLICATED, ALL_REPLICAS_BAD, NEGATIVE_SIZE, REPLICA_MISMATCH]
2025-12-02 19:41:57,791 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Feature
versioned 2 finalized successfully.
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.ReconSchemaVersionTableManager: Updated
schema version to '3'.
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: MLV
updated to: 3
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.NSSummaryAggregatedTotalsUpgrade:
Triggering asynchronous NSSummary tree rebuild for materialized totals (upgrade
action).
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl: Queueing
task reinitialization event due to: MANUAL_TRIGGER (retry attempt count: 0)
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl: Attempting
checkpoint creation (retry attempt: 1)
2025-12-02 19:41:57,794 ERROR
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Failed
to finalize feature 3. Rolling back changes.
java.lang.NullPointerException: Cannot invoke
"org.apache.hadoop.ozone.recon.recovery.ReconOMMetadataManager.getStore()"
because "omMetaManager" is null
at
org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.cleanTempCheckPointPath(ReconTaskControllerImpl.java:624)
at
org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.createOMCheckpoint(ReconTaskControllerImpl.java:607)
at
org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.queueReInitializationEvent(ReconTaskControllerImpl.java:503)
at
org.apache.hadoop.ozone.recon.upgrade.NSSummaryAggregatedTotalsUpgrade.execute(NSSummaryAggregatedTotalsUpgrade.java:54)
at
org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager.finalizeLayoutFeatures(ReconLayoutVersionManager.java:101)
at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:177)
at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:73)
at picocli.CommandLine.executeUserObject(CommandLine.java:2031)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at
picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2469)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2423)
at
picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2425)
at picocli.CommandLine.execute(CommandLine.java:2174)
at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:89)
at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:80)
at org.apache.hadoop.ozone.recon.ReconServer.main(ReconServer.java:95)
2025-12-02 19:41:57,796 ERROR
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Failed
to finalize layout features: Cannot invoke
"org.apache.hadoop.ozone.recon.recovery.ReconOMMetadataManager.getStore()"
because "omMetaManager" is null
```
So after setting it explicitly from the setter I was able to fix this
problem of NULL pointer
##########
hadoop-ozone/recon/src/main/java/org/apache/hadoop/ozone/recon/spi/impl/OzoneManagerServiceProviderImpl.java:
##########
@@ -272,6 +272,7 @@ public void start() {
throw new RuntimeException(runtimeException);
}
}
+ reconTaskController.updateOMMetadataManager(omMetadataManager);
Review Comment:
I had actually gotten this error when I was trying to fix this problem with
a different fix -
This is the following code for ReconServer.java class of my earlier
iteration of fix where I kept the `start()` before Upgrade logic.
```
LOG.debug("Now starting all services of Recon...");
this.start();
this.isStarted = true;
LOG.info("Initializing support of Recon Features...");
FeatureProvider.initFeatureSupport(this.configuration);
LOG.info("Finalizing Layout Features.");
ReconSchemaVersionTableManager versionTableManager =
(ReconSchemaVersionTableManager)this.injector.getInstance(ReconSchemaVersionTableManager.class);
DataSource dataSource =
(DataSource)this.injector.getInstance(DataSource.class);
ReconLayoutVersionManager layoutVersionManager = new
ReconLayoutVersionManager(versionTableManager, reconContext, dataSource);
layoutVersionManager.finalizeLayoutFeatures();
LOG.info("Recon schema versioning completed.");
LOG.info("Recon server initialized successfully!");
```
I got the following error in the logs while trying to rebuild the NSSummary
tree -
```
2025-12-02 19:41:57,739 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Feature
versioned 0 finalized successfully.
2025-12-02 19:41:57,751 INFO
[main]-org.apache.hadoop.ozone.recon.ReconSchemaVersionTableManager: Updated
schema version to '1'.
2025-12-02 19:41:57,751 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: MLV
updated to: 1
2025-12-02 19:41:57,753 INFO
[main]-org.apache.ozone.recon.schema.SqlDbUtils: RECON_TASK_STATUS table
already exists, skipping creation.
2025-12-02 19:41:57,754 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconTaskStatusTableUpgradeAction:
Adding 'last_task_run_status' column to task status table
2025-12-02 19:41:57,757 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconTaskStatusTableUpgradeAction:
Adding 'is_current_task_running' column to task status table
2025-12-02 19:41:57,765 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconTaskStatusTableUpgradeAction:
Updated 9 rows with default value for new columns
2025-12-02 19:41:57,779 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Feature
versioned 1 finalized successfully.
2025-12-02 19:41:57,780 INFO
[main]-org.apache.hadoop.ozone.recon.ReconSchemaVersionTableManager: Updated
schema version to '2'.
2025-12-02 19:41:57,780 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: MLV
updated to: 2
2025-12-02 19:41:57,784 INFO
[main]-org.apache.ozone.recon.schema.SqlDbUtils: UNHEALTHY_CONTAINERS table
already exists, skipping creation.
2025-12-02 19:41:57,790 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.UnhealthyContainerReplicaMismatchAction:
Added the updated constraint to the UNHEALTHY_CONTAINERS table for enum state
values: [MISSING, EMPTY_MISSING, UNDER_REPLICATED, OVER_REPLICATED,
MIS_REPLICATED, ALL_REPLICAS_BAD, NEGATIVE_SIZE, REPLICA_MISMATCH]
2025-12-02 19:41:57,791 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Feature
versioned 2 finalized successfully.
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.ReconSchemaVersionTableManager: Updated
schema version to '3'.
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: MLV
updated to: 3
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.upgrade.NSSummaryAggregatedTotalsUpgrade:
Triggering asynchronous NSSummary tree rebuild for materialized totals (upgrade
action).
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl: Queueing
task reinitialization event due to: MANUAL_TRIGGER (retry attempt count: 0)
2025-12-02 19:41:57,792 INFO
[main]-org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl: Attempting
checkpoint creation (retry attempt: 1)
2025-12-02 19:41:57,794 ERROR
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Failed
to finalize feature 3. Rolling back changes.
java.lang.NullPointerException: Cannot invoke
"org.apache.hadoop.ozone.recon.recovery.ReconOMMetadataManager.getStore()"
because "omMetaManager" is null
at
org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.cleanTempCheckPointPath(ReconTaskControllerImpl.java:624)
at
org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.createOMCheckpoint(ReconTaskControllerImpl.java:607)
at
org.apache.hadoop.ozone.recon.tasks.ReconTaskControllerImpl.queueReInitializationEvent(ReconTaskControllerImpl.java:503)
at
org.apache.hadoop.ozone.recon.upgrade.NSSummaryAggregatedTotalsUpgrade.execute(NSSummaryAggregatedTotalsUpgrade.java:54)
at
org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager.finalizeLayoutFeatures(ReconLayoutVersionManager.java:101)
at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:177)
at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:73)
at picocli.CommandLine.executeUserObject(CommandLine.java:2031)
at picocli.CommandLine.access$1500(CommandLine.java:148)
at
picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2469)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2461)
at picocli.CommandLine$RunLast.handle(CommandLine.java:2423)
at
picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2277)
at picocli.CommandLine$RunLast.execute(CommandLine.java:2425)
at picocli.CommandLine.execute(CommandLine.java:2174)
at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:89)
at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:80)
at org.apache.hadoop.ozone.recon.ReconServer.main(ReconServer.java:95)
2025-12-02 19:41:57,796 ERROR
[main]-org.apache.hadoop.ozone.recon.upgrade.ReconLayoutVersionManager: Failed
to finalize layout features: Cannot invoke
"org.apache.hadoop.ozone.recon.recovery.ReconOMMetadataManager.getStore()"
because "omMetaManager" is null
```
So after setting it explicitly from the setter I was able to fix this
problem of NULL pointer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
