Copilot commented on code in PR #19126:
URL: https://github.com/apache/iotdb/pull/19126#discussion_r4476886048
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/protocol/thrift/impl/DataNodeInternalRPCServiceImpl.java:
##########
@@ -2651,13 +2651,7 @@ void sampleDiskLoad(TLoadSample loadSample) {
commonConfig.getDiskSpaceWarningThreshold());
commonConfig.setNodeStatusWithReason(NodeStatus.ReadOnly,
NodeStatus.DISK_FULL);
} else {
- synchronized (commonConfig) {
- // A concurrent Manual/Stopping/error transition must not be cleared
by disk recovery.
- if (NodeStatus.ReadOnly.equals(commonConfig.getNodeStatus())
- && NodeStatus.DISK_FULL.equals(commonConfig.getStatusReason())) {
- commonConfig.setNodeStatus(NodeStatus.Running);
- }
- }
+ commonConfig.tryRecoverFromDiskFull();
Review Comment:
Recovery now runs retrying filesystem probes inside the heartbeat RPC: three
candidates reporting zero free bytes incur at least 30 seconds of sleeps in
`JVMCommonUtils.hasSpace()`. ConfigNode skips further heartbeats while the
request is pending, so this can exceed the default 20-second fixed/cold-start
failure threshold and mark a reachable read-only node `Unknown`. Move recovery
probes off the heartbeat thread or use bounded, non-retrying probes, keeping
the node read-only while checks are pending.
##########
iotdb-core/node-commons/src/main/java/org/apache/iotdb/commons/conf/CommonConfig.java:
##########
@@ -861,13 +868,41 @@ public void setNodeStatus(NodeStatus newStatus) {
setNodeStatusWithReason(newStatus, null);
}
+ /** Records the recovery condition together with the disk-full transition. */
+ public synchronized void setReadOnlyForDiskFull(Object source,
BooleanSupplier recoveryCheck) {
+ diskFullRecoveryChecks.put(source, recoveryCheck);
Review Comment:
Adding a writable data directory through hot reload can leave the node stuck
in `ReadOnly(DiskFull)` while the original directories remain full.
`TierManager.resetFolders()` replaces the managers, but retained callbacks
still check only the old candidates, even when the new manager and aggregate
disk check are healthy. Retire or replace the superseded group's registrations,
including constructor-failure entries, during reinitialization without clearing
unrelated failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]