qzsee commented on code in PR #69642:
URL: https://github.com/apache/doris/pull/69642#discussion_r4594311296
##########
fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBEnvironment.java:
##########
@@ -228,6 +236,47 @@ public void setup(File envHome, String selfNodeName,
String selfNodeHostPort,
}
}
}
+ if (replicatedEnvironment == null || epochDB == null) {
+ // e.g. still got InsufficientLogException at the last retry
+ LOG.error("failed to open replicated environment after {} retries.
will exit.", RETRY_TIME);
+ exitHook.accept(-1);
+ }
+ }
+
+ /*
+ * Handle InsufficientLogException thrown by an opened environment: exit
the process WITHOUT NetworkRestore.
+ *
+ * Once the opened environment is invalidated, every thread which touches
it (replayer, http/rpc threads, ...)
+ * gets the same exception. Running NetworkRestore in this process is
unsafe:
+ * - NetworkRestore deletes and renames log files in the environment
directory, concurrent restores by
+ * different threads corrupt the directory;
+ * - the invalidated environment is still open, other threads hold its
handles and its bdbje daemon threads
+ * may still touch the log files;
+ * - the in-memory meta may have replayed logs which have been rolled
back, it must be rebuilt anyway.
+ *
+ * After restart, opening the environment throws InsufficientLogException
again because the cause still
+ * exists, and setup() performs the restore in a clean state. If no
exception is thrown after restart, the
+ * node can sync up with the master normally and no restore is needed.
+ */
+ public static void exitForNetworkRestore(InsufficientLogException
insufficientLogEx, String reason) {
+ synchronized (NETWORK_RESTORE_LOCK) {
+ LOG.error("bdb environment needs network restore, thread: {},
reason: {}. "
+ + "will exit, and restore when opening bdb environment
after restart.",
+ Thread.currentThread().getName(), reason,
insufficientLogEx);
+ exitHook.accept(-1);
Review Comment:
The hang only happens when thrift_server_type=SIMPLE. The other cases where
it exits directly also hang. We can open a separate PR to fix this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]