qzsee commented on code in PR #69642:
URL: https://github.com/apache/doris/pull/69642#discussion_r4594311296


##########
fe/fe-core/src/main/java/org/apache/doris/journal/bdbje/BDBEnvironment.java:
##########
@@ -228,6 +236,47 @@ public void setup(File envHome, String selfNodeName, 
String selfNodeHostPort,
                 }
             }
         }
+        if (replicatedEnvironment == null || epochDB == null) {
+            // e.g. still got InsufficientLogException at the last retry
+            LOG.error("failed to open replicated environment after {} retries. 
will exit.", RETRY_TIME);
+            exitHook.accept(-1);
+        }
+    }
+
+    /*
+     * Handle InsufficientLogException thrown by an opened environment: exit 
the process WITHOUT NetworkRestore.
+     *
+     * Once the opened environment is invalidated, every thread which touches 
it (replayer, http/rpc threads, ...)
+     * gets the same exception. Running NetworkRestore in this process is 
unsafe:
+     * - NetworkRestore deletes and renames log files in the environment 
directory, concurrent restores by
+     *   different threads corrupt the directory;
+     * - the invalidated environment is still open, other threads hold its 
handles and its bdbje daemon threads
+     *   may still touch the log files;
+     * - the in-memory meta may have replayed logs which have been rolled 
back, it must be rebuilt anyway.
+     *
+     * After restart, opening the environment throws InsufficientLogException 
again because the cause still
+     * exists, and setup() performs the restore in a clean state. If no 
exception is thrown after restart, the
+     * node can sync up with the master normally and no restore is needed.
+     */
+    public static void exitForNetworkRestore(InsufficientLogException 
insufficientLogEx, String reason) {
+        synchronized (NETWORK_RESTORE_LOCK) {
+            LOG.error("bdb environment needs network restore, thread: {}, 
reason: {}. "
+                    + "will exit, and restore when opening bdb environment 
after restart.",
+                    Thread.currentThread().getName(), reason, 
insufficientLogEx);
+            exitHook.accept(-1);

Review Comment:
   The hang only happens when thrift_server_type=SIMPLE. The other cases where 
it exits directly also hang. We can open a separate PR to fix this issue.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to