bitflicker64 commented on code in PR #3477:
URL: https://github.com/apache/hugegraph/pull/3477#discussion_r4563947080


##########
hugegraph-cluster-test/hugegraph-clustertest-minicluster/src/main/java/org/apache/hugegraph/ct/node/AbstractNodeWrapper.java:
##########
@@ -154,8 +157,10 @@ public boolean isStarted() {
             return false;
         }
 
-        try (Scanner sc = new Scanner(Paths.get(getLogPath()),
-                                      StandardCharsets.UTF_8.name())) {
+        try (InputStream log = Files.newInputStream(Paths.get(getLogPath()))) {
+            // Skip what an earlier run of this node wrote before a restart
+            log.skipNBytes(this.startupLogOffset);
+            Scanner sc = new Scanner(log, StandardCharsets.UTF_8.name());
             while (sc.hasNextLine()) {
                 String line = sc.nextLine();
                 if (line.contains(startLine)) return true;

Review Comment:
   Not changing this. The `Scanner` only wraps `log`, and `log` is the resource 
of the enclosing `try`, so the file handle is closed on every path, the early 
`return true` included. What the `Scanner` holds besides that is a heap buffer 
that goes away with the local variable. Listing it as a resource too would 
create it before `log.skipNBytes(...)`, which is correct only because `Scanner` 
reads lazily.
   



##########
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/cache/CachedSchemaTransactionV2.java:
##########
@@ -500,13 +740,160 @@ private void loadAllSchema() {
     @Override
     public void clear() {
         // Clear schema info firstly
-        super.clear();
-        this.clearCache(false);
-        this.notifySchemaCacheClear();
+        try {
+            super.clear();
+        } finally {
+            // A clear that threw may have removed part of the schema
+            try {
+                this.clearCache(false);
+            } finally {
+                this.publishClearEvent();
+            }
+        }
+    }
+
+    /**
+     * Clear events of this Server that could not be published.
+     * <p>
+     * A clear event is sent after its schema write is committed, so a failed
+     * publish must not fail the write. The graph is kept here instead and the
+     * event is sent again: after a delay that doubles up to a limit, and at
+     * once when the clear-event watch becomes live. One event says "drop your
+     * caches of this graph", so one successful publish covers every earlier
+     * failure for that graph.
+     * <p>
+     * The set lives in memory. If the Server stops before a retry succeeds,
+     * the publish it does on the next graph open covers it. If the Server
+     * never returns, the other Servers keep what they cached until the next
+     * event for that graph or their next watch restart.
+     */
+    static final class PendingClearEvents {
+
+        private static final long RETRY_MIN_DELAY = 1000L;
+        private static final long RETRY_MAX_DELAY = 30000L;
+
+        private final ConcurrentMap<GraphRef, Long> pending =
+                new ConcurrentHashMap<>();
+        private final AtomicLong failures = new AtomicLong();
+
+        // Guarded by this
+        private ScheduledExecutorService executor;
+        private ScheduledFuture<?> retry;
+        private long retryDelay = RETRY_MIN_DELAY;
+        // The number of the newest scheduled retry
+        private long retryEpoch;
+
+        void publish(String graphSpace, String graph) {
+            try {
+                clearEventPublisher.accept(graphSpace, graph);
+            } catch (RuntimeException e) {
+                LOG.warn("Failed to publish the schema cache clear event of " +
+                         "graph '{}/{}', it will be sent again",
+                         graphSpace, graph, e);
+                /*
+                 * The number tells a retry whether the graph failed again
+                 * after the publish the retry is about to count as done.
+                 */
+                this.pending.put(new GraphRef(graphSpace, graph),
+                                 this.failures.incrementAndGet());
+                this.scheduleRetry(false);
+            }
+        }
+
+        void retryNow() {
+            this.scheduleRetry(true);
+        }
+
+        int size() {
+            return this.pending.size();
+        }
+
+        private synchronized void scheduleRetry(boolean now) {
+            if (this.pending.isEmpty()) {
+                return;
+            }
+            if (this.retry != null && !this.retry.isDone()) {
+                if (!now) {
+                    return;
+                }
+                this.retry.cancel(false);
+            }
+            if (this.executor == null) {
+                this.executor = Executors.newSingleThreadScheduledExecutor(r 
-> {
+                    Thread thread = new Thread(r, "schema-cache-clear-retry");
+                    thread.setDaemon(true);
+                    return thread;
+                });
+            }

Review Comment:
   Valid, left as it is. The executor is created only after a first failed 
publish and is one daemon thread that sits idle once the pending set is empty, 
so it holds nothing and does not block shutdown. Shutting it down and creating 
it again would add a lifecycle to the retry path, which already has to handle a 
retry that runs while a watch start schedules a new one. The classes in Server 
core that need a scheduler each own one the same way. It would be worth 
changing if Server gets a shared scheduler for jobs like this.
   



##########
hugegraph-server/hugegraph-core/src/main/java/org/apache/hugegraph/backend/cache/CachedSchemaTransactionV2.java:
##########
@@ -500,13 +740,160 @@ private void loadAllSchema() {
     @Override
     public void clear() {
         // Clear schema info firstly
-        super.clear();
-        this.clearCache(false);
-        this.notifySchemaCacheClear();
+        try {
+            super.clear();
+        } finally {
+            // A clear that threw may have removed part of the schema
+            try {
+                this.clearCache(false);
+            } finally {
+                this.publishClearEvent();
+            }
+        }
+    }
+
+    /**
+     * Clear events of this Server that could not be published.
+     * <p>
+     * A clear event is sent after its schema write is committed, so a failed
+     * publish must not fail the write. The graph is kept here instead and the
+     * event is sent again: after a delay that doubles up to a limit, and at
+     * once when the clear-event watch becomes live. One event says "drop your
+     * caches of this graph", so one successful publish covers every earlier
+     * failure for that graph.
+     * <p>
+     * The set lives in memory. If the Server stops before a retry succeeds,
+     * the publish it does on the next graph open covers it. If the Server
+     * never returns, the other Servers keep what they cached until the next
+     * event for that graph or their next watch restart.
+     */
+    static final class PendingClearEvents {
+
+        private static final long RETRY_MIN_DELAY = 1000L;
+        private static final long RETRY_MAX_DELAY = 30000L;
+
+        private final ConcurrentMap<GraphRef, Long> pending =
+                new ConcurrentHashMap<>();
+        private final AtomicLong failures = new AtomicLong();
+
+        // Guarded by this
+        private ScheduledExecutorService executor;
+        private ScheduledFuture<?> retry;
+        private long retryDelay = RETRY_MIN_DELAY;
+        // The number of the newest scheduled retry
+        private long retryEpoch;
+
+        void publish(String graphSpace, String graph) {
+            try {
+                clearEventPublisher.accept(graphSpace, graph);
+            } catch (RuntimeException e) {
+                LOG.warn("Failed to publish the schema cache clear event of " +
+                         "graph '{}/{}', it will be sent again",
+                         graphSpace, graph, e);
+                /*
+                 * The number tells a retry whether the graph failed again
+                 * after the publish the retry is about to count as done.
+                 */
+                this.pending.put(new GraphRef(graphSpace, graph),
+                                 this.failures.incrementAndGet());
+                this.scheduleRetry(false);
+            }
+        }
+
+        void retryNow() {
+            this.scheduleRetry(true);
+        }
+
+        int size() {
+            return this.pending.size();
+        }
+
+        private synchronized void scheduleRetry(boolean now) {
+            if (this.pending.isEmpty()) {
+                return;
+            }
+            if (this.retry != null && !this.retry.isDone()) {
+                if (!now) {
+                    return;
+                }
+                this.retry.cancel(false);
+            }
+            if (this.executor == null) {
+                this.executor = Executors.newSingleThreadScheduledExecutor(r 
-> {
+                    Thread thread = new Thread(r, "schema-cache-clear-retry");
+                    thread.setDaemon(true);
+                    return thread;
+                });
+            }
+            long delay = now ? 0L : this.retryDelay;
+            long epoch = ++this.retryEpoch;
+            this.retry = this.executor.schedule(() -> this.retryPending(epoch),
+                                                delay, TimeUnit.MILLISECONDS);
+        }
+
+        private void retryPending(long epoch) {
+            List<Map.Entry<GraphRef, Long>> entries =
+                    new ArrayList<>(this.pending.entrySet());
+            for (Map.Entry<GraphRef, Long> entry : entries) {
+                GraphRef graph = entry.getKey();
+                try {
+                    clearEventPublisher.accept(graph.graphSpace(),
+                                               graph.graph());
+                    // Keep it if the graph failed again in the meantime
+                    this.pending.remove(graph, entry.getValue());
+                } catch (RuntimeException e) {
+                    LOG.warn("Failed again to publish the schema cache " +
+                             "clear event of graph '{}/{}': {}",
+                             graph.graphSpace(), graph.graph(), e.toString());
+                }
+            }
+            synchronized (this) {
+                if (epoch != this.retryEpoch) {
+                    /*
+                     * A watch start scheduled a newer retry while this one
+                     * was running: that one goes on, or there would be two.
+                     */
+                    return;
+                }
+                this.retry = null;
+                if (this.pending.isEmpty()) {
+                    this.retryDelay = RETRY_MIN_DELAY;
+                    return;
+                }

Review Comment:
   Same point as the comment on the executor field above, answered there: one 
idle daemon thread after a first failed publish, left as it is.
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to