hongzhi-gao commented on code in PR #18848:
URL: https://github.com/apache/iotdb/pull/18848#discussion_r4290366591
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/flush/MemTableFlushTask.java:
##########
@@ -382,6 +407,68 @@ public static void recordFlushPointsMetricInternal(
WRITING_METRICS.recordFlushCost(WritingMetrics.FLUSH_STAGE_IO, ioTime);
};
+ /** Tracks actual worker exit, including cancellation before the executor
starts it. */
+ private final class FlushSubTask implements Runnable {
+ private final Runnable task;
+ private final CompletableFuture<Void> finished = new CompletableFuture<>();
+ private Thread runner;
+ private boolean cancelled;
+
+ private FlushSubTask(Runnable task) {
+ this.task = task;
+ }
+
+ @Override
+ public void run() {
+ synchronized (this) {
+ if (cancelled) {
+ return;
+ }
+ runner = Thread.currentThread();
+ }
+ try {
+ task.run();
+ } catch (Throwable failure) {
+ subTaskFailure.completeExceptionally(
+ failure instanceof CompletionException || failure instanceof
UncheckedIOException
+ ? failure.getCause()
+ : failure);
+ } finally {
+ synchronized (this) {
+ runner = null;
+ finished.complete(null);
+ }
+ }
+ }
+
+ private synchronized boolean isCancelled() {
+ return cancelled;
+ }
+
+ private synchronized void cancel() {
+ cancelled = true;
+ if (runner != null) {
+ runner.interrupt();
+ } else {
+ finished.complete(null);
+ }
+ }
+
+ private boolean awaitExit() {
+ boolean interrupted = false;
+ while (true) {
+ try {
+ finished.get();
+ return interrupted;
+ } catch (InterruptedException e) {
+ interrupted = true;
+ } catch (ExecutionException e) {
+ throw new AssertionError(e);
+ }
+ }
+ }
Review Comment:
Replaced the exit wait with a CountDownLatch, so worker failures are
propagated through subTaskFailure to syncFlushMemTable rather than being
represented as an AssertionError in cleanup. The caller still has to wait after
interruption: cancellation only requests worker interruption, and the worker
may still be using the writer/buffers. Returning here would release its
flush-memory reservation before those resources stop being used. Cleanup waits
for both workers to exit, then restores the caller's interrupt flag and
propagates the original failure/interruption. The regression now also
interrupts the caller again during cancellation and checks that the reservation
remains live until the I/O worker exits.
Implemented in
[f1945e5201](https://github.com/apache/iotdb/pull/18848/commits/f1945e52015f4d3b99ffc036b9c903592feb3f82).
Validation: 87 focused tests passed; full-reactor test compilation passed
for both the default and Chinese locales; all 4 IoTDBWriteMemoryLifecycleIT
cases passed in 1 ConfigNode + 1 DataNode, including forced-restart WAL
recovery. Independent Codex review found no actionable issues.
##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/storageengine/dataregion/memtable/IMemTable.java:
##########
@@ -77,7 +79,13 @@ void writeAlignedRow(
*/
long memSize();
- /** only used when mem control enabled */
+ void setWriteMemoryManager(WriteMemoryManager manager);
+
+ WriteMemoryBlock getArrayMemoryBlock();
+
+ WriteMemoryBlock getBinaryMemoryBlock();
+
+ /** Adds array/TVList structural memory; Binary payload has a separate
block. */
void addTVListRamCost(long cost);
Review Comment:
Added getStructureMemoryBlock() and the MEMTABLE_STRUCTURE category. Aligned
TVList container estimates are now accounted separately from arrays/bitmaps and
Binary payloads, using the existing before/after reconciliation for working and
sorted TVLists. This keeps the aggregate estimate unchanged rather than adding
a second charge. The block participates in attach, clear and release; WAL
snapshots retain the existing aggregate field and recover it into the
recovered-total block. Extended coverage checks category totals, WAL-failure
rollback, clear/release, and both old/current snapshot readers.
Implemented in
[f1945e5201](https://github.com/apache/iotdb/pull/18848/commits/f1945e52015f4d3b99ffc036b9c903592feb3f82).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]