Caideyipi opened a new pull request, #18808: URL: https://github.com/apache/iotdb/pull/18808
## Description Consensus subscription queues can stop delivering data while WAL lag remains positive. Existing poll/progress timestamps do not measure gaps between data events or show how long a prefetch round has been running. The shared subscription allocator also intentionally admits one entry larger than its entire budget when otherwise empty, but currently exposes neither this exception nor the resulting node overcommit. Add queue timing gauges, node allocator gauges, and diagnostic warnings so operators can correlate delivery gaps with prefetch work and memory admission. ### Behavior and metric semantics - Track data delivery with monotonic time when a data event is returned by `poll`. Empty polls, watermarks, and ACKs do not reset the data idle timer. Expose the last and maximum delivery interval and the current data idle time, in milliseconds. - Expose an ongoing prefetch round's duration (including queue lock acquisition), its maximum duration, and time since the last completed round. Gauge reads remain available while the worker is blocked. Completed rounds lasting at least 10 seconds produce a diagnostic warning. - Export node subscription memory used, limit, overcommit bytes, and the cumulative oversized-entry admission count directly from the shared allocator. These account for retained subscription data; temporary materialization and WAL/JVM allocations require separate JVM metrics. - Preserve the existing single oversized-entry progress exception: the budget must be positive and otherwise empty. Warn on its first occurrence and at most once every 30 seconds thereafter; other queues can wait until the entry is released. Removing this exception would prevent replay from advancing past that entry. - Include timing and allocator readings in queue core reports. Bind/unbind the new gauges across metric service restarts. Add matching English and Chinese warning messages. Delivery idle gauges can grow when there is no data or no polling. Interpret them together with queue active/initialized state, WAL lag, and polling activity. A full JVM pause also suspends its exporter. This change supplies diagnostics for subsequent investigation; it does not change client idle timeouts or establish the cause of a storage/GC stall. ### Validation - Full 53-module reactor compilation passed for both default English and `with-zh-locale` builds. The English build used `mvn test-compile -DskipTests`; the Chinese build used the full `test` lifecycle with only the selected subscription tests enabled. - All 14 tests passed in each locale across `SubscriptionMemoryManagerTest`, `ConsensusPrefetchingQueueDataNodeMemoryTest`, `ConsensusPrefetchingQueueWalBackpressureTest`, `ConsensusSubscriptionPrefetchingQueueMetricsTest`, and `SubscriptionMetricsRestartTest`. - An expanded 54-test run also included `ConsensusPrefetchingQueueTest`. Six failures in that class reproduced at the same assertions with unmodified production sources from base commit `eb0e3f1f2080e34bf5593bfd1cf63b62ecdba2e0`: `testTabletMemoryReleasedAfterAck`, `testLateAckDoesNotWaitForPrefetchReadLock`, `testFilteredEmptyEntryAdvancesProgressWithoutEvent`, `testAtEndInitializationPreservesPendingQueueRegistrationBoundary`, `testFilteredEmptyEntryDoesNotAdvanceAcrossUncommittedData`, and `testRequeueDoesNotIncrementNackCount`. The expanded suite is not reported as passing. - Spotless, `git diff --check`, English/Chinese message-key and placeholder parity, deterministic message hashes, and the scan for added raw log literals passed. ### Self-review - [x] Reviewed concurrent reads and writes; timing reads do not acquire the queue lock. - [x] Documented metric semantics and the oversized-entry progress exception in code and this description. - [x] Added tests for data gaps across watermarks/ACKs, timing during blocked prefetch, monotonic clock wrap, cross-queue admission/recovery, and metric service restart/removal. - [x] Included Apache license headers and matching localized messages. ### Key changed classes `ConsensusPrefetchingQueue`, `SubscriptionQueueTimeTracker`, `ConsensusSubscriptionPrefetchingQueueMetrics`, `SubscriptionMetrics`, and `SubscriptionMemoryManager`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
