yuqi1129 opened a new pull request, #14128: URL: https://github.com/apache/gravitino/pull/14128
### What changes were proposed in this pull request? Use the documented `gravitino.lineage.queueCapacity` as the total capacity, with `sinkQueueCapacity` retained as a compatibility alias. The documented key takes precedence when both are set. Validate the capacity as positive. Convert the total into the top-level listener queue capacity so each isolated lineage sink gets `max(1, total / sinkCount)` slots. Apply generated settings after copying lineage properties so the original total cannot overwrite the divided value. Document rounding and the legacy key. ### Why are the changes needed? The generated per-sink settings are ignored by `EventListenerManager`. Queues therefore use either the 3000-slot listener default or the undivided total, causing incorrect memory allocation and backpressure thresholds. Fix: #14127 ### Does this PR introduce _any_ user-facing change? Both lineage queue capacity keys now control the documented total across sinks (default 10000). The documented key wins when both are present. Each queue retains a minimum of one slot; division remainders are unused. Non-positive values are rejected. The existing 90% high watermark mechanism is unchanged. ### How was this patch tested? - Added regression tests and verified failures before the production fix. - All 32 lineage tests passed with `./gradlew :lineage:test -PskipITs`. - Eight real-queue scenarios block sinks with latches and check the high watermark boundary for single/multiple sinks, both keys, precedence, defaults, rounding, and the one-slot minimum. - Configuration tests cover alias resolution and invalid capacities. - Module Spotless checks and `git diff --check` passed. - No external lineage backend or Docker integration tests were run. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
