yuqi1129 opened a new issue, #14127: URL: https://github.com/apache/gravitino/issues/14127
### Version main branch (`23c489d7612860da40358b0b3c819b07acf0910a`) ### Describe what's wrong The lineage queue capacity configuration is not applied according to the documentation: - `LineageConfig` defines `sinkQueueCapacity`, but the documentation names `gravitino.lineage.queueCapacity`. - `LineageSinkManager` divides the configured total across sinks and writes `<sink>.queueCapacity`. - `EventListenerManager` reads only the top-level `queueCapacity`. The generated per-sink capacities are ignored. As a result, `sinkQueueCapacity` leaves each queue at the listener default of 3000. The documented `queueCapacity` instead sets each queue to the full total, multiplying the expected allocation when multiple sinks are configured. This also moves the high watermark and HTTP 429 backpressure threshold away from the configured value. ### Error message and/or stacktrace No exception occurs during initialization. With two blocked sinks and `queueCapacity=20`, each queue should hold 10 events and enter high watermark at 10 queued events. It does not: each queue is allocated 20 slots instead. With `sinkQueueCapacity=20`, each queue retains 3000 slots. ### How to reproduce 1. Configure two lineage sinks and set `gravitino.lineage.queueCapacity=20`. 2. Block both sinks while they process their first event. 3. Dispatch 10 additional events into both queues. 4. Observe that the manager has not entered high watermark, although each queue should have a capacity of 10 and a high watermark threshold of 9. 5. Repeat with `gravitino.lineage.sinkQueueCapacity=20`; the divided capacity is also ignored. ### Additional context Regression tests driving real asynchronous queues reproduce the mismatch without an external lineage backend. Existing map-conversion tests check generated configuration but do not verify that the queues consume it. Keep the documented key, accept the old key as an alias, and pass the divided capacity to the setting the listener manager actually reads. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
