XiaoHongbo-Hope commented on PR #696: URL: https://github.com/apache/paimon-rust/pull/696#issuecomment-5929143231
Added a bounded Jindo read gate in `bf2c86b`: - new `fs.jindo.max.concurrent.reads` option, default `8` per OSS operator - the permit is held until the blocking Jindo C call actually returns, including when the async waiter is cancelled - invalid/non-positive limits fail configuration Motivation from a fixed single-node 8-rank/8-worker read workload with official pyjindosdk 6.10.401: | Build | Measured throughput | Warm-up | Jindo warnings | | --- | ---: | ---: | ---: | | #696 without gate, four rounds | 267.388 / 266.481 / 268.879 / 267.838 samples/s | 14.595 / 14.219 / 14.385 / 103.819 s | 4,357 connect, 4 request | | same workload with limit 8, two rounds | 265.806 / 263.424 samples/s | 9.126 / 9.463 s | 148 connect | The observed throughput cost was about 1.1%, while connection warnings dropped substantially and the reproduced warm-up tail did not recur in the bounded runs. This gate limits request fan-out; it does not replace an independent JindoSDK DNS/connection-pool fix. Local validation: `cargo test -p paimon --features storage-jindo storage_jindo --lib` (9 passed, 1 existing SDK-dependent test ignored), Clippy with `-D warnings`, rustfmt, and diff checks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
