Asthenia0412 opened a new pull request, #67963:
URL: https://github.com/apache/doris/pull/67963
## Proposed changes
This bumps the Paimon dependency from 1.3.1 to 1.4.2 (matching branch-4.1 /
the 4.1.4 release) to fix the thread leak reported in #67898.
### Root cause
The leaking `rs_normal` threads on external Paimon catalog queries are not
Doris scheduler threads. Triaging on the 4.1.3 build showed Doris's remote scan
pool stays correctly bounded (`r_scan real_num:8`, cap 256) while
`/proc/<pid>/task` grows. The same TIDs are AWS SDK v2
`sdk-ScheduledExecutor-<n>-<m>` threads from Paimon async clients created per
scan and never `close()`d. Paimon 1.4.2 properly closes those executors; the
reporter verified the leak disappears after upgrading to 4.1.4 (+40 → +0 SDK
executor instances per 20 queries per BE).
### API adaptations for 1.4.2
`paimon-scanner` had to follow two Paimon 1.4.2 API changes:
- `FallbackReadFileStoreTable.fallback()` → `other()`, and the constructor
now takes a `wrappedFirst` flag. The `true` value preserves the 1.3.1 two-arg
constructor's fallback (wrapped-first) semantics.
- `IOManager` gained `pickTempDir()`; the test `TestIOManager` implements it.
## Further comments
- Compile verified for all three `${paimon.version}` consumers:
`paimon-scanner`, `preload-extensions`, `fe-connector-paimon`.
- All 47 paimon-scanner unit tests pass (including the adapted
`FallbackReadFileStoreTable` manifest-parallelism and serialization cases).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
