SteNicholas commented on PR #574: URL: https://github.com/apache/paimon-cpp/pull/574#issuecomment-6946800691
Thanks @lxy-9602 for [pointing this out](https://github.com/apache/paimon-cpp/pull/574#issuecomment-6945581225). You're right for the Linux shared-library build with bundled Arrow: core and the Parquet plugin each link Arrow with its symbols hidden, so this setter changes the plugin's internal CPU pool, not the application's or core spill's pool. My description of the effect as unconditionally process-wide was too broad. To answer the motivation question directly, this proposal came from source inspection, as noted in #562. I don't have a production failure or measured performance regression to cite here. The concrete behavior motivating it is within the same Parquet plugin: constructing reader A with a thread count of 6 sets the shared plugin pool to 6, then constructing reader B with default options resets it to 3 while A may still be reading. Thus, a per-reader option changes the effective pool capacity for other readers, depending on construction order. This is a configuration side effect; it does not by itself establish a data race or performance regression. I also checked the test setup: changing the test dependency from `paimon_shared` to `paimon_static` makes the spill and Parquet checks use the same Arrow instance. Those tests cover that linkage arrangement, but do not substantiate the cross-component interference claim for the bundled shared-library build. The PR's motivation and documentation need to distinguish these configurations, and validation should cover the shared-library setup, including how its isolated plugin pool is configured once the legacy numeric option stops setting capacity. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
