SteNicholas commented on PR #574:
URL: https://github.com/apache/paimon-cpp/pull/574#issuecomment-6946800691

   Thanks @lxy-9602 for [pointing this 
out](https://github.com/apache/paimon-cpp/pull/574#issuecomment-6945581225). 
You're right for the Linux shared-library build with bundled Arrow: core and 
the Parquet plugin each link Arrow with its symbols hidden, so this setter 
changes the plugin's internal CPU pool, not the application's or core spill's 
pool. My description of the effect as unconditionally process-wide was too 
broad.
   
   To answer the motivation question directly, this proposal came from source 
inspection, as noted in #562. I don't have a production failure or measured 
performance regression to cite here.
   
   The concrete behavior motivating it is within the same Parquet plugin: 
constructing reader A with a thread count of 6 sets the shared plugin pool to 
6, then constructing reader B with default options resets it to 3 while A may 
still be reading. Thus, a per-reader option changes the effective pool capacity 
for other readers, depending on construction order. This is a configuration 
side effect; it does not by itself establish a data race or performance 
regression.
   
   I also checked the test setup: changing the test dependency from 
`paimon_shared` to `paimon_static` makes the spill and Parquet checks use the 
same Arrow instance. Those tests cover that linkage arrangement, but do not 
substantiate the cross-component interference claim for the bundled 
shared-library build. The PR's motivation and documentation need to distinguish 
these configurations, and validation should cover the shared-library setup, 
including how its isolated plugin pool is configured once the legacy numeric 
option stops setting capacity.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to