lucasfang commented on PR #574:
URL: https://github.com/apache/paimon-cpp/pull/574#issuecomment-6967596269

   > We generally set the thread count via query options. Although the readers 
overwrite the plugin pool capacity, they still use the same thread count in our 
current setup, so we haven’t seen issues with this behavior.
   > 
   > The current pr modifies the default CPU capacity and could impact query 
performance—especially for wide schemas with thousands of columns. Without a 
production problem driving this change, I’d prefer to keep the current behavior 
for now. Alternatively, we could preserve the existing default behavior and 
introduce an option to disable pool resizing. What do you think, 
[@lucasfang](https://github.com/lucasfang)?
   
   I agree that we should preserve the current default behavior. Since Arrow 
symbols are hidden within the plugin, 
arrow::SetCpuThreadPoolCapacity(executor_thread_count) only affects the 
plugin’s internal Arrow thread pool.
   
   This PR effectively switches to Arrow’s default thread count, which could 
change query performance for existing workloads, especially those with 
thousands of columns. Without a concrete production issue motivating that 
change, I’d prefer to keep the existing behavior.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to