lucasfang commented on PR #574: URL: https://github.com/apache/paimon-cpp/pull/574#issuecomment-6967596269
> We generally set the thread count via query options. Although the readers overwrite the plugin pool capacity, they still use the same thread count in our current setup, so we haven’t seen issues with this behavior. > > The current pr modifies the default CPU capacity and could impact query performance—especially for wide schemas with thousands of columns. Without a production problem driving this change, I’d prefer to keep the current behavior for now. Alternatively, we could preserve the existing default behavior and introduce an option to disable pool resizing. What do you think, [@lucasfang](https://github.com/lucasfang)? I agree that we should preserve the current default behavior. Since Arrow symbols are hidden within the plugin, arrow::SetCpuThreadPoolCapacity(executor_thread_count) only affects the plugin’s internal Arrow thread pool. This PR effectively switches to Arrow’s default thread count, which could change query performance for existing workloads, especially those with thousands of columns. Without a concrete production issue motivating that change, I’d prefer to keep the existing behavior. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
