pitrou commented on issue #40301: URL: https://github.com/apache/arrow/issues/40301#issuecomment-2045365707
I'm honestly not sure what this has to do with work stealing. If I read correctly: * a new ScalarMemoTable is created when converting each column (either eagerly or lazily, this might or might not matter) * columns are converted in parallel for smaller execution times * a lot of memory allocations may occur because of this What I would like to know is: 1. What is the concern? The total number of allocations? The max allocated working set during the workload? 2. Is it a practical concern? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
