zhztheplayer opened a new issue, #11514: URL: https://github.com/apache/incubator-gluten/issues/11514
### Description Currently ColumnarBatchOutIterator returned by `NativePlanEvaluator#createKernelWithBatchIterator` is disposable. It could only serve one input stream and output the result data one single time. We'd add support for Velox's task barrier to make it reusable. Fixing this will significantly improve performance of the [ongoing native Delta statistics writer PR](https://github.com/apache/incubator-gluten/issues/11169) where we are creating a new Velox task for every single written partition, which can become super slow when there are thousands of partitions to write. Reference, Velox task barrier: https://facebookincubator.github.io/velox/develop/task-barrier.html ### Gluten version None -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
