andygrove opened a new issue, #2152: URL: https://github.com/apache/datafusion-comet/issues/2152
### What is the problem the feature request solves? As discovered in https://github.com/apache/datafusion-comet/issues/2147, the `native_iceberg_compat` scan currently re-uses some legacy code that uses mutable Parquet buffers. As a result, we often need to perform a deep copy of all batches produced from this scan, which impacts performance. There is also the risk of memory corruption if our logic for determining when to make copies is not correct. ### Describe the potential solution I would like `natice_iceberg_compat` scans to stop using mutable Parquet buffers so that we can stop making copies of the output batches. ### Additional context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
