andygrove commented on issue #1995: URL: https://github.com/apache/datafusion-comet/issues/1995#issuecomment-3211160863
Based on recent learnings, I wonder if we should reconsider this work? We discovered that we always need to a deep copy in `ScanExec` when we fetch a batch from the JVM because we currently do not implement Arrow FFI correctly, and the JVM still has references to the arrays after handing them off to native code. We may still want to insert `CopyExec` for other reasons such as unpacking dictionaries. What do you think @mbutrovich? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
