bharath-techie commented on issue #14816: URL: https://github.com/apache/datafusion/issues/14816#issuecomment-2680694026
Thanks @chenkovsky for confirming. We are new to datafusion , but at high level looks like this feature will need a deeper integration in the ParquetExec flow and we might need changes in `ParquetRecordBatchStream` in `arrow-rs` as it performs pruning and at datafusion layer we might not be able to figure out the actual row ids because of it. Experts can comment on this / see if there are any other ways that they can think of. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
