niebayes opened a new issue, #20132: URL: https://github.com/apache/datafusion/issues/20132
I have found that arrow-rs has supported reading virtual columns, including RowGroupIndex and RowNumber, since v57.0.0. I propose to add a method `with_virtual_columns` to FileScanConfigBuilder to let user decide which virtual columns should be read. The virtual columns, like RowNumber, could be used to implement prewhere, late materialized topk, and secondary indexes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
