ranflarion opened a new pull request, #27486: URL: https://github.com/apache/datafusion/pull/27486
## Which issue does this PR close? - Closes #27481. ## Rationale for this change A `PhysicalExprAdapterFactory` receives the Parquet file's schema metadata (the footer's key-value metadata) in `physical_file_schema`, except when the scan also reads a virtual column such as `row_number`, where it receives none. An adapter that reads that metadata to decide how to adapt a file stops working as soon as a row number is requested. ## What changes are included in this PR? `append_fields`, which adds the virtual fields to a schema, keeps the base schema's metadata instead of building its result with `Schema::new`. It has three callers: the physical file schema handed to the adapter factory and the simplifier, the logical file schema with virtual columns, and the decoder's stream schema. Each now carries the metadata it already has on the path without virtual columns (`build_projection_read_plan` builds the projected schema with the file schema's metadata). ## What is the testing strategy for this PR? `test_row_index_keeps_file_schema_metadata_for_expr_adapter` writes a file with a key-value entry and opens it through an adapter factory that records the entry it sees, with and without a `row_number` column. On main the read with the virtual column records `None`. ## Are there any user-facing changes? No API changes. The schemas the opener derives for a scan with virtual columns keep their metadata. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
