andygrove opened a new issue, #26306: URL: https://github.com/apache/datafusion/issues/26306
### Is your feature request related to a problem or challenge? The test module in [core/src/datasource/physical_plan/csv.rs](https://github.com/apache/datafusion/blob/0d79252f61bd8777990b2749c216ffd07079c30d/datafusion/core/src/datasource/physical_plan/csv.rs#L65-L86) defines a private `aggr_test_schema()` that is a verbatim copy of the public [`datafusion::test_util::aggr_test_schema()`](https://github.com/apache/datafusion/blob/0d79252f61bd8777990b2749c216ffd07079c30d/datafusion/core/src/test_util/mod.rs#L92-L112). If the shared schema changes (for example new columns or metadata on `aggregate_test_100.csv`), the CSV tests keep using the stale copy. The same module also repeats about 35 lines of scan setup in every rstest case (session context, `partitioned_file_groups`, `CsvOptions`, `TableSchema`, `CsvSource`, `FileScanConfigBuilder`), differing only in projection, limit and the expected snapshot ([L130-L442](https://github.com/apache/datafusion/blob/0d79252f61bd8777990b2749c216ffd07079c30d/datafusion/core/src/datasource/physical_plan/csv.rs#L130-L442)). `json.rs` ([L213-L372](https://github.com/apache/datafusion/blob/0d79252f61bd8777990b2749c216ffd07079c30d/datafusion/core/src/datasource/physical_plan/json.rs#L213-L372)) does the same with `prepare_store` and `JsonSource`. ### Describe the solution you'd like 1. Delete the local `aggr_test_schema` in the csv.rs tests and import `crate::test_util::aggr_test_schema`. Keep `aggr_test_schema_with_missing_col` as a derivation of it. 2. Optionally add a test helper such as `csv_scan_config(file_compression_type, file_schema, |builder| ...) -> FileScanConfig` that does the shared setup, so each rstest case only applies its projection or limit and asserts. Do the same for json.rs. ### Describe alternatives you've considered Do step 1 only. ### Additional context Step 1 is a small, self-contained change and a good first issue. Line references are pinned to commit 0d79252f61bd. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
