kamilk-pal opened a new issue, #27613:
URL: https://github.com/apache/datafusion/issues/27613
### Is your feature request related to a problem or challenge?
Adding a field to any struct generated by `config_namespace!`
(`ExecutionOptions`, `OptimizerOptions`, `ParquetOptions`, `CsvOptions`, ...)
is a semver-breaking change. These are plain structs with all-`pub` fields, so
downstream code can build or match them exhaustively. For example, #23296 added
`enable_file_stream_work_stealing` to `ExecutionOptions` in 54.1.0, and this
compiles against 54.0.0 but not 54.1.0:
```rust
let ExecutionOptions { batch_size: _, /* ...every other 54.0.0 field... */ }
= options;
// error[E0027]: pattern does not mention field
`enable_file_stream_work_stealing`
```
The structs used to be protected against this:
- #4771 introduced `config_namespace!` with [`#[non_exhaustive]` on the
generated
struct](https://github.com/apache/datafusion/blob/21169b9c10ddcdd80d71fc4b19a284962b36049f/datafusion/core/src/config.rs#L113-L115).
- #9382 replaced it [with
`PartialEq`](https://github.com/apache/datafusion/commit/27b8a49a10875bdfbcae134b5e4a54f22c567513#diff-2f697c310af1a48521829d8bd665cf64b6415fbf88edd370efa30b1ed686354dL114).
I couldn't find any discussion of the removal there. It looks like a side
effect of the changes. The macro's [doc
comment](https://github.com/apache/datafusion/blob/477c7b9ebc88b879d5b8c8d60b21ba4a80593b9d/datafusion/common/src/config.rs#L56-L62)
still shows `#[non_exhaustive]`.
### Describe the solution you'd like
Emit `#[non_exhaustive]` from `config_namespace!` again, so that adding a
config option isn't a breaking change. This is itself a breaking change. Struct
literals, even ones that use `..Default::default()`, would no longer compile
outside `datafusion-common`, and callers would start from `Default::default()`
and set fields.
### Describe alternatives you've considered
- Keep the current structs. Document that struct literals for config structs
aren't covered by semver. Explicitly allow for adding config fields -- semver
breaking changes -- in minor/patch releases.
### Additional context
- Came up in #27562, which discusses adding a config option on `branch-54`.
- Was discussed in #27308, resulting in not adding a config field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]