kamilk-pal opened a new issue, #27613:
URL: https://github.com/apache/datafusion/issues/27613

   ### Is your feature request related to a problem or challenge?
   
   Adding a field to any struct generated by `config_namespace!` 
(`ExecutionOptions`, `OptimizerOptions`, `ParquetOptions`, `CsvOptions`, ...) 
is a semver-breaking change. These are plain structs with all-`pub` fields, so 
downstream code can build or match them exhaustively. For example, #23296 added 
`enable_file_stream_work_stealing` to `ExecutionOptions` in 54.1.0, and this 
compiles against 54.0.0 but not 54.1.0:
   
   ```rust
   let ExecutionOptions { batch_size: _, /* ...every other 54.0.0 field... */ } 
= options;
   // error[E0027]: pattern does not mention field 
`enable_file_stream_work_stealing`
   ```
   
   The structs used to be protected against this:
   
   - #4771 introduced `config_namespace!` with [`#[non_exhaustive]` on the 
generated 
struct](https://github.com/apache/datafusion/blob/21169b9c10ddcdd80d71fc4b19a284962b36049f/datafusion/core/src/config.rs#L113-L115).
   - #9382 replaced it [with 
`PartialEq`](https://github.com/apache/datafusion/commit/27b8a49a10875bdfbcae134b5e4a54f22c567513#diff-2f697c310af1a48521829d8bd665cf64b6415fbf88edd370efa30b1ed686354dL114).
 I couldn't find any discussion of the removal there. It looks like a side 
effect of the changes. The macro's [doc 
comment](https://github.com/apache/datafusion/blob/477c7b9ebc88b879d5b8c8d60b21ba4a80593b9d/datafusion/common/src/config.rs#L56-L62)
 still shows `#[non_exhaustive]`.
   
   ### Describe the solution you'd like
   
   Emit `#[non_exhaustive]` from `config_namespace!` again, so that adding a 
config option isn't a breaking change. This is itself a breaking change. Struct 
literals, even ones that use `..Default::default()`, would no longer compile 
outside `datafusion-common`, and callers would start from `Default::default()` 
and set fields.
   
   ### Describe alternatives you've considered
   
   - Keep the current structs. Document that struct literals for config structs 
aren't covered by semver. Explicitly allow for adding config fields -- semver 
breaking changes -- in minor/patch releases.
   
   ### Additional context
   
   - Came up in #27562, which discusses adding a config option on `branch-54`. 
   - Was discussed in #27308, resulting in not adding a config field.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to