alamb commented on a change in pull request #2029:
URL: https://github.com/apache/arrow-datafusion/pull/2029#discussion_r830580391
##########
File path: datafusion/src/execution/context.rs
##########
@@ -846,39 +773,14 @@ pub struct SessionConfig {
/// parallel using the provided `target_partitions` level
pub repartition_windows: bool,
/// Should DataFusion parquet reader using the predicate to prune data
- parquet_pruning: bool,
- /// Runtime configurations such as memory threshold and local disk for
spill
- pub runtime: RuntimeConfig,
+ pub parquet_pruning: bool,
Review comment:
@yjshen I agree that the interface with getters would be good;
```rust
pub fn get_bool(&self, key: &str) -> Result<bool> {
self.get(key).and_then(Value::into_bool)
}
```
Among other things that would allow us to change the implementation (e.g. to
properties or typed fields) without issue
I do think typed fields have some nice properties (like being able to do
possibly earlier error detection) but in order to support extensions/customized
properties we definitely need to support aribtrary key/values as well
In terms of `TaskProperties` I personally prefer something like this:
```rust
pub struct TaskProperties {
///SessionConfig
config: SessionConfig,
/// Additional Name-value pairs of task properties
properties: HashMap<String, String>,
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
