andygrove opened a new issue, #387:
URL: https://github.com/apache/datafusion-spark/issues/387

   ## Is your feature request related to a problem or challenge?
   
   Spark expressions take the session settings they depend on as constructor 
parameters, such as `failOnError`/`evalMode` and `timeZoneId` 
(`TimeZoneAwareExpression`). Here each function reads them from `ConfigOptions` 
in its own way:
   
   - ANSI mode is read as the `bool` 
`config_options.execution.enable_ansi_mode` in 16 files (`abs`, `divide`, 
`elt`, `make_date`, `modulus`, `negative`, `next_day`, `round`, ...). 
`parse_url` uses `EvalMode::from_config`, which was added for this purpose.
   - It is read in `invoke_with_args` by most functions, in `simplify` by 
`nanvl` and `positive`, and in `with_updated_config` by `SparkInputTypes`. Only 
the last of these can affect planning, for example coercion and nullability.
   - The session time zone is read directly by `date_trunc`, `to_json` and 
`cast`. Spark's other time-zone-aware expressions (`hour`, `minute`, `second`, 
`date_format`, `from_unixtime`, ...) still need to be checked against this.
   
   ## Describe the solution you'd like
   
   - Use `EvalMode::from_config` everywhere ANSI mode is read, so that `try_*` 
variants can share the code with `EvalMode::Try`.
   - Add one helper that reads the session time zone, using the same parsing as 
`timezone_utils.rs`, for the time-zone-aware functions.
   - Document where each setting should be read: in `with_updated_config` if it 
affects planning, otherwise in `invoke_with_args`.
   
   ## Describe alternatives you've considered
   
   Leave it as is. Each read is correct, but the inconsistency makes it harder 
to check a function against Spark and to add `try_*` variants.
   
   ## Additional context
   
   Related: #88 (`EvalMode` for CAST), #300 (lost `SparkOptions`), #25 
(`next_day` and the ANSI default). This is a small, low-priority cleanup.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to