uros-b commented on code in PR #56966:
URL: https://github.com/apache/spark/pull/56966#discussion_r3524907701
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/XmlInferSchema.scala:
##########
@@ -682,16 +682,20 @@ class XmlInferSchema(private val options: XmlOptions,
private val caseSensitive:
} else if (isTimeTypeEnabled && isTime(field)) {
// TIME is tried ahead of date/timestamp, matching the ordering of the
previous cascade.
TimeType(TimeType.DEFAULT_PRECISION)
- } else {
+ } else if (options.preferDate) {
tryParseDate(field)
+ } else {
+ tryParseTimestampNTZ(field)
}
}
private def tryParseTime(field: String): DataType = {
if (isTimeTypeEnabled && isTime(field)) {
TimeType(TimeType.DEFAULT_PRECISION)
- } else {
+ } else if (options.preferDate) {
tryParseDate(field)
+ } else {
+ tryParseTimestampNTZ(field)
}
Review Comment:
Possible coverage gaps:
- No test asserts a TIME-shaped value still infers TimeType when
preferDate=false, so a future refactor moving the TIME guard inside the
preferDate branch would go uncaught.
- The tryParseTime temporal re-entry point (incremental path with
preferDate=false) is untested, and the "incremental == legacy" parity test runs
only with default preferDate=true.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
