MaxGekk commented on code in PR #56916:
URL: https://github.com/apache/spark/pull/56916#discussion_r3504578703
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/xml/XmlInferSchema.scala:
##########
@@ -542,14 +542,13 @@ class XmlInferSchema(private val options: XmlOptions,
private val caseSensitive:
if ((SQLConf.get.legacyTimeParserPolicy == LegacyBehaviorPolicy.LEGACY ||
timestampType == TimestampNTZType) &&
timestampNTZFormatter.parseWithoutTimeZoneOptional(field,
false).isDefined) {
- if (SQLConf.get.timestampNanosTypesEnabled) {
- // Prefer nanosecond type when the fractional seconds part has more
than 6 digits,
- // indicating sub-microsecond precision that cannot be represented
by TimestampNTZType.
- val nanosOpt =
- timestampNTZFormatter.parseWithoutTimeZoneNanosOptional(field, 9,
false)
- nanosOpt.filter(_.nanosWithinMicro != 0).foreach { _ =>
- return Some(TimestampNTZNanosType(9))
- }
+ // Prefer nanosecond type when the fractional seconds part has more
than 6 digits,
Review Comment:
Nit (pre-existing wording, but you're moving this comment so it's a natural
moment): "more than 6 digits" doesn't quite match the `nanosWithinMicro != 0`
test below it. A 7-digit fraction with a trailing zero — e.g. `.1234560` →
123456000 ns → `nanosWithinMicro == 0` — has more than 6 digits yet stays
`TimestampNTZType`; the actual trigger is a *nonzero* sub-microsecond
remainder, not the digit count. Maybe: `// Prefer nanosecond type when there is
a nonzero sub-microsecond component (nanosWithinMicro != 0) that
TimestampNTZType cannot represent.`
The refactor itself is exactly equivalent — nice cleanup.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
