uros-db commented on code in PR #56355:
URL: https://github.com/apache/spark/pull/56355#discussion_r3372320830
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/ToStringBase.scala:
##########
@@ -246,18 +238,17 @@ trait ToStringBase { self: UnaryExpression with
TimeZoneAwareExpression =>
ctx.addReferenceObj("timestampNTZFormatter", timestampNTZFormatter),
timestampNTZFormatter.getClass)
(c, evPrim) => code"$evPrim = UTF8String.fromString($tf.format($c));"
- case t: TimestampLTZNanosType =>
- val tf = JavaCode.global(
- ctx.addReferenceObj("timestampFormatter", timestampFormatter),
- timestampFormatter.getClass)
- (c, evPrim) =>
- code"$evPrim = UTF8String.fromString($tf.formatNanos($c,
${t.precision}));"
- case t: TimestampNTZNanosType =>
- val tf = JavaCode.global(
- ctx.addReferenceObj("timestampNTZFormatter", timestampNTZFormatter),
- timestampNTZFormatter.getClass)
- (c, evPrim) =>
- code"$evPrim =
UTF8String.fromString($tf.formatWithoutTimeZoneNanos($c, ${t.precision}));"
+ case _: TimestampNTZNanosType | _: TimestampLTZNanosType =>
+ // Route nanosecond timestamp cast-to-string through the Types
Framework: emit a runtime
+ // call into the ops reference object. The cast's session zone is
threaded into the lookup
+ // so LTZ carries it; NTZ is zone-independent (SPARK-57285).
+ val ops = TypeApiOps(from, zoneId).get
Review Comment:
In the new codegen path and the interpreted path falls through to
castToStringDefault, whose nanos cases were deleted, so it now lands on the
generic terminal case: `case _ => o => UTF8String.fromString(o.toString)`.
TypeApiOps(...) returns None whenever typesFrameworkEnabled == false. The
intended invariant ("nanos types imply the framework is on") is only enforced
at set-time, and only on one flag:
```
.checkValue(
enabled => !enabled || SQLConf.get.typesFrameworkEnabled,
"REQUIREMENT",
_ => Map("confRequirement" ->
(s"'${TYPES_FRAMEWORK_ENABLED.key}' must be true to enable the
nanosecond " +
"timestamp types.")))
```
TYPES_FRAMEWORK_ENABLED has no symmetric guard, so a session can set
timestampNanosTypes.enabled=true, materialize nanos values, then set
types.framework.enabled=false. In that (admittedly unusual, internal-flag)
state, casting a nanos value to string would:
- interpreted: silently produce TimestampNanosVal.toString (wrong output)
instead of a formatted timestamp;
- codegen: throw NoSuchElementException from .get.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
