davidm-db commented on code in PR #56439: URL: https://github.com/apache/spark/pull/56439#discussion_r3405602810
########## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/types/ops/TimeTypeOps.scala: ########## @@ -92,7 +94,19 @@ case class TimeTypeOps(override val t: TimeType) extends TimeTypeApiOps(t) with // ==================== Optional Operations ==================== + // Building a TIME serializer/deserializer requires the TIME type to be enabled. The framework + // dispatch at the head of Serializer/DeserializerBuildHelper routes LocalTimeEncoder through + // these methods, so this is the single gate on the encoder path. Throw rather than return None + // so the dispatch surfaces UNSUPPORTED_TIME_TYPE instead of falling through to the default + // match, which no longer handles LocalTimeEncoder. Review Comment: Done -- switched to "does not handle" so it reads correctly after merge. Thanks! ########## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/types/ops/TimeTypeOpsSuite.scala: ########## @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.spark.sql.catalyst.types.ops + +import org.apache.spark.SparkFunSuite +import org.apache.spark.sql.AnalysisException +import org.apache.spark.sql.catalyst.{DeserializerBuildHelper, SerializerBuildHelper} +import org.apache.spark.sql.catalyst.encoders.AgnosticEncoders.LocalTimeEncoder +import org.apache.spark.sql.catalyst.plans.SQLHelper +import org.apache.spark.sql.internal.SQLConf + +/** + * Tests for the Types Framework wiring of TimeType. + * + * The TIME-type-enabled gate lives in TimeTypeOps.createSerializer / createDeserializer; the + * framework dispatch at the head of Serializer/DeserializerBuildHelper routes LocalTimeEncoder + * through it. Disabling spark.sql.timeType.enabled must therefore still reject building a TIME + * serializer/deserializer with UNSUPPORTED_TIME_TYPE, rather than silently producing one. + */ +class TimeTypeOpsSuite extends SparkFunSuite with SQLHelper { Review Comment: Added -- `TimeTypeOpsSuite` now also asserts the enabled path builds a TIME serializer/deserializer (returning the expected `TimeType` / `LocalTime`), so the gate is exercised in both directions. Thanks for the suggestion! ########## sql/api/src/main/scala/org/apache/spark/sql/types/ops/TypeApiOps.scala: ########## @@ -187,23 +186,19 @@ object TypeApiOps { def apply( dt: DataType, zoneId: => ZoneId = SparkDateTimeUtils.getZoneId(SqlApiConf.get.sessionLocalTimeZone)) - : Option[TypeApiOps] = { - if (!SqlApiConf.get.typesFrameworkEnabled) return None - dt match { - case tt: TimeType => Some(new TimeTypeApiOps(tt)) - case t: TimestampNTZNanosType => Some(new TimestampNTZNanosTypeApiOps(t)) - case t: TimestampLTZNanosType => Some(new TimestampLTZNanosTypeApiOps(t, zoneId)) - // Add new types here - single registration point - case _ => None - } + : Option[TypeApiOps] = dt match { Review Comment: Good catch. That comment documents the legacy `Row.toJsonDefault` TIME arm, which this flag removal makes unreachable -- `TypeApiOps` now always resolves for TIME, so Row JSON renders through `formatExternal` and never falls back here. It's one of the wider legacy TIME fallbacks I discussed with Max in the `SerializerBuildHelper` thread (`HiveResult.toHiveString`, the Connect converters, ...); I'm keeping this PR flag-specific and removing all of them together in a TIME-specific follow-up, where I'd update this comment alongside deleting the dead arm. Happy to do just the one-line comment fix here instead if you'd prefer. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
