MaxGekk commented on PR #56622: URL: https://github.com/apache/spark/pull/56622#issuecomment-4757395995
The MICROS/NANOS bifurcation for `TimeType` is duplicated into three fallback branches that are effectively dead: `ParquetTypeOps(TimeType)` always returns `Some(TimeTypeParquetOps(...))`, so the framework-first dispatch (`convertField` / `makeWriter` / `newConverter`) always routes `TimeType` to `TimeTypeParquetOps` and these `case _: TimeType` fallbacks never run. The live logic is in `TimeTypeParquetOps`. - `ParquetSchemaConverter.scala:716` (`convertFieldDefault`) - `ParquetWriteSupport.scala:310` (`makeWriterDefault`) - `ParquetRowConverter.scala:529` (`newConverterDefault`) I'd prefer not to expand the dead copies further in this PR. Tracking the cleanup (drop the shadowed `TimeType` fallbacks so the framework op is the single source of truth, or confirm they're intentionally kept during the types-framework migration) under [SPARK-57582](https://issues.apache.org/jira/browse/SPARK-57582), a sub-task of the Types framework umbrella [SPARK-53504](https://issues.apache.org/jira/browse/SPARK-53504). cc @davidm-db -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
