gengliangwang commented on pull request #30902: URL: https://github.com/apache/spark/pull/30902#issuecomment-750019909
Hi @saikocat , thanks for reporting the issue. As per Avro spec https://avro.apache.org/docs/1.8.2/spec.html#Time+%28millisecond+precision%29: ``` Time (microsecond precision) The time-micros logical type represents a time of day, with no reference to a particular calendar, time zone or date, with a precision of one microsecond. A time-micros logical type annotates an Avro long, where the long stores the number of microseconds after midnight, 00:00:00.000000. Timestamp (millisecond precision) The timestamp-millis logical type represents an instant on the global timeline, independent of a particular time zone or calendar, with a precision of one millisecond. A timestamp-millis logical type annotates an Avro long, where the long stores the number of milliseconds from the unix epoch, 1 January 1970 00:00:00.000 UTC. ``` So, we can't map the time-millis/time-micros type to Timestamp type. Instead, we should map it as `CalendarIntervalType`. In Spark, `CalendarIntervalType` is internal type and we can't use it for table creatation. So we didn't support the Avro logical type time-millis and time-micros. In the current stage, I am +0 for supporting it. cc @cloud-fan ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
