acvictor commented on PR #11609:
URL:
https://github.com/apache/incubator-gluten/pull/11609#issuecomment-3908770799
> Hi @acvictor, has the RowToVeloxColumnarExec transition for the
TimestampNTZ type already been supported with this PR? Thanks.
Thank you for the review!
This PR fixes the SparkArrowUtil Arrow type mapping so that the RTC
transitions at fallback boundaries don't crash with
UnsupportedOperationException. The flow is:
1. The new FallbackByTimestampNTZ validator detects TimestampNTZ in a
plan node's schema and forces a fallback to Spark's row-based execution.
2. At the fallback boundary, RowToVeloxColumnarExec calls
SparkArrowUtil.toArrowSchema() to convert the schema. Before this PR, that call
would throw on timestamp_ntz `java.lang.UnsupportedOperationException:
Unsupported data type: timestamp_ntz at
org.apache.spark.sql.utils.SparkArrowUtil$.toArrowType(SparkArrowUtil.scala:58)`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
