Max Gekk created SPARK-57556:
--------------------------------
Summary: Support the TIME data type in Hive SerDe interop
(HiveInspectors)
Key: SPARK-57556
URL: https://issues.apache.org/jira/browse/SPARK-57556
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 4.3.0
Reporter: Max Gekk
h2. What
Handle {{TimeType}} in Hive interop so Hive-serde tables and Hive UDFs do not
fail on TIME,
or, if full support is not feasible, produce a clear, documented error.
h2. Why
{{HiveInspectors}} has no {{TimeType}} case, so object-inspector creation and
TypeInfo
mapping fall through. Hive-serde tables and Hive ORC already reject TIME
(SPARK-51590), but
the behavior is implicit. Hive itself has no TIME type, so a faithful mapping
is limited.
h2. Scope
Option A (preferred if viable): map {{TimeType}} through a Hive-representable
carrier
(e.g. string or {{TIMESTAMP}}) in {{HiveInspectors}} (toInspector / unwrap /
wrap / TypeInfo
paths) with documented semantics.
Option B (fallback): detect {{TimeType}} early and raise
{{UNSUPPORTED_DATA_TYPE_FOR_DATASOURCE}} (or a dedicated Hive error) with a
clear message,
and document the limitation on the TIME docs page.
h2. Acceptance criteria
* No {{MatchError}}/internal error when a TIME column reaches Hive serde paths.
* Behavior (support or explicit error) is covered by tests in the hive module
and documented.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
