[
https://issues.apache.org/jira/browse/HIVE-30166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-30166:
----------------------------------
Summary: Iceberg: report VARIANT columns as VARIANT to clients that declare
VARIANT support (was: Iceberg: report VARIANT instead of
struct<metadata:binary,value:binary> as the result type of variant columns )
> Iceberg: report VARIANT columns as VARIANT to clients that declare VARIANT
> support
> ----------------------------------------------------------------------------------
>
> Key: HIVE-30166
> URL: https://issues.apache.org/jira/browse/HIVE-30166
> Project: Hive
> Issue Type: Sub-task
> Reporter: Denys Kuzmenko
> Priority: Major
>
> Since HIVE-29770, {{select v}} returns a variant as JSON, but the column type
> reported for it is still Hive's internal struct:
>
> {code:sql}
>
>
> create table t (v variant) stored by iceberg tblproperties
> ('format-version'='3');
>
> insert into t values (parse_json('{"name":"John","age":30}'));
>
>
> select v from t; -- {"age":30,"name":"John"}
>
>
> select typeof(v) from t; -- struct<metadata:binary,value:binary>
>
>
> {code}
>
>
> For {{select v}}, JDBC {{ResultSetMetaData}} returns {{getColumnTypeName}}
> = struct and {{getColumnType}} = {{Types.STRUCT}}. This does not depend on
> {{hive.serdes.using.metastore.for.schema}}: the result is the same with
> {{HiveIcebergSerDe}} in the list.
>
> *Cause:* {{VariantObjectInspector extends StandardStructObjectInspector}}, so
> it inherits {{Category.STRUCT}} and the struct type name. The planner types
> table columns from the SerDe ObjectInspector, so {{variant}} ->
> {{VariantTypeInfo}} -> {{VariantObjectInspector}} -> {{StructTypeInfo}}.
> {{VariantTypeInfo}} and {{Category.VARIANT}} exist (HIVE-29184), but the
> inspector
> never reports them.
>
>
>
>
>
> *Compatibility:* once the result schema says VARIANT, HS2 sends
> {{TTypeId.VARIANT_TYPE}} (24, added by HIVE-29770) in GetResultSetMetadata.
> Every released driver fails to decode an unknown type id
> ({{TProtocolException: Required field 'type' is unset!}}). So HS2 must report
> VARIANT only to sessions that negotiated {{HIVE_CLI_SERVICE_PROTOCOL_V12}}
> and keep reporting struct to older ones. Today the highest server-side gate
> is V9 (serialize-in-tasks).
>
>
>
>
> Changing the inspector's category or type name affects every consumer that
> handles it as a struct. That includes the HIVE-29770 JSON decode in
> {{SerDeUtils}}, which recognises a variant by the struct type name.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
