[
https://issues.apache.org/jira/browse/HIVE-25553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17419620#comment-17419620
]
Adesh Kumar Rao commented on HIVE-25553:
----------------------------------------
[~pvary] [~kgyrtkirk] [~ShubhamChaurasia] These changes are backward
incompatible (not using list to store map).
But since this is being used internally by llap (and creating hive tables with
arrow format is not supported?), it should not cause any issues.
Let me know if you have any concerns.
> Support Map data-type in Arrow format
> -------------------------------------
>
> Key: HIVE-25553
> URL: https://issues.apache.org/jira/browse/HIVE-25553
> Project: Hive
> Issue Type: Improvement
> Components: llap, Serializers/Deserializers
> Reporter: Adesh Kumar Rao
> Assignee: Adesh Kumar Rao
> Priority: Major
> Fix For: 4.0.0
>
>
> Currently ArrowColumnarBatchSerDe converts map datatype as a list of structs
> data-type (where stuct is containing the key-value pair of the map). This
> causes issues when reading Map datatype using llap-ext-client as it reads a
> list of structs instead.
> HiveWarehouseConnector which uses the llap-ext-client throws exception when
> the schema (containing Map data type) is different from actual data (list of
> structs).
>
> Fixing this issue requires upgrading arrow version (where map data-type is
> supported), modifying ArrowColumnarBatchSerDe and corresponding
> Serializer/Deserializer to not use list as a workaround for map and use the
> arrow map data-type instead.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
