[
https://issues.apache.org/jira/browse/HIVE-9500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14298694#comment-14298694
]
Aihua Xu commented on HIVE-9500:
--------------------------------
[~thejas] Yes, LaxySimpleSerDe is used. This limitation actually applies for
not only avro format. The simple case of column <array<array...<array<int>>...>
with too many levels would fail.
I added a configuration and that approach seems to work. But the better
approach seems to completely remove such limitation since the number of levels
only affects how to choose the separators. Instead of preselecting the list of
separators, we should dynamically generate such list. This way, we can
deprecate the configuration "hive.serialization.extend.nesting.levels" as well.
> Support nested structs over 24 levels.
> --------------------------------------
>
> Key: HIVE-9500
> URL: https://issues.apache.org/jira/browse/HIVE-9500
> Project: Hive
> Issue Type: Improvement
> Reporter: Aihua Xu
> Labels: SerDe
>
> Customer has deeply nested avro structure and is receiving the following
> error when performing queries.
> 15/01/09 20:59:29 ERROR ql.Driver: FAILED: SemanticException
> org.apache.hadoop.hive.serde2.SerDeException: Number of levels of nesting
> supported for LazySimpleSerde is 23 Unable to work with level 24
> Currently we support up to 24 levels of nested structs when
> hive.serialization.extend.nesting.levels is set to true, while the customers
> have the requirement to support more than that.
> It would be better to make the supported levels configurable or completely
> removed (i.e., we can support any number of levels).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
