[
https://issues.apache.org/jira/browse/HIVE-12299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981697#comment-14981697
]
Sergey Shelukhin commented on HIVE-12299:
-----------------------------------------
You can check HIVE-11985. Are you using something similar, like Avro, or
creating this struct manually? The root cause is DB engine limitations,
metastore currently stores data in RDBMS and 4000 is the lowest common
denominator for maximum varchar length.
However, for the external schema use-case, HIVE-11985 will eliminate the
metastore storage requirement.
> Hive Column Data Type definition in schema limited to 4000 characters - too
> small
> ---------------------------------------------------------------------------------
>
> Key: HIVE-12299
> URL: https://issues.apache.org/jira/browse/HIVE-12299
> Project: Hive
> Issue Type: Improvement
> Components: Hive
> Reporter: Lakshmi Ramakrishnan
>
> The data type definitions in the table schema are limited to 4K characters -
> as per the code here:
> https://github.com/apache/hive/tree/master/metastore/scripts/upgrade/derby
> I checked quickly and all meta stores have similar schema definition for the
> column type.
> Is there any reason why this limit is low? We had a table that had defined a
> struct, which had over 200 columns and the column names were rather verbose
> (for readability). This caused a non-obvious failure like
> FAILED: IllegalArgumentException Error: : expected at the end of
> 'string:array<struct<xyz:string, .....'. Is it possible to make this
> configurable or at least increase it to something much higher?
> Additionally, there is no validation error that communicates this limitation
> to the user, it required non-trivial debugging and looking into the table
> definitions when it failed trying to parse what was essentially a truncated
> type.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
