[
https://issues.apache.org/jira/browse/HIVE-12274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137015#comment-15137015
]
Elliot West commented on HIVE-12274:
------------------------------------
Hi, I have attached some sample code to reproduce. The snippet below shows the
error we see. Note that the ellipsis are my own addition for brevity:
{code}
hive> CREATE EXTERNAL TABLE IF NOT EXISTS mytests.mybigtable
> (
> bigstruct STRUCT <
> myField1 : STRING,
...
> myField1000 : STRING
> >
> )
> PARTITIONED BY (yymmdd STRING)
> ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.JsonSerde'
> STORED AS SEQUENCEFILE
> LOCATION '/path-to/mybigtable/';
OK
Time taken: 4.86 seconds
hive> desc mytests.mybigtable;
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. Error: type expected at the position
3996 of
'struct<myField1:string,myField2:string,myField3:string,myField4:string,myField5:string,myField6:string,
... ,myField228:stri' but 'stri' is found.
hive>
{code}
> Increase width of columns used for general configuration in the metastore.
> --------------------------------------------------------------------------
>
> Key: HIVE-12274
> URL: https://issues.apache.org/jira/browse/HIVE-12274
> Project: Hive
> Issue Type: Improvement
> Components: Metastore
> Affects Versions: 2.0.0
> Reporter: Elliot West
> Assignee: Sushanth Sowmyan
> Labels: metastore
>
> This issue is very similar in principle to HIVE-1364. We are hitting a limit
> when processing JSON data that has a large nested schema. The struct
> definition is truncated when inserted into the metastore database column
> {{COLUMNS_V2.YPE_NAME}} as it is greater than 4000 characters in length.
> Given that the purpose of these columns is to hold very loosely defined
> configuration values it seems rather limiting to impose such a relatively low
> length bound. One can imagine that valid use cases will arise where
> reasonable parameter/property values exceed the current limit. Can these
> columns not use CLOB-like types as for example as used by
> {{TBLS.VIEW_EXPANDED_TEXT}}? It would seem that suitable type equivalents
> exist for all targeted database platforms:
> * MySQL: {{mediumtext}}
> * Postgres: {{text}}
> * Oracle: {{CLOB}}
> * Derby: {{LONG VARCHAR}}
> I'd suggest that the candidates for type change are:
> * {{COLUMNS_V2.TYPE_NAME}}
> * {{TABLE_PARAMS.PARAM_VALUE}}
> * {{SERDE_PARAMS.PARAM_VALUE}}
> * {{SD_PARAMS.PARAM_VALUE}}
> Finally, will this limitation persist in the work resulting from HIVE-9452?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
