I hope it’s not used for anything cause otherwise it would be impossible to support these kinds of schemas without painful metastore schema changes to use CLOB/varchar(MAX) and hoping DN supports it well :)
On 15/9/28, 19:38, "kulkarni.swar...@gmail.com" <kulkarni.swar...@gmail.com> wrote: >Sergey, > >Is your table a partitioned or a non-partitioned one? I have usually seen >this problem manifest itself for partitioned tables and that is mostly >where the pruning bites. So if you now try to add a partition to this >table, you might see an exception like: > >java.sql.BatchUpdateException: Data truncation: Data too long for column >'TYPE_NAME' at row 1) > >The "TYPE_NAME" is not actually a definition of the Avro schema. Instead, >it is a definition of the type structure in Hive terms. I assume it is >used for things such as validating the query before it is executed, etc. > >On Mon, Sep 28, 2015 at 7:38 PM, Chaoyu Tang <ctang...@gmail.com> wrote: > >> Yes, when you described the avro table, what you get back was actually >>from >> your avro schema instead of database table. The avro table is NOT >> considered as a metastore backed SerDe. But that it has its columns >> populated to DB (e.g. HIVE-6308 >> <https://issues.apache.org/jira/browse/HIVE-6308>) is mainly for column >> statistics purpose, which obviously is not applicable to your case which >> has a type name > 100kb. >> >> Chaoyu >> >> On Mon, Sep 28, 2015 at 8:12 PM, Sergey Shelukhin >><ser...@hortonworks.com> >> wrote: >> >> > Hi. >> > I noticed that when I create an Avro table using a very large schema >> file, >> > mysql metastore silently truncates the TYPE_NAME in COLUMNS_V2 table >>to >> > the size of varchar (4000); however, when I do describe on the table, >>it >> > still displays the whole type name (around 100Kb long) that I presume >>it >> > gets from deserializer. >> > Is the value in TYPE_NAME used for anything for Avro tables? >> > >> > >> > > > >-- >Swarnim
