jarredhj0214 opened a new issue, #12038:
URL: https://github.com/apache/gravitino/issues/12038
### Version
main branch
### Describe what's wrong
When importing an external table, Gravitino stores column metadata in
`table_column_version_info`.
The `column_comment` column is currently defined as `VARCHAR(256)` in the
relational store schema. If an external table contains a column comment longer
than 256 characters, the import fails with a MySQL data truncation error.
Column comments are part of table schema metadata and may come from external
systems where the comment length is not limited to 256 characters. The current
limit is too restrictive and can block table import.
### Error message and/or stacktrace
Caused by: java.io.IOException:
com.mysql.cj.jdbc.exceptions.MysqlDataTruncation: Data truncation: Data too
long for column 'column_comment' at row 29
at
org.apache.gravitino.storage.relational.converters.MySQLExceptionConverter.toGravitinoException(MySQLExceptionConverter.java:44)
at
org.apache.gravitino.storage.relational.utils.ExceptionUtils.checkSQLException(ExceptionUtils.java:33)
at
org.apache.gravitino.storage.relational.service.TableMetaService.insertTable_aroundBody6(TableMetaService.java:153)
at
org.apache.gravitino.storage.relational.service.TableMetaService.insertTable(TableMetaService.java:1)
at
org.apache.gravitino.storage.relational.JDBCBackend.insert(JDBCBackend.java:185)
at
org.apache.gravitino.storage.relational.RelationalEntityStore.put(RelationalEntityStore.java:151)
at
org.apache.gravitino.catalog.TableOperationDispatcher.importTable(TableOperationDispatcher.java:521)
### How to reproduce
Use Gravitino with MySQL as the relational store.
Prepare an external table that contains at least one column whose comment is
longer than 256 characters.
Import the table into Gravitino.
The import fails when inserting column metadata into
table_column_version_info.
The related schema definition is:
`column_comment` VARCHAR(256) DEFAULT '' COMMENT 'column comment'
### Additional context
table_column_version_info is used to store versioned column metadata and
reconstruct table schema snapshots by table version. Therefore, column_comment
is necessary in this table, but the current VARCHAR(256) length is too small
for imported external metadata.
A possible fix is to increase the column length, for example to
VARCHAR(4096), or change it to TEXT. The change should be applied consistently
to MySQL, PostgreSQL, and H2 schema files and upgrade scripts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
