jackye1995 opened a new issue #3325: URL: https://github.com/apache/iceberg/issues/3325
In #3048 we added a copy of Iceberg schema to Glue, with information in the comment section for each column. In fact, Glue supports column parameters (https://docs.aws.amazon.com/glue/latest/webapi/API_Column.html). Instead of description, here is a proposal to categorize information into the following column parameters for better information presentation purpose: - `iceberg.content`, can be `column-root`, `column-nested` or `partition-field` - `iceberg.field.type.id`, storing the Iceberg `TypeID`, such as `MAP` - `iceberg.field.type.string`, string representation of the type, such as `map<string,string>` - `iceberg.field.id`, storing integer ID of the field - `iceberg.field.optional`, `true` if optional otherwise `false` - `iceberg.partition.transform`, storing partition transform string for `partition-field`, such as `bucket[16]` - `iceberg.partition.field-id`, storing field ID of the partition field - `iceberg.partition.source-id`, storing source ID of the partition field The column `comment` field should just store the `doc` of the field. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
