electrum commented on issue #2068: URL: https://github.com/apache/iceberg/issues/2068#issuecomment-759169871
Another concern is that the migrated files will violate the Iceberg specification, since they won't have the required Iceberg column IDs, and may also be missing the identity partitioning columns. If migrated files are officially supported in Iceberg, then exactly what is allowed and how to read them should be covered by the specification. We have this problem today for Hive migrated tables. They are, at least in theory, supported in Trino (formerly PrestoSQL), but as second class citizen, since there are no tests or specification to know it's correct. I only know about this because @Parth-Brahmbhatt originally wrote the connector based on Netflix's internal usage and told me about how it's supposed to work. Any future Iceberg implementations are likely to miss this detail. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
