SinghAsDev commented on pull request #4018: URL: https://github.com/apache/iceberg/pull/4018#issuecomment-1028175781
@szehon-ho that is a good point and I was also thinking about it. However, AFAIK, among all the clients Iceberg is used from, only Spark and Hive support external table concept, right? As such, I feel it should be OK to just limit this to Hive catalog. What do you think? > When creating external tables using the Hive engine, Hive appends external.table.purge=true to the table properties behind the scenes in HiveIcebergMetaHook. This goes against user expectations even in Hive 4.0+. I'll prepare a PR to fix that up on the Hive engine side (which should fix your Hive test failures presumably). @marton-bod this is exactly I was planning to do test failures here as well. Let me know if you fix it as a separate PR and I can rebase. > On the HiveCatalog-level, how do we want to handle different combinations of external/managed tables with the gc.enabled flag. I think it makes sense to disallow the MANAGED + gc.enabled=false combination. But I would not disallow the EXTERNAL + gc.enabled=true combination, exactly because Hive 4.0 makes this combination valid via the external.table.purge=true flag @marton-bod for disallowing EXTERNAL + gc.enabled=true, I think it makes sense for Hive 4.0+, but below that I think it is expected to not allow that, right? Maybe we can make this configurable. Would that be OK with you? @szehon-ho @marton-bod @pvary @rdblue I think we are at least aligned on the issue and the need for changing the behavior? If not, please let me know if I can provide more info. Since this change is a behavior change, I am curious is there a set of steps like user communication that we usually do or do we restrict such changes to dot releases? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
