wypoon commented on pull request #4017: URL: https://github.com/apache/iceberg/pull/4017#issuecomment-1028233193
@rdblue it is true that if your Hadoop `Configuration` has iceberg.engine.hive.enabled set to true for all your writers, this is not an issue. I'd argue that as a matter of logical consistency, the table property should be set if the table is created as Hive-enabled, so that a future writer (whose configuration we may not be in control of) that updates the table does not drop that Hive-enabledness. [In `HiveTableOperations#doCommit`, we check if `hiveEngineEnabled` by checking if "engine.hive.enabled" is in the `TableMetadata`'s properties and using its value if so, else use the value of "iceberg.engine.hive.enabled" in the Hadoop `Configuration`, **defaulting to false**. Thus there are two routes to `hiveEngineEnabled` being true, because "engine.hive.enabled" is in the table properties, or because of the Hadoop `Configuration`. What I'm proposing is that we always then encode it in the table properties, so future commits will keep the table being Hive-enabled, regardless of the writer's Hadoop `Configuration`. Otherwise, `hiveEngineEnabled` could be true due to the `Configuration`, and not write the property to `TableMetadata`, and another writer comes along with a different `Configuration`, and `hiveEngineEnabled` is now false from the `TableMetadata`.] -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
