wypoon commented on PR #4793: URL: https://github.com/apache/hive/pull/4793#issuecomment-1768742869
Maybe I'm missing something or my understanding is incorrect, but I don't see how this change enables Hive to read Iceberg tables written by other engines that do not set engine.hive.enabled to true. Other engines that use `HiveCatalog` use `HiveTableOperations` from the Iceberg project, not the one here, so if they don't set engine.hive.enabled to true, then the Hive client does not set the storage handler and the SerDe, InputFormat and OutputFormat to the ones used by Hive for Iceberg tables. Unless there is separate change that enables Hive to read Iceberg tables without relying on those things being set correctly in the HMS. To @pvary's comment, I am not sure if Spark needs to have a jar with HiveIcebergStorageHandler, HiveIcebergSerDe, etc in its classpath when its Hive client makes requests to the HMS on an Iceberg table. In Cloudera's platform, we put the Iceberg Hive runtime jar in Spark's classpath, so it in fact does have those classes in its classpath (but obviously that's not necessarily true for other vendors/platforms). However, we didn't used to do that in earlier days, and Spark was still able to work with Iceberg tables with engine.hive.enabled set to true then (the one problem I recall is with DROP DATABASE ... CASCADE, which fails for some reason). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
