jerryshao commented on issue #370: Iceberg failed to work with embedded metastore, which is by default in Spark URL: https://github.com/apache/incubator-iceberg/issues/370#issuecomment-521917078 After spending a bit time on this issue, seems this problem is more complicated than I thought, it is not just "HIVE_LOCKS" table doesn't exist. A more severe problem is that embedded metastore only allows one Hive client, if Spark HiveExternalCatalog created hive client, then Iceberg cannot create another, vice versa. A simple fix would be to reject using Iceberg datasource when Spark is configured with embedded metastore. A more complicated fix would be to share the same hive client with Spark, which requires some reflections to get hive client object from Spark. @rdblue what's your opinion on it? I would suggest to use a simple solution.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
