okumin commented on PR #4444: URL: https://github.com/apache/hive/pull/4444#issuecomment-1771157733
@dengzhhu653 Thanks for the meaningful insight! > 1. Can we just expose SessionHiveMetaStoreClient in the Hive class? > 2. This makes sense to me, the SessionHiveMetastoreClient acts as a wrapper for the IMetaStoreClient implementation If I understand correctly, it is a problem resolved by [HIVE-27473](https://issues.apache.org/jira/browse/HIVE-27473). I agree users might want to use adaptors to add some capabilities. In addition, they can be optional in my opinion. I mean if a user wants to use the convenient adaptors, they can use it in the following way. ``` IMetaStoreClient customMetaStoreClient = ...; IMetaStoreClient metaStoreClientWithCache = new MetaStoreClientWithCache(customMetaStoreClient); IMetaStoreClient metaStoreClientWithTmpTable = new MetaStoreClientWithTmpTable(metaStoreClientWithCache); ``` If the user doesn't need any or all of the additional capabilities or features, they don't have to wrap their client. Everything, e.g. what to support or how to implement, is up to the user as long as their plugin is integrated with Hive via the primitive interface, IMetaStoreClient. I expect this doesn't sacrifice convenience so much, keeping things explicit and flexible. > 3. IDataConnectorProvider in HMS can plugin different datasources nowadays It seems to be a really lovely feature. I also guess it may not fit into our use case. That's because we would need to verify the security, availability, and performance of both HMS and our metadata service. To be fair, I am very confident with the capabilities of HMS. But we still always need to keep all endpoints well-tested. @zabetak > What is a bit worrisome is that there are lots of calls to RetryingMetaStoreClient.getProxy methods outside the Hive class itself. It seems that the factory will not have any effect on those. Isn't this a problem? I didn't notice that point, and I am listing use cases... - RetryingMetaStoreClient.getProxy - UpgradeTool - HiveClientCache of HCatalog - Constructor of HiveMetaStoreClient, including inheritance - HiveClientCache - MsckOperation & Msck - HiveStrictManagedMigration - PartitionManagementTask - SmokeTest - Constructor of SessionHiveMetaStoreClient - Hive.java Some might not be needed for a 3rd party metastore but I feel some could be valuable. Now, I guess it could be better to think of usages outside Hive.java, too. I can PoC once we can agree with the gluing API... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org For additional commands, e-mail: gitbox-h...@hive.apache.org
