[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-712658826 Hi @cloud-fan , in the previous discussion, we are confused about whether or not the hive client is thread-safe. Actually, I missed a change in HiveClientImpl -- changing the

[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-709898787 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-709796207 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-709727243 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-654836968 In my opinion, hive client (metastore clien) is not thread-safe. I don't find any documentation but I find a patch

[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-654796439 I think it's safe to use database level R/W locks here with thread-unsafe hive-client. Use one object lock is over kill in `HiveExternalCatalog`. You never know how slow a

[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-654693688 Gentle ping @dongjoon-hyun @cloud-fan @wangyum @rxin @maropu This is an automated message from the Apache

[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-651006218 > Please put that on the PR description. That will be a commit log permanantly. Updated. This is an

[GitHub] [spark] LantaoJin commented on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin commented on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-650868622 > Hi, @LantaoJin . Since this PR aims to use multiple locks instead of a global one, could you add some explanation briefly about how this PR is deadlock-free? Thanks.