[GitHub] [spark] LantaoJin edited a comment on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin edited a comment on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-654836968 AFAIK, hive client (metastore clien) is not thread-safe. I don't find any documentation but I find a patch

[GitHub] [spark] LantaoJin edited a comment on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin edited a comment on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-654796439 I think it's safe to use database level R/W locks here with thread-unsafe hive-client. Use one object lock is over kill in `HiveExternalCatalog`. You never know how

[GitHub] [spark] LantaoJin edited a comment on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin edited a comment on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-650868622 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] LantaoJin edited a comment on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin edited a comment on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-650868622 > Hi, @LantaoJin . Since this PR aims to use multiple locks instead of a global one, could you add some explanation briefly about how this PR is deadlock-free? Thanks.

[GitHub] [spark] LantaoJin edited a comment on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin edited a comment on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-654796439 I think it's safe to use database level R/W locks here with thread-unsafe hive-client. Use one object lock is over kill in `HiveExternalCatalog`. You never know how

[GitHub] [spark] LantaoJin edited a comment on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin edited a comment on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-650868622 > Hi, @LantaoJin . Since this PR aims to use multiple locks instead of a global one, could you add some explanation briefly about how this PR is deadlock-free? Thanks.

[GitHub] [spark] LantaoJin edited a comment on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin edited a comment on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-650868622 > Hi, @LantaoJin . Since this PR aims to use multiple locks instead of a global one, could you add some explanation briefly about how this PR is deadlock-free? Thanks.

[GitHub] [spark] LantaoJin edited a comment on pull request #28938: [SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog

LantaoJin edited a comment on pull request #28938: URL: https://github.com/apache/spark/pull/28938#issuecomment-650868622 > Hi, @LantaoJin . Since this PR aims to use multiple locks instead of a global one, could you add some explanation briefly about how this PR is deadlock-free? Thanks.