baibaichen commented on pull request #34292: URL: https://github.com/apache/spark/pull/34292#issuecomment-946361281
@HyukjinKwon Reducing the scope of a synchronized block is always a good idea. I don't understand your question, do you mean to add a test to show the deadlock? As @cloud-fan said the current Apache Spark can't hit this deadlock, so the current unit test is enough. In our case, we have implemented an external catalog, and there is a thread that directly calls `SessionCatalog.getTempViewOrPermanentTableMetadata` and holds the lock of `SessionCatalog`. It eventually goes into our external catalog, unfortunately, we then call some functions of `SparkSession`, e.g. `sql`. When it calls `CatalogManager.currentNamespace`, it tries to hold the lock of `CatalogManager`. In the meantime, there are some query threads that execute sqls via `DataFrame` interface. This is how deadlock occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
