amanraj2520 commented on issue #6212: URL: https://github.com/apache/kyuubi/issues/6212#issuecomment-2024481588
Sure @yaooqinn Will do that. @pan3793 After some debugging found that the SolrZkClient creates threads with daemon as false, that's why they are non-daemon threads  So as per my understanding solr client runs on top of spark driver and when spark context is shutdown, ideally there should not be any solr client thread running right? Do you think making it as daemon as done for the PolicyRefresher thread in Ranger in this - https://issues.apache.org/jira/browse/RANGER-1083 work? I will give it a try and update this thread. So in a nutshell the issue is this that the non-daemon thread of SolrZkClient are not exiting thereby leading to Spark Driver to run even if spark context is shutdown. Ideally this issue will not be for long running spark-shells or livy/zeppelin sessions, since they end by manual intervention of the end user. But in cases of spark-submit where the sparkContext has to run for a definite amount of time, SparkContext needs to have a way to end the SolrZkClient threads. This issue might not have impacted other services of Ranger (like hive, yarn etc) since they are long running sessions as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
