[
https://issues.apache.org/jira/browse/RANGER-5012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jirar updated RANGER-5012:
--------------------------
Description:
In our environment, we use Kyuubi Spark AuthZ Plugin as a ranger plugin for
spark, enable elasticsearch audit as well.After we submit spark task by
spark-submit, the spark task would not exit even if the task is already
finished.
As we can know that spark driver main thread is finished, but there are some
threads that were not daemon are still running, the JVM for spark driver cannot
invoke shutdown hook, the JVM would exist all the way.
In the jstack information, we get these threads is still in runnable status, as
we expect, they are not daemon threads
!image-2024-11-29-13-27-33-947.png!
finally we find these threads were created by ElasticSearchAuditDestination
Solution: set thread as daemon in RestClientBuilder, it is in
ranger-plugins-audit
was:
In our environment, we use Kyuubi Spark AuthZ Plugin as a ranger plugin for
spark, enable elasticsearch audit as well.After we submit spark task by
spark-submit, the spark task would not exit even if the task is already
finished.
As we can get that spark driver main thread is finished, but there are some
threads that were not daemon are still running, the JVM for spark driver cannot
invoke shutdown hook, the JVM would exist all the way.
In the jstack information, we get these threads is still in runnable status, as
we expect, they are not daemon threads
!image-2024-11-29-13-27-33-947.png!
finally we find these threads were created by ElasticSearchAuditDestination
> Elasticsearch Audit Thread may cause main thread stucked
> --------------------------------------------------------
>
> Key: RANGER-5012
> URL: https://issues.apache.org/jira/browse/RANGER-5012
> Project: Ranger
> Issue Type: Bug
> Components: audit
> Affects Versions: 2.3.0
> Reporter: jirar
> Priority: Major
> Labels: patch
> Fix For: 2.3.0
>
> Attachments: image-2024-11-29-13-27-33-947.png
>
>
> In our environment, we use Kyuubi Spark AuthZ Plugin as a ranger plugin for
> spark, enable elasticsearch audit as well.After we submit spark task by
> spark-submit, the spark task would not exit even if the task is already
> finished.
> As we can know that spark driver main thread is finished, but there are some
> threads that were not daemon are still running, the JVM for spark driver
> cannot invoke shutdown hook, the JVM would exist all the way.
> In the jstack information, we get these threads is still in runnable status,
> as we expect, they are not daemon threads
> !image-2024-11-29-13-27-33-947.png!
> finally we find these threads were created by ElasticSearchAuditDestination
> Solution: set thread as daemon in RestClientBuilder, it is in
> ranger-plugins-audit
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
