This is an automated email from the ASF dual-hosted git repository.
ulyssesyou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new c1e5545 [KYUUBI #1043] Let spark history logger handle events
asynchronously
c1e5545 is described below
commit c1e5545c648fbd9d62c562014447eea205849094
Author: timothy65535 <[email protected]>
AuthorDate: Tue Sep 7 09:46:53 2021 +0800
[KYUUBI #1043] Let spark history logger handle events asynchronously
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in
https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your
PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
> there seems to be no good way to solve the synchronization problem.
For `EventLoggerType.SPARK`
This looks simple, Please send a separate PR to solve this issue
```git
- sc.eventLogger.foreach(_.onOtherEvent(kyuubiEvent))
+ sc.listenerBus.post(kyuubiEvent)
```
_Originally posted by yaooqinn in
https://github.com/apache/incubator-kyuubi/issues/1023#issuecomment-913558016_
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run
test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests)
locally before make a pull request
Closes #1044 from timothy65535/1043.
Closes #1043
0ea1d4f7 [timothy65535] [KYUUBI #1043] Let spark history logger handle
events asynchronously
Authored-by: timothy65535 <[email protected]>
Signed-off-by: ulysses-you <[email protected]>
---
.../src/main/scala/org/apache/spark/kyuubi/SparkContextHelper.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/SparkContextHelper.scala
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/SparkContextHelper.scala
index dd10718..667bd98 100644
---
a/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/SparkContextHelper.scala
+++
b/externals/kyuubi-spark-sql-engine/src/main/scala/org/apache/spark/kyuubi/SparkContextHelper.scala
@@ -38,6 +38,6 @@ object SparkContextHelper {
*/
private class SparkHistoryEventLogger(sc: SparkContext) extends
EventLogger[KyuubiSparkEvent] {
override def logEvent(kyuubiEvent: KyuubiSparkEvent): Unit = {
- sc.eventLogger.foreach(_.onOtherEvent(kyuubiEvent))
+ sc.listenerBus.post(kyuubiEvent)
}
}
