[GitHub] [spark] gengliangwang commented on a diff in pull request #39385: [SPARK-41882][CORE][SQL][UI] Add tests for `SQLAppStatusStore` with RocksDB backend and fix some bugs
gengliangwang commented on code in PR #39385:
URL: https://github.com/apache/spark/pull/39385#discussion_r1062756776
##
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala:
##
@@ -1007,6 +1004,36 @@ class SQLAppStatusListenerSuite extends
SharedSparkSession with JsonTestUtils
}
}
+class SQLAppStatusListenerWithInMemoryStoreSuite extends
SQLAppStatusListenerSuite {
+ override protected def createStatusStore(): SQLAppStatusStore = {
+val conf = sparkContext.conf
+kvstore = new ElementTrackingStore(new InMemoryStore, conf)
+val listener = new SQLAppStatusListener(conf, kvstore, live = true)
+new SQLAppStatusStore(kvstore, Some(listener))
+ }
+}
+class SQLAppStatusListenerWithRocksDBBackendSuite extends
SQLAppStatusListenerSuite {
+
Review Comment:
It's ok to have this one
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on a diff in pull request #39385: [SPARK-41882][CORE][SQL][UI] Add tests for `SQLAppStatusStore` with RocksDB backend and fix some bugs
gengliangwang commented on code in PR #39385:
URL: https://github.com/apache/spark/pull/39385#discussion_r1062043632
##
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/AllExecutionsPageSuite.scala:
##
@@ -178,3 +177,35 @@ class AllExecutionsPageSuite extends SharedSparkSession
with BeforeAndAfter {
}
}
+class AllExecutionsPageWithInMemoryStoreSuite extends AllExecutionsPageSuite {
+ override protected def createStatusStore: SQLAppStatusStore = {
+val conf = sparkContext.conf
+kvstore = new ElementTrackingStore(new InMemoryStore, conf)
+val listener = new SQLAppStatusListener(conf, kvstore, live = true)
+new SQLAppStatusStore(kvstore, Some(listener))
+ }
+}
+
+class AllExecutionsPageWithRocksDBBackendSuite extends AllExecutionsPageSuite {
+ // TODO: SPARK-41882 remove this field after RocksDB can automatically
cleanup
Review Comment:
FYI I am going to merge https://github.com/apache/spark/pull/39226 right
after the test passes. @LuciferYang please wait until
https://github.com/apache/spark/pull/39226 is merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on a diff in pull request #39385: [SPARK-41882][CORE][SQL][UI] Add tests for `SQLAppStatusStore` with RocksDB backend and fix some bugs
gengliangwang commented on code in PR #39385:
URL: https://github.com/apache/spark/pull/39385#discussion_r1062043632
##
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/AllExecutionsPageSuite.scala:
##
@@ -178,3 +177,35 @@ class AllExecutionsPageSuite extends SharedSparkSession
with BeforeAndAfter {
}
}
+class AllExecutionsPageWithInMemoryStoreSuite extends AllExecutionsPageSuite {
+ override protected def createStatusStore: SQLAppStatusStore = {
+val conf = sparkContext.conf
+kvstore = new ElementTrackingStore(new InMemoryStore, conf)
+val listener = new SQLAppStatusListener(conf, kvstore, live = true)
+new SQLAppStatusStore(kvstore, Some(listener))
+ }
+}
+
+class AllExecutionsPageWithRocksDBBackendSuite extends AllExecutionsPageSuite {
+ // TODO: SPARK-41882 remove this field after RocksDB can automatically
cleanup
Review Comment:
FYI I am going to merge https://github.com/apache/spark/pull/39226 first.
@LuciferYang please wait until https://github.com/apache/spark/pull/39226 is
merged.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on a diff in pull request #39385: [SPARK-41882][CORE][SQL][UI] Add tests for `SQLAppStatusStore` with RocksDB backend and fix some bugs
gengliangwang commented on code in PR #39385:
URL: https://github.com/apache/spark/pull/39385#discussion_r1061839322
##
sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLAppStatusListenerSuite.scala:
##
@@ -1007,6 +1004,36 @@ class SQLAppStatusListenerSuite extends
SharedSparkSession with JsonTestUtils
}
}
+class SQLAppStatusListenerWithInMemoryStoreSuite extends
SQLAppStatusListenerSuite {
+ protected def createStatusStore(): SQLAppStatusStore = {
Review Comment:
```suggestion
override protected def createStatusStore(): SQLAppStatusStore = {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
