HeartSaVioR commented on code in PR #42066:
URL: https://github.com/apache/spark/pull/42066#discussion_r1267562812
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -1852,6 +1852,24 @@ object SQLConf {
.createWithDefault(
"org.apache.spark.sql.execution.streaming.state.HDFSBackedStateStoreProvider")
+ val ENABLE_STATE_STORE_MAINTENANCE_THREAD_POOL =
+
buildConf("spark.sql.streaming.stateStore.enableStateStoreMaintenanceThreadPool")
+ .internal()
+ .doc("Whether background maintenance tasks are executed by single thread
or thread pool")
Review Comment:
Shall we provide the context to inform the reason the old mechanism is set
to default? Something along the line (please refine the sentence): the thread
pool is consuming more resource than single thread, which might affect the
performance of the query. Enable this if you observe the issue with the
maintenance task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
