dongjoon-hyun commented on a change in pull request #25333: [SPARK-28597][SS]
Add config to retry spark streaming's meta log when it met error
URL: https://github.com/apache/spark/pull/25333#discussion_r313162619
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -312,6 +312,13 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val STREAMING_META_DATA_NUM_RETRIES =
buildConf("spark.sql.streaming.metadata.num.retries")
+ .doc("The number of meta data log retries.")
+ .intConf
+ .checkValue(_ > 0, "The minimum number of metadata retries " +
+ "must be a positive integer.")
+ .createWithDefault(3)
+
Review comment:
Could you put this near the other `spark.sql.streaming.XXX` configuration in
this file?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
