uros-b commented on code in PR #56675:
URL: https://github.com/apache/spark/pull/56675#discussion_r3462735311
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -3138,8 +3138,9 @@ object SQLConf {
.createWithDefault(false)
val MIN_BATCHES_TO_RETAIN =
buildConf("spark.sql.streaming.minBatchesToRetain")
- .internal()
Review Comment:
Minor note regarding the config value: there is currently no `.checkValue(_
> 0)` guard on the config. Positivity is enforced only downstream by
`require(minLogEntriesToMaintain > 0, "minBatchesToRetain has to be positive")`
at StreamExecution.scala:91, so a non-positive value fails late with a
require-style error rather than at config-set time. Since this PR is now
promoting the config to public, adding a `.checkValue` would be a reasonable
accompanying hardening.
From the doc perspective, we seem to omit the positive-value constraint
there too. Now that the config is public, a one-line note that the value must
be positive would help users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
