Github user zsxwing commented on a diff in the pull request:
https://github.com/apache/spark/pull/21701#discussion_r200761267
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -875,6 +875,16 @@ object SQLConf {
.stringConf
.createWithDefault("org.apache.spark.sql.execution.streaming.ManifestFileCommitProtocol")
+ val STREAMING_MULTIPLE_WATERMARK_POLICY =
+ buildConf("spark.sql.streaming.multipleWatermarkPolicy")
+ .doc("Policy to calculate the global watermark value when there are
multiple watermark " +
+ "operators in a streaming query. The default value is 'min' which
chooses " +
+ "the minimum watermark reported across multiple operators." +
+ "Note: This configuration cannot be changed between query restarts
from the same " +
+ "checkpoint location.")
+ .stringConf
--- End diff --
You can use `checkValue` to add validation for this conf.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
