Jungtaek Lim created SPARK-43161:
------------------------------------
Summary: Throw error in planning phase when the query redefines
watermark for a stream
Key: SPARK-43161
URL: https://issues.apache.org/jira/browse/SPARK-43161
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.5.0
Reporter: Jungtaek Lim
With SPARK-42376, we started to disallow redefinition of watermark in a single
stream. But the check is actually performed in watermark propagator, which does
not throw an exception in the batch 0.
Since we tend to give a fast feedback for improper usage, throwing exception
should also happen in planning phase, ideally in UnsupportedOperationChecker.
In addition, we don't seem to document this restriction. Also update the
structured streaming doc.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
