Trevor Christman created SPARK-38861:
----------------------------------------
Summary: GroupState.getCurrentWatermarkMs() on static Dataframe
throws exception but should return -1
Key: SPARK-38861
URL: https://issues.apache.org/jira/browse/SPARK-38861
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.1.2
Reporter: Trevor Christman
As per documentation GroupState.getCurrentWatermarkMs() returns -1 when
operating on a static Dataframe, as found here:
[https://spark.apache.org/docs/3.1.3/api/scala/org/apache/spark/sql/streaming/GroupState.html#getCurrentWatermarkMs():Long]
In reality this function throws UnsupportedOperationException as documented
when operating on a DataFrame without a watermark set. This happens even if
the static DataFrame had a (useless) watermark configured.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
