Jungtaek Lim created SPARK-29450:
------------------------------------
Summary: [SS] In streaming aggregation, metric for output rows is
not measured in append mode
Key: SPARK-29450
URL: https://issues.apache.org/jira/browse/SPARK-29450
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 2.4.4, 3.0.0
Reporter: Jungtaek Lim
The bug is reported in dev. mailing list. Credit to [~jlaskowski]. Quoting here:
{quote}
I've just noticed that the number of output rows metric of StateStoreSaveExec
physical operator does not seem to be measured for Append output mode. In other
words, whatever happens before or after StateStoreSaveExec operator the metric
is always 0.
It is measured for the other modes - Complete and Update.
{quote}
https://lists.apache.org/thread.html/eec318f7ff84c700bffc8bdb7c95c0dcaffb2494ac7ccd7a7c7c6588@%3Cdev.spark.apache.org%3E
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
