[
https://issues.apache.org/jira/browse/SPARK-43973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang reassigned SPARK-43973:
--------------------------------------
Assignee: Kris Mok
> Structured Streaming UI should display failed queries correctly
> ---------------------------------------------------------------
>
> Key: SPARK-43973
> URL: https://issues.apache.org/jira/browse/SPARK-43973
> Project: Spark
> Issue Type: Bug
> Components: Web UI
> Affects Versions: 3.1.0, 3.2.0, 3.3.0, 3.4.0
> Reporter: Kris Mok
> Assignee: Kris Mok
> Priority: Major
>
> The Structured Streaming UI is designed to be able to show a query's status
> (active/finished/failed) and if failed, the error message.
> Due to a bug in the implementation, the error message in
> {{QueryTerminatedEvent}} isn't being tracked by the UI data, so in turn the
> UI always shows failed queries as "finished".
> Example:
> {code:scala}
> implicit val ctx = spark.sqlContext
> import org.apache.spark.sql.execution.streaming.MemoryStream
> spark.conf.set("spark.sql.ansi.enabled", "true")
> val inputData = MemoryStream[(Int, Int)]
> val df = inputData.toDF().selectExpr("_1 / _2 as a")
> inputData.addData((1, 2), (3, 4), (5, 6), (7, 0))
> val testQuery =
> df.writeStream.format("memory").queryName("kristest").outputMode("append").start
> testQuery.processAllAvailable()
> {code}
> Here we intentionally fail a query, but the Spark UI's Structured Streaming
> tab would show this as "FINISHED" without any errors, which is wrong.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
