Xinyao Zhang created SPARK-60536:
------------------------------------
Summary: Investigate JSON suite task-completion timeouts with
RocksDB as the UI backend
Key: SPARK-60536
URL: https://issues.apache.org/jira/browse/SPARK-60536
Project: Spark
Issue Type: Sub-task
Components: SQL
Affects Versions: 5.0.0
Reporter: Xinyao Zhang
Parent: SPARK-60531.
The RocksDB-as-UI-backend master run on 2026-10-23 reports 63 failed tests
across JsonV2Suite (16), JsonUnsafeRowSuite (16), JsonV1Suite (16), and
JsonLegacyTimeParserSuite (15). The workflow uses JDK 17, Scala 2.13, and
LIVE_UI_LOCAL_STORE_DIR=/tmp/kvStore.
The repeated failure is a 10-second timeout in QueryTest.waitForTasksToFinish,
reported at QueryTest.scala:634 in that run:
"Last failure message: 1 did not equal 0."
This helper checks that the sum of numRunningTasks reported by
statusTracker.getExecutorInfos is zero. Failures affect unrelated JSON cases,
including MapType handling and date/timestamp writes, so this subtask groups
the shared task-completion symptom.
Evidence: https://github.com/apache/spark/actions/runs/39910364472
Tested commit: b19dd54a05291bb5eaa012e7622289edf6807768.
Investigate whether a task remains active or task-status reporting remains
stale with this backend, and compare with the default UI backend. Local
reproduction, root cause, and production impact are not yet established.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
