wy created SPARK-41683:
--------------------------
Summary: Spark UI: In jobs API, numActiveStages can be negative in
some cases
Key: SPARK-41683
URL: https://issues.apache.org/jira/browse/SPARK-41683
Project: Spark
Issue Type: Bug
Components: Spark Core, Web UI
Affects Versions: 3.3.1
Reporter: wy
In API /api/v1/applications/.../jobs, for each job, there's one property called
numActiveStages, telling how many active stages of the job.
It is calculated by the following code. When stage is submitted,
numActiveStages += 1. When job is ended, for each pending stage,
numActiveStages -= 1.
!image-2022-12-22-15-58-30-833.png!
!image-2022-12-22-15-56-05-110.png!
However, there's possibility that there's no stage submitted event for some
skipped stages. In this case, when job ends, numActiveStages may be minus.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
