callmepandey opened a new pull request, #53803:
URL: https://github.com/apache/spark/pull/53803
### What changes were proposed in this pull request?
Added `parentIds: collection.Seq[Int]` field to `StageData` class to track
parent stage dependencies. This information was already available in
`StageInfo` but not exposed through the public API.
### Why are the changes needed?
The `parentIds` field is necessary for tracking stage dependencies in the
Spark UI and other status-related tools. External monitoring systems and UIs
can now understand the DAG structure of stages.
### Does this PR introduce any user-facing change?
Yes. The REST API `/api/v1/applications/{appId}/stages` now includes
`parentIds` in the response:
```json
{
"stageId": 5,
"parentIds": [2, 3],
...
}
```
### How was this patch tested?
- Added unit test for `parentIds` serialization/deserialization in
`KVStoreProtobufSerializerSuite`
- Added test case for empty `parentIds` (root stages)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
