Yicong-Huang opened a new pull request, #58840:
URL: https://github.com/apache/spark/pull/58840
### What changes were proposed in this pull request?
Example 2 in the `DataFrameReader.json` docstring writes two rows into two
separate directories (`d1`, `d2`) and then reads them back with
`spark.read.json([d1, d2]).show()` without any ordering. This PR adds
`.sort("name")` before `.show()` so the output is deterministic. The expected
output block is unchanged, since `Alice`/`Bob` is already in sorted order.
### Why are the changes needed?
Reading multiple files/directories back produces rows in a nondeterministic
order, so the doctest fails intermittently (~50% of the time). This makes the
`pyspark.sql.connect.readwriter` doctest module (which reuses these docstrings)
flaky. Sorting makes the example deterministic, consistent with Example 4 and
the parquet/text examples in the same file that already call
`.sort(...)`/`.orderBy(...)`.
### Does this PR introduce _any_ user-facing change?
No. Docstring example only.
### How was this patch tested?
Existing doctests. The expected output is unchanged because the sorted order
matches the previously expected rows.
### Was this patch authored or co-authored using generative AI tooling?
No
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
