zhengruifeng opened a new pull request, #57154: URL: https://github.com/apache/spark/pull/57154
### What changes were proposed in this pull request? Add a `## Python Test Base Classes` section to `AGENTS.md` (`CLAUDE.md` is a symlink to it), right after the existing `## Scala Test Base Classes` section and mirroring its structure: a short intro, an ASCII "adds capability on top of the previous" ladder, and a scope/base/notes table. It covers: - the classic ladder `unittest.TestCase` -> `PySparkBaseTestCase` -> `ReusedPySparkTestCase` -> `ReusedSQLTestCase` -> `PandasOnSparkTestCase`, including the fresh-vs-shared `SparkContext` distinction (`PySparkTestCase` vs `ReusedPySparkTestCase`); - a `### Spark Connect test bases` subsection for `PlanOnlyTestFixture`, `ReusedConnectTestCase`, and `ReusedMixedTestCase`, noting the `should_test_connect` auto-skip; - a `### Mixins and helpers` subsection for `SQLTestUtils`, `PySparkErrorTestUtils`, the standalone `assertDataFrameEqual` / `assertSchemaEqual`, and the off-ladder domain bases (`SparkSessionTestCase`, `MLlibTestCase`, `PySparkStreamingTestCase`). ### Why are the changes needed? `AGENTS.md` already guides contributors (and AI agents) to pick the lowest suitable Scala test base class, but has no equivalent for PySpark. The Python test bases under `python/pyspark/testing/` form a similar ladder that is easy to get wrong (e.g. subclassing a heavier base than needed, or missing the Connect-specific bases). Documenting them fills the adjacent gap. ### Does this PR introduce _any_ user-facing change? No. Documentation only. ### How was this patch tested? No code changes. Verified the described class hierarchy and capabilities against the sources under `python/pyspark/testing/` (`utils.py`, `sqlutils.py`, `connectutils.py`, `pandasutils.py`, `mlutils.py`, `mllibutils.py`, `streamingutils.py`), and spot-checked that only em-dashes (U+2014) were added as non-ASCII, consistent with the rest of the file. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code (Opus 4.8) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
