[
https://issues.apache.org/jira/browse/SPARK-58059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-58059.
----------------------------------
Resolution: Not A Problem
> Reduce runtime of pyspark.pandas.frame doctests
> -----------------------------------------------
>
> Key: SPARK-58059
> URL: https://issues.apache.org/jira/browse/SPARK-58059
> Project: Spark
> Issue Type: Test
> Components: PySpark
> Affects Versions: 5.0.0
> Reporter: Hyukjin Kwon
> Priority: Major
>
> pyspark.pandas.frame runs as a single doctest module with ~800 doctest
> statements spanning the public DataFrame API, making it one of the slower
> PySpark test targets. Unlike class-based test suites, a doctest module cannot
> be split by simply moving test classes, and bulk-skipping public-API doctests
> would reduce user-facing documentation coverage.
> This is a follow-up to SPARK-58058 (which split several class-based PySpark
> suites). The goal here is to reduce the wall-clock runtime of the
> pyspark.pandas.frame doctests without losing meaningful coverage, e.g. by
> relocating the heaviest doctests into dedicated test cases that can run in
> parallel, or trimming redundant/expensive doctest examples where appropriate.
> Approach to be discussed given the doc-coverage tradeoff.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
