This is an automated email from the ASF dual-hosted git repository.

gaogaotiantian pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-4.x by this push:
     new 82177686fa4e [SPARK-60139][PYTHON] Remove dead code 
UNSUPPORTED_COMBINATIONS
82177686fa4e is described below

commit 82177686fa4e0e3c3fbf3ddfe98a7aa0c1863141
Author: Tim Hsiung <[email protected]>
AuthorDate: Thu Oct 29 15:14:58 2026 -0700

    [SPARK-60139][PYTHON] Remove dead code UNSUPPORTED_COMBINATIONS
    
    ### What changes were proposed in this pull request?
    
    Remove the unused `UNSUPPORTED_COMBINATIONS` constant from 
`python/pyspark/install.py`, together with its import and dead loop in 
`test_checked_versions`.
    
    This supersedes the original annotation-only approach. The existing 
supported-version checks and explicit test rejecting Hive 1.2 remain unchanged.
    
    ### Why are the changes needed?
    
    History investigation shows that this constant has been unused by 
production code since September 2020:
    
    - **September 23, 2020:** [SPARK-32017, commit 
942f577b6e3](https://github.com/apache/spark/commit/942f577b6e34a4f24203dfcad53f5fb98232a6e4)
 introduced the constant to reject Hive 1.2 paired with either `without-hadoop` 
or Hadoop 3.2.
    - **September 24, 2020:** [SPARK-32982, commit 
688d016c7ac](https://github.com/apache/spark/commit/688d016c7acc4b9d96d75b40123be9f40b7b2693)
 removed Hive 1.2 support, emptied the list, and deleted the production 
membership check in `checked_versions`. The empty declaration and test loop 
were left behind.
    - **September 27, 2020:** [SPARK-32714, commit 
c65b64552f9](https://github.com/apache/spark/commit/c65b64552f947a7eaf4f379edbdce05daa923363)
 added `# type: ignore` to silence mypy's missing-annotation error for the 
empty list.
    
    Before this cleanup, a repository-wide search found only the declaration, 
the test import, and the test loop. The loop executes zero iterations because 
the list is empty. Removing these leftovers is preferable to adding a type 
annotation to dead code.
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    The following focused tests passed both before and after removal (2 tests):
    
    ```bash
    source .venv/bin/activate
    PYTHONPATH="$PWD/python:$PWD/python/lib/py4j-0.10.9.9-src.zip" python -m 
unittest \
      
pyspark.tests.test_install_spark.SparkInstallationTestCase.test_checked_versions
 \
      
pyspark.tests.test_install_spark.SparkInstallationTestCase.test_package_name
    ```
    
    Additional checks passed:
    
    - Ruff lint and format checks on both changed files.
    - Standalone mypy checking of the full `install.py` source using 
`python/mypy.ini`, without importing the parent PySpark package.
    - `git diff --check`.
    - Search confirmed no remaining `UNSUPPORTED_COMBINATIONS` references under 
`python`.
    
    The network-dependent installation/download test was not run. An earlier 
package-aware mypy invocation encountered 23 errors elsewhere in PySpark; this 
is not a claim that the full package type check passes.
    
    ### Was this patch authored or co-authored using generative AI tooling?
    
    Generated-by: pi 1.1.0
    
    Closes #59343 from bearomorphism/fix-unsupported-combinations-typing.
    
    Authored-by: Tim Hsiung <[email protected]>
    Signed-off-by: Tian Gao <[email protected]>
    (cherry picked from commit 3877d03b697335319bd99e8c47ce9e6c8a02daad)
    Signed-off-by: Tian Gao <[email protected]>
---
 python/pyspark/install.py                  | 1 -
 python/pyspark/tests/test_install_spark.py | 9 ---------
 2 files changed, 10 deletions(-)

diff --git a/python/pyspark/install.py b/python/pyspark/install.py
index c2ae38e05c14..4ed4d578eeb0 100644
--- a/python/pyspark/install.py
+++ b/python/pyspark/install.py
@@ -35,7 +35,6 @@ DEFAULT_HADOOP = "hadoop3"
 DEFAULT_HIVE = "hive2.3"
 SUPPORTED_HADOOP_VERSIONS = ["hadoop3", "without-hadoop"]
 SUPPORTED_HIVE_VERSIONS = ["hive2.3"]
-UNSUPPORTED_COMBINATIONS = []  # type: ignore
 
 # Official ASF-controlled hosts serving release checksums. Integrity metadata
 # must come from these origins, never from a (community) mirror, so that a
diff --git a/python/pyspark/tests/test_install_spark.py 
b/python/pyspark/tests/test_install_spark.py
index 19332897d2c3..12c4d9c4e01b 100644
--- a/python/pyspark/tests/test_install_spark.py
+++ b/python/pyspark/tests/test_install_spark.py
@@ -26,7 +26,6 @@ import urllib.request
 from pyspark.install import (
     DEFAULT_HADOOP,
     DEFAULT_HIVE,
-    UNSUPPORTED_COMBINATIONS,
     _extract_tar,
     _parse_sha512_checksum,
     _verify_checksum,
@@ -190,14 +189,6 @@ class SparkInstallationTestCase(unittest.TestCase):
         )
 
         # Negative test cases
-        for hadoop_version, hive_version in UNSUPPORTED_COMBINATIONS:
-            with self.assertRaisesRegex(RuntimeError, "Hive.*should.*Hadoop"):
-                checked_versions(
-                    spark_version=test_version,
-                    hadoop_version=hadoop_version,
-                    hive_version=hive_version,
-                )
-
         with self.assertRaisesRegex(RuntimeError, "Spark version should start 
with 'spark-'"):
             checked_versions(
                 spark_version="malformed", hadoop_version=DEFAULT_HADOOP, 
hive_version=DEFAULT_HIVE


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to