QCLyu opened a new pull request, #11495:
URL: https://github.com/apache/incubator-gluten/pull/11495
## What changes are proposed in this pull request?
This PR removes remaining Spark 3.2-specific compatibility code from the
codebase, completing the Spark 3.2 deprecation.
Changes:
- Removed lteSpark32 from SparkVersionUtil.scala and updated lteSpark33 to
use direct version comparison
- Removed Spark 3.2-specific code paths from:
- SparkTaskUtil.scala - Spark 3.2 TaskContext constructor path
- SparkPlanUtil.scala - Spark 3.2-specific supportsRowBased implementation
- GlutenCostEvaluator.scala - Spark 3.2-specific CostEvaluator
instantiation
- Convention.scala - Spark 3.2-specific row type handling
- Removed Spark 3.2 test case from MiscOperatorSuite.scala
- Deleted entire shims/spark32 directory including:
- ColumnarArrayShim.java
- ParquetFooterReaderShim.scala
- Cleaned up unused imports (SparkVersionUtil, SparkShimLoader,
AnalysisException)
The codebase now only supports Spark 3.3 and later versions.
## How was this patch tested?
Verified compilation succeeds with all unused imports removed
Existing unit tests should continue to pass (Spark 3.3+ only)
Manual verification that no references to lteSpark32 or Spark 3.2-specific
code remain in the codebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
