baibaichen opened a new issue, #11340:
URL: https://github.com/apache/incubator-gluten/issues/11340
### Description
This issue tracks five key compatibility fixes and improvements aimed at
proactively adapting to upstream Spark changes and the upcoming 4.1 upgrade,
minimizing future review and merge overhead:
1. **Enable SPARK_TESTING Environment Variable**
Spark unit tests set the environment variable `SPARK_TESTING=1` by default.
To stay aligned with Spark's internal testing behavior, we now explicitly set
`SPARK_TESTING=1` in Gluten's test workflows.
Ref: [#53344](https://github.com/apache/spark/pull/53344)
2. **Update commons-collections Dependency to 4.5.0**
Follows Spark's upgrade by removing version 3.x and migrating to 4.x,
bringing better support for Java generics and ensuring dependency consistency.
Ref: [#52743](https://github.com/apache/spark/pull/52743)
3. **Add GeographyVal and GeometryVal Support**
Adds support for new geospatial data types (GEOGRAPHY and GEOMETRY) as
proposed and implemented in SPARK-51658, enriching row accessor interfaces for
upcoming geospatial analytics.
Ref: [SPARK-51658](https://issues.apache.org/jira/browse/SPARK-51658)
4. **Make `reserveNewColumn` Public**
Adjusts method visibility ahead of time to match the signature changes in
future Spark versions and lessen upgrade friction.
Ref: [#52557](https://github.com/apache/spark/pull/52557)
5. **Remove JetBrains `@NotNull` Annotations**
Removes incompatible `@NotNull` annotations to resolve compilation issues
surfaced by Spark 4.1 and ORC 2.2.0 upgrades.
Ref: [#51676](https://github.com/apache/spark/pull/51676)
### Gluten version
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
