goutamadwant opened a new pull request, #11307:
URL: https://github.com/apache/seatunnel/pull/11307
### Purpose of this pull request
Fixes #11128.
This updates the Spark 3 runtime line to Spark 3.5.8.
Changes included:
- Update the Spark 3 starter and Spark translation modules to use Spark 3.5.8
- Replace the removed `RowEncoder.apply(...)` usage with `Encoders.row(...)`
- Explicitly keep Spark commit coordinator behavior in `SeaTunnelBatchWrite`
- Exclude Spark 3.5 SLF4J 2 bridge dependencies that conflict with SeaTunnel
logging
- Align the managed Jackson version with Spark 3.5's Jackson Scala module
- Update Spark docs to mention Spark 3.5.x support
### Does this PR introduce _any_ user-facing change?
Yes.
The Spark 3 starter is now built against Spark 3.5.8 instead of the older
Spark 3.3 dependency line.
### How was this patch tested?
Verified the Spark 3 starter and translation modules with Spark 3.5.8:
```shell
JAVA_HOME=$(/usr/libexec/java_home -v 11) PATH="$JAVA_HOME/bin:$PATH" ./mvnw
-q -pl
seatunnel-core/seatunnel-spark-starter/seatunnel-spark-3-starter,seatunnel-translation/seatunnel-translation-spark/seatunnel-translation-spark-3.3
-am -Dtest=SparkSinkTest,SparkCommandArgsTest,MultiTableSinkTest
-DfailIfNoTests=false -DskipIT package
```
The selected Spark tests passed:
- `SparkSinkTest`
- `SparkCommandArgsTest`
- `MultiTableSinkTest`
Verified the distribution package assembly:
```shell
JAVA_HOME=$(/usr/libexec/java_home -v 11) PATH="$JAVA_HOME/bin:$PATH" ./mvnw
-q -pl seatunnel-dist -am -DskipTests -DskipIT package
```
Also ran:
```shell
git diff --check
```
### Check list
* [ ] If any new Jar binary package adding in your PR, please add License
Notice according
[New License
Guide](https://github.com/apache/seatunnel/blob/dev/docs/en/developer/new-license.md)
* [x] If necessary, please update the documentation to describe the new
feature. https://github.com/apache/seatunnel/tree/dev/docs
* [ ] If necessary, please update `incompatible-changes.md` to describe the
incompatibility caused by this PR.
* [ ] If you are contributing the connector code, please check that the
following files are updated:
1. Update
[plugin-mapping.properties](https://github.com/apache/seatunnel/blob/dev/plugin-mapping.properties)
and add new connector information in it
2. Update the pom file of
[seatunnel-dist](https://github.com/apache/seatunnel/blob/dev/seatunnel-dist/pom.xml)
3. Add ci label in
[label-scope-conf](https://github.com/apache/seatunnel/blob/dev/.github/workflows/labeler/label-scope-conf.yml)
4. Add e2e testcase in
[seatunnel-e2e](https://github.com/apache/seatunnel/tree/dev/seatunnel-e2e/seatunnel-connector-v2-e2e/)
5. Update connector
[plugin_config](https://github.com/apache/seatunnel/blob/dev/config/plugin_config)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
