msmygit opened a new pull request, #1389:
URL: https://github.com/apache/cassandra-spark-connector/pull/1389
…er to `4.19.2`
Upgrade the connector to Apache Spark `4.1.2` and Cassandra Java driver
`4.19.2`. Spark `4.x` requires Java 17 and Scala 2.13, so Scala 2.12 and Java
8/11 are dropped.
Build:
* Spark 4.1.2, Cassandra Java driver 4.19.2, Jetty 11.0.26, Scala 2.13.17
* Build Scala 2.13 only; target Java 17 (-release 17 / javac 17)
* Add JDK 17 --add-opens java options for forked (test) JVMs
* Remove obsolete -XX:MaxPermSize from the sbt launcher
* Remove unused scala-2.12 source directories
Spark 4 API migration (connector):
* Use org.apache.spark.sql.classic.{Strategy, SparkSession, Dataset} where
the Connect/Classic split moved the implementations
* Convert Column <-> Expression via classic
ClassicConversions/ColumnConversions
* DataSourceV2Relation gained a timeTravelSpec field; update pattern matches
* Replace removed Dataset.sqlContext / SQLContext.conf with sparkSession /
sessionState.conf
* Adapt to error-class based exception constructors (AnalysisException,
NoSuch/AlreadyExists exceptions)
Docs/CI: update README version matrix, quick start and other docs to Spark
4.1 / Scala 2.13 / Java 17, add CHANGES entry, and run CI on Java 17 + Scala
2.13.
# Description
## How did the Spark Cassandra Connector Work or Not Work Before this Patch
Describe the problem, or state of the project that this patch fixes. Explain
why this is a problem if this isn't obvious.
Example:
"When I read from tables with 3 INTS I get a ThreeIntException(). This is
a problem because I often want to read from a table with three integers."
## General Design of the patch
How the fix is accomplished, were new parameters or classes added? Why did
you
pursue this particular fix?
Example: "I removed the incorrect assertion which would throw the
ThreeIntException. This exception was incorrectly added and the assertion is
not actually needed."
Fixes:
[CASSANALYTICS-173](https://issues.apache.org/jira/browse/CASSANALYTICS-173)
`gh-pages` PR is
https://github.com/apache/cassandra-spark-connector/pull/1388
# How Has This Been Tested?
Almost all changes and especially bug fixes will require a test to be added
to either the integration or Unit Tests. Any tests added will be automatically
run on travis when the pull request is pushed to github. Be sure to run suites
locally as well.
# Checklist:
- [x] I have a ticket in the
[JIRA](https://issues.apache.org/jira/projects/CASSANALYTICS)
- [x] I have performed a self-review of my own code
- [x] Locally all tests pass (make sure tests fail without your patch)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
