echauchot opened a new pull request, #19586:
URL: https://github.com/apache/flink/pull/19586
## What is the purpose of the change
Make current Flink Cassandra connector support all the latest Cassandra
versions supported by Apache Cassandra (3.026, 3.111.12, 4.0.3 until now)
## Brief change log
**Versions**: I was able to address all Cassandra versions with cassandra
4.x lib and driver 3.x without prod code change. There was a big refactor of
the driver in 4.x that totally changes the driver API so I preferred sticking
to latest driver 3.x that allows to call Cassandra 4.x
** Bug uncovering ** Migrating to Cassandra 4.x uncovered a race condition
in the tests between the asynchronous writes and the junit assertions. So I
introduced a `sink#setSynchronousWrites()` method defaulting to false for
backward compatibility. And I put all the sinks in the tests to write
synchronously. That way we are sure that, in the tests, writes are finished
before asserting on their result (no more race condition).
**Tests**: I first tried to use parameterized tests but the parameters are
injected after the testContainers ClassRule that creates the container is
evaluated. As this ClassRule is mandatory for Cassandra testConainer, I could
not use parameterized tests. So I switched to a hierarchy of classes with an
ITCase per Cassandra version : the subClasses do only the container management
and the actual tests remain in CassandraConnectorITCase which was renamed (and
put abstract) because it is no more the IT entry point.
**Commits history** I think the history is clean: I isolated support of all
Cassandra versions in a single commit. I also isolated synchronous write
support of every sink for easy revert. The final commit is to fix the race
condition in all the tests.
## Verifying this change
This change added tests and can be verified as follows:
Cassandra30ConnectorITCase (Cassandra 3.0.26), Cassandra311ConnectorITCase
(Cassandra 3.11.12), Cassandra40ConnectorITCase (Cassandra 4.0.3)
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): yes upgrade
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: (yes / no / don't know): no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? yes ability for sync
writes with timeout
- If yes, how is the feature documented? javadoc
R: @MartijnVisser
CC: @zentol
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
