spuru9 commented on code in PR #56:
URL: 
https://github.com/apache/flink-connector-cassandra/pull/56#discussion_r4633069720


##########
flink-connector-cassandra/pom.xml:
##########
@@ -38,16 +38,15 @@ under the License.
        <!-- Allow users to pass custom connector versions -->
        <properties>
                <scala.binary.version>2.12</scala.binary.version>
-               <scala-library.version>2.12.7</scala-library.version>
                <slf4j.version>1.7.36</slf4j.version>
                <junit5.version>5.8.1</junit5.version>
-               <assertj.version>3.23.1</assertj.version>
+               <assertj.version>3.27.3</assertj.version>
                <archunit.version>1.0.0</archunit.version>
                <!-- For dependency convergence we exclude testContainers 
commons-compress transitive dependency and keep only flink's.
                Don't downgrade testcontainers otherwise the wrong 
commons-compress transitive dependency will lead to NoClassDefFound-->
                <testcontainers.version>1.21.4</testcontainers.version>
-               <mockito.version>3.4.6</mockito.version>
-               <byte-buddy.version>1.12.10</byte-buddy.version>
+               <mockito.version>5.19.0</mockito.version>

Review Comment:
   Mockito 5.19 needs objenesis 3.3 and Flink 2.3's Kryo needs 3.4, but the 
`objenesis` 2.6 convergence pin further down still forces the old one. Could we 
bump it to 3.4 to match Flink? `junit5.version` 5.8.1 is also well behind Flink 
2.3's 5.11.4.



##########
pom.xml:
##########
@@ -168,4 +168,28 @@ under the License.
                        </plugin>
                </plugins>
        </build>
+
+       <profiles>
+               <profile>
+                       <id>java21</id>
+                       <activation>
+                               <jdk>[21,)</jdk>
+                       </activation>
+                       <build>
+                               <pluginManagement>
+                                       <plugins>
+                                               <plugin>
+                                                       
<groupId>com.diffplug.spotless</groupId>
+                                                       
<artifactId>spotless-maven-plugin</artifactId>
+                                                       <configuration>
+                                                               <!-- 
google-java-format 1.7 from the parent does not run on Java 21.
+                                                                       Same as 
the java21 profile in flink-connector-parent 2.0.0. -->
+                                                               
<skip>true</skip>
+                                                       </configuration>
+                                               </plugin>
+                                       </plugins>
+                               </pluginManagement>
+                       </build>
+               </profile>
+       </profiles>
 </project>

Review Comment:
   Not a blocker, but kafka, jdbc, es and mongodb are already on 
`flink-connector-parent` 2.0.0. It ships this exact profile, and 1.1.0 still 
defaults to a Java 8 target. Any reason not to bump the parent instead of 
copying the profile? A follow-up is fine too.



##########
flink-connector-cassandra/src/test/java/org/apache/flink/streaming/connectors/cassandra/CassandraConnectorITCase.java:
##########
@@ -456,6 +449,41 @@ void testCassandraCommitter() throws Exception {
         cc1.close();
     }
 
+    @Test
+    void testCassandraTupleWriteAheadSinkInJob() throws Exception {
+        StreamExecutionEnvironment env = 
StreamExecutionEnvironment.getExecutionEnvironment();
+        env.setParallelism(1);
+        env.enableCheckpointing(100);
+        addWriteAheadSink(env);
+        env.execute();
+
+        ResultSet rs =
+                cassandraTestEnvironment.executeRequestWithTimeout(
+                        injectTableName(SELECT_DATA_QUERY));
+        assertThat(rs.all()).hasSameSizeAs(collection);
+    }
+
+    @Test
+    void testCassandraTupleWriteAheadSinkRequiresCheckpointing() throws 
Exception {
+        StreamExecutionEnvironment env = 
StreamExecutionEnvironment.getExecutionEnvironment();
+        env.setParallelism(1);
+        addWriteAheadSink(env);
+
+        assertThatThrownBy(env::execute)
+                .satisfies(
+                        anyCauseMatches(
+                                IllegalStateException.class,
+                                "The write-ahead log requires checkpointing to 
be enabled."));
+    }
+
+    private void addWriteAheadSink(StreamExecutionEnvironment env) throws 
Exception {
+        CassandraSink.addSink(env.fromData(collection))
+                .setQuery(injectTableName(INSERT_DATA_QUERY))
+                
.setClusterBuilder(cassandraTestEnvironment.getBuilderForWriting())
+                .enableWriteAheadLog()
+                .build();
+    }

Review Comment:
   `CassandraRowWriteAheadSink` got the same checkpointing check but has no 
test at all. Could this helper take the input stream, so the two job-level 
tests also run with `rowCollection`?



##########
flink-connector-cassandra/src/test/java/org/apache/flink/streaming/connectors/cassandra/CassandraConnectorITCase.java:
##########
@@ -682,114 +668,4 @@ void testCassandraBatchRowFormat() throws Exception {
         List<com.datastax.driver.core.Row> rows = rs.all();
         assertThat(rows).hasSameSizeAs(rowCollection);
     }
-
-    @Test
-    void testCassandraScalaTupleAtLeastOnceSinkBuilderDetection() throws 
Exception {
-        Class<scala.Tuple1<String>> c =
-                (Class<scala.Tuple1<String>>) new 
scala.Tuple1<>("hello").getClass();
-        Seq<TypeInformation<?>> typeInfos =
-                JavaConverters.asScalaBufferConverter(
-                                Collections.<TypeInformation<?>>singletonList(
-                                        BasicTypeInfo.STRING_TYPE_INFO))
-                        .asScala();
-        Seq<String> fieldNames =
-                
JavaConverters.asScalaBufferConverter(Collections.singletonList("_1")).asScala();
-
-        CaseClassTypeInfo<scala.Tuple1<String>> typeInfo =
-                new CaseClassTypeInfo<scala.Tuple1<String>>(c, null, 
typeInfos, fieldNames) {
-                    @Override
-                    public TypeSerializer<scala.Tuple1<String>> 
createSerializer(
-                            ExecutionConfig config) {
-                        return null;
-                    }
-                };
-
-        StreamExecutionEnvironment env = 
StreamExecutionEnvironment.getExecutionEnvironment();
-        DataStream<scala.Tuple1<String>> input =
-                env.fromElements(new 
scala.Tuple1<>("hello")).returns(typeInfo);
-
-        CassandraSink.CassandraSinkBuilder<scala.Tuple1<String>> sinkBuilder =
-                CassandraSink.addSink(input);
-        
assertThat(sinkBuilder).isInstanceOf(CassandraSink.CassandraScalaProductSinkBuilder.class);
-    }
-
-    @Test
-    void testCassandraScalaTupleAtLeastSink() throws Exception {
-        CassandraScalaProductSink<scala.Tuple3<String, Integer, Integer>> sink 
=
-                new CassandraScalaProductSink<>(
-                        injectTableName(INSERT_DATA_QUERY),
-                        cassandraTestEnvironment.getBuilderForWriting());
-
-        List<scala.Tuple3<String, Integer, Integer>> scalaTupleCollection = 
new ArrayList<>(20);
-        for (int i = 0; i < 20; i++) {
-            scalaTupleCollection.add(new 
scala.Tuple3<>(UUID.randomUUID().toString(), i, 0));
-        }
-        try {
-            sink.open(new Configuration());
-            for (scala.Tuple3<String, Integer, Integer> value : 
scalaTupleCollection) {
-                sink.invoke(value, SinkContextUtil.forTimestamp(0));
-            }
-        } finally {
-            sink.close();
-        }
-
-        ResultSet rs =
-                cassandraTestEnvironment.executeRequestWithTimeout(
-                        injectTableName(SELECT_DATA_QUERY));
-        List<com.datastax.driver.core.Row> rows = rs.all();
-        assertThat(rows).hasSameSizeAs(scalaTupleCollection);
-
-        for (com.datastax.driver.core.Row row : rows) {
-            scalaTupleCollection.remove(
-                    new scala.Tuple3<>(
-                            row.getString(TUPLE_ID_FIELD),
-                            row.getInt(TUPLE_COUNTER_FIELD),
-                            row.getInt(TUPLE_BATCHID_FIELD)));
-        }
-        assertThat(scalaTupleCollection).isEmpty();
-    }
-
-    @Test
-    void testCassandraScalaTuplePartialColumnUpdate() throws Exception {

Review Comment:
   This was the only test for `setIgnoreNullFields(true)`, and the tuple and 
row sinks still go through that path. Could you port it to a `Tuple3` with 
`CassandraTupleSink` instead of dropping it?



##########
flink-connector-cassandra/pom.xml:
##########
@@ -38,16 +38,15 @@ under the License.
        <!-- Allow users to pass custom connector versions -->
        <properties>
                <scala.binary.version>2.12</scala.binary.version>

Review Comment:
   Nothing in the module depends on Scala anymore, so this property now only 
feeds the `_2.12` suffix on the artifactId. Since 4.0 is a breaking release 
anyway, do we want to drop the suffix (here or in its own ticket)?



##########
flink-connector-cassandra/src/test/java/org/apache/flink/streaming/connectors/cassandra/CassandraConnectorITCase.java:
##########
@@ -289,7 +282,7 @@ protected CassandraTupleWriteAheadSink<Tuple3<String, 
Integer, Integer>> createS
         return new CassandraTupleWriteAheadSink<>(
                 injectTableName(INSERT_DATA_QUERY),
                 TypeExtractor.getForObject(new Tuple3<>("", 0, 0))
-                        .createSerializer(new ExecutionConfig()),
+                        .createSerializer(new SerializerConfigImpl()),

Review Comment:
   Heads-up, this predates your change: the five tests inherited from 
`WriteAheadSinkTestBase` use JUnit 4 `@Test`, so they never run (CI shows 
`Tests run: 11` here). If someone revives them, they'll now fail in `open()`, 
since the harness's job config has no checkpoint interval. Worth a follow-up 
ticket to move the base to JUnit 5?



##########
flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java:
##########
@@ -223,7 +208,10 @@ public static <IN> CassandraSinkBuilder<IN> 
addSink(DataStream<IN> input) {
                             tupleInput
                                     .getType()
                                     .createSerializer(
-                                            
tupleInput.getExecutionEnvironment().getConfig()));
+                                            tupleInput
+                                                    .getExecutionEnvironment()
+                                                    .getConfig()
+                                                    .getSerializerConfig()));

Review Comment:
   nit: `getExecutionEnvironment().getConfig().getSerializerConfig()` is now 
spelled out three times. A local `SerializerConfig` at the top of `addSink` 
would shorten all three branches.



##########
flink-connector-cassandra/src/main/java/org/apache/flink/streaming/connectors/cassandra/CassandraSink.java:
##########
@@ -233,23 +221,19 @@ public static <IN> CassandraSinkBuilder<IN> 
addSink(DataStream<IN> input) {
                             rowInput.getType(),
                             rowInput.getType()
                                     .createSerializer(
-                                            
rowInput.getExecutionEnvironment().getConfig()));
+                                            rowInput.getExecutionEnvironment()
+                                                    .getConfig()
+                                                    .getSerializerConfig()));
         }
         if (typeInfo instanceof PojoTypeInfo) {
             return new CassandraPojoSinkBuilder<>(
                     input,
                     input.getType(),
-                    
input.getType().createSerializer(input.getExecutionEnvironment().getConfig()));
-        }
-        if (typeInfo instanceof CaseClassTypeInfo) {
-            DataStream<Product> productInput = (DataStream<Product>) input;
-            return (CassandraSinkBuilder<IN>)
-                    new CassandraScalaProductSinkBuilder<>(
-                            productInput,
-                            productInput.getType(),
-                            productInput
-                                    .getType()
-                                    
.createSerializer(input.getExecutionEnvironment().getConfig()));
+                    input.getType()
+                            .createSerializer(
+                                    input.getExecutionEnvironment()
+                                            .getConfig()
+                                            .getSerializerConfig()));
         }
         throw new IllegalArgumentException(
                 "No support for the type of the given DataStream: " + 
input.getType());

Review Comment:
   nit: Scala case-class streams land here now. Maybe list the supported types 
(Tuple, Row, POJO) in the message so the error says what to do?



##########
flink-connector-cassandra/pom.xml:
##########
@@ -341,19 +329,13 @@ under the License.
                                <version>${byte-buddy.version}</version>
                        </dependency>
 
-                       <!-- For dependency convergence -->
                        <dependency>
-                               <groupId>com.esotericsoftware.kryo</groupId>
-                               <artifactId>kryo</artifactId>
-                               <version>2.24.0</version>
+                               <groupId>net.bytebuddy</groupId>
+                               <artifactId>byte-buddy-agent</artifactId>
+                               <version>${byte-buddy.version}</version>
                        </dependency>
 
-                       <!-- For dependency convergence -->
-                       <dependency>
-                               <groupId>org.scala-lang</groupId>
-                               <artifactId>scala-library</artifactId>
-                               <version>${scala-library.version}</version>
-                       </dependency>
+
 

Review Comment:
   nit: blank lines left over from the removed dependencies (same at lines 66 
and 74).
   
   ```suggestion
   
   ```



##########
pom.xml:
##########
@@ -42,7 +42,7 @@ under the License.
        </scm>
 
        <properties>
-               <flink.version>1.20.0</flink.version>
+               <flink.version>2.3.0</flink.version>

Review Comment:
   Make sense. But we don't have any support for 2.2 Flink for this connector. 
Probably adding this point to release notes also works.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to