echauchot commented on PR #22667: URL: https://github.com/apache/flink/pull/22667#issuecomment-1564460268
Tested it on Cassandra connector and it reports these violations: 1. If I put back IOUtils use in the connector it raises `Method <org.apache.flink.connector.cassandra.source.enumerator.CassandraEnumeratorStateSerializer.deserialize(int, [B)> calls method <org.apache.flink.util.IOUtils.readFully(java.io.InputStream, [B, int, int)> in (CassandraEnumeratorStateSerializer.java:90) ` => This is what we wanted 2. On current code base, it raises: `java.lang.AssertionError: Architecture Violation [Priority: MEDIUM] - Rule 'Connector production code must not depend on non-public API outside of connector packages' was violated (16 times): Constructor <org.apache.flink.connector.cassandra.source.CassandraSource.<init>(org.apache.flink.streaming.connectors.cassandra.ClusterBuilder, long, java.lang.Class, java.lang.String, org.apache.flink.streaming.connectors.cassandra.MapperOptions)> calls method <org.apache.flink.api.java.ClosureCleaner.clean(java.lang.Object, org.apache.flink.api.common.ExecutionConfig$ClosureCleanerLevel, boolean)> in (CassandraSource.java:138) Constructor <org.apache.flink.connector.cassandra.source.CassandraSource.<init>(org.apache.flink.streaming.connectors.cassandra.ClusterBuilder, long, java.lang.Class, java.lang.String, org.apache.flink.streaming.connectors.cassandra.MapperOptions)> calls method <org.apache.flink.util.Preconditions.checkNotNull(java.lang.Object, java.lang.String)> in (CassandraSource.java:124) Constructor <org.apache.flink.connector.cassandra.source.CassandraSource.<init>(org.apache.flink.streaming.connectors.cassandra.ClusterBuilder, long, java.lang.Class, java.lang.String, org.apache.flink.streaming.connectors.cassandra.MapperOptions)> calls method <org.apache.flink.util.Preconditions.checkNotNull(java.lang.Object, java.lang.String)> in (CassandraSource.java:125) Constructor <org.apache.flink.connector.cassandra.source.CassandraSource.<init>(org.apache.flink.streaming.connectors.cassandra.ClusterBuilder, long, java.lang.Class, java.lang.String, org.apache.flink.streaming.connectors.cassandra.MapperOptions)> calls method <org.apache.flink.util.Preconditions.checkNotNull(java.lang.Object, java.lang.String)> in (CassandraSource.java:126) Constructor <org.apache.flink.connector.cassandra.source.CassandraSource.<init>(org.apache.flink.streaming.connectors.cassandra.ClusterBuilder, long, java.lang.Class, java.lang.String, org.apache.flink.streaming.connectors.cassandra.MapperOptions)> calls method <org.apache.flink.util.Preconditions.checkState(boolean, java.lang.String, [Ljava.lang.Object;)> in (CassandraSource.java:127) Constructor <org.apache.flink.connector.cassandra.source.CassandraSource.<init>(org.apache.flink.streaming.connectors.cassandra.ClusterBuilder, long, java.lang.Class, java.lang.String, org.apache.flink.streaming.connectors.cassandra.MapperOptions)> gets field <org.apache.flink.api.common.ExecutionConfig$ClosureCleanerLevel.RECURSIVE> in (CassandraSource.java:138) Field <org.apache.flink.connector.cassandra.source.split.CassandraSplitSerializer.SERIALIZER_CACHE> has generic type <java.lang.ThreadLocal<org.apache.flink.core.memory.DataOutputSerializer>> with type argument depending on <org.apache.flink.core.memory.DataOutputSerializer> in (CassandraSplitSerializer.java:0) Method <org.apache.flink.connector.cassandra.source.CassandraSource.checkQueryValidity(java.lang.String)> calls method <org.apache.flink.util.Preconditions.checkState(boolean, java.lang.Object)> in (CassandraSource.java:145) Method <org.apache.flink.connector.cassandra.source.CassandraSource.checkQueryValidity(java.lang.String)> calls method <org.apache.flink.util.Preconditions.checkState(boolean, java.lang.Object)> in (CassandraSource.java:149) Method <org.apache.flink.connector.cassandra.source.CassandraSource.checkQueryValidity(java.lang.String)> is annotated with <org.apache.flink.annotation.VisibleForTesting> in (CassandraSource.java:0) Method <org.apache.flink.connector.cassandra.source.reader.CassandraSplitReader.generateRangeQuery(java.lang.String, java.lang.String)> is annotated with <org.apache.flink.annotation.VisibleForTesting> in (CassandraSplitReader.java:0) Method <org.apache.flink.connector.cassandra.source.split.CassandraSplitSerializer.deserialize(int, [B)> calls constructor <org.apache.flink.core.memory.DataInputDeserializer.<init>([B)> in (CassandraSplitSerializer.java:57) Method <org.apache.flink.connector.cassandra.source.split.CassandraSplitSerializer.serialize(org.apache.flink.connector.cassandra.source.split.CassandraSplit)> calls method <org.apache.flink.core.memory.DataOutputSerializer.clear()> in (CassandraSplitSerializer.java:51) Method <org.apache.flink.connector.cassandra.source.split.CassandraSplitSerializer.serialize(org.apache.flink.connector.cassandra.source.split.CassandraSplit)> calls method <org.apache.flink.core.memory.DataOutputSerializer.getCopyOfBuffer()> in (CassandraSplitSerializer.java:50) Method <org.apache.flink.connector.cassandra.source.split.SplitsGenerator.estimateTableSize()> is annotated with <org.apache.flink.annotation.VisibleForTesting> in (SplitsGenerator.java:0) Static Initializer <org.apache.flink.connector.cassandra.source.split.CassandraSplitSerializer.<clinit>()> calls constructor <org.apache.flink.core.memory.DataOutputSerializer.<init>(int)> in (CassandraSplitSerializer.java:34) ` => These are calls to ClosureCleaner, checkState and checkNotNull, VisibleForTesting which should be added as violation exceptions => The only discutable part is the dependency on `org.apache.flink.core.memory.DataInputDeserializer` and `org.apache.flink.core.memory.DataOutputSerializer` which are internal APIs but we could definitely use [ByteArray | Object]Streams JDK counterparts instead in the Cassandra code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
