BK202503 commented on PR #22536: URL: https://github.com/apache/kafka/pull/22536#issuecomment-5207215941
> I left a comment on Jira. Please take a look. https://issues.apache.org/jira/browse/KAFKA-20656 (I have no account above link to add comment, so write on the comment below.) @chia7712 Real producers of ByteBuffer-valued BYTES Structs: * debezium JdbcValueConverters.java (lines 916, 919, 922): every BLOB/BYTEA/BINARY/RAW column is wrapped in a ByteBuffer before being written via Struct.put(...). * debezium LogicalDecodingMessageMonitor.java (lines 95, 125, 127): logical decoding messages are wrapped with ByteBuffer.wrap(...) and emitted as BYTES. * confluentinc/kafka-connect-jdbc GenericDatabaseDialect.java (lines 1911–1922, 2159–2172): the sink already assumes BYTES values may be ByteBuffers and safely handles them by slicing before reading. * Aiven-Open/jdbc-connector-for-apache-kafka carries the same implementation in its fork. * mongo-kafka BytesFieldConverter: similarly assumes BYTES may be a ByteBuffer, although it currently calls .array(), which has the same issue on the sink side. Consumers are already working around the Struct.getBytes(...) limitation. For example, debezium S3LargeMessagePostProcessor.java (lines 289–297) uses duplicate(), remaining(), and get(byte[]) instead of calling .array() directly. There is also precedent within Kafka itself. KAFKA-16844 (PR #16101) updated ByteArrayConverter to accept ByteBuffer, following the same compatibility principle. Regarding allocateDirect(...) inside connectors, I couldn't find any evidence of connectors producing direct ByteBuffers. We can therefore drop the direct-buffer test if desired and keep only: * sliced ByteBuffer test coverage, and * byte[]/ByteBuffer behavioral equivalence tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
