taoran92 opened a new pull request, #11046:
URL: https://github.com/apache/paimon/pull/11046

   ### Purpose
   
   Fixes #11045.
   
   Kafka Connect's `JsonConverter` emits UTF-8 JSON bytes, but the CDC 
deserializer decoded them using the JVM's default charset. This corrupts 
non-ASCII text on non-UTF-8 JVMs, including MySQL snapshot and binlog records. 
Decode these bytes explicitly as UTF-8, and use the same charset when logging 
invalid JSON in the JSON and Kafka Debezium deserializers.
   
   ### Tests
   
   - Add a regression test that starts fresh JVMs with UTF-8 and US-ASCII 
default charsets and checks non-ASCII text and escaped characters with schema 
inclusion enabled and disabled.
   - Passed the targeted Maven tests and standard validation checks:
   
     ```shell
     mvn -o -B -ntp -pl paimon-flink/paimon-flink-cdc -Pflink1 \
       -DwildcardSuites=none \
       
-Dtest=CdcDebeziumDeserializationSchemaTest,KafkaDebeziumJsonDeserializationSchemaTest
 test
     ```
   
   - Verified the regression test entry point on JDK 8 and JDK 11: the parent 
commit fails with US-ASCII; the current commit passes with both charsets. All 
changed Java files also compile with `javac --release 8`.
   - MySQL/Flink end-to-end validation was not run.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to