taoran92 opened a new issue, #10972:
URL: https://github.com/apache/paimon/issues/10972

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   ### Paimon version
   
   `master` (`2.2-SNAPSHOT`), commit `484bfab6cffdd17acf4621261b9d613d615403bf`.
   
   ### Compute Engine
   
   Flink 1.20.4, Flink CDC 3.5.0, PostgreSQL with `pgoutput`, and JDK 11 
(UTF-8).
   
   ### Minimal reproduce step
   
   1. On a PostgreSQL server with logical replication enabled, create a table 
containing non-UTF-8 binary data:
   
   ```sql
   CREATE TABLE public.binary_values (
       id INT PRIMARY KEY,
       payload BYTEA
   );
   ALTER TABLE public.binary_values REPLICA IDENTITY FULL;
   
   INSERT INTO public.binary_values VALUES (1, decode('00ff8041', 'hex'));
   ```
   
   2. Synchronize the table using `postgres_sync_table` with checkpointing 
enabled.
   3. After a checkpoint completes, read the target `payload` column and 
compare its bytes with the source. The same problem occurs for binary values 
inserted or updated after the initial snapshot.
   
   ### What doesn't meet your expectations?
   
   PostgreSQL `BYTEA` values should be preserved byte for byte in the Paimon 
`BYTES` column. Instead, bytes that are invalid UTF-8 are silently replaced 
during conversion, corrupting both snapshot and WAL records.
   
   ### Anything else?
   
   Follow-up to #10879.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to