taoran92 opened a new issue, #11045:
URL: https://github.com/apache/paimon/issues/11045

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   ### Paimon version
   
   master (`4b8e50df3178f15ccafe6c76cf2b543229b8f4e1`)
   
   ### Compute Engine
   
   Flink 1.20.4, Flink CDC 3.5.0, JDK 11; MySQL 8.0.
   
   ### Minimal reproduce step
   
   1. Set `env.java.opts.all: -Dfile.encoding=US-ASCII` before starting a local 
Flink cluster, so the setting applies to the TaskManagers as well as the 
client. Install the [MySQL CDC 
dependencies](https://paimon.apache.org/docs/master/cdc-ingestion/mysql-cdc/#prerequisites)
 and enable MySQL row-based binlogging.
   
   2. Using a utf8mb4 MySQL connection, create the source table:
   
   ```sql
   CREATE DATABASE cdc_repro CHARACTER SET utf8mb4;
   USE cdc_repro;
   CREATE TABLE texts (id INT PRIMARY KEY, msg VARCHAR(200), rev INT)
       CHARACTER SET utf8mb4;
   INSERT INTO texts VALUES (1, 'δΈ­ζ–‡πŸ˜€', 0);
   ```
   
   3. Run `mysql_sync_table` for `cdc_repro.texts` into an empty Paimon 
warehouse, with checkpointing enabled and `--mysql_conf 
jdbc.properties.characterEncoding=UTF-8`. Wait for the initial row to be 
committed and read its `msg` value from a UTF-8 query environment.
   
   4. Using a utf8mb4 MySQL connection, update and insert rows, then read 
Paimon again after the changes are committed:
   
   ```sql
   UPDATE cdc_repro.texts SET msg = 'ζ›΄ζ–°δΈ­ζ–‡πŸ˜€', rev = 1 WHERE id = 1;
   INSERT INTO cdc_repro.texts VALUES (2, 'ζ–°ε’žδΈ­ζ–‡πŸ˜€', 1);
   ```
   
   ### What doesn't meet your expectations?
   
   Expected: The initial row preserves `δΈ­ζ–‡πŸ˜€`; after the changes, the rows are 
`(1, 'ζ›΄ζ–°δΈ­ζ–‡πŸ˜€', 1)` and `(2, 'ζ–°ε’žδΈ­ζ–‡πŸ˜€', 1)`.
   
   Actual: With the CDC JVM's default charset set to `US-ASCII`, Chinese 
characters are replaced with `οΏ½` in persisted Paimon values during both the 
initial snapshot and binlog updates/inserts. The job keeps running, and the 
emoji remains intact in this example.
   
   ### Are you willing to submit a PR?
   
   - [x] I'm willing to submit a PR!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to