homesickjava opened a new issue, #1587:
URL: https://github.com/apache/incubator-paimon/issues/1587

   ### Search before asking
   
   - [X] I searched in the 
[issues](https://github.com/apache/incubator-paimon/issues) and found nothing 
similar.
   
   
   ### Paimon version
   
   0.5
   
   ### Compute Engine
   
   flink
   
   ### Minimal reproduce step
   
   I have met this data bias when sync data from mysql to paimon. and my 
command is as following, and I have 800000 records in mysql table. 
   
   ./flink run -Dscan.mode=latest -Dstreaming-read-mode=file   
-Dexecution.checkpointing.interval=1s  
-Dclassloader.check-leaked-classloader=false 
paimon-flink-action-0.5-SNAPSHOT.jar mysql-sync-table --warehouse 
hdfs://pcosmo-gyos03:8020/user/root/paimon --database paimon_test --table 
paimon_sync_wangke  --computed-columns 
'current_login_time=avg(current_login_time)'   --mysql-conf 
hostname=10.206.32.3 --mysql-conf port=9006  --mysql-conf username=root  
--mysql-conf password=123456  --mysql-conf database-name=paimontest  
--mysql-conf table-name='wangke' --catalog-conf metastore=hive --catalog-conf 
uri=thrift://10.206.32.3:9083 --table-conf write-manifest-cache=1024mb 
--table-conf manifest.full-compaction-threshold-size=16mb --table-conf 
scan.manifest.parallelism=4 --table-conf scan.split-enumerator.batch-size=4 
--table-conf bucket=4 --table-conf changelog-producer=input --table-conf 
sink.parallelism=4  --table-conf execution.checkpointing.timeout=60min 
--table-conf execution.checkpo
 inting.interval=5min --table-conf full-compaction.delta-commits=3 --table-conf 
changelog-producer=full-compaction --table-conf scan.mode=compacted-full   
--table-conf num-sorted-run.stop-trigger=2147483647 --table-conf 
sort-spill-threshold=10 --table-conf  write-buffer-size=4096mb  --table-conf 
taskmanager.memory.managed.size=1m --table-conf write-buffer-spillable=true 
   
   
   
![image](https://github.com/apache/incubator-paimon/assets/8140615/be7cfd6d-5eac-4186-bd2f-8fcd29a5b073)
   
   
   ### What doesn't meet your expectations?
   
   I want to solve this data bias
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [X] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to