gaogao110 opened a new issue, #67123: URL: https://github.com/apache/doris/issues/67123
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 4.1.3 ### What's Wrong? When synchronizing PostgreSQL incremental data via Doris Streaming Job, the import throughput is extremely low and cannot reach business expectations. Server overall resources are abundant: the machine has 60.7% idle CPU and plenty of free memory, no swap pressure, and IO wait is low. However, the doris_be process consistently occupies 100% single-core CPU, which becomes the strict bottleneck of CDC import. The upstream Flink task CPU load is only 6.3%, which proves that the data source side is not the bottleneck. The performance bottleneck is completely inside the Doris BE write logic. Top System Status %Cpu(s): 24.3 us, 4.7 sy, 0.0 ni, 60.7 id, 6.9 wa, 0.9 hi, 2.5 si, 0.0 st MiB Mem : 31301.9 total, 12756.9 free, 16703.9 used, 2284.1 buff/cache MiB Swap: 0.0 total, 0.0 free, 0.0 used. 14598.0 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 3923736 root 20 0 31.1g 1.5g 256252 S 102.0 5.0 51:40.35 doris_be 3928325 root 20 0 16.5g 7.6g 19520 S 6.3 24.8 13:12.08 java - BE single-core full load, multi-core idle, typical single-thread serialization bottleneck - Upstream Flink Java task almost no pressure - Memory is sufficient, no IO bottleneck <img width="2822" height="474" alt="Image" src="https://github.com/user-attachments/assets/f88b269e-3be2-40db-af30-f71f7dab8c21"; /> Perf Flame Graph Analysis CPU hotspot is concentrated in the single-thread write path of FragmentMgrAsyn asynchronous thread pool: FragmentMgrAsyn └── doris::AsyncResultWriter::process_block └── doris::VTabletWriter::write └── doris::VRowDistribution::generate_rows_distribution └── doris::VRowDistribution::__generate_rows_distribution_for_non_auto_partition ├── doris::OlapTableFinder::find_tables └── doris::Block::dump_data └── doris::ColumnWithTypeAndName::to_string └── doris::IDataType::to_string Two major CPU overheads: 1. Non-auto partition routing: For standard Range partition tables (not AUTO-RANGE), every row needs full tablet metadata lookup via find_tables, which causes heavy per-row computation. 2. Block serialization overhead: Frequent dump_data / to_string conversion consumes massive user CPU during CDC stream load writing. <img width="2398" height="1298" alt="Image" src="https://github.com/user-attachments/assets/ec0ed3d2-d9a8-4444-8fd5-c82430f98713"; /> ### What You Expected? fix it ### How to Reproduce? To Reproduce 1. Create a standard Range partitioned table (non AUTO-RANGE) 2. Create PostgreSQL CDC streaming job to consume incremental data 3. Generate continuous PG incremental traffic 4. Observe low import throughput and 100% single-core BE CPU usage ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
