acelyc111 opened a new pull request #3814:
URL: https://github.com/apache/incubator-doris/pull/3814
Trace util is helpful for diagnosing compaction performance problems,
we can get trace log for base compaction like:
```
W0610 11:26:33.804431 56452 storage_engine.cpp:552] Trace:
0610 11:23:03.727535 (+ 0us) storage_engine.cpp:554] start to perform
base compaction
0610 11:23:03.728961 (+ 1426us) storage_engine.cpp:560] found best tablet
546859
0610 11:23:03.728963 (+ 2us) base_compaction.cpp:40] got base compaction
lock
0610 11:23:03.729029 (+ 66us) base_compaction.cpp:44] rowsets picked
0610 11:24:51.784439 (+108055410us) compaction.cpp:46] got concurrency lock
and start to do compaction
0610 11:24:51.784818 (+ 379us) compaction.cpp:74] prepare finished
0610 11:26:33.359265 (+101574447us) compaction.cpp:87] merge rowsets finished
0610 11:26:33.484481 (+125216us) compaction.cpp:102] output rowset built
0610 11:26:33.484482 (+ 1us) compaction.cpp:106] check correctness
finished
0610 11:26:33.513197 (+ 28715us) compaction.cpp:110] modify rowsets finished
0610 11:26:33.513300 (+ 103us) base_compaction.cpp:49] compaction finished
0610 11:26:33.513441 (+ 141us) base_compaction.cpp:56] unused rowsets have
been moved to GC queue
Metrics:
{"filtered_rows":0,"input_row_num":3346807,"input_rowsets_count":42,"input_rowsets_data_size":1256413170,"input_segments_num":44,"merge_rowsets_latency_us":101574444,"merged_rows":0,"output_row_num":3346807,"output_rowset_data_size":1228439659,"output_segments_num":6}
```
for cumulative compaction like:
```
W0610 11:14:18.714366 56468 storage_engine.cpp:518] Trace:
0610 11:14:08.068484 (+ 0us) storage_engine.cpp:520] start to perform
cumulative compaction
0610 11:14:08.069844 (+ 1360us) storage_engine.cpp:526] found best tablet
547083
0610 11:14:08.069846 (+ 2us) cumulative_compaction.cpp:42] got
cumulative compaction lock
0610 11:14:08.069947 (+ 101us) cumulative_compaction.cpp:46] calculated
cumulative point
0610 11:14:08.070141 (+ 194us) cumulative_compaction.cpp:50] rowsets picked
0610 11:14:08.070143 (+ 2us) compaction.cpp:46] got concurrency lock and
start to do compaction
0610 11:14:08.070518 (+ 375us) compaction.cpp:74] prepare finished
0610 11:14:15.389893 (+7319375us) compaction.cpp:87] merge rowsets finished
0610 11:14:15.390916 (+ 1023us) compaction.cpp:102] output rowset built
0610 11:14:15.390917 (+ 1us) compaction.cpp:106] check correctness
finished
0610 11:14:15.409460 (+ 18543us) compaction.cpp:110] modify rowsets finished
0610 11:14:15.409496 (+ 36us) cumulative_compaction.cpp:55] compaction
finished
0610 11:14:15.410138 (+ 642us) cumulative_compaction.cpp:65] unused
rowsets have been moved to GC queue
Metrics:
{"filtered_rows":0,"input_row_num":136707,"input_rowsets_count":302,"input_rowsets_data_size":76617836,"input_segments_num":302,"merge_rowsets_latency_us":7319372,"merged_rows":0,"output_row_num":136707,"output_rowset_data_size":53893280,"output_segments_num":1}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
