Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
morningman closed pull request #63310: [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon URL: https://github.com/apache/doris/pull/63310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4745016711 ClickBench: Total hot run time: 25.41 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 31f5e7b8e267f8386f1547e9f8cca674c4a7765e, data reload: false query1 0.010.000.01 query2 0.100.050.05 query3 0.260.140.14 query4 1.610.130.13 query5 0.250.230.22 query6 1.301.071.05 query7 0.040.010.00 query8 0.100.030.03 query9 0.380.330.32 query10 0.580.580.58 query11 0.200.140.14 query12 0.180.160.14 query13 0.480.480.49 query14 1.030.991.01 query15 0.650.610.62 query16 0.340.350.32 query17 1.131.151.15 query18 0.230.220.22 query19 2.001.911.92 query20 0.010.010.02 query21 15.44 0.220.15 query22 4.810.050.06 query23 16.12 0.300.12 query24 3.040.470.27 query25 0.100.060.05 query26 0.730.200.16 query27 0.050.040.03 query28 3.430.930.57 query29 12.51 4.313.47 query30 0.270.150.15 query31 2.780.600.32 query32 3.220.600.50 query33 3.143.233.22 query34 15.62 4.283.56 query35 3.553.533.53 query36 0.560.430.42 query37 0.100.060.07 query38 0.050.040.03 query39 0.040.020.02 query40 0.190.160.15 query41 0.090.040.03 query42 0.040.030.03 query43 0.040.030.03 Total cold run time: 96.8 s Total hot run time: 25.41 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4744983032 TPC-DS: Total hot run time: 175566 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 31f5e7b8e267f8386f1547e9f8cca674c4a7765e, data reload: false query5 4301629 476 476 query6 447 195 180 180 query7 4817569 316 316 query8 373 218 215 215 query9 8772407241014072 query10 451 319 268 268 query11 5913236721302130 query12 160 102 97 97 query13 1261610 448 448 query14 6373540350695069 query14_14425438243624362 query15 206 199 175 175 query16 978 466 459 459 query17 942 708 583 583 query18 2428485 377 377 query19 207 196 147 147 query20 113 115 108 108 query21 222 136 119 119 query22 13634 13576 13396 13396 query23 17476 16624 16124 16124 query23_116281 16227 16286 16227 query24 7400181012941294 query24_11318131313181313 query25 588 474 407 407 query26 1304307 170 170 query27 2711566 341 341 query28 4491203820062006 query29 1135595 476 476 query30 312 235 198 198 query31 11031061949 949 query32 130 60 57 57 query33 522 317 247 247 query34 11831127692 692 query35 811 773 671 671 query36 1413143912491249 query37 157 113 90 90 query38 3200312530453045 query39 941 920 890 890 query39_1875 899 871 871 query40 215 119 98 98 query41 64 64 59 59 query42 91 96 95 95 query43 322 328 273 273 query44 1424772 775 772 query45 194 182 173 173 query46 10491242724 724 query47 2323238522322232 query48 408 397 300 300 query49 620 467 336 336 query50 1064351 255 255 query51 4405437043224322 query52 87 86 75 75 query53 253 268 195 195 query54 265 227 189 189 query55 79 73 69 69 query56 234 221 215 215 query57 1430143013301330 query58 246 214 200 200 query59 1600164714711471 query60 286 247 227 227 query61 155 142 143 142 query62 704 651 587 587 query63 228 188 194 188 query64 2540732 586 586 query65 4907480947714771 query66 1813480 336 336 query67 29789 29102 29633 29102 query68 30671622962 962 query69 411 291 265 265 query70 1064978 950 950 query71 297 224 213 213 query72 2916253522682268 query73 840 749 426 426 query74 5123499847444744 query75 2647259522352235 query76 23011185810 810 query77 357 382 316 316 query78 12438 12477 11845 11845 query79 14641186755 755 query80 1081482 385 385 query81 498 275 241 241 query82 825 153 124 124 query83 350 276 247 247 query84 313 146 113 113 query85 905 519 420 420 query86 418 307 269 269 query87 3404331931863186 query88 3668278427702770 query89 424 378 331 331 query90 1805191 187 187 query91 175 156 136 136 query92 65 57 56 56 query93 14951452973 973 query94 625 354 320 320 query95 685 383 355 355 query96 1123804 377 377 query97 2709268425352535 query98 225 226 196 196 query99 1176117510331033 Total cold run time: 261445 ms Total hot run time: 175566 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4744888390 TPC-H: Total hot run time: 29377 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 31f5e7b8e267f8386f1547e9f8cca674c4a7765e, data reload: false -- Round 1 -- q1 17593 400340154003 q2 2042322 194 194 q3 10325 1423816 816 q4 4686472 337 337 q5 7512860 582 582 q6 185 170 141 141 q7 789 838 625 625 q8 9355161515921592 q9 5828452744944494 q10 6773178815631563 q11 438 282 245 245 q12 627 422 289 289 q13 18126 335927562756 q14 266 261 248 248 q15 q16 783 786 723 723 q17 1040964 887 887 q18 7087574856785678 q19 1303162310741074 q20 495 400 270 270 q21 5924264425612561 q22 438 360 299 299 Total cold run time: 101615 ms Total hot run time: 29377 ms - Round 2, with runtime_filter_mode=off - q1 4451429242664266 q2 354 360 235 235 q3 4558497843924392 q4 2103216213831383 q5 4470431143164311 q6 234 176 130 130 q7 1715180519151805 q8 2648221923232219 q9 8136859679697969 q10 4835476743514351 q11 586 414 388 388 q12 740 755 550 550 q13 3242374529282928 q14 295 320 289 289 q15 q16 712 719 638 638 q17 1344131613271316 q18 8055742772387238 q19 1167113110931093 q20 2216222419381938 q21 5326475744974497 q22 494 478 392 392 Total cold run time: 57681 ms Total hot run time: 52328 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
github-actions[bot] commented on code in PR #63310:
URL: https://github.com/apache/doris/pull/63310#discussion_r3436948233
##
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java:
##
@@ -316,6 +326,7 @@ private void eraseDatabase(long currentTimeMs, int keepNum)
{
if (dbInfo == null) {
continue;
}
+eraseTablesForDatabase(dbInfo);
Database db = dbInfo.getDb();
Review Comment:
This fixes the normal expired-database path, but the parallel same-name
database eviction path still bypasses this by-db cleanup.
`eraseDatabaseWithSameName()` below still calls `eraseAllTables(dbInfo)`, which
only removes tables listed in `RecycleDatabaseInfo`; that list is built from
the tables present at database-drop time. Concrete sequence: drop table `t_old`
from db `d`, later drop the remaining tables and db `d`, then exceed
`max_same_name_catalog_trash_num` with more databases named `d`. The same-name
eviction can log `OP_ERASE_DB` for the old db after min latency while `t_old`
remains in `idToTable` with a missing db parent. This is distinct from the
earlier normal-expired-db thread because that path now calls
`eraseTablesForDatabase`, but the same-name db path still uses the old limited
helper. Please route `eraseDatabaseWithSameName()` through the same db-id child
cleanup and add a same-name database test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4743408390 # FE Regression Coverage Report Increment line coverage ` 90.62% (58/64)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
yx-keith commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4743352596 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4743269091 # FE Regression Coverage Report Increment line coverage ` 90.62% (58/64)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4743142686 # FE Regression Coverage Report Increment line coverage ` 90.62% (58/64)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4743173089 # FE Regression Coverage Report Increment line coverage ` 90.62% (58/64)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4741917230 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4741838679 # FE Regression Coverage Report Increment line coverage ` 90.62% (58/64)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_31f5e7b8e267f8386f1547e9f8cca674c4a7765e_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4740525076 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4740173841 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
yx-keith commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4737275285 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on code in PR #63310:
URL: https://github.com/apache/doris/pull/63310#discussion_r3432537350
##
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java:
##
@@ -467,6 +479,7 @@ private void eraseTable(long currentTimeMs, int keepNum) {
if (tableInfo == null) {
continue;
}
+erasePartitionsForTable(tableId);
Review Comment:
I already fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4733602631 # FE Regression Coverage Report Increment line coverage ` 89.71% (61/68)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_c92c60e6d644d9e325f2adc1fcf690eae9e6b7eb_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_c92c60e6d644d9e325f2adc1fcf690eae9e6b7eb_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4732309940 ClickBench: Total hot run time: 25.35 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit c92c60e6d644d9e325f2adc1fcf690eae9e6b7eb, data reload: false query1 0.000.000.00 query2 0.100.050.05 query3 0.250.140.14 query4 1.610.150.14 query5 0.240.230.23 query6 1.271.071.04 query7 0.030.010.01 query8 0.060.040.03 query9 0.380.320.32 query10 0.590.560.56 query11 0.190.140.14 query12 0.180.140.14 query13 0.470.470.48 query14 1.011.011.01 query15 0.610.600.58 query16 0.320.320.31 query17 1.081.101.06 query18 0.230.210.22 query19 2.051.961.95 query20 0.020.010.01 query21 15.43 0.190.13 query22 4.960.060.05 query23 16.13 0.310.12 query24 2.930.420.31 query25 0.090.060.04 query26 0.730.220.14 query27 0.040.040.03 query28 3.520.890.56 query29 12.51 4.343.49 query30 0.280.140.16 query31 2.780.610.31 query32 3.230.600.49 query33 3.223.293.38 query34 15.60 4.183.55 query35 3.563.513.54 query36 0.550.430.43 query37 0.090.070.06 query38 0.040.040.04 query39 0.030.020.02 query40 0.170.160.15 query41 0.090.030.03 query42 0.040.030.03 query43 0.040.030.03 Total cold run time: 96.75 s Total hot run time: 25.35 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4732254714 TPC-DS: Total hot run time: 176126 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c92c60e6d644d9e325f2adc1fcf690eae9e6b7eb, data reload: false query5 4322637 473 473 query6 428 189 178 178 query7 4827536 307 307 query8 365 229 196 196 query9 8740417341014101 query10 439 301 255 255 query11 5940234721372137 query12 158 104 106 104 query13 1348596 436 436 query14 6403541851125112 query14_14411441544144414 query15 209 200 175 175 query16 996 460 452 452 query17 1133724 582 582 query18 2576479 352 352 query19 201 188 149 149 query20 118 109 109 109 query21 220 143 118 118 query22 13791 13623 13490 13490 query23 17515 16623 16239 16239 query23_116318 16333 16332 16332 query24 7548180113451345 query24_11323132513501325 query25 565 472 394 394 query26 1314305 173 173 query27 2681517 348 348 query28 4464205520822055 query29 1088646 506 506 query30 319 242 197 197 query31 11581060957 957 query32 100 63 62 62 query33 525 325 266 266 query34 12051177688 688 query35 765 786 730 730 query36 1419139011951195 query37 160 121 89 89 query38 3280315530593059 query39 950 931 883 883 query39_1878 888 876 876 query40 220 121 102 102 query41 64 62 61 61 query42 98 95 95 95 query43 331 324 289 289 query44 1468783 771 771 query45 190 184 171 171 query46 10741216760 760 query47 2331237021842184 query48 427 421 291 291 query49 617 467 357 357 query50 1023359 259 259 query51 4301427542524252 query52 88 89 77 77 query53 245 268 197 197 query54 270 217 201 201 query55 77 75 74 74 query56 236 233 208 208 query57 1454141113131313 query58 244 209 233 209 query59 1575161614681468 query60 286 251 228 228 query61 158 146 149 146 query62 697 648 570 570 query63 240 197 198 197 query64 2485776 616 616 query65 4890480747904790 query66 1770450 377 377 query67 29877 30018 29563 29563 query68 31861504895 895 query69 404 303 272 272 query70 1149979 992 979 query71 292 236 211 211 query72 2987264622902290 query73 877 760 428 428 query74 5120497348034803 query75 2622260322122212 query76 22841206791 791 query77 376 384 292 292 query78 12522 12537 11876 11876 query79 14801150797 797 query80 1296476 407 407 query81 530 278 236 236 query82 603 162 125 125 query83 325 277 255 255 query84 268 148 116 116 query85 892 491 402 402 query86 437 293 290 290 query87 3418337331753175 query88 3719281928132813 query89 437 387 330 330 query90 1877201 190 190 query91 173 163 138 138 query92 66 58 56 56 query93 16301483859 859 query94 717 347 322 322 query95 673 488 343 343 query96 1075890 347 347 query97 2675268325602560 query98 221 219 202 202 query99 1166121310321032 Total cold run time: 262503 ms Total hot run time: 176126 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4732134618 TPC-H: Total hot run time: 28900 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit c92c60e6d644d9e325f2adc1fcf690eae9e6b7eb, data reload: false -- Round 1 -- q1 17664 407540424042 q2 1987305 190 190 q3 10372 1381850 850 q4 4679469 335 335 q5 7490869 569 569 q6 193 179 140 140 q7 762 836 617 617 q8 9412164716211621 q9 6399448644994486 q10 6867176214981498 q11 440 274 246 246 q12 633 419 298 298 q13 18141 336928382838 q14 285 259 244 244 q15 q16 779 781 702 702 q17 17591127763 763 q18 6899578755025502 q19 15431356949 949 q20 481 401 271 271 q21 5990259924342434 q22 431 358 305 305 Total cold run time: 103206 ms Total hot run time: 28900 ms - Round 2, with runtime_filter_mode=off - q1 4393429642694269 q2 326 356 233 233 q3 4635495244054405 q4 2050216713921392 q5 4447430043694300 q6 226 175 128 128 q7 1688217816911691 q8 2483228320922092 q9 7968793579977935 q10 4771478343124312 q11 574 420 550 420 q12 727 746 546 546 q13 3327354831233123 q14 289 304 298 298 q15 q16 732 736 639 639 q17 1386139613621362 q18 7964755270177017 q19 1153109010991090 q20 221219391939 q21 5292460745534553 q22 510 458 410 410 Total cold run time: 57163 ms Total hot run time: 52154 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4731449022 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4730015690 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4729711949 # FE Regression Coverage Report Increment line coverage ` 88.06% (59/67)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_4b977d0b4c19914042cee8e3f06c8358f43aa43a_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_4b977d0b4c19914042cee8e3f06c8358f43aa43a_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
github-actions[bot] commented on code in PR #63310:
URL: https://github.com/apache/doris/pull/63310#discussion_r3427248747
##
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java:
##
@@ -574,7 +587,128 @@ public void replayEraseTable(long tableId) {
}
}
-private void erasePartition(long currentTimeMs, int keepNum) {
+/**
+ * Erase all partitions belonging to the given table.
+ * This handles partitions that were not expired when erasePartition()
collected
+ * its expired IDs, but became expired later due to time advancement
during processing.
+ *
+ * This method acquires its own locks with fine granularity:
+ *
+ * Read lock to collect partition IDs
+ * Write lock for each individual partition cleanup
+ *
+ *
+ * @param tableId the table ID whose partitions need to be erased
+ */
+private void erasePartitionsForTable(long tableId) {
+// 1. Collect orphan partition IDs under read lock (fast, non-blocking)
+List partitionIds = new ArrayList<>();
+readLock();
+try {
+for (Map.Entry entry :
idToPartition.entrySet()) {
+if (entry.getValue().getTableId() == tableId) {
+partitionIds.add(entry.getKey());
+}
+}
+} finally {
+readUnlock();
+}
+
+// 2. Clean each orphan partition with individual write lock (fine
granularity)
+for (Long partitionId : partitionIds) {
+writeLock();
+try {
+RecyclePartitionInfo pInfo = idToPartition.remove(partitionId);
+if (pInfo == null) {
+continue;
+}
+Partition partition = pInfo.getPartition();
+Env.getCurrentEnv().onErasePartition(partition);
+idToRecycleTime.remove(partitionId);
+
+dbTblIdPartitionNameToIds.computeIfPresent(
+Pair.of(pInfo.getDbId(), pInfo.getTableId()), (pair,
partitionMap) -> {
+partitionMap.computeIfPresent(partition.getName(),
(name, idSet) -> {
+idSet.remove(partitionId);
+return idSet.isEmpty() ? null : idSet;
+});
+return partitionMap.isEmpty() ? null :
partitionMap;
+});
+
+
Env.getCurrentEnv().getEditLog().logErasePartition(partitionId);
+LOG.info("erase orphan partition[{}] when erasing table[{}]",
partitionId, tableId);
+} finally {
+writeUnlock();
+}
+}
+}
+
+/**
+ * Erase all tables belonging to the given database.
+ * This handles tables that were not expired when eraseTable() collected
+ * its expired IDs, but became expired later due to time advancement
during processing.
+ *
+ * This method acquires its own locks with fine granularity:
+ *
+ * Read lock to collect table IDs
+ * For each table, erase its partitions then erase the table
itself
+ *
+ *
+ * @param dbInfo the RecycleDatabaseInfo containing the database and its
table metadata
+ */
+private void eraseTablesForDatabase(RecycleDatabaseInfo dbInfo) {
+long dbId = dbInfo.getDb().getId();
+Set tableNames = Sets.newHashSet(dbInfo.getTableNames());
+Set tableIdSet = Sets.newHashSet(dbInfo.getTableIds());
+
+// 1. Collect orphan table IDs under read lock
+List tableIds = new ArrayList<>();
+readLock();
+try {
+for (Map.Entry entry :
idToTable.entrySet()) {
+RecycleTableInfo tableInfo = entry.getValue();
+if (tableInfo.getDbId() == dbId
+&& tableNames.contains(tableInfo.getTable().getName())
Review Comment:
This still skips recycled tables from the same database if they were dropped
before the database drop. Concrete sequence: drop table `t_old` in db `d`, then
later drop database `d`. `RecycleDatabaseInfo` is built from `db.getTables()`
at the database drop, so it does not contain `t_old`. When the db expires, this
helper skips `t_old`, then `eraseDatabase()` removes/logs the db, leaving
`idToTable` with a dbId that is no longer in the catalog or `idToDatabase`.
That table is no longer recoverable and will be rebuilt from image as a
recycle-bin table whose database is missing. For database parent erasure,
please erase all recycled tables/partitions with this `dbId` before logging
`OP_ERASE_DB`, or otherwise explicitly clean the remaining children by dbId.
This is distinct from the existing thread's current-table drop-db case because
it covers tables that were already recycled before the database drop and
therefore are not in `dbInfo.getTableIds()`.
##
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4728462262 ClickBench: Total hot run time: 25.23 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 4b977d0b4c19914042cee8e3f06c8358f43aa43a, data reload: false query1 0.010.010.01 query2 0.100.050.05 query3 0.260.140.13 query4 1.600.130.15 query5 0.250.220.22 query6 1.271.091.07 query7 0.040.010.01 query8 0.060.040.04 query9 0.370.310.30 query10 0.540.550.55 query11 0.220.140.14 query12 0.180.140.15 query13 0.470.470.48 query14 1.011.000.99 query15 0.610.590.59 query16 0.320.330.33 query17 1.061.101.11 query18 0.230.210.21 query19 2.021.932.03 query20 0.020.010.01 query21 15.45 0.220.14 query22 4.710.060.05 query23 16.13 0.320.13 query24 2.980.430.33 query25 0.110.050.04 query26 0.740.210.15 query27 0.040.030.04 query28 3.540.960.52 query29 12.47 4.293.45 query30 0.270.150.15 query31 2.770.600.31 query32 3.220.590.48 query33 3.243.193.33 query34 15.87 4.203.51 query35 3.473.543.56 query36 0.550.430.42 query37 0.090.070.06 query38 0.050.040.03 query39 0.040.030.03 query40 0.180.170.16 query41 0.080.040.03 query42 0.030.030.03 query43 0.040.030.04 Total cold run time: 96.71 s Total hot run time: 25.23 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4728408466 TPC-DS: Total hot run time: 175911 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 4b977d0b4c19914042cee8e3f06c8358f43aa43a, data reload: false query5 4299629 473 473 query6 435 192 173 173 query7 4816561 308 308 query8 361 219 225 219 query9 8754416441084108 query10 457 306 260 260 query11 5844236921162116 query12 155 103 100 100 query13 1263606 469 469 query14 6629540550885088 query14_14381438143924381 query15 208 200 183 183 query16 1052458 451 451 query17 1141738 600 600 query18 2739517 357 357 query19 258 187 149 149 query20 114 109 105 105 query21 219 144 129 129 query22 13649 13684 13386 13386 query23 17523 16670 16208 16208 query23_116256 16308 16295 16295 query24 7488181513051305 query24_11338133113431331 query25 579 460 406 406 query26 1293314 176 176 query27 2575557 358 358 query28 4351210220652065 query29 1085642 511 511 query30 311 237 207 207 query31 11361088964 964 query32 102 61 62 61 query33 532 328 254 254 query34 11971212660 660 query35 745 816 664 664 query36 1370138812381238 query37 148 108 89 89 query38 3221313430553055 query39 917 926 894 894 query39_1899 853 869 853 query40 221 118 100 100 query41 63 62 62 62 query42 93 93 97 93 query43 328 328 287 287 query44 1441780 793 780 query45 194 189 176 176 query46 10601232764 764 query47 2343238621952195 query48 400 385 309 309 query49 634 460 346 346 query50 969 364 254 254 query51 4367434142874287 query52 88 89 77 77 query53 255 259 186 186 query54 265 216 204 204 query55 81 74 69 69 query56 242 224 204 204 query57 1437143713371337 query58 231 206 206 206 query59 1558167614421442 query60 288 241 226 226 query61 157 153 153 153 query62 684 644 583 583 query63 236 189 194 189 query64 2458765 595 595 query65 4849473047514730 query66 1711506 336 336 query67 29747 29683 29541 29541 query68 32901572999 999 query69 421 296 292 292 query70 1038976 951 951 query71 295 236 218 218 query72 2949265824142414 query73 846 806 425 425 query74 5094500247654765 query75 2638258422532253 query76 23691209807 807 query77 356 375 279 279 query78 12415 12563 11854 11854 query79 13411244767 767 query80 616 483 374 374 query81 460 282 241 241 query82 604 160 124 124 query83 346 276 247 247 query84 263 147 113 113 query85 867 519 416 416 query86 357 309 263 263 query87 3370332731383138 query88 3700280027752775 query89 423 393 328 328 query90 1973183 176 176 query91 176 165 136 136 query92 72 61 57 57 query93 14841440939 939 query94 550 354 322 322 query95 680 489 346 346 query96 1138811 338 338 query97 2715270625472547 query98 212 210 202 202 query99 1176115910311031 Total cold run time: 260736 ms Total hot run time: 175911 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4728290930 TPC-H: Total hot run time: 29537 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 4b977d0b4c19914042cee8e3f06c8358f43aa43a, data reload: false -- Round 1 -- q1 17778 405540844055 q2 2011318 188 188 q3 10316 1472823 823 q4 4682475 341 341 q5 7499848 568 568 q6 185 170 136 136 q7 770 853 624 624 q8 9363163016381630 q9 5719452445514524 q10 6759179015401540 q11 435 276 243 243 q12 632 420 299 299 q13 18110 347528082808 q14 272 258 245 245 q15 q16 782 779 706 706 q17 1022914 959 914 q18 7709585455525552 q19 1341119510321032 q20 494 427 262 262 q21 6574290927262726 q22 458 379 321 321 Total cold run time: 102911 ms Total hot run time: 29537 ms - Round 2, with runtime_filter_mode=off - q1 5073497149344934 q2 348 382 233 233 q3 4959522846894689 q4 2087213714501450 q5 4934470647594706 q6 236 182 132 132 q7 1855174915451545 q8 2406209621272096 q9 7580740373617361 q10 4758463242224222 q11 533 386 356 356 q12 733 743 523 523 q13 3007341527662766 q14 278 295 260 260 q15 q16 683 711 632 632 q17 1297128212541254 q18 7352695768166816 q19 1130113511281128 q20 2228221319361936 q21 5320456645674566 q22 557 447 401 401 Total cold run time: 57354 ms Total hot run time: 52006 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
morningman commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4728042061 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
morningman commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4728041328 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on code in PR #63310:
URL: https://github.com/apache/doris/pull/63310#discussion_r3426559995
##
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java:
##
@@ -67,6 +69,14 @@ public class CatalogRecycleBin extends MasterDaemon
implements Writable {
private final ReentrantReadWriteLock lock = new
ReentrantReadWriteLock(true);
+// Injectable clock source, defaults to system clock
+private LongSupplier clock = System::currentTimeMillis;
Review Comment:
I will fix this issue.
##
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java:
##
@@ -1393,13 +1403,14 @@ public void addTabletToInvertedIndex() {
@Override
protected void runAfterCatalogReady() {
-long currentTimeMs = System.currentTimeMillis();
// should follow the partition/table/db order
// in case of partition(table) is still in recycle bin but table(db)
is missing
+// Each erase method gets its own currentTimeMs to avoid using a stale
timestamp,
+// since previous erase operations may take significant time.
int keepNum = Config.max_same_name_catalog_trash_num;
-erasePartition(currentTimeMs, keepNum);
-eraseTable(currentTimeMs, keepNum);
-eraseDatabase(currentTimeMs, keepNum);
+erasePartition(clock.getAsLong(), keepNum);
+eraseTable(clock.getAsLong(), keepNum);
+eraseDatabase(clock.getAsLong(), keepNum);
Review Comment:
I will fix this issue.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
github-actions[bot] commented on code in PR #63310:
URL: https://github.com/apache/doris/pull/63310#discussion_r3425103907
##
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java:
##
@@ -67,6 +69,14 @@ public class CatalogRecycleBin extends MasterDaemon
implements Writable {
private final ReentrantReadWriteLock lock = new
ReentrantReadWriteLock(true);
+// Injectable clock source, defaults to system clock
+private LongSupplier clock = System::currentTimeMillis;
Review Comment:
`CatalogRecycleBin.write()` still serializes `this` with
`GsonUtils.GSON.toJson(this)` and `readFieldsWithGson()` immediately calls
`GsonUtils.GSON.fromJson(..., CatalogRecycleBin.class)` on that JSON. Because
this new test hook is not `transient` or otherwise excluded, the image JSON now
contains a `LongSupplier` implementation. On image load Gson has to deserialize
that interface field even though the returned object is ignored, which can fail
and at minimum persists a non-deterministic test-only clock implementation.
Please keep the injected clock out of the persisted image.
```suggestion
private transient LongSupplier clock = System::currentTimeMillis;
```
##
fe/fe-core/src/main/java/org/apache/doris/catalog/CatalogRecycleBin.java:
##
@@ -1393,13 +1403,14 @@ public void addTabletToInvertedIndex() {
@Override
protected void runAfterCatalogReady() {
-long currentTimeMs = System.currentTimeMillis();
// should follow the partition/table/db order
// in case of partition(table) is still in recycle bin but table(db)
is missing
+// Each erase method gets its own currentTimeMs to avoid using a stale
timestamp,
+// since previous erase operations may take significant time.
int keepNum = Config.max_same_name_catalog_trash_num;
-erasePartition(currentTimeMs, keepNum);
-eraseTable(currentTimeMs, keepNum);
-eraseDatabase(currentTimeMs, keepNum);
+erasePartition(clock.getAsLong(), keepNum);
+eraseTable(clock.getAsLong(), keepNum);
+eraseDatabase(clock.getAsLong(), keepNum);
Review Comment:
This fresh timestamp can remove a parent after its child phase already
skipped the child, which violates the invariant in the comment above. Concrete
drop-database case: tables are recycled before the db in
`InternalCatalog.unprotectDropDb()`/`recycleDatabase()`. If `eraseTable()` is
called at `tableRecycleTime + expire - 1` it leaves the table in `idToTable`;
if that phase then takes long enough, this later
`eraseDatabase(clock.getAsLong(), ...)` can see the db as expired and the
normal expired branch only removes `idToDatabase`/logs `OP_ERASE_DB` (it does
not call `eraseAllTables`). After replay or image rebuild the table remains in
the recycle bin with its database already erased, exactly what the
partition/table/db ordering is supposed to prevent. The same boundary exists
between partition and table. Please either use a consistent cutoff for
dependent child/parent phases, or make parent erasure check/erase remaining
children before logging the parent erase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
morningman commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4725167466 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
github-actions[bot] commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4725000871 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
github-actions[bot] commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4725000657 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4717858446 # FE Regression Coverage Report Increment line coverage ` 77.78% (7/9)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_c777b85bc8a8cb928dba405a4b684eebd819e8b9_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_c777b85bc8a8cb928dba405a4b684eebd819e8b9_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4716575397 ClickBench: Total hot run time: 25.28 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit c777b85bc8a8cb928dba405a4b684eebd819e8b9, data reload: false query1 0.010.010.01 query2 0.100.050.05 query3 0.260.140.14 query4 1.610.130.13 query5 0.240.230.23 query6 1.181.051.04 query7 0.040.010.01 query8 0.050.040.04 query9 0.400.340.31 query10 0.570.580.56 query11 0.200.150.14 query12 0.190.150.15 query13 0.490.470.47 query14 1.001.021.03 query15 0.620.600.60 query16 0.320.330.33 query17 1.091.091.13 query18 0.240.220.22 query19 2.102.011.91 query20 0.020.010.00 query21 15.44 0.200.13 query22 4.980.050.05 query23 16.14 0.320.12 query24 2.900.410.33 query25 0.110.060.04 query26 0.720.210.14 query27 0.040.040.04 query28 3.540.950.52 query29 12.47 4.323.49 query30 0.270.150.15 query31 2.760.610.32 query32 3.220.600.49 query33 3.173.213.26 query34 15.82 4.173.50 query35 3.523.543.51 query36 0.560.440.43 query37 0.090.060.07 query38 0.050.030.04 query39 0.040.030.03 query40 0.180.160.16 query41 0.080.030.03 query42 0.030.020.02 query43 0.040.030.04 Total cold run time: 96.9 s Total hot run time: 25.28 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4716537274 TPC-DS: Total hot run time: 175377 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c777b85bc8a8cb928dba405a4b684eebd819e8b9, data reload: false query5 4324625 464 464 query6 432 204 176 176 query7 4862557 313 313 query8 372 213 195 195 query9 8762402740534027 query10 440 308 253 253 query11 5911233821652165 query12 156 99 99 99 query13 1265594 452 452 query14 6394538450565056 query14_14365440543644364 query15 204 195 176 176 query16 1019460 475 460 query17 1091697 558 558 query18 2500484 334 334 query19 195 184 134 134 query20 111 108 102 102 query21 213 135 118 118 query22 13734 13718 13335 13335 query23 17343 16579 16067 16067 query23_116291 16269 16324 16269 query24 7430177013171317 query24_11309131413111311 query25 528 435 366 366 query26 1282314 166 166 query27 2682551 338 338 query28 4449204520202020 query29 1049624 494 494 query30 312 237 197 197 query31 11181075961 961 query32 104 63 58 58 query33 504 309 236 236 query34 11951174670 670 query35 768 776 678 678 query36 1355141511951195 query37 155 105 90 90 query38 3213314430473047 query39 907 907 901 901 query39_1886 880 876 876 query40 225 124 104 104 query41 72 68 67 67 query42 97 100 100 100 query43 326 329 286 286 query44 1444785 785 785 query45 195 186 174 174 query46 10901214775 775 query47 2369235022092209 query48 414 421 298 298 query49 696 483 344 344 query50 971 355 261 261 query51 4344434542784278 query52 87 88 76 76 query53 248 280 186 186 query54 284 210 184 184 query55 76 74 67 67 query56 241 218 216 216 query57 1433141413091309 query58 232 214 208 208 query59 1553165014861486 query60 285 249 222 222 query61 149 151 148 148 query62 686 652 582 582 query63 231 190 193 190 query64 2509767 616 616 query65 4855482347924792 query66 1776460 340 340 query67 30059 29653 29526 29526 query68 31521512916 916 query69 399 317 261 261 query70 1094979 963 963 query71 295 233 214 214 query72 2906274324662466 query73 843 775 454 454 query74 5151496047644764 query75 2632261022582258 query76 23401180770 770 query77 384 377 280 280 query78 12412 12458 11824 11824 query79 11781179741 741 query80 521 469 381 381 query81 448 278 241 241 query82 240 159 125 125 query83 269 278 251 251 query84 268 144 115 115 query85 824 516 418 418 query86 330 306 274 274 query87 3345335132183218 query88 3717279728122797 query89 407 385 329 329 query90 2147186 181 181 query91 172 163 132 132 query92 66 59 56 56 query93 14271624993 993 query94 538 349 306 306 query95 677 371 434 371 query96 1053838 371 371 query97 2720270925752575 query98 213 207 199 199 query99 1160115610281028 Total cold run time: 259902 ms Total hot run time: 175377 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4716448170 TPC-H: Total hot run time: 29009 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit c777b85bc8a8cb928dba405a4b684eebd819e8b9, data reload: false -- Round 1 -- q1 17651 399639583958 q2 1988315 191 191 q3 10380 1400833 833 q4 4679470 346 346 q5 7482856 567 567 q6 202 183 145 145 q7 757 847 608 608 q8 10161 163116591631 q9 6319450945414509 q10 6810181115411541 q11 438 273 238 238 q12 661 421 296 296 q13 18175 347927952795 q14 284 268 253 253 q15 q16 786 765 701 701 q17 16091191774 774 q18 6797585755805580 q19 2714138610881088 q20 514 404 263 263 q21 5964257224032403 q22 432 356 289 289 Total cold run time: 104803 ms Total hot run time: 29009 ms - Round 2, with runtime_filter_mode=off - q1 4409422242484222 q2 330 358 223 223 q3 4624497343994399 q4 2032216413711371 q5 4422429342864286 q6 235 178 125 125 q7 2250192115821582 q8 2475216420892089 q9 8083785979697859 q10 4807474644884488 q11 573 410 374 374 q12 760 746 571 571 q13 3242374330083008 q14 303 298 298 298 q15 q16 714 715 632 632 q17 1358131713181317 q18 7843730067816781 q19 1115112911231123 q20 2202220819621962 q21 5328454444294429 q22 520 439 390 390 Total cold run time: 57625 ms Total hot run time: 51529 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4716223332 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4710849762 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4710299170 # FE Regression Coverage Report Increment line coverage ` 77.78% (7/9)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_a15360e85ac725af3a16308e33e1996bd355f920_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_a15360e85ac725af3a16308e33e1996bd355f920_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4710168037 # FE UT Coverage Report Increment line coverage ` 100.00% (9/9)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_a15360e85ac725af3a16308e33e1996bd355f920/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_a15360e85ac725af3a16308e33e1996bd355f920/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4709789165 TPC-DS: Total hot run time: 168965 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a15360e85ac725af3a16308e33e1996bd355f920, data reload: false query5 4305631 470 470 query6 451 191 171 171 query7 4880572 298 298 query8 357 209 191 191 query9 8742398039773977 query10 442 322 253 253 query11 5852230721752175 query12 157 100 95 95 query13 1280595 430 430 query14 6385537750695069 query14_14365437243454345 query15 205 202 178 178 query16 1025446 439 439 query17 1132713 582 582 query18 2461477 362 362 query19 210 190 144 144 query20 113 107 106 106 query21 225 145 118 118 query22 13576 13482 13449 13449 query23 17231 16498 16067 16067 query23_116191 16363 16273 16273 query24 7581175213171317 query24_11326131413201314 query25 594 450 402 402 query26 1307312 168 168 query27 2723531 339 339 query28 4438202520172017 query29 1085625 490 490 query30 308 236 195 195 query31 11141081946 946 query32 109 62 60 60 query33 522 317 261 261 query34 12081130669 669 query35 787 785 683 683 query36 1423139812831283 query37 154 98 86 86 query38 3251318431163116 query39 963 972 968 968 query39_1936 914 910 910 query40 224 121 101 101 query41 65 61 61 61 query42 95 92 91 91 query43 318 323 280 280 query44 query45 200 183 178 178 query46 11231223709 709 query47 2398237122712271 query48 391 425 291 291 query49 614 458 331 331 query50 942 335 260 260 query51 4327435343044304 query52 86 86 74 74 query53 242 263 193 193 query54 270 218 215 215 query55 79 77 69 69 query56 216 214 213 213 query57 1449140212981298 query58 249 203 212 203 query59 1544165614201420 query60 286 240 225 225 query61 154 149 141 141 query62 695 659 594 594 query63 226 180 188 180 query64 2536738 581 581 query65 query66 1799458 329 329 query67 29847 29677 29595 29595 query68 query69 434 303 266 266 query70 950 963 910 910 query71 307 218 205 205 query72 2929264023502350 query73 877 820 429 429 query74 5127497547514751 query75 2664256222292229 query76 23401130798 798 query77 356 379 294 294 query78 12388 12380 11858 11858 query79 14671009756 756 query80 1270449 366 366 query81 518 272 237 237 query82 720 156 120 120 query83 349 284 240 240 query84 query85 888 491 407 407 query86 435 305 291 291 query87 3385339831653165 query88 3650274527502745 query89 443 373 322 322 query90 1926175 166 166 query91 201 156 134 134 query92 66 62 56 56 query93 15861512846 846 query94 753 320 310 310 query95 679 370 420 370 query96 1031755 333 333 query97 2678268325812581 query98 210 220 199 199 query99 1132118010361036 Total cold run time: 251914 ms Total hot run time: 168965 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4709696881 TPC-H: Total hot run time: 28297 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit a15360e85ac725af3a16308e33e1996bd355f920, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17696 398139313931 q2 q3 10842 1361780 780 q4 4682472 339 339 q5 7563851 596 596 q6 179 165 135 135 q7 782 814 635 635 q8 9815156914641464 q9 7012443444504434 q10 6805181314981498 q11 432 268 243 243 q12 642 421 289 289 q13 18158 365627722772 q14 268 258 240 240 q15 q16 825 781 700 700 q17 1180946 824 824 q18 6930578155055505 q19 1252129310781078 q20 495 409 253 253 q21 5769266822812281 q22 432 357 300 300 Total cold run time: 101759 ms Total hot run time: 28297 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4329421142314211 q2 q3 4474494143744374 q4 2065213813861386 q5 4386428042784278 q6 226 172 125 125 q7 1715160618611606 q8 2530218921072107 q9 7771781077497749 q10 4773477543244324 q11 580 444 367 367 q12 749 749 550 550 q13 3456352528952895 q14 300 301 266 266 q15 q16 764 759 663 663 q17 1323132112961296 q18 8076731470607060 q19 1187109911011099 q20 2207222519351935 q21 5218454543834383 q22 493 476 413 413 Total cold run time: 56622 ms Total hot run time: 51087 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4709445557 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4709321374 TPC-DS: Total hot run time: 168866 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit a15360e85ac725af3a16308e33e1996bd355f920, data reload: false query5 4332627 476 476 query6 429 192 175 175 query7 4860568 312 312 query8 369 223 199 199 query9 8774404040434040 query10 441 312 260 260 query11 5941233821832183 query12 162 102 99 99 query13 1265607 418 418 query14 6386542950835083 query14_14404443643884388 query15 205 198 175 175 query16 1000456 421 421 query17 1121721 584 584 query18 2655494 359 359 query19 202 196 149 149 query20 113 111 109 109 query21 224 150 121 121 query22 13634 13585 13366 13366 query23 17418 16501 16233 16233 query23_116195 16228 16295 16228 query24 7480177212661266 query24_11314131413171314 query25 581 468 399 399 query26 1298310 178 178 query27 2675570 332 332 query28 4463201720412017 query29 1081646 501 501 query30 313 231 202 202 query31 11051075965 965 query32 111 66 60 60 query33 543 328 261 261 query34 11781132669 669 query35 761 811 681 681 query36 1459135912561256 query37 160 102 87 87 query38 3193313130303030 query39 913 909 908 908 query39_1867 873 874 873 query40 227 118 100 100 query41 63 73 59 59 query42 92 92 98 92 query43 315 320 272 272 query44 query45 200 185 176 176 query46 10611230760 760 query47 2307233821802180 query48 366 421 290 290 query49 619 490 335 335 query50 1020350 257 257 query51 4377438542124212 query52 86 86 76 76 query53 255 266 193 193 query54 262 216 217 216 query55 80 74 70 70 query56 227 225 222 222 query57 1433141313341334 query58 234 209 201 201 query59 1599163913771377 query60 285 244 230 230 query61 158 152 157 152 query62 702 660 589 589 query63 229 184 188 184 query64 2493763 616 616 query65 query66 1743451 340 340 query67 29763 29657 28924 28924 query68 query69 412 306 271 271 query70 992 933 942 933 query71 297 222 208 208 query72 2987260623332333 query73 836 766 454 454 query74 5122494347924792 query75 2650258122672267 query76 22941136776 776 query77 352 367 289 289 query78 12517 12563 12023 12023 query79 14351089772 772 query80 921 465 388 388 query81 501 288 247 247 query82 583 158 118 118 query83 325 273 251 251 query84 query85 882 537 424 424 query86 400 303 298 298 query87 3402341231933193 query88 3602274327332733 query89 417 387 327 327 query90 1867189 178 178 query91 173 156 130 130 query92 67 62 56 56 query93 14341490862 862 query94 628 357 319 319 query95 680 408 445 408 query96 1099803 328 328 query97 2708267126222622 query98 208 204 205 204 query99 1117117210041004 Total cold run time: 251091 ms Total hot run time: 168866 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4709208412 TPC-H: Total hot run time: 28825 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit a15360e85ac725af3a16308e33e1996bd355f920, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17825 410640394039 q2 q3 10757 1401815 815 q4 4724477 355 355 q5 7878894 567 567 q6 209 169 140 140 q7 827 842 617 617 q8 10433 162616281626 q9 6822448545024485 q10 6806183115041504 q11 436 263 247 247 q12 676 426 295 295 q13 18177 334527882788 q14 263 252 236 236 q15 q16 822 765 705 705 q17 1000884 1027884 q18 7154577955365536 q19 12211307 q20 509 408 265 265 q21 5706257323132313 q22 449 350 297 297 Total cold run time: 102694 ms Total hot run time: 28825 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4348424942734249 q2 q3 4555495943014301 q4 2106222613811381 q5 4487432343044304 q6 251 176 129 129 q7 1717212815901590 q8 2491223220722072 q9 7924788478997884 q10 4813478243324332 q11 576 432 542 432 q12 757 748 556 556 q13 3222371529972997 q14 303 324 287 287 q15 q16 716 753 645 645 q17 1352133413151315 q18 7788724267546754 q19 1098106311011063 q20 2217220419731973 q21 5257503644584458 q22 542 486 399 399 Total cold run time: 56520 ms Total hot run time: 51121 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4708522903 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4708521917 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4699132947 # FE Regression Coverage Report Increment line coverage ` 66.67% (4/6)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_621a300298116963c26e950558d255700bcfca15_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_621a300298116963c26e950558d255700bcfca15_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4698777246 TPC-DS: Total hot run time: 168029 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 621a300298116963c26e950558d255700bcfca15, data reload: false query5 4318613 480 480 query6 442 190 169 169 query7 4845552 316 316 query8 364 215 203 203 query9 8771397439753974 query10 428 310 251 251 query11 5942235221622162 query12 152 102 96 96 query13 1245650 428 428 query14 6385530550145014 query14_14336432443144314 query15 198 198 175 175 query16 1020432 414 414 query17 1083672 539 539 query18 2526450 335 335 query19 189 178 133 133 query20 114 107 106 106 query21 216 138 116 116 query22 13667 13519 13399 13399 query23 17164 16424 16062 16062 query23_116198 16260 16222 16222 query24 7568178712941294 query24_11315128613151286 query25 552 468 402 402 query26 1295311 162 162 query27 2686543 346 346 query28 4501199920041999 query29 1079612 489 489 query30 310 237 198 198 query31 11021072939 939 query32 108 61 59 59 query33 553 328 252 252 query34 11871142648 648 query35 752 793 676 676 query36 1385140912441244 query37 157 109 100 100 query38 3206311530403040 query39 926 920 912 912 query39_1895 881 894 881 query40 222 126 105 105 query41 72 64 68 64 query42 95 96 101 96 query43 314 321 282 282 query44 query45 198 189 191 189 query46 10881201713 713 query47 2383239922462246 query48 393 415 300 300 query49 640 470 360 360 query50 997 350 267 267 query51 4309426242284228 query52 92 90 78 78 query53 248 264 193 193 query54 279 226 206 206 query55 78 80 73 73 query56 247 229 231 229 query57 1424140913121312 query58 286 217 216 216 query59 1556162614581458 query60 290 260 232 232 query61 178 203 149 149 query62 705 645 566 566 query63 225 182 182 182 query64 2507764 615 615 query65 query66 1757454 338 338 query67 29756 29620 29464 29464 query68 query69 440 302 257 257 query70 954 944 955 944 query71 288 219 206 206 query72 2774261223462346 query73 856 790 433 433 query74 5088493847904790 query75 2616255622182218 query76 23321151762 762 query77 345 374 282 282 query78 12380 12474 11754 11754 query79 14041072770 770 query80 1286465 377 377 query81 516 275 245 245 query82 598 154 121 121 query83 344 276 244 244 query84 query85 884 506 406 406 query86 432 297 279 279 query87 3401332532093209 query88 3637273527362735 query89 424 386 324 324 query90 1969185 175 175 query91 166 158 136 136 query92 62 62 56 56 query93 15771482889 889 query94 724 357 295 295 query95 672 458 334 334 query96 1140793 341 341 query97 2721267625632563 query98 207 199 201 199 query99 1155116510311031 Total cold run time: 251374 ms Total hot run time: 168029 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4698750263 TPC-H: Total hot run time: 29041 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 621a300298116963c26e950558d255700bcfca15, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17696 403739493949 q2 q3 10758 1353802 802 q4 4679477 345 345 q5 7513863 567 567 q6 175 168 136 136 q7 766 820 630 630 q8 9365146816761468 q9 5835452244764476 q10 6782174315311531 q11 440 263 246 246 q12 631 428 296 296 q13 18117 341727892789 q14 264 261 236 236 q15 q16 821 771 702 702 q17 989 965 941 941 q18 7001564755005500 q19 1321124711061106 q20 535 390 263 263 q21 6229279627412741 q22 446 387 317 317 Total cold run time: 100363 ms Total hot run time: 29041 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 5078458847154588 q2 q3 4852516046814681 q4 2110219213581358 q5 4820476546434643 q6 239 179 133 133 q7 1836171115701570 q8 2397208620692069 q9 7943761573797379 q10 4701465242174217 q11 522 378 347 347 q12 726 752 532 532 q13 3027335527552755 q14 278 288 248 248 q15 q16 673 693 607 607 q17 1270124612341234 q18 7327673267976732 q19 1115106310971063 q20 2214221019371937 q21 5614448343094309 q22 532 465 406 406 Total cold run time: 57274 ms Total hot run time: 50808 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4698701401 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4698517478 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4698516603 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
morningman commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4610750129 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4603890244 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4517358263 # FE Regression Coverage Report Increment line coverage ` 8.82% (3/34)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_98c43256eae708c582b6c51e66fe460dafae051a_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_98c43256eae708c582b6c51e66fe460dafae051a_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4516659948 # FE UT Coverage Report Increment line coverage ` 100.00% (3/3)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/63310_98c43256eae708c582b6c51e66fe460dafae051a/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/63310_98c43256eae708c582b6c51e66fe460dafae051a/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4516488278 TPC-DS: Total hot run time: 169884 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 98c43256eae708c582b6c51e66fe460dafae051a, data reload: false query5 4310654 536 536 query6 359 212 192 192 query7 4256554 314 314 query8 328 238 208 208 query9 8824401940254019 query10 442 364 298 298 query11 5776239322072207 query12 181 128 125 125 query13 1347617 415 415 query14 5917532750475047 query14_14368434343764343 query15 213 203 186 186 query16 1013459 453 453 query17 1101732 605 605 query18 2536499 366 366 query19 224 208 170 170 query20 147 133 128 128 query21 218 141 119 119 query22 13771 13535 13375 13375 query23 17276 16365 16081 16081 query23_116290 16298 16137 16137 query24 7469174713121312 query24_11297128313191283 query25 574 504 433 433 query26 1330319 171 171 query27 2679543 329 329 query28 4460195419701954 query29 1028638 521 521 query30 311 237 202 202 query31 11221070946 946 query32 92 76 77 76 query33 545 370 316 316 query34 11611130635 635 query35 774 785 696 696 query36 1370134611721172 query37 158 107 92 92 query38 3202317531103110 query39 938 934 924 924 query39_1870 885 887 885 query40 235 151 128 128 query41 72 70 68 68 query42 113 111 111 111 query43 328 336 289 289 query44 query45 223 205 199 199 query46 10591185737 737 query47 2298232321822182 query48 398 415 312 312 query49 739 520 406 406 query50 1008353 245 245 query51 4407433342494249 query52 108 108 98 98 query53 259 292 211 211 query54 328 291 279 279 query55 99 92 88 88 query56 331 332 315 315 query57 1401140113031303 query58 314 315 270 270 query59 1565164214131413 query60 315 344 309 309 query61 162 155 154 154 query62 668 626 553 553 query63 242 199 205 199 query64 2403781 637 637 query65 query66 1733486 347 347 query67 30355 29982 29734 29734 query68 query69 447 346 314 314 query70 980 1020986 986 query71 306 277 272 272 query72 2984270624052405 query73 836 751 453 453 query74 5060497147444744 query75 2691259722592259 query76 22701152818 818 query77 405 403 336 336 query78 12276 12230 11577 11577 query79 14711082734 734 query80 1237535 458 458 query81 513 277 241 241 query82 1341158 126 126 query83 355 287 243 243 query84 263 138 113 113 query85 930 573 444 444 query86 433 343 314 314 query87 3482334532203220 query88 3513267226882672 query89 447 385 333 333 query90 1802179 190 179 query91 175 169 140 140 query92 82 78 74 74 query93 14451436943 943 query94 629 360 303 303 query95 683 468 336 336 query96 998 759 342 342 query97 2712269125622562 query98 245 226 222 222 query99 11321121987 987 Total cold run time: 254513 ms Total hot run time: 169884 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4516384310 TPC-H: Total hot run time: 31429 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 98c43256eae708c582b6c51e66fe460dafae051a, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17605 401939373937 q2 q3 10815 1406819 819 q4 4690469 343 343 q5 7567225821072107 q6 261 176 135 135 q7 962 764 654 654 q8 9409168416301630 q9 6968494049284928 q10 6452208517851785 q11 452 282 246 246 q12 692 421 293 293 q13 18186 330728112811 q14 262 253 233 233 q15 q16 815 768 708 708 q17 1002878 841 841 q18 6835580155835583 q19 1183123712291229 q20 530 444 281 281 q21 5903271225542554 q22 455 384 312 312 Total cold run time: 101044 ms Total hot run time: 31429 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4579462747234627 q2 q3 4946518046184618 q4 2128218914091409 q5 4756474047034703 q6 234 178 134 134 q7 1869170815521552 q8 2446190319231903 q9 7312728672287228 q10 4490441139923992 q11 523 383 352 352 q12 704 718 519 519 q13 3049333428652865 q14 270 283 248 248 q15 q16 687 715 623 623 q17 1259123212261226 q18 7233682768536827 q19 1091111510981098 q20 2218221119421942 q21 5278461444674467 q22 537 457 395 395 Total cold run time: 55609 ms Total hot run time: 50728 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4515788817 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
heguanhui commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4514248107 rub buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](fe) Fix stale timestamp in CatalogRecycleBin erase daemon [doris]
hello-stephen commented on PR #63310: URL: https://github.com/apache/doris/pull/63310#issuecomment-4467046977 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
