Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
dataroaring merged PR #53451: URL: https://github.com/apache/doris/pull/53451 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
doris-robot commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3084555316 ClickBench: Total hot run time: 33.1 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit cc320ada823515fef9dbae1900e4243c31a6db12, data reload: false query1 0.030.030.03 query2 0.100.050.06 query3 0.290.060.07 query4 1.600.100.08 query5 0.430.410.41 query6 1.160.660.66 query7 0.020.020.02 query8 0.060.050.06 query9 0.650.530.54 query10 0.560.570.59 query11 0.250.130.12 query12 0.250.130.13 query13 0.640.630.64 query14 0.800.830.83 query15 0.970.890.89 query16 0.380.390.38 query17 1.031.121.06 query18 0.240.240.24 query19 2.041.811.95 query20 0.010.020.02 query21 15.38 0.980.67 query22 0.931.010.80 query23 14.69 1.540.76 query24 5.500.570.27 query25 0.170.090.09 query26 0.560.230.18 query27 0.080.090.08 query28 11.01 1.190.58 query29 12.57 4.003.39 query30 3.093.043.00 query31 2.840.600.44 query32 3.240.610.51 query33 3.113.163.10 query34 16.76 5.474.69 query35 4.824.784.79 query36 0.660.520.52 query37 0.200.180.18 query38 0.180.150.15 query39 0.050.050.04 query40 0.200.180.16 query41 0.110.050.05 query42 0.060.050.06 query43 0.050.050.04 Total cold run time: 107.77 s Total hot run time: 33.1 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
doris-robot commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3084538773 TPC-DS: Total hot run time: 185532 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit cc320ada823515fef9dbae1900e4243c31a6db12, data reload: false query1 972 380 410 380 query2 6513167216391639 query3 6741213 219 213 query4 26128 23833 22934 22934 query5 4349594 429 429 query6 311 210 210 210 query7 4618490 286 286 query8 269 221 225 221 query9 8602258825622562 query10 476 324 263 263 query11 15288 15015 14746 14746 query12 151 109 107 107 query13 1660519 398 398 query14 8578575257805752 query15 199 187 176 176 query16 7115615 451 451 query17 909 690 565 565 query18 1967390 291 291 query19 187 184 167 167 query20 120 112 111 111 query21 207 125 109 109 query22 3995418540504050 query23 33979 33214 32980 32980 query24 8442237723732373 query25 571 459 391 391 query26 1236274 143 143 query27 2758506 335 335 query28 4366211320822082 query29 766 542 426 426 query30 283 215 183 183 query31 927 843 747 747 query32 69 63 63 63 query33 549 341 310 310 query34 776 837 512 512 query35 827 813 740 740 query36 967 1001890 890 query37 110 94 71 71 query38 4121409640824082 query39 1525144714421442 query40 224 117 101 101 query41 56 56 56 56 query42 118 105 100 100 query43 495 499 473 473 query44 1301804 811 804 query45 176 167 171 167 query46 830 1002620 620 query47 1743181817461746 query48 379 429 302 302 query49 718 465 382 382 query50 625 686 399 399 query51 5456545653775377 query52 103 104 91 91 query53 220 251 174 174 query54 567 570 533 533 query55 80 84 81 81 query56 293 316 294 294 query57 1182119711401140 query58 265 249 256 249 query59 2537264225822582 query60 336 324 299 299 query61 126 123 146 123 query62 795 704 672 672 query63 221 185 184 184 query64 4433997 652 652 query65 4267422542074207 query66 1133426 317 317 query67 15762 15599 15538 15538 query68 4818885 525 525 query69 490 301 268 268 query70 1168111010961096 query71 419 322 303 303 query72 5288470948334709 query73 675 612 353 353 query74 9257906386518651 query75 3216320226722672 query76 31661142795 795 query77 479 383 289 289 query78 10104 10186 92569256 query79 953 819 577 577 query80 1196517 454 454 query81 529 251 219 219 query82 398 125 104 104 query83 342 315 242 242 query84 292 94 86 86 query85 781 349 310 310 query86 379 326 271 271 query87 4512448643244324 query88 2879228822812281 query89 388 330 281 281 query90 1728213 205 205 query91 131 138 115 115 query92 66 58 59 58 query93 993 938 571 571 query94 658 396 308 308 query95 371 296 283 283 query96 484 582 293 293 query97 2656275426692669 query98 239 207 199 199 query99 1322141212721272 Total cold run time: 266445 ms Total hot run time: 185532 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
doris-robot commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3084502526 TPC-H: Total hot run time: 33706 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit cc320ada823515fef9dbae1900e4243c31a6db12, data reload: false -- Round 1 -- q1 17692 542551475147 q2 1925272 173 173 q3 10652 1291722 722 q4 10285 1095517 517 q5 8087268123072307 q6 180 158 129 129 q7 898 750 614 614 q8 9314132710801080 q9 6891501450405014 q10 6889239319441944 q11 518 296 256 256 q12 346 346 217 217 q13 17778 366231023102 q14 228 228 220 220 q15 544 474 469 469 q16 418 458 369 369 q17 570 860 352 352 q18 7774730569986998 q19 1215943 558 558 q20 334 340 215 215 q21 3790257623412341 q22 10371021962 962 Total cold run time: 107365 ms Total hot run time: 33706 ms - Round 2, with runtime_filter_mode=off - q1 5145508150755075 q2 241 341 215 215 q3 2215266722902290 q4 1348175313031303 q5 4195446045594460 q6 213 180 135 135 q7 2012191718311831 q8 2644254125832541 q9 7298729672497249 q10 3088327228292829 q11 596 514 502 502 q12 688 749 642 642 q13 3779388631793179 q14 300 303 282 282 q15 523 473 478 473 q16 469 509 461 461 q17 1224156114211421 q18 8105763977537639 q19 817 815 942 815 q20 1974204119471947 q21 4675443242054205 q22 10791026992 992 Total cold run time: 52628 ms Total hot run time: 50486 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
github-actions[bot] commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3084342599 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
github-actions[bot] commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3084342722 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
dataroaring commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3084318162 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
gavinchou commented on code in PR #53451:
URL: https://github.com/apache/doris/pull/53451#discussion_r2213503413
##
fe/fe-core/src/main/java/org/apache/doris/transaction/PublishVersionDaemon.java:
##
@@ -246,8 +246,14 @@ private void tryFinishTxnAsync(TransactionState
transactionState, GlobalTransact
transactionState.getTransactionId(),
transactionState.getDbId(), transactionState.getTransactionId());
try {
dbExecutors.get((int) (transactionState.getDbId() %
Config.publish_thread_pool_num)).execute(() -> {
-tryFinishTxnSync(transactionState, globalTransactionMgr);
-publishingTxnIds.remove(transactionState.getTransactionId());
+try {
+tryFinishTxnSync(transactionState, globalTransactionMgr);
+
publishingTxnIds.remove(transactionState.getTransactionId());
+} catch (Throwable e) {
+LOG.warn("failed to finish transaction {}, dbId: {},
txnId: {}, exception: {}",
Review Comment:
多了最后一个{}
exception 会默认打印
##
fe/fe-core/src/main/java/org/apache/doris/transaction/PublishVersionDaemon.java:
##
@@ -246,8 +246,14 @@ private void tryFinishTxnAsync(TransactionState
transactionState, GlobalTransact
transactionState.getTransactionId(),
transactionState.getDbId(), transactionState.getTransactionId());
try {
dbExecutors.get((int) (transactionState.getDbId() %
Config.publish_thread_pool_num)).execute(() -> {
-tryFinishTxnSync(transactionState, globalTransactionMgr);
-publishingTxnIds.remove(transactionState.getTransactionId());
+try {
+tryFinishTxnSync(transactionState, globalTransactionMgr);
+
publishingTxnIds.remove(transactionState.getTransactionId());
+} catch (Throwable e) {
+LOG.warn("failed to finish transaction {}, dbId: {},
txnId: {}, exception: {}",
Review Comment:
多了最后一个{}
exception 会默认打印, 不需要 placeholder
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
doris-robot commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3083276290 ClickBench: Total hot run time: 33.31 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 73bcfce8916d660075f4921d2f1bd60a35569e30, data reload: false query1 0.050.030.03 query2 0.080.040.04 query3 0.240.070.08 query4 1.630.110.11 query5 0.450.430.42 query6 1.160.660.68 query7 0.020.020.01 query8 0.050.040.04 query9 0.590.530.51 query10 0.570.570.57 query11 0.160.110.11 query12 0.150.110.12 query13 0.650.620.61 query14 0.830.820.82 query15 0.920.880.87 query16 0.390.410.39 query17 1.071.061.05 query18 0.220.220.22 query19 2.151.841.89 query20 0.010.010.01 query21 15.36 0.930.54 query22 0.761.230.76 query23 14.82 1.390.66 query24 6.711.361.62 query25 0.480.310.07 query26 0.540.160.16 query27 0.060.060.06 query28 10.31 0.870.44 query29 12.54 4.023.34 query30 3.113.063.02 query31 2.830.580.39 query32 3.230.550.48 query33 3.263.073.14 query34 16.16 5.414.79 query35 4.834.864.85 query36 0.710.500.49 query37 0.090.070.07 query38 0.050.040.04 query39 0.030.020.03 query40 0.180.130.14 query41 0.080.030.03 query42 0.030.030.02 query43 0.040.030.03 Total cold run time: 107.6 s Total hot run time: 33.31 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
doris-robot commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3083258178 TPC-DS: Total hot run time: 187653 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 73bcfce8916d660075f4921d2f1bd60a35569e30, data reload: false query1 994 374 394 374 query2 6552171017171710 query3 6738217 217 217 query4 27131 23742 23599 23599 query5 4341601 432 432 query6 314 216 201 201 query7 4632496 289 289 query8 272 220 217 217 query9 8637262526202620 query10 490 308 272 272 query11 15750 15071 14758 14758 query12 155 107 107 107 query13 1661547 419 419 query14 8979595159675951 query15 210 200 175 175 query16 7150659 463 463 query17 1189696 569 569 query18 1967402 295 295 query19 191 188 159 159 query20 121 124 114 114 query21 216 120 105 105 query22 4121429740344034 query23 34076 2 33128 33128 query24 8516241024322410 query25 535 502 410 410 query26 1249266 150 150 query27 2765513 340 340 query28 4342214221192119 query29 758 561 440 440 query30 290 214 184 184 query31 909 841 771 771 query32 77 73 60 60 query33 550 370 306 306 query34 823 845 522 522 query35 786 816 738 738 query36 971 1029927 927 query37 114 103 78 78 query38 4245413540604060 query39 1507148314101410 query40 218 126 113 113 query41 64 60 62 60 query42 124 108 108 108 query43 526 513 487 487 query44 1370834 832 832 query45 180 175 171 171 query46 902 1025654 654 query47 1768183717081708 query48 393 437 308 308 query49 709 482 403 403 query50 662 724 402 402 query51 5541556556445565 query52 116 106 98 98 query53 233 260 179 179 query54 591 561 522 522 query55 82 81 82 81 query56 307 289 293 289 query57 1194119311231123 query58 264 267 255 255 query59 2569261225042504 query60 360 320 315 315 query61 133 124 128 124 query62 812 744 653 653 query63 230 194 187 187 query64 4297993 653 653 query65 4279423542004200 query66 1125411 323 323 query67 16274 15554 15678 15554 query68 6693909 539 539 query69 486 309 276 276 query70 1233118211271127 query71 413 319 307 307 query72 5664474548794745 query73 674 678 351 351 query74 8981912789328932 query75 3189321127972797 query76 32021207761 761 query77 469 383 299 299 query78 10113 10241 93399339 query79 1932831 586 586 query80 657 528 454 454 query81 501 259 223 223 query82 413 128 103 103 query83 247 248 239 239 query84 253 109 90 90 query85 787 352 353 352 query86 379 314 298 298 query87 4436453043514351 query88 2936226822872268 query89 380 327 284 284 query90 1824212 210 210 query91 146 140 110 110 query92 73 60 58 58 query93 1230986 594 594 query94 627 430 289 289 query95 381 298 289 289 query96 502 562 283 283 query97 2657281426322632 query98 232 208 208 208 query99 1344139012551255 Total cold run time: 272701 ms Total hot run time: 187653 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
doris-robot commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3083219890 TPC-H: Total hot run time: 34485 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 73bcfce8916d660075f4921d2f1bd60a35569e30, data reload: false -- Round 1 -- q1 17738 534552215221 q2 1939291 183 183 q3 10668 1342771 771 q4 10349 1043545 545 q5 9186246723882388 q6 203 161 130 130 q7 929 761 613 613 q8 9331135911731173 q9 7524515051105110 q10 6996241219831983 q11 481 302 280 280 q12 356 364 231 231 q13 17795 378631083108 q14 241 229 231 229 q15 557 482 497 482 q16 443 447 379 379 q17 626 903 373 373 q18 7415704671787046 q19 1327939 595 595 q20 342 349 223 223 q21 4031326424522452 q22 10711032970 970 Total cold run time: 109548 ms Total hot run time: 34485 ms - Round 2, with runtime_filter_mode=off - q1 5379519152095191 q2 252 324 223 223 q3 2167268722972297 q4 1369181013461346 q5 4733463545984598 q6 214 165 133 133 q7 2093205118651865 q8 2707264326752643 q9 7338723671267126 q10 3118334228592859 q11 591 542 492 492 q12 712 818 662 662 q13 3783419833583358 q14 319 321 303 303 q15 523 498 491 491 q16 458 528 446 446 q17 1242160714211421 q18 8048770076157615 q19 901 855 935 855 q20 2019193518421842 q21 4733443643164316 q22 10871025974 974 Total cold run time: 53786 ms Total hot run time: 51056 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
Thearas commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3083092986 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](editlog) catch exception in publish thread and remove synchronized in BackupJob::isDone [doris]
dataroaring commented on PR #53451: URL: https://github.com/apache/doris/pull/53451#issuecomment-3083092379 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
