Re: [PR] [improve] Make the schema change memory space adaptive [doris]
xiaokang commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2100020028 #34350 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
xiaokang merged PR #34515: URL: https://github.com/apache/doris/pull/34515 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
github-actions[bot] commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2099898708 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
github-actions[bot] commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2099898657 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
doris-robot commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2099744305 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit f739f98b68a7ba7271c8e038fbbdf99612b8ce37 with default session variables Stream load json: 20 seconds loaded 2358488459 Bytes, about 112 MB/s Stream load orc: 58 seconds loaded 1101869774 Bytes, about 18 MB/s Stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s Insert into select: 21.5 seconds inserted 1000 Rows, about 465K ops/s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
doris-robot commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2099741814 ClickBench: Total hot run time: 31.27 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit f739f98b68a7ba7271c8e038fbbdf99612b8ce37, data reload: false query1 0.020.030.02 query2 0.060.030.02 query3 0.250.050.06 query4 1.780.070.07 query5 0.530.520.52 query6 1.230.620.60 query7 0.020.010.01 query8 0.030.020.03 query9 0.530.460.49 query10 0.540.540.53 query11 0.120.090.09 query12 0.100.090.08 query13 0.620.610.62 query14 0.790.780.79 query15 0.800.760.78 query16 0.390.360.35 query17 0.980.991.00 query18 0.230.240.25 query19 1.931.871.86 query20 0.010.010.01 query21 15.49 0.560.55 query22 1.702.711.95 query23 17.22 1.030.89 query24 5.261.421.18 query25 0.390.080.05 query26 0.580.160.15 query27 0.040.050.04 query28 7.640.720.72 query29 12.69 2.332.32 query30 0.620.520.53 query31 2.820.390.37 query32 3.370.500.50 query33 3.073.113.05 query34 15.26 4.794.80 query35 4.874.884.84 query36 1.041.021.03 query37 0.060.040.04 query38 0.030.020.02 query39 0.020.010.01 query40 0.170.140.14 query41 0.060.020.02 query42 0.020.020.01 query43 0.020.020.02 Total cold run time: 103.4 s Total hot run time: 31.27 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
doris-robot commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2099737584 TPC-DS: Total hot run time: 202312 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit f739f98b68a7ba7271c8e038fbbdf99612b8ce37, data reload: false query1 922 391 374 374 query2 6524291927292729 query3 6910208 203 203 query4 19900 17986 17823 17823 query5 19731 650764386438 query6 281 218 222 218 query7 4155305 299 299 query8 265 232 221 221 query9 3128271026202620 query10 405 294 318 294 query11 11352 10705 10746 10705 query12 114 80 72 72 query13 5572691 651 651 query14 17818 13949 13538 13538 query15 356 227 229 227 query16 6453275 261 261 query17 17281456870 870 query18 2313401 397 397 query19 201 142 147 142 query20 79 70 76 70 query21 190 96 87 87 query22 5239485450254854 query23 32638 31917 31701 31701 query24 7040650965196509 query25 512 433 435 433 query26 632 164 155 155 query27 2057293 291 291 query28 6201235623212321 query29 2867275228232752 query30 233 160 159 159 query31 890 757 732 732 query32 65 59 64 59 query33 395 250 250 250 query34 835 468 467 467 query35 1118907 818 818 query36 1303128711461146 query37 86 66 59 59 query38 3058289829262898 query39 1358132213191319 query40 272 92 94 92 query41 37 36 34 34 query42 90 84 83 83 query43 759 735 753 735 query44 1112710 709 709 query45 246 230 229 229 query46 1215969 971 969 query47 2100171619241716 query48 1005701 685 685 query49 631 378 363 363 query50 866 602 600 600 query51 4702466646444644 query52 91 76 81 76 query53 439 316 310 310 query54 2667242824752428 query55 91 84 72 72 query56 192 209 202 202 query57 1341119811051105 query58 208 194 201 194 query59 4071397340183973 query60 209 200 202 200 query61 84 82 82 82 query62 776 468 493 468 query63 482 336 333 333 query64 2545146914301430 query65 3623355035403540 query66 803 375 376 375 query67 15996 15135 15428 15135 query68 9546638 663 638 query69 580 352 339 339 query70 1753141913661366 query71 424 306 302 302 query72 6470339434393394 query73 745 318 319 318 query74 6419588458375837 query75 5329369035853585 query76 5798114911991149 query77 1011249 253 249 query78 12652 11583 12037 11583 query79 7636642 656 642 query80 1003385 398 385 query81 455 231 227 227 query82 806 102 96 96 query83 165 132 130 130 query84 251 69 70 69 query85 848 293 290 290 query86 336 301 297 297 query87 3216303930353035 query88 4477235123842351 query89 391 290 330 290 query90 1896203 203 203 query91 167 140 136 136 query92 60 53 50 50 query93 6369588 560 560 query94 695 200 196 196 query95 1097103910521039 query96 652 339 329 329 query97 6509632363566323 query98 187 177 170 170 query99 2847790 889 790 Total cold run time: 312367 ms Total hot run time: 202312 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
doris-robot commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2099736769 TeamCity be ut coverage result: Function Coverage: 37.81% (8078/21365) Line Coverage: 29.45% (65939/223899) Region Coverage: 28.92% (33946/117383) Branch Coverage: 24.77% (17418/70308) Coverage Report: http://coverage.selectdb-in.cc/coverage/f739f98b68a7ba7271c8e038fbbdf99612b8ce37_f739f98b68a7ba7271c8e038fbbdf99612b8ce37/report/index.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
doris-robot commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2099727277 TPC-H: Total hot run time: 50182 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit f739f98b68a7ba7271c8e038fbbdf99612b8ce37, data reload: false -- Round 1 -- q1 17539 435943344334 q2 2030162 142 142 q3 10265 196019521952 q4 10105 124113101241 q5 8452415339433943 q6 238 123 123 123 q7 2049160115661566 q8 9301275228182752 q9 11092 11129 10571 10571 q10 8703353635413536 q11 421 235 237 235 q12 463 302 311 302 q13 18341 397540933975 q14 355 321 324 321 q15 503 461 458 458 q16 688 568 569 568 q17 1138968 981 968 q18 7395687470406874 q19 1705156115141514 q20 539 326 325 325 q21 4511416040914091 q22 497 393 391 391 Total cold run time: 116330 ms Total hot run time: 50182 ms - Round 2, with runtime_filter_mode=off - q1 4300436743384338 q2 319 216 222 216 q3 4177415341284128 q4 2775275327602753 q5 7180714771777147 q6 233 120 118 118 q7 3290283328082808 q8 4352452144814481 q9 16909 16835 16738 16738 q10 4236431142334233 q11 763 670 707 670 q12 1019845 851 845 q13 4910373737533737 q14 453 420 414 414 q15 504 461 446 446 q16 738 682 672 672 q17 3916385438793854 q18 8776874687278727 q19 1731173016701670 q20 2391217721132113 q21 8491857485078507 q22 1049932 989 932 Total cold run time: 82512 ms Total hot run time: 79547 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
Lchangliang commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2099684267 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
github-actions[bot] commented on code in PR #34515:
URL: https://github.com/apache/doris/pull/34515#discussion_r1593348260
##
be/src/olap/storage_engine.h:
##
@@ -233,6 +233,8 @@ class StorageEngine {
std::set get_broken_paths() { return _broken_paths; }
+int64_t memory_limitation_bytes_per_thread_for_schema_change() const;
Review Comment:
warning: function 'memory_limitation_bytes_per_thread_for_schema_change'
should be marked [[nodiscard]] [modernize-use-nodiscard]
```suggestion
[[nodiscard]] int64_t
memory_limitation_bytes_per_thread_for_schema_change() const;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
doris-robot commented on PR #34515: URL: https://github.com/apache/doris/pull/34515#issuecomment-2099677731 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
dataroaring merged PR #34350: URL: https://github.com/apache/doris/pull/34350 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
github-actions[bot] commented on PR #34350: URL: https://github.com/apache/doris/pull/34350#issuecomment-2087801366 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
github-actions[bot] commented on PR #34350: URL: https://github.com/apache/doris/pull/34350#issuecomment-2085319933 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
dataroaring commented on PR #34350: URL: https://github.com/apache/doris/pull/34350#issuecomment-2085173086 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
github-actions[bot] commented on PR #34350: URL: https://github.com/apache/doris/pull/34350#issuecomment-2084602517 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improve] Make the schema change memory space adaptive [doris]
doris-robot commented on PR #34350: URL: https://github.com/apache/doris/pull/34350#issuecomment-2084592379 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
