subject:"Re\: \[PR\] \[fix\]\(mow\) modify delete bitmap storage format in ms \[doris\]"

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-29 Thread via GitHub



bobhan1 commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2309853583


##
regression-test/suites/unique_with_mow_p0/cloud_delete_bitmap/test_cu_compaction.groovy:
##
@@ -0,0 +1,138 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import org.apache.doris.regression.suite.ClusterOptions
+
+suite("test_cu_compaction", "docker") {
+def options = new ClusterOptions()
+options.beConfigs += [
+'delete_bitmap_store_version=2',
+'delete_bitmap_max_bytes_store_in_fdb=-1',
+'enable_delete_bitmap_store_v2_check_correctness=true',
+'enable_debug_points=true',
+'enable_java_support=false',
+ // trash
+'pending_data_expire_time_sec=0',
+'path_gc_check_interval_second=1',
+'trash_file_expire_time_sec=0',
+'tablet_rowset_stale_sweep_time_sec=1',
+'min_garbage_sweep_interval=1'
+]
+options.setFeNum(1)
+options.setBeNum(1)
+options.cloudMode = true
+
+def checkRowsetNum = { tablet, rowsetNum ->
+String compactionUrl = tablet["CompactionStatus"]
+def (code, out, err) = curl("GET", compactionUrl)
+logger.info("Show tablets status: code=" + code + ", out=" + out + ", 
err=" + err)
+assertEquals(code, 0)
+def tabletJson = parseJson(out.trim())
+assert tabletJson.rowsets instanceof List
+return tabletJson.rowsets.size() <= rowsetNum
+}
+
+docker(options) {
+sql """
+CREATE TABLE test_cu_compaction (
+`k` int(11) NOT NULL,
+`v` int(11) NOT NULL
+) ENGINE=OLAP
+unique KEY(`k`)
+DISTRIBUTED BY HASH(`k`) BUCKETS 1
+PROPERTIES (
+"disable_auto_compaction"="true",
+"replication_num" = "1"
+); 
+"""
+
+def backendId_to_backendIP = [:]
+def backendId_to_backendHttpPort = [:]
+def backendId_to_params = [string: [:]]
+getBackendIpHttpPort(backendId_to_backendIP, 
backendId_to_backendHttpPort)
+
+def tablets = sql_return_maparray "show tablets from 
test_cu_compaction"
+assertEquals(1, tablets.size())
+def tablet = tablets[0]
+String tablet_id = tablet.TabletId
+def backend_id = tablet.BackendId
+
+def triggerCompaction = { start_version, end_version, rowsetNum ->
+
GetDebugPoint().enableDebugPointForAllBEs("CloudSizeBasedCumulativeCompactionPolicy::pick_input_rowsets.set_input_rowsets",
+[tablet_id: "${tablet.TabletId}", start_version: 
start_version, end_version: end_version])
+def (code, out, err) = 
be_run_cumulative_compaction(backendId_to_backendIP.get(backend_id), 
backendId_to_backendHttpPort.get(backend_id), tablet_id)
+logger.info("Run compaction: code=" + code + ", out=" + out + ", 
err=" + err)
+assertEquals(code, 0)
+def compactJson = parseJson(out.trim())
+logger.info("compact json: " + compactJson)
+
+// check rowset num
+def check_result = false
+for (int i = 0; i < 20; i++) {
+check_result = checkRowsetNum(tablet, rowsetNum)
+if (check_result) {
+break
+}
+sleep(100)
+}
+assertTrue(check_result)
+}
+
+// v2
+sql """ insert into test_cu_compaction values(11, 11), (12, 12), (13, 
13), (14, 14), (15, 15), (16, 16), (17, 17), (18, 18), (19, 19) """
+// v3
+sql """ insert into test_cu_compaction values(21, 21), (22, 22), (23, 
23), (24, 24), (25, 25), (26, 26), (27, 27), (28, 28), (29, 29) """
+// v4
+sql """ insert into test_cu_compaction values(31, 31), (32, 32), (33, 
33), (34, 34), (35, 35), (36, 36), (37, 37), (38, 38), (39, 39) """
+// v5
+sql """ insert into test_cu_compaction values(41, 41), (42, 42), (43, 
43), (44, 44), (45, 45), (46, 46), (47, 47), (48, 48), (49, 49),
+   (11, 41), (21, 41), (31, 41) """
+// v6
+sql """ insert into test_cu_compaction

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-29 Thread via GitHub



dataroaring merged PR #53908:
URL: https://github.com/apache/doris/pull/53908


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-29 Thread via GitHub



github-actions[bot] commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3236021636

   PR approved by at least one committer and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-29 Thread via GitHub



github-actions[bot] commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3236021734

   PR approved by anyone and no changes requested.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-28 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3232891931

   # BE Regression && UT Coverage Report
   Increment line coverage `37.44% (246/657)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_1adb79f37daaf3e46764c68808633f3ef3231b4b_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_1adb79f37daaf3e46764c68808633f3ef3231b4b_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 70.68% (23010/32556) |
   | Line Coverage | 56.95% (239168/419945) |
   | Region Coverage   | 52.42% (198939/379503) |
   | Branch Coverage   | 54.06% (85835/158766) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-28 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3232571278

   # FE UT Coverage Report
   Increment line coverage `` :tada:
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_1adb79f37daaf3e46764c68808633f3ef3231b4b/fe_increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_1adb79f37daaf3e46764c68808633f3ef3231b4b/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-28 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3232511625

   
   
   TPC-DS: Total hot run time: 187173 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 1adb79f37daaf3e46764c68808633f3ef3231b4b, 
data reload: false
   
   query1   1045449 402 402
   query2   6581179817601760
   query3   6750227 227 227
   query4   26153   23372   22983   22983
   query5   4436663 518 518
   query6   354 265 257 257
   query7   4653540 307 307
   query8   310 265 256 256
   query9   8625289928952895
   query10  467 376 303 303
   query11  15845   15133   14910   14910
   query12  187 124 129 124
   query13  1697598 440 440
   query14  9934592958915891
   query15  215 198 189 189
   query16  7672647 514 514
   query17  1482772 658 658
   query18  2073497 340 340
   query19  202 197 172 172
   query20  134 133 119 119
   query21  223 131 119 119
   query22  4133412839663966
   query23  33891   33107   32944   32944
   query24  7831239524072395
   query25  570 525 458 458
   query26  1243282 169 169
   query27  2683522 362 362
   query28  4194226922342234
   query29  784 625 517 517
   query30  293 226 202 202
   query31  916 817 746 746
   query32  90  85  85  85
   query33  568 388 361 361
   query34  789 840 560 560
   query35  844 842 756 756
   query36  951 1022921 921
   query37  129 114 95  95
   query38  4121402039703970
   query39  1486144414591444
   query40  232 139 134 134
   query41  72  70  72  70
   query42  129 111 117 111
   query43  514 490 487 487
   query44  1337881 910 881
   query45  187 182 174 174
   query46  867 1023648 648
   query47  1783180617491749
   query48  402 433 340 340
   query49  747 509 428 428
   query50  649 679 404 404
   query51  4240416041924160
   query52  120 117 106 106
   query53  249 261 201 201
   query54  623 613 552 552
   query55  92  94  97  94
   query56  363 343 312 312
   query57  1199121011271127
   query58  298 293 292 292
   query59  2668277426402640
   query60  371 359 363 359
   query61  168 164 161 161
   query62  808 755 664 664
   query63  233 210 202 202
   query64  43571196844 844
   query65  4352423942074207
   query66  1085451 357 357
   query67  15730   15566   15110   15110
   query68  8761939 590 590
   query69  498 337 304 304
   query70  1210112111841121
   query71  490 353 316 316
   query72  5797506851975068
   query73  793 664 364 364
   query74  9265892889438928
   query75  3929319026682668
   query76  36661160744 744
   query77  790 403 337 337
   query78  9668974388498849
   query79  2104820 600 600
   query80  646 595 518 518
   query81  541 264 229 229
   query82  416 156 111 111
   query83  268 266 254 254
   query84  264 112 90  90
   query85  896 487 447 447
   query86  367 327 331 327
   query87  4322431042064206
   query88  33902235
   query89  387 340 292 292
   query90  1919231 232 231
   query91  165 183 140 140
   query92  95  78  70  70
   query93  1728998 663 663
   query94  708 426 341 341
   query95  414 338 335 335
   query96  484 588 286 286
   query97  2670268625672567
   query98  260 236 223 223
   query99  1418141313491349
   Total cold run time: 276758 ms
   Total hot run time: 187173 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-28 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3232561145

   # BE UT Coverage Report
   Increment line coverage `15.17% (103/679)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/1adb79f37daaf3e46764c68808633f3ef3231b4b_1adb79f37daaf3e46764c68808633f3ef3231b4b/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/1adb79f37daaf3e46764c68808633f3ef3231b4b_1adb79f37daaf3e46764c68808633f3ef3231b4b/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 51.79% (17162/33138) |
   | Line Coverage | 37.23% (156382/420077) |
   | Region Coverage   | 31.88% (119240/373994) |
   | Branch Coverage   | 33.22% (52417/157787) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-28 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3232534689

   
   
   ClickBench: Total hot run time: 32.65 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 1adb79f37daaf3e46764c68808633f3ef3231b4b, 
data reload: false
   
   query1   0.060.060.05
   query2   0.090.060.05
   query3   0.260.080.08
   query4   1.610.120.12
   query5   0.460.410.40
   query6   1.170.640.64
   query7   0.030.030.03
   query8   0.050.050.05
   query9   0.620.530.52
   query10  0.570.590.58
   query11  0.170.110.12
   query12  0.160.130.12
   query13  0.630.630.63
   query14  0.810.880.82
   query15  0.880.870.85
   query16  0.400.410.39
   query17  1.051.051.06
   query18  0.220.210.21
   query19  1.931.861.80
   query20  0.020.010.01
   query21  15.40   1.010.59
   query22  0.791.270.72
   query23  14.77   1.460.63
   query24  6.731.030.62
   query25  0.500.170.14
   query26  0.630.160.14
   query27  0.060.050.06
   query28  9.190.960.43
   query29  12.79   3.903.21
   query30  3.123.043.05
   query31  2.820.600.39
   query32  3.240.560.47
   query33  3.073.013.15
   query34  16.22   5.564.91
   query35  4.944.914.97
   query36  0.710.520.50
   query37  0.100.080.08
   query38  0.060.040.04
   query39  0.040.030.03
   query40  0.180.150.14
   query41  0.090.030.03
   query42  0.040.030.03
   query43  0.050.030.04
   Total cold run time: 106.73 s
   Total hot run time: 32.65 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-28 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3232470031

   
   
   TPC-H: Total hot run time: 34291 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 1adb79f37daaf3e46764c68808633f3ef3231b4b, 
data reload: false
   
   -- Round 1 --
   q1   16463   526451335133
   q2   2019352 213 213
   q3   97661319719 719
   q4   97671048528 528
   q5   7321239423492349
   q6   188 170 142 142
   q7   938 769 639 639
   q8   9027136310791079
   q9   6966521551855185
   q10  6897241820012001
   q11  502 303 280 280
   q12  385 365 230 230
   q13  17406   363230343034
   q14  239 245 230 230
   q15  561 505 513 505
   q16  446 443 376 376
   q17  595 884 369 369
   q18  7466729071447144
   q19  1090950 559 559
   q20  344 349 228 228
   q21  3796321023262326
   q22  1126105010221022
   Total cold run time: 103308 ms
   Total hot run time: 34291 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5205519851595159
   q2   253 340 234 234
   q3   2155269923582358
   q4   1393179913301330
   q5   4249410247014102
   q6   224 176 138 138
   q7   2092205118021802
   q8   2637268625782578
   q9   7555743174277427
   q10  3100341028862886
   q11  594 531 519 519
   q12  742 814 674 674
   q13  3499393732973297
   q14  281 321 282 282
   q15  553 516 509 509
   q16  481 514 453 453
   q17  1166159114241424
   q18  7919796675807580
   q19  870 873 904 873
   q20  2032209218811881
   q21  5043454643684368
   q22  1112104810191019
   Total cold run time: 53155 ms
   Total hot run time: 50893 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-28 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3232328884

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-28 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3232409334

   # Cloud UT Coverage Report
   Increment line coverage `76.69% (398/519)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/1adb79f37daaf3e46764c68808633f3ef3231b4b_1adb79f37daaf3e46764c68808633f3ef3231b4b_cloud/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/1adb79f37daaf3e46764c68808633f3ef3231b4b_1adb79f37daaf3e46764c68808633f3ef3231b4b_cloud/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 83.76% (1439/1718) |
   | Line Coverage | 67.40% (25738/38188) |
   | Region Coverage   | 68.44% (12797/18698) |
   | Branch Coverage   | 58.30% (6809/11680) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3232100899

   # BE UT Coverage Report
   Increment line coverage `15.17% (103/679)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/fddbaa02ec8ed36fe4389cfda7abab36353d535e_fddbaa02ec8ed36fe4389cfda7abab36353d535e/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/fddbaa02ec8ed36fe4389cfda7abab36353d535e_fddbaa02ec8ed36fe4389cfda7abab36353d535e/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 51.79% (17162/33138) |
   | Line Coverage | 37.23% (156408/420072) |
   | Region Coverage   | 31.93% (119398/373968) |
   | Branch Coverage   | 33.24% (52438/157775) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3231935859

   
   
   ClickBench: Total hot run time: 33.64 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit fddbaa02ec8ed36fe4389cfda7abab36353d535e, 
data reload: false
   
   query1   0.040.040.04
   query2   0.120.080.06
   query3   0.300.070.08
   query4   1.610.090.08
   query5   0.440.420.41
   query6   1.160.640.64
   query7   0.030.030.02
   query8   0.070.060.06
   query9   0.650.540.53
   query10  0.590.590.60
   query11  0.260.130.13
   query12  0.250.140.14
   query13  0.670.640.66
   query14  0.840.870.85
   query15  0.960.870.89
   query16  0.400.400.39
   query17  1.071.071.06
   query18  0.240.220.23
   query19  2.041.801.86
   query20  0.020.020.01
   query21  15.40   1.030.72
   query22  0.951.140.97
   query23  14.69   1.560.80
   query24  5.160.620.32
   query25  0.180.100.09
   query26  0.560.210.17
   query27  0.090.090.10
   query28  11.12   1.160.58
   query29  12.58   4.083.35
   query30  3.113.043.08
   query31  2.830.630.42
   query32  3.240.600.51
   query33  3.133.163.08
   query34  16.45   5.544.78
   query35  4.894.854.89
   query36  0.680.530.51
   query37  0.220.190.18
   query38  0.170.170.16
   query39  0.060.050.05
   query40  0.210.200.18
   query41  0.120.060.06
   query42  0.070.060.06
   query43  0.060.060.06
   Total cold run time: 107.73 s
   Total hot run time: 33.64 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3231921284

   
   
   TPC-DS: Total hot run time: 186649 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit fddbaa02ec8ed36fe4389cfda7abab36353d535e, 
data reload: false
   
   query1   1069439 435 435
   query2   6565172117801721
   query3   6754231 228 228
   query4   26628   23637   22875   22875
   query5   4382639 506 506
   query6   352 238 217 217
   query7   4674504 306 306
   query8   298 263 235 235
   query9   8606287728992877
   query10  480 364 291 291
   query11  15749   15038   14751   14751
   query12  181 126 120 120
   query13  1676562 438 438
   query14  8655586358295829
   query15  216 190 185 185
   query16  7202661 502 502
   query17  1253768 662 662
   query18  2036454 353 353
   query19  212 212 190 190
   query20  135 128 125 125
   query21  222 134 114 114
   query22  4189436640264026
   query23  33933   33066   33220   33066
   query24  8125247924382438
   query25  585 501 452 452
   query26  1260279 167 167
   query27  2736514 355 355
   query28  4415224922262226
   query29  797 589 495 495
   query30  293 235 212 212
   query31  908 814 806 806
   query32  86  83  81  81
   query33  578 385 371 371
   query34  792 852 537 537
   query35  839 840 750 750
   query36  994 998 947 947
   query37  125 109 94  94
   query38  4155397739893977
   query39  1490144514591445
   query40  233 138 134 134
   query41  72  70  62  62
   query42  125 113 115 113
   query43  504 528 475 475
   query44  1349882 861 861
   query45  182 174 178 174
   query46  886 1016655 655
   query47  1746180117371737
   query48  403 432 336 336
   query49  748 517 417 417
   query50  653 678 406 406
   query51  4046413941464139
   query52  124 131 115 115
   query53  245 283 201 201
   query54  629 631 558 558
   query55  100 93  88  88
   query56  335 334 322 322
   query57  1212122811481148
   query58  289 281 286 281
   query59  2660279626632663
   query60  361 353 351 351
   query61  164 154 156 154
   query62  819 747 674 674
   query63  233 204 201 201
   query64  44351123830 830
   query65  4329420542634205
   query66  1159443 366 366
   query67  15709   15292   15369   15292
   query68  8924917 590 590
   query69  493 330 294 294
   query70  1269113411231123
   query71  470 348 322 322
   query72  5651496047494749
   query73  742 616 376 376
   query74  9073904787868786
   query75  4232308726072607
   query76  37121227775 775
   query77  818 396 324 324
   query78  9527979889118911
   query79  2120847 602 602
   query80  707 586 510 510
   query81  479 263 227 227
   query82  241 142 108 108
   query83  306 268 252 252
   query84  312 119 100 100
   query85  852 457 421 421
   query86  356 310 302 302
   query87  4344433741904190
   query88  2850225922602259
   query89  412 337 290 290
   query90  2096234 237 234
   query91  163 167 143 143
   query92  91  76  76  76
   query93  1418996 661 661
   query94  695 415 333 333
   query95  428 344 329 329
   query96  487 602 287 287
   query97  2710273625952595
   query98  241 223 218 218
   query99  1438137612921292
   Total cold run time: 275052 ms
   Total hot run time: 186649 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3231900053

   
   
   TPC-H: Total hot run time: 34479 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit fddbaa02ec8ed36fe4389cfda7abab36353d535e, 
data reload: false
   
   -- Round 1 --
   q1   17282   530251325132
   q2   2004328 205 205
   q3   10033   1325747 747
   q4   10243   1064549 549
   q5   7532247923512351
   q6   186 171 144 144
   q7   911 774 660 660
   q8   9336135011411141
   q9   7144516451315131
   q10  6969239919961996
   q11  485 311 289 289
   q12  364 360 223 223
   q13  17797   368030393039
   q14  237 251 220 220
   q15  570 525 496 496
   q16  452 437 396 396
   q17  615 871 361 361
   q18  7920732771557155
   q19  1096972 580 580
   q20  344 371 253 253
   q21  4112318724132413
   q22  10431025998 998
   Total cold run time: 106675 ms
   Total hot run time: 34479 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5181513451275127
   q2   255 336 234 234
   q3   2184268423112311
   q4   1479179213411341
   q5   4276455045884550
   q6   225 173 132 132
   q7   2096200018111811
   q8   2714269225672567
   q9   7565725674057256
   q10  3163324829982998
   q11  583 515 502 502
   q12  791 958 639 639
   q13  3488401733403340
   q14  279 300 288 288
   q15  535 475 469 469
   q16  490 520 444 444
   q17  1188161113841384
   q18  7949790076937693
   q19  892 844 967 844
   q20  2082227718881888
   q21  4986439644354396
   q22  10731074989 989
   Total cold run time: 53474 ms
   Total hot run time: 51203 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3231847781

   # FE UT Coverage Report
   Increment line coverage `` :tada:
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_fddbaa02ec8ed36fe4389cfda7abab36353d535e/fe_increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_fddbaa02ec8ed36fe4389cfda7abab36353d535e/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3231747986

   # Cloud UT Coverage Report
   Increment line coverage `76.69% (398/519)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/fddbaa02ec8ed36fe4389cfda7abab36353d535e_fddbaa02ec8ed36fe4389cfda7abab36353d535e_cloud/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/fddbaa02ec8ed36fe4389cfda7abab36353d535e_fddbaa02ec8ed36fe4389cfda7abab36353d535e_cloud/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 83.76% (1439/1718) |
   | Line Coverage | 67.42% (25748/38188) |
   | Region Coverage   | 68.47% (12803/18698) |
   | Branch Coverage   | 58.23% (6801/11680) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3231707753

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3228913389

   # BE Regression && UT Coverage Report
   Increment line coverage `13.02% (85/653)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 56.26% (18317/32556) |
   | Line Coverage | 41.54% (174424/419931) |
   | Region Coverage   | 35.76% (135697/379451) |
   | Branch Coverage   | 37.03% (58788/158742) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3228440734

   run cloud_p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3228462780

   # BE Regression && UT Coverage Report
   Increment line coverage `13.02% (85/653)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 56.26% (18317/32556) |
   | Line Coverage | 41.54% (174424/419931) |
   | Region Coverage   | 35.76% (135697/379451) |
   | Branch Coverage   | 37.03% (58788/158742) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3228224392

   # BE UT Coverage Report
   Increment line coverage `15.26% (103/675)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 51.79% (17161/33138) |
   | Line Coverage | 37.23% (156385/420063) |
   | Region Coverage   | 31.91% (119309/373942) |
   | Branch Coverage   | 33.22% (52413/157763) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227962320

   
   
   TPC-DS: Total hot run time: 187461 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 769d3a01874c9bbb5bb89c35fbe72d7a868ace9b, 
data reload: false
   
   query1   1060462 417 417
   query2   6547179018431790
   query3   6754233 224 224
   query4   26474   23785   23193   23193
   query5   5423694 532 532
   query6   345 261 225 225
   query7   4647521 311 311
   query8   300 254 246 246
   query9   8669289628802880
   query10  534 353 304 304
   query11  16084   15086   15173   15086
   query12  185 126 127 126
   query13  1678593 433 433
   query14  10881   591058815881
   query15  247 199 173 173
   query16  7672667 495 495
   query17  1591760 627 627
   query18  2084439 347 347
   query19  199 199 174 174
   query20  131 125 123 123
   query21  224 128 114 114
   query22  4021429440104010
   query23  33956   33043   33203   33043
   query24  7883240623882388
   query25  597 612 493 493
   query26  1252289 178 178
   query27  2709528 356 356
   query28  4415229322552255
   query29  809 643 516 516
   query30  299 227 206 206
   query31  906 813 763 763
   query32  95  86  88  86
   query33  603 412 376 376
   query34  814 857 536 536
   query35  832 864 755 755
   query36  974 1048916 916
   query37  133 135 96  96
   query38  4076410139863986
   query39  1514144114311431
   query40  230 142 142 142
   query41  79  72  71  71
   query42  143 131 125 125
   query43  536 530 490 490
   query44  1419885 878 878
   query45  187 176 176 176
   query46  877 1024656 656
   query47  1766182017051705
   query48  406 448 331 331
   query49  757 522 423 423
   query50  685 700 413 413
   query51  4147415641524152
   query52  124 116 109 109
   query53  262 278 204 204
   query54  627 621 549 549
   query55  100 95  96  95
   query56  361 330 349 330
   query57  1249120311451145
   query58  288 289 285 285
   query59  2613277526552655
   query60  360 357 354 354
   query61  179 155 164 155
   query62  790 730 659 659
   query63  243 206 197 197
   query64  43961165839 839
   query65  4323426542234223
   query66  1073441 349 349
   query67  15523   15241   14975   14975
   query68  9039940 603 603
   query69  523 336 304 304
   query70  1313110311261103
   query71  509 366 343 343
   query72  5857508152145081
   query73  768 691 366 366
   query74  9012898686568656
   query75  4277308827002700
   query76  37471149769 769
   query77  815 435 343 343
   query78  9614956788578857
   query79  3597874 594 594
   query80  713 590 613 590
   query81  491 258 229 229
   query82  477 143 116 116
   query83  294 282 256 256
   query84  308 116 98  98
   query85  865 474 443 443
   query86  345 316 309 309
   query87  4364441342124212
   query88  3293227122802271
   query89  447 326 293 293
   query90  2006236 243 236
   query91  166 160 132 132
   query92  93  82  72  72
   query93  26281008658 658
   query94  692 429 334 334
   query95  417 344 333 333
   query96  490 601 282 282
   query97  2684269325952595
   query98  249 223 219 219
   query99  1466138512881288
   Total cold run time: 282919 ms
   Total hot run time: 187461 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227983226

   
   
   ClickBench: Total hot run time: 33.24 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 769d3a01874c9bbb5bb89c35fbe72d7a868ace9b, 
data reload: false
   
   query1   0.060.040.05
   query2   0.100.050.06
   query3   0.250.090.09
   query4   1.610.110.11
   query5   0.450.450.42
   query6   1.190.630.68
   query7   0.030.030.02
   query8   0.060.040.04
   query9   0.600.540.53
   query10  0.590.580.57
   query11  0.170.120.12
   query12  0.150.130.13
   query13  0.630.640.61
   query14  0.800.850.85
   query15  0.880.870.85
   query16  0.400.420.40
   query17  1.031.071.07
   query18  0.210.200.20
   query19  1.891.831.84
   query20  0.020.020.01
   query21  15.44   0.950.60
   query22  0.771.240.77
   query23  14.77   1.390.62
   query24  6.521.871.10
   query25  0.500.260.17
   query26  0.570.160.13
   query27  0.060.050.06
   query28  9.620.910.44
   query29  12.60   3.913.23
   query30  3.142.982.96
   query31  2.840.590.38
   query32  3.240.560.46
   query33  3.163.063.16
   query34  15.93   5.484.92
   query35  4.944.904.93
   query36  0.720.510.49
   query37  0.100.080.08
   query38  0.060.050.04
   query39  0.040.030.03
   query40  0.180.150.14
   query41  0.080.030.03
   query42  0.050.030.03
   query43  0.040.040.04
   Total cold run time: 106.49 s
   Total hot run time: 33.24 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227916950

   
   
   TPC-H: Total hot run time: 34495 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 769d3a01874c9bbb5bb89c35fbe72d7a868ace9b, 
data reload: false
   
   -- Round 1 --
   q1   17604   530351305130
   q2   1994325 239 239
   q3   10230   1299754 754
   q4   10222   1039534 534
   q5   7476246724132413
   q6   187 173 138 138
   q7   941 766 644 644
   q8   9361137711491149
   q9   7058508953455089
   q10  6917242519881988
   q11  480 302 286 286
   q12  348 374 227 227
   q13  17771   366430763076
   q14  251 264 228 228
   q15  557 509 503 503
   q16  431 462 380 380
   q17  596 897 368 368
   q18  7694722872007200
   q19  898 963 595 595
   q20  363 355 234 234
   q21  3863319523432343
   q22  10601008977 977
   Total cold run time: 106302 ms
   Total hot run time: 34495 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5225515952745159
   q2   251 328 231 231
   q3   2183273823812381
   q4   1363180514031403
   q5   4231456346444563
   q6   225 183 146 146
   q7   2073204818211821
   q8   2712269027092690
   q9   7505743674387436
   q10  3139336729212921
   q11  575 526 518 518
   q12  719 801 622 622
   q13  3585392833653365
   q14  296 300 266 266
   q15  588 476 501 476
   q16  466 517 486 486
   q17  1256156813921392
   q18  7992761975907590
   q19  835 877 936 877
   q20  1965206119271927
   q21  5083433843514338
   q22  10871046996 996
   Total cold run time: 53354 ms
   Total hot run time: 51604 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227894396

   # FE UT Coverage Report
   Increment line coverage `` :tada:
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b/fe_increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227807204

   # Cloud UT Coverage Report
   Increment line coverage `76.69% (398/519)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_cloud/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_769d3a01874c9bbb5bb89c35fbe72d7a868ace9b_cloud/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 83.76% (1439/1718) |
   | Line Coverage | 67.42% (25745/38188) |
   | Region Coverage   | 68.44% (12796/18698) |
   | Branch Coverage   | 58.29% (6808/11680) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227746799

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227737059

   # BE Regression && UT Coverage Report
   Increment line coverage `65.08% (425/653)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_eff1fe124fdc3133bbf6115f77dfa0e5ab330cd7_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_eff1fe124fdc3133bbf6115f77dfa0e5ab330cd7_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 70.56% (22971/32556) |
   | Line Coverage | 56.81% (238555/419916) |
   | Region Coverage   | 52.12% (197774/379450) |
   | Branch Coverage   | 53.87% (85512/158740) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227415948

   # BE UT Coverage Report
   Increment line coverage `15.26% (103/675)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/eff1fe124fdc3133bbf6115f77dfa0e5ab330cd7_eff1fe124fdc3133bbf6115f77dfa0e5ab330cd7/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/eff1fe124fdc3133bbf6115f77dfa0e5ab330cd7_eff1fe124fdc3133bbf6115f77dfa0e5ab330cd7/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 51.78% (17160/33138) |
   | Line Coverage | 37.22% (156359/420048) |
   | Region Coverage   | 31.88% (119204/373941) |
   | Branch Coverage   | 33.22% (52401/157761) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227361924

   
   
   ClickBench: Total hot run time: 33.67 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit eff1fe124fdc3133bbf6115f77dfa0e5ab330cd7, 
data reload: false
   
   query1   0.050.050.05
   query2   0.120.060.07
   query3   0.290.080.07
   query4   1.610.090.09
   query5   0.420.420.42
   query6   1.180.640.64
   query7   0.030.020.03
   query8   0.070.060.07
   query9   0.660.530.54
   query10  0.580.580.60
   query11  0.260.130.14
   query12  0.270.140.14
   query13  0.660.640.64
   query14  0.830.830.85
   query15  0.970.870.87
   query16  0.400.390.38
   query17  1.051.081.08
   query18  0.240.220.22
   query19  1.981.831.80
   query20  0.010.010.02
   query21  15.40   1.030.72
   query22  0.941.110.87
   query23  14.71   1.490.78
   query24  4.800.590.34
   query25  0.180.110.10
   query26  0.540.220.18
   query27  0.100.090.09
   query28  11.18   1.160.57
   query29  12.54   4.013.34
   query30  3.163.043.03
   query31  2.810.610.44
   query32  3.240.610.50
   query33  3.233.173.17
   query34  16.88   5.514.78
   query35  4.894.964.89
   query36  0.680.520.51
   query37  0.210.180.18
   query38  0.180.160.17
   query39  0.050.050.05
   query40  0.200.180.18
   query41  0.100.060.07
   query42  0.070.070.06
   query43  0.060.060.05
   Total cold run time: 107.83 s
   Total hot run time: 33.67 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227343179

   
   
   TPC-DS: Total hot run time: 186670 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit eff1fe124fdc3133bbf6115f77dfa0e5ab330cd7, 
data reload: false
   
   query1   1047435 436 435
   query2   6607181217831783
   query3   6754228 225 225
   query4   26515   23932   23412   23412
   query5   4407653 512 512
   query6   359 245 237 237
   query7   4647508 304 304
   query8   310 273 241 241
   query9   8657292828982898
   query10  484 329 306 306
   query11  15806   15034   14708   14708
   query12  171 124 119 119
   query13  1689587 434 434
   query14  8743590357715771
   query15  227 197 176 176
   query16  7162657 482 482
   query17  957 755 631 631
   query18  2004438 342 342
   query19  202 198 175 175
   query20  129 127 123 123
   query21  223 132 110 110
   query22  4163422040004000
   query23  33827   32656   32885   32656
   query24  7950240424022402
   query25  596 529 448 448
   query26  1242277 173 173
   query27  2738512 355 355
   query28  4167230122552255
   query29  802 607 497 497
   query30  292 231 203 203
   query31  919 811 746 746
   query32  92  87  79  79
   query33  599 406 381 381
   query34  824 871 526 526
   query35  833 831 773 773
   query36  982 989 925 925
   query37  141 119 92  92
   query38  4104402740694027
   query39  1486140814241408
   query40  230 135 129 129
   query41  71  64  65  64
   query42  128 115 120 115
   query43  530 529 479 479
   query44  1398879 871 871
   query45  184 174 175 174
   query46  886 1015651 651
   query47  1731182117501750
   query48  417 441 348 348
   query49  767 518 422 422
   query50  666 682 410 410
   query51  4051412840644064
   query52  124 119 109 109
   query53  258 305 221 221
   query54  619 645 551 551
   query55  94  89  93  89
   query56  342 332 316 316
   query57  1191119111401140
   query58  289 292 278 278
   query59  2616281526412641
   query60  362 366 351 351
   query61  170 162 169 162
   query62  793 723 662 662
   query63  247 198 196 196
   query64  41981159867 867
   query65  4305417942284179
   query66  1215450 356 356
   query67  15613   15230   15078   15078
   query68  7949937 586 586
   query69  517 338 305 305
   query70  1202115611071107
   query71  452 348 340 340
   query72  5732512151355121
   query73  681 654 360 360
   query74  8898893388638863
   query75  3389311826282628
   query76  34991172747 747
   query77  812 437 337 337
   query78  9528964089578957
   query79  2183814 596 596
   query80  654 638 519 519
   query81  493 267 227 227
   query82  401 147 115 115
   query83  260 270 245 245
   query84  303 110 100 100
   query85  933 482 437 437
   query86  376 341 304 304
   query87  4351435941264126
   query88  34812263
   query89  405 323 292 292
   query90  1946231 220 220
   query91  163 172 140 140
   query92  91  79  74  74
   query93  1886998 648 648
   query94  677 467 329 329
   query95  415 338 330 330
   query96  493 581 280 280
   query97  2653272025622562
   query98  259 224 218 218
   query99  1377143212921292
   Total cold run time: 272811 ms
   Total hot run time: 186670 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227306741

   
   
   TPC-H: Total hot run time: 34244 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit eff1fe124fdc3133bbf6115f77dfa0e5ab330cd7, 
data reload: false
   
   -- Round 1 --
   q1   16446   524450865086
   q2   1995350 212 212
   q3   97321311738 738
   q4   96791036533 533
   q5   7189239123752375
   q6   185 166 136 136
   q7   920 763 636 636
   q8   8950135711921192
   q9   7016515351585153
   q10  6911241219941994
   q11  479 306 280 280
   q12  351 365 232 232
   q13  17394   368329932993
   q14  240 246 226 226
   q15  560 505 502 502
   q16  452 432 387 387
   q17  591 859 370 370
   q18  7502709270377037
   q19  1090970 577 577
   q20  346 340 229 229
   q21  3941321923622362
   q22  10721049994 994
   Total cold run time: 103041 ms
   Total hot run time: 34244 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5171512351295123
   q2   261 329 234 234
   q3   2211266023272327
   q4   1374177213721372
   q5   4436408844174088
   q6   220 171 135 135
   q7   2088195218951895
   q8   2744264726782647
   q9   7456730772837283
   q10  3132332129152915
   q11  589 569 507 507
   q12  698 793 647 647
   q13  3505394632953295
   q14  291 311 295 295
   q15  532 477 490 477
   q16  473 730 433 433
   q17  1151155014141414
   q18  7940765576137613
   q19  820 812 848 812
   q20  1978203519311931
   q21  480141814181
   q22  1059101910061006
   Total cold run time: 52930 ms
   Total hot run time: 50630 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-27 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3227203647

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-26 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3224439636

   # BE Regression && UT Coverage Report
   Increment line coverage `63.25% (413/653)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_c06c2e0d1a7fc0e9ad35bce1d6b9ca8753e95787_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_c06c2e0d1a7fc0e9ad35bce1d6b9ca8753e95787_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 70.72% (23024/32556) |
   | Line Coverage | 56.98% (239271/419916) |
   | Region Coverage   | 52.47% (199098/379450) |
   | Branch Coverage   | 54.10% (85884/158740) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-26 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3224222014

   # BE UT Coverage Report
   Increment line coverage `15.26% (103/675)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/c06c2e0d1a7fc0e9ad35bce1d6b9ca8753e95787_c06c2e0d1a7fc0e9ad35bce1d6b9ca8753e95787/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/c06c2e0d1a7fc0e9ad35bce1d6b9ca8753e95787_c06c2e0d1a7fc0e9ad35bce1d6b9ca8753e95787/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 51.78% (17160/33138) |
   | Line Coverage | 37.22% (156340/420048) |
   | Region Coverage   | 31.88% (119219/373941) |
   | Branch Coverage   | 33.21% (52392/157761) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-26 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3223887393

   # FE UT Coverage Report
   Increment line coverage `` :tada:
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_c06c2e0d1a7fc0e9ad35bce1d6b9ca8753e95787/fe_increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_c06c2e0d1a7fc0e9ad35bce1d6b9ca8753e95787/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-26 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3223695303

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-25 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3219720029

   # BE Regression && UT Coverage Report
   Increment line coverage `65.94% (426/646)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_bbeaeb7633a15819a904f20ba95a2dcae06fd7b0_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_bbeaeb7633a15819a904f20ba95a2dcae06fd7b0_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 70.74% (23016/32535) |
   | Line Coverage | 57.00% (239250/419729) |
   | Region Coverage   | 52.37% (198751/379500) |
   | Branch Coverage   | 54.10% (85860/158706) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-25 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3219527328

   # BE UT Coverage Report
   Increment line coverage `14.97% (100/668)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/bbeaeb7633a15819a904f20ba95a2dcae06fd7b0_bbeaeb7633a15819a904f20ba95a2dcae06fd7b0/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/bbeaeb7633a15819a904f20ba95a2dcae06fd7b0_bbeaeb7633a15819a904f20ba95a2dcae06fd7b0/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 51.76% (17149/33129) |
   | Line Coverage | 37.20% (156222/419946) |
   | Region Coverage   | 31.87% (119203/374074) |
   | Branch Coverage   | 33.20% (52374/157748) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-25 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3219317102

   
   
   ClickBench: Total hot run time: 33.25 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit bbeaeb7633a15819a904f20ba95a2dcae06fd7b0, 
data reload: false
   
   query1   0.040.040.03
   query2   0.080.050.04
   query3   0.230.070.07
   query4   1.630.110.11
   query5   0.410.400.40
   query6   1.160.640.65
   query7   0.030.020.02
   query8   0.050.040.04
   query9   0.600.510.51
   query10  0.580.560.57
   query11  0.160.100.11
   query12  0.150.120.11
   query13  0.630.600.61
   query14  0.790.850.82
   query15  0.870.870.85
   query16  0.380.390.38
   query17  1.031.041.02
   query18  0.210.200.20
   query19  1.891.921.77
   query20  0.010.000.01
   query21  15.43   0.950.59
   query22  0.771.160.73
   query23  14.84   1.370.61
   query24  6.481.601.38
   query25  0.470.260.19
   query26  0.580.170.13
   query27  0.050.050.05
   query28  9.870.950.43
   query29  12.72   3.973.27
   query30  3.113.083.03
   query31  2.810.580.37
   query32  3.240.570.48
   query33  3.053.083.15
   query34  15.79   5.464.88
   query35  4.894.914.95
   query36  0.680.510.50
   query37  0.090.070.07
   query38  0.060.050.03
   query39  0.030.030.02
   query40  0.170.150.14
   query41  0.080.030.03
   query42  0.040.030.03
   query43  0.040.030.03
   Total cold run time: 106.22 s
   Total hot run time: 33.25 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-25 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3219301698

   
   
   TPC-DS: Total hot run time: 184532 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit bbeaeb7633a15819a904f20ba95a2dcae06fd7b0, 
data reload: false
   
   query1   977 400 417 400
   query2   6547173817251725
   query3   6753232 211 211
   query4   26257   23734   23026   23026
   query5   4795638 514 514
   query6   329 221 213 213
   query7   4637504 286 286
   query8   275 253 226 226
   query9   8654288928712871
   query10  504 329 274 274
   query11  15685   15103   14781   14781
   query12  164 115 115 115
   query13  1659566 431 431
   query14  9432587558245824
   query15  209 188 171 171
   query16  7650673 506 506
   query17  1471734 590 590
   query18  2037419 328 328
   query19  202 195 161 161
   query20  126 120 126 120
   query21  219 122 109 109
   query22  4029426540764076
   query23  34044   32773   32632   32632
   query24  8009235023552350
   query25  537 469 429 429
   query26  1233267 164 164
   query27  2701498 344 344
   query28  4263225722382238
   query29  723 559 463 463
   query30  289 222 190 190
   query31  922 792 726 726
   query32  86  74  78  74
   query33  560 367 334 334
   query34  809 842 505 505
   query35  803 821 751 751
   query36  967 1012933 933
   query37  121 105 87  87
   query38  4070401340204013
   query39  1498142713861386
   query40  220 127 112 112
   query41  78  55  53  53
   query42  126 112 113 112
   query43  498 500 469 469
   query44  1369855 850 850
   query45  177 169 168 168
   query46  869 1007630 630
   query47  1726180417121712
   query48  389 421 316 316
   query49  729 470 387 387
   query50  641 678 402 402
   query51  4111406241494062
   query52  118 116 104 104
   query53  237 271 189 189
   query54  589 603 523 523
   query55  113 82  88  82
   query56  315 304 302 302
   query57  1176119111061106
   query58  279 274 278 274
   query59  2592273626162616
   query60  359 329 315 315
   query61  134 128 125 125
   query62  775 707 672 672
   query63  228 200 200 200
   query64  42041022703 703
   query65  4330421042004200
   query66  1077413 330 330
   query67  15607   15239   15039   15039
   query68  8258929 582 582
   query69  493 312 279 279
   query70  1178115111261126
   query71  512 329 309 309
   query72  5569468446614661
   query73  690 569 350 350
   query74  9111902389188918
   query75  3882304926142614
   query76  37771149767 767
   query77  796 408 326 326
   query78  9656963788958895
   query79  2571855 610 610
   query80  652 549 485 485
   query81  483 249 222 222
   query82  428 137 110 110
   query83  296 252 236 236
   query84  293 97  88  88
   query85  791 374 334 334
   query86  341 320 312 312
   query87  4314414541214121
   query88  3331220922102209
   query89  401 318 283 283
   query90  1977226 223 223
   query91  143 142 114 114
   query92  87  70  66  66
   query93  1716970 647 647
   query94  674 398 309 309
   query95  394 324 315 315
   query96  522 573 275 275
   query97  2645268826052605
   query98  244 223 216 216
   query99  1447145312951295
   Total cold run time: 274984 ms
   Total hot run time: 184532 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-25 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3219264673

   
   
   TPC-H: Total hot run time: 34066 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit bbeaeb7633a15819a904f20ba95a2dcae06fd7b0, 
data reload: false
   
   -- Round 1 --
   q1   17216   523651025102
   q2   1900293 178 178
   q3   10110   1298735 735
   q4   10207   1015530 530
   q5   7599237923592359
   q6   178 159 131 131
   q7   901 734 596 596
   q8   9307133311321132
   q9   7010512551355125
   q10  6946241319781978
   q11  498 295 264 264
   q12  355 355 212 212
   q13  17780   360630133013
   q14  238 234 236 234
   q15  574 486 495 486
   q16  423 414 363 363
   q17  592 861 363 363
   q18  7548710471087104
   q19  1229951 578 578
   q20  335 339 227 227
   q21  3894320823902390
   q22  10941045966 966
   Total cold run time: 105934 ms
   Total hot run time: 34066 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5231516851275127
   q2   243 322 225 225
   q3   2207274323522352
   q4   1346179213521352
   q5   4226410140934093
   q6   208 163 124 124
   q7   1892179816661666
   q8   2486245123972397
   q9   6840679467656765
   q10  2971315227002700
   q11  566 516 489 489
   q12  665 718 607 607
   q13  3252361130073007
   q14  276 311 256 256
   q15  514 464 476 464
   q16  424 459 417 417
   q17  1112146613631363
   q18  7210705470277027
   q19  793 839 896 839
   q20  1901196318311831
   q21  4869430043884300
   q22  1067106210051005
   Total cold run time: 50299 ms
   Total hot run time: 48406 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-25 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3219105719

   # Cloud UT Coverage Report
   Increment line coverage `81.77% (323/395)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/bbeaeb7633a15819a904f20ba95a2dcae06fd7b0_bbeaeb7633a15819a904f20ba95a2dcae06fd7b0_cloud/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/bbeaeb7633a15819a904f20ba95a2dcae06fd7b0_bbeaeb7633a15819a904f20ba95a2dcae06fd7b0_cloud/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 83.76% (1424/1700) |
   | Line Coverage | 67.11% (25324/37736) |
   | Region Coverage   | 68.04% (12576/18484) |
   | Branch Coverage   | 57.90% (6665/11512) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-24 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3219049669

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-22 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3214603041

   # BE Regression && UT Coverage Report
   Increment line coverage `58.82% (370/629)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_2e627f6ea7aa89829d934818605016b9cb723dbc_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_2e627f6ea7aa89829d934818605016b9cb723dbc_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 70.65% (22986/32534) |
   | Line Coverage | 56.91% (238870/419717) |
   | Region Coverage   | 52.34% (198642/379529) |
   | Branch Coverage   | 53.98% (85675/158708) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-22 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3214339488

   # BE UT Coverage Report
   Increment line coverage `2.07% (13/629)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/2e627f6ea7aa89829d934818605016b9cb723dbc_2e627f6ea7aa89829d934818605016b9cb723dbc/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/2e627f6ea7aa89829d934818605016b9cb723dbc_2e627f6ea7aa89829d934818605016b9cb723dbc/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 51.73% (17137/33128) |
   | Line Coverage | 37.18% (156141/419912) |
   | Region Coverage   | 31.86% (119160/374066) |
   | Branch Coverage   | 33.18% (52339/157731) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-22 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3214186868

   
   
   ClickBench: Total hot run time: 33.37 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 2e627f6ea7aa89829d934818605016b9cb723dbc, 
data reload: false
   
   query1   0.050.030.03
   query2   0.120.060.05
   query3   0.290.060.07
   query4   1.600.090.08
   query5   0.420.400.41
   query6   1.170.650.64
   query7   0.030.020.02
   query8   0.060.050.05
   query9   0.630.530.52
   query10  0.580.580.58
   query11  0.260.120.13
   query12  0.250.140.14
   query13  0.650.630.63
   query14  0.830.850.83
   query15  0.970.860.88
   query16  0.380.370.38
   query17  1.061.051.08
   query18  0.230.210.23
   query19  1.971.871.85
   query20  0.020.020.02
   query21  15.37   1.020.71
   query22  0.931.170.87
   query23  14.72   1.610.75
   query24  5.150.620.33
   query25  0.160.100.09
   query26  0.540.210.18
   query27  0.090.090.09
   query28  11.04   1.230.57
   query29  12.54   4.073.39
   query30  3.163.113.04
   query31  2.810.620.42
   query32  3.230.600.50
   query33  3.223.103.11
   query34  16.50   5.514.78
   query35  4.934.854.87
   query36  0.650.530.50
   query37  0.200.180.19
   query38  0.180.160.16
   query39  0.050.040.04
   query40  0.200.180.17
   query41  0.100.060.05
   query42  0.060.050.05
   query43  0.060.040.04
   Total cold run time: 107.46 s
   Total hot run time: 33.37 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-22 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3214167920

   
   
   TPC-DS: Total hot run time: 183591 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 2e627f6ea7aa89829d934818605016b9cb723dbc, 
data reload: false
   
   query1   1012382 402 382
   query2   6539176316981698
   query3   6743220 224 220
   query4   26296   23648   22803   22803
   query5   4387610 488 488
   query6   314 221 223 221
   query7   4636495 288 288
   query8   281 235 237 235
   query9   8596287728982877
   query10  466 329 278 278
   query11  15881   15100   14659   14659
   query12  161 116 117 116
   query13  1686560 425 425
   query14  9255568855775577
   query15  198 187 171 171
   query16  7683644 475 475
   query17  1199735 587 587
   query18  2080395 319 319
   query19  189 177 157 157
   query20  122 166 116 116
   query21  203 119 106 106
   query22  4147425842674258
   query23  33730   32682   33028   32682
   query24  7976233923632339
   query25  531 522 401 401
   query26  1228265 157 157
   query27  2718510 359 359
   query28  4474225622112211
   query29  731 554 439 439
   query30  285 215 187 187
   query31  891 800 730 730
   query32  82  74  76  74
   query33  547 364 376 364
   query34  790 834 507 507
   query35  804 811 751 751
   query36  976 1022899 899
   query37  127 108 89  89
   query38  4071400739513951
   query39  1476140514131405
   query40  218 122 112 112
   query41  58  57  52  52
   query42  117 106 113 106
   query43  488 490 477 477
   query44  1388853 841 841
   query45  178 168 164 164
   query46  848 1002633 633
   query47  1780182817251725
   query48  382 443 311 311
   query49  730 471 393 393
   query50  669 679 397 397
   query51  4204411040994099
   query52  116 108 99  99
   query53  233 255 191 191
   query54  593 581 514 514
   query55  89  80  91  80
   query56  319 310 301 301
   query57  1181118511321132
   query58  279 269 268 268
   query59  2649273925992599
   query60  355 340 323 323
   query61  122 123 122 122
   query62  810 724 656 656
   query63  222 185 186 185
   query64  42581030734 734
   query65  4294420942584209
   query66  1077406 326 326
   query67  15388   15364   15127   15127
   query68  8746898 577 577
   query69  459 333 292 292
   query70  1203115911091109
   query71  462 323 308 308
   query72  5305467245534553
   query73  722 550 342 342
   query74  9287880188768801
   query75  4019305726062606
   query76  37661132722 722
   query77  796 396 313 313
   query78  9628966588488848
   query79  2292819 587 587
   query80  605 542 474 474
   query81  488 253 229 229
   query82  469 134 112 112
   query83  255 249 234 234
   query84  246 98  88  88
   query85  849 368 385 368
   query86  386 271 290 271
   query87  4268434041304130
   query88  3388218721542154
   query89  386 319 288 288
   query90  1894231 230 230
   query91  144 145 115 115
   query92  83  74  66  66
   query93  1740985 643 643
   query94  658 401 304 304
   query95  383 305 297 297
   query96  488 571 268 268
   query97  2726270625552555
   query98  251 209 211 209
   query99  1341140113211321
   Total cold run time: 274289 ms
   Total hot run time: 183591 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-22 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3214137994

   
   
   TPC-H: Total hot run time: 33645 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 2e627f6ea7aa89829d934818605016b9cb723dbc, 
data reload: false
   
   -- Round 1 --
   q1   17208   518949994999
   q2   1918292 177 177
   q3   10115   1295709 709
   q4   10202   1007523 523
   q5   7587245323022302
   q6   176 166 133 133
   q7   913 745 595 595
   q8   9298129710651065
   q9   7072514751145114
   q10  6938239019881988
   q11  492 283 271 271
   q12  335 349 218 218
   q13  17775   364129812981
   q14  229 250 222 222
   q15  573 476 477 476
   q16  428 421 366 366
   q17  599 872 357 357
   q18  7423710170387038
   q19  1091949 554 554
   q20  345 327 221 221
   q21  3788318523522352
   q22  10891043984 984
   Total cold run time: 105594 ms
   Total hot run time: 33645 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5111508650785078
   q2   241 320 223 223
   q3   2200271123212321
   q4   1364175713131313
   q5   4201433045624330
   q6   211 169 132 132
   q7   2057198118241824
   q8   2586267626232623
   q9   7437739773727372
   q10  3148326728502850
   q11  670 588 508 508
   q12  686 782 620 620
   q13  3460392932673267
   q14  290 339 293 293
   q15  528 474 468 468
   q16  492 509 445 445
   q17  1274164913591359
   q18  7877760278887602
   q19  810 869 975 869
   q20  2010205118251825
   q21  4795438143514351
   q22  11021033994 994
   Total cold run time: 52550 ms
   Total hot run time: 50667 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-22 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3213975902

   # Cloud UT Coverage Report
   Increment line coverage `81.77% (323/395)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/2e627f6ea7aa89829d934818605016b9cb723dbc_2e627f6ea7aa89829d934818605016b9cb723dbc_cloud/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/2e627f6ea7aa89829d934818605016b9cb723dbc_2e627f6ea7aa89829d934818605016b9cb723dbc_cloud/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 83.75% (1422/1698) |
   | Line Coverage | 67.11% (25164/37498) |
   | Region Coverage   | 68.08% (12525/18398) |
   | Branch Coverage   | 57.91% (6638/11462) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-22 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3213896946

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-21 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3212992604

   # BE UT Coverage Report
   Increment line coverage `1.84% (13/707)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/e7d8daf66e4a14a613c0d9a8a15fd407bd66444c_e7d8daf66e4a14a613c0d9a8a15fd407bd66444c/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/e7d8daf66e4a14a613c0d9a8a15fd407bd66444c_e7d8daf66e4a14a613c0d9a8a15fd407bd66444c/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 59.56% (16790/28190) |
   | Line Coverage | 48.43% (152781/315461) |
   | Region Coverage   | 37.34% (116120/310953) |
   | Branch Coverage   | 40.33% (51278/127148) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-21 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3212880715

   # FE UT Coverage Report
   Increment line coverage `` :tada:
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_e7d8daf66e4a14a613c0d9a8a15fd407bd66444c/fe_increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_e7d8daf66e4a14a613c0d9a8a15fd407bd66444c/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-21 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3212764689

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-21 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3210545607

   
   
   ClickBench: Total hot run time: 33.45 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit d3dff4c720e37f309a2727bb54f116ed1af6d039, 
data reload: false
   
   query1   0.040.030.03
   query2   0.110.060.05
   query3   0.310.060.06
   query4   1.600.080.07
   query5   0.420.410.41
   query6   1.170.650.64
   query7   0.020.020.02
   query8   0.060.050.05
   query9   0.650.530.54
   query10  0.590.580.59
   query11  0.250.130.13
   query12  0.250.130.14
   query13  0.640.620.63
   query14  0.830.830.84
   query15  0.960.880.88
   query16  0.380.380.38
   query17  1.051.031.05
   query18  0.230.220.23
   query19  1.961.821.84
   query20  0.020.010.02
   query21  15.38   1.020.73
   query22  0.931.100.91
   query23  14.70   1.530.79
   query24  4.970.600.32
   query25  0.170.110.09
   query26  0.540.220.18
   query27  0.090.090.09
   query28  11.08   1.200.59
   query29  12.63   4.003.37
   query30  3.153.083.00
   query31  2.830.600.43
   query32  3.240.590.50
   query33  3.133.093.07
   query34  16.72   5.484.80
   query35  4.834.854.91
   query36  0.650.540.52
   query37  0.220.190.18
   query38  0.190.180.17
   query39  0.060.050.05
   query40  0.210.170.17
   query41  0.100.060.05
   query42  0.060.060.05
   query43  0.060.050.05
   Total cold run time: 107.48 s
   Total hot run time: 33.45 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-21 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3210525569

   
   
   TPC-DS: Total hot run time: 184889 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit d3dff4c720e37f309a2727bb54f116ed1af6d039, 
data reload: false
   
   query1   972 395 406 395
   query2   6547175117461746
   query3   6743232 215 215
   query4   26171   23707   22984   22984
   query5   4343650 488 488
   query6   302 226 209 209
   query7   4625517 298 298
   query8   271 242 235 235
   query9   8584287629152876
   query10  469 332 273 273
   query11  15762   15480   14755   14755
   query12  176 144 112 112
   query13  1646559 399 399
   query14  10198   574457785744
   query15  203 185 162 162
   query16  7629621 457 457
   query17  1391687 570 570
   query18  2023417 327 327
   query19  189 194 160 160
   query20  120 114 114 114
   query21  209 121 110 110
   query22  4031416540344034
   query23  33665   32820   32994   32820
   query24  7799235923552355
   query25  521 462 407 407
   query26  1238301 156 156
   query27  2713494 345 345
   query28  4308228122172217
   query29  747 581 451 451
   query30  285 226 188 188
   query31  899 826 749 749
   query32  83  74  71  71
   query33  564 370 355 355
   query34  786 833 508 508
   query35  790 828 740 740
   query36  941 1025916 916
   query37  123 110 86  86
   query38  4007406640064006
   query39  1485141014261410
   query40  216 126 117 117
   query41  59  56  54  54
   query42  121 107 112 107
   query43  509 506 459 459
   query44  1340860 864 860
   query45  173 166 160 160
   query46  861 1023644 644
   query47  1751181017021702
   query48  388 414 313 313
   query49  720 494 384 384
   query50  682 682 400 400
   query51  4166419340764076
   query52  110 104 98  98
   query53  241 263 200 200
   query54  594 588 526 526
   query55  93  82  90  82
   query56  324 301 299 299
   query57  1162120711241124
   query58  281 274 271 271
   query59  2635279125852585
   query60  344 332 317 317
   query61  127 124 121 121
   query62  780 757 657 657
   query63  230 195 191 191
   query64  41891050802 802
   query65  4320422542024202
   query66  1051445 316 316
   query67  15459   15455   15178   15178
   query68  8126955 585 585
   query69  465 321 293 293
   query70  1206111211381112
   query71  467 326 331 326
   query72  5577482449044824
   query73  685 655 357 357
   query74  9014915289318931
   query75  3337307726362636
   query76  34041139720 720
   query77  561 383 328 328
   query78  9545956589028902
   query79  1624833 595 595
   query80  601 543 494 494
   query81  515 251 219 219
   query82  219 136 110 110
   query83  255 254 251 251
   query84  256 107 90  90
   query85  845 447 415 415
   query86  378 317 320 317
   query87  4347435141554155
   query88  2783222522052205
   query89  388 313 283 283
   query90  1954227 222 222
   query91  140 141 116 116
   query92  85  72  68  68
   query93  1772980 642 642
   query94  738 404 303 303
   query95  389 306 312 306
   query96  484 599 275 275
   query97  2613271226362636
   query98  239 229 212 212
   query99  1330145612731273
   Total cold run time: 270826 ms
   Total hot run time: 184889 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-21 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3210483144

   
   
   TPC-H: Total hot run time: 34011 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit d3dff4c720e37f309a2727bb54f116ed1af6d039, 
data reload: false
   
   -- Round 1 --
   q1   17019   517550915091
   q2   1915272 175 175
   q3   10096   1299728 728
   q4   10200   995 520 520
   q5   7602238323542354
   q6   180 161 133 133
   q7   901 761 600 600
   q8   9308132011421142
   q9   7065523651005100
   q10  6908238219551955
   q11  458 296 268 268
   q12  355 354 225 225
   q13  17781   370730823082
   q14  229 232 216 216
   q15  552 482 488 482
   q16  433 427 378 378
   q17  595 866 362 362
   q18  7856707771407077
   q19  1094955 567 567
   q20  339 346 221 221
   q21  4182321123752375
   q22  11081042960 960
   Total cold run time: 106176 ms
   Total hot run time: 34011 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5156513152885131
   q2   243 329 218 218
   q3   2153272523222322
   q4   1345175013341334
   q5   4247449144634463
   q6   218 175 132 132
   q7   2025200018321832
   q8   2663266626722666
   q9   7339726673827266
   q10  3132326129252925
   q11  563 513 512 512
   q12  681 768 635 635
   q13  3499395534043404
   q14  285 322 287 287
   q15  530 610 549 549
   q16  460 496 431 431
   q17  1161160413631363
   q18  7984771875067506
   q19  856 815 899 815
   q20  1886204618061806
   q21  4883427043104270
   q22  1073102710171017
   Total cold run time: 52382 ms
   Total hot run time: 50884 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-21 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3210383560

   # FE UT Coverage Report
   Increment line coverage `` :tada:
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_d3dff4c720e37f309a2727bb54f116ed1af6d039/fe_increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_d3dff4c720e37f309a2727bb54f116ed1af6d039/fe_report/index.html)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-21 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3210114304

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-21 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3210267354

   # Cloud UT Coverage Report
   Increment line coverage `81.77% (323/395)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/d3dff4c720e37f309a2727bb54f116ed1af6d039_d3dff4c720e37f309a2727bb54f116ed1af6d039_cloud/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/d3dff4c720e37f309a2727bb54f116ed1af6d039_d3dff4c720e37f309a2727bb54f116ed1af6d039_cloud/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 83.73% (1420/1696) |
   | Line Coverage | 67.00% (25020/37341) |
   | Region Coverage   | 67.90% (12446/18330) |
   | Branch Coverage   | 57.76% (6590/11410) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-20 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2287236692


##
be/src/cloud/delete_bitmap_file_reader.cpp:
##
@@ -0,0 +1,120 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "cloud/delete_bitmap_file_reader.h"
+
+#include "cloud/delete_bitmap_file_writer.h"
+#include "common/status.h"
+#include "io/fs/file_reader.h"
+#include "util/coding.h"
+#include "util/crc32c.h"
+
+namespace doris {
+#include "common/compile_check_begin.h"
+
+DeleteBitmapFileReader::DeleteBitmapFileReader(int64_t tablet_id, const 
std::string& rowset_id,

Review Comment:
   add ut for file reader and file writer.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2287155473


##
be/src/cloud/cloud_cumulative_compaction.cpp:
##
@@ -394,7 +394,7 @@ Status CloudCumulativeCompaction::modify_rowsets() {
 }
 // agg delete bitmap for pre rowsets
 if (config::enable_agg_and_remove_pre_rowsets_delete_bitmap &&
-_tablet->keys_type() == KeysType::UNIQUE_KEYS &&
+config::delete_bitmap_store_version == 1 && _tablet->keys_type() == 
KeysType::UNIQUE_KEYS &&

Review Comment:
   If we agg it only for v1, I am not sure if agg works for v2.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2287152270


##
be/src/cloud/cloud_full_compaction.cpp:
##
@@ -356,8 +356,25 @@ Status 
CloudFullCompaction::_cloud_full_compaction_update_delete_bitmap(int64_t
   
delete_bitmap));
 }
 }
-RETURN_IF_ERROR(_engine.meta_mgr().update_delete_bitmap(*cloud_tablet(), 
-1, initiator,
-
delete_bitmap.get()));
+auto delete_bitmap_size = delete_bitmap->delete_bitmap.size();
+DeleteBitmapPtr delete_bitmap_v2 = 
std::make_shared(*delete_bitmap);
+if (config::delete_bitmap_store_version >= 2) {
+std::vector> 
pre_rowset_ids_to_segment_num;
+for (const auto& [rowset_version, rowset_ptr] : 
cloud_tablet()->rowset_map()) {
+if (rowset_version.second < _output_rowset->start_version() ||
+rowset_version.first > _output_rowset->end_version()) {
+pre_rowset_ids_to_segment_num.emplace_back(
+std::make_pair(rowset_ptr->rowset_id(), 
rowset_ptr->num_segments()));
+}
+}
+_tablet->tablet_meta()->delete_bitmap().subset(
+pre_rowset_ids_to_segment_num, 
_input_rowsets.front()->start_version(),
+_input_rowsets.back()->end_version(), delete_bitmap_v2.get());
+}

Review Comment:
   same as code in cloud_tablet.cpp afer line 1217. Maybe can let it as a 
static method of cloud tablet and call it here.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2287137422


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -1004,6 +1105,204 @@ Status 
CloudMetaMgr::sync_tablet_delete_bitmap(CloudTablet* tablet, int64_t old_
 return Status::OK();
 }
 
+Status CloudMetaMgr::sync_tablet_delete_bitmap_v2(
+CloudTablet* tablet, int64_t old_max_version, std::ranges::range 
auto&& rs_metas,
+const TabletStatsPB& stats, const TabletIndexPB& idx, DeleteBitmap* 
delete_bitmap,
+bool full_sync, SyncRowsetStats* sync_stats,
+std::map rowset_to_resource, bool all_sync) {
+// TODO support sync_tablet_delete_bitmap_by_cache, now sync from ms to 
check the correctness
+
+GetDeleteBitmapRequest req;
+GetDeleteBitmapResponse res;
+req.set_cloud_unique_id(config::cloud_unique_id);
+req.set_tablet_id(tablet->tablet_id());
+req.set_store_version(2);
+if (all_sync) {
+for (auto& [rowset_id, _] : rowset_to_resource) {
+req.add_rowset_ids(rowset_id);
+}
+} else {
+DCHECK_EQ(rowset_to_resource.size(), 0);
+req.set_base_compaction_cnt(stats.base_compaction_cnt());
+req.set_cumulative_compaction_cnt(stats.cumulative_compaction_cnt());
+req.set_cumulative_point(stats.cumulative_point());
+// When there are many delete bitmaps that need to be synchronized, it
+// may take a longer time, especially when loading the tablet for the
+// first time, so set a relatively long timeout time.
+*(req.mutable_idx()) = idx;
+for (const auto& rs_meta : rs_metas) {
+req.add_rowset_ids(rs_meta.rowset_id_v2());
+rowset_to_resource[rs_meta.rowset_id_v2()] = rs_meta.resource_id();
+}
+}
+
+if (sync_stats) {
+sync_stats->get_remote_delete_bitmap_rowsets_num += 
req.rowset_ids_size();
+}
+
+VLOG_DEBUG << "send GetDeleteBitmapRequest: " << req.ShortDebugString();
+
+auto start = std::chrono::steady_clock::now();
+auto st = retry_rpc("get delete bitmap", req, &res, 
&MetaService_Stub::get_delete_bitmap);
+auto end = std::chrono::steady_clock::now();
+if (st.code() == ErrorCode::THRIFT_RPC_ERROR) {
+return st;
+}
+
+if (res.status().code() == MetaServiceCode::TABLET_NOT_FOUND) {
+return Status::NotFound("failed to get delete bitmap: {}", 
res.status().msg());
+}
+// The delete bitmap of stale rowsets will be removed when commit 
compaction job,
+// then delete bitmap of stale rowsets cannot be obtained. But the rowsets 
obtained
+// by sync_tablet_rowsets may include these stale rowsets. When this case 
happend, the
+// error code of ROWSETS_EXPIRED will be returned, we need to retry sync 
rowsets again.
+//
+// Be query thread meta-service  Be compaction thread
+//  || |
+//  |get rowset  | |
+//  |--->| |
+//  |return get rowset   | |
+//  |<---| |
+//  ||commit job   |
+//  ||<|
+//  ||return commit job|
+//  ||>|
+//  |  get delete bitmap | |
+//  |--->| |
+//  |  return get delete bitmap  | |
+//  |<---| |
+//  || |
+if (res.status().code() == MetaServiceCode::ROWSETS_EXPIRED) {
+return Status::Error("failed to get 
delete bitmap: {}",
+
res.status().msg());
+}
+if (res.status().code() != MetaServiceCode::OK) {
+return Status::Error("failed to get 
delete bitmap: {}",
+   
res.status().msg());
+}
+const auto& rowset_ids = res.rowset_ids();
+const auto& delete_bitmap_storages = res.delete_bitmap_storages();
+if (rowset_ids.size() != delete_bitmap_storages.size()) {
+return Status::Error(
+"get delete bitmap data wrong, 
rowset_ids.size={},delete_bitmap_storages.size={}",
+rowset_ids.size(), delete_bitmap_storages.size());
+}
+if (sync_stats) {
+sync_stats->get_remote_delete_bitmap_rpc_ns +=
+std::chrono::duration_cast(end - 
start).count();
+sync_stats->get_remote_delete_bitmap_key_count += 
delete_bitmap_storages.size();
+}
+
+

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2286928033


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -711,6 +724,89 @@ Status 
CloudMetaMgr::sync_tablet_rowsets_unlocked(CloudTablet* tablet,
 .tag("new_rowsets(rowset,count,cardinality)",
  fmt::format("[{}]", fmt::join(new_rowset_msgs, ", 
")));
 }
+if (check_delete_bitmap_v2) {
+int64_t new_max_version =
+std::max(old_max_version, 
resp.rowset_meta().rbegin()->end_version());
+// rowset_id, num_segments
+std::vector> all_rowsets;
+std::map rowset_to_resource;
+for (const auto& rs_meta : resp.rowset_meta()) {
+RowsetId rowset_id;
+rowset_id.init(rs_meta.rowset_id_v2());
+all_rowsets.emplace_back(std::make_pair(rowset_id, 
rs_meta.num_segments()));
+rowset_to_resource[rs_meta.rowset_id_v2()] = 
rs_meta.resource_id();
+}
+if (old_max_version > 0) {
+RowsetIdUnorderedSet all_rs_ids;
+RETURN_IF_ERROR(tablet->get_all_rs_id(old_max_version, 
&all_rs_ids));
+for (auto& rowset : 
tablet->get_rowset_by_ids(&all_rs_ids)) {
+all_rowsets.emplace_back(
+std::make_pair(rowset->rowset_id(), 
rowset->num_segments()));
+rowset_to_resource[rowset->rowset_id().to_string()] =
+rowset->rowset_meta()->resource_id();
+}
+}
+
+auto compare_delete_bitmap = [&](DeleteBitmap* delete_bitmap, 
int version) {
+bool success = true;
+for (auto& [rs_id, num_segments] : all_rowsets) {
+for (int seg_id = 0; seg_id < num_segments; ++seg_id) {
+DeleteBitmap::BitmapKey key = {rs_id, seg_id, 
new_max_version};
+auto dm1 = 
tablet->tablet_meta()->delete_bitmap().get_agg(key);
+auto dm2 = 
delete_bitmap->get_agg_without_cache(key);
+if (*dm1 != *dm2) {
+success = false;
+LOG(WARNING) << "failed to check delete bitmap 
correctness by v"
+ << std::to_string(version)
+ << ", tablet_id=" << 
tablet->tablet_id()
+ << ", rowset_id=" << 
rs_id.to_string()
+ << ", segment_id=" << seg_id
+ << ", max_version=" << 
new_max_version
+ << ". size1=" << 
dm1->cardinality()
+ << ", size2=" << 
dm2->cardinality();
+}
+}
+}
+if (success) {
+LOG(INFO) << "succeed to check delete bitmap 
correctness by v"
+  << std::to_string(version)
+  << ", tablet_id=" << tablet->tablet_id()
+  << ", max_version=" << new_max_version;
+}
+};
+
+// check v2 delete bitmap in ms
+if (check_delete_bitmap_v2) {
+DeleteBitmap full_delete_bitmap(tablet_id);
+auto status = sync_tablet_delete_bitmap_v2(

Review Comment:
   delete bitmap is synced already above.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2286908537


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -711,6 +724,89 @@ Status 
CloudMetaMgr::sync_tablet_rowsets_unlocked(CloudTablet* tablet,
 .tag("new_rowsets(rowset,count,cardinality)",
  fmt::format("[{}]", fmt::join(new_rowset_msgs, ", 
")));
 }
+if (check_delete_bitmap_v2) {

Review Comment:
   The code in this branch only depends on old_max_version and resp, so we'd 
better move this branch to a independent function.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2286856193


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -1384,15 +1690,108 @@ Status CloudMetaMgr::update_delete_bitmap(const 
CloudTablet& tablet, int64_t loc
 if (next_visible_version > 0) {
 req.set_next_visible_version(next_visible_version);
 }
-for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
-req.add_rowset_ids(std::get<0>(key).to_string());
-req.add_segment_ids(std::get<1>(key));
-req.add_versions(std::get<2>(key));
-// To save space, convert array and bitmap containers to run containers
-bitmap.runOptimize();
-std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
-bitmap.write(bitmap_data.data());
-*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+
+bool write_v1 =
+config::delete_bitmap_store_version == 1 || 
config::delete_bitmap_store_version == 3;
+bool write_v2 = config::delete_bitmap_store_version >= 2;
+// write v1 kvs
+if (write_v1) {
+for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
+req.add_rowset_ids(std::get<0>(key).to_string());
+req.add_segment_ids(std::get<1>(key));
+req.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+}
+}
+
+// write v2 kvs
+if (write_v2) {
+// TODO skip update if delete bitmap is empty
+//  it's ok for compaction and schema change, but for load, ms check 
partition version
+auto add_delete_bitmap = [](DeleteBitmapPB& delete_bitmap_pb,
+const DeleteBitmap::BitmapKey& key, 
roaring::Roaring& bitmap) {
+delete_bitmap_pb.add_rowset_ids(std::get<0>(key).to_string());
+delete_bitmap_pb.add_segment_ids(std::get<1>(key));
+delete_bitmap_pb.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(delete_bitmap_pb.add_segment_delete_bitmaps()) = 
std::move(bitmap_data);
+};
+auto handle_rowset_delete_bitmap = [&](std::string& rowset_id,
+   DeleteBitmapPB& 
delete_bitmap_pb) {
+std::stringstream ss;
+DeleteBitmapStoragePB delete_bitmap_storage;
+for (int i = 0; i < delete_bitmap_pb.rowset_ids_size(); i++) {
+ss << "{rid=" << delete_bitmap_pb.rowset_ids(i)
+   << ", sid=" << delete_bitmap_pb.segment_ids(i)
+   << ", ver=" << delete_bitmap_pb.versions(i) << "}, ";
+}
+if (config::delete_bitmap_max_bytes_store_in_fdb >= 0 &&
+delete_bitmap_pb.ByteSizeLong() > 
config::delete_bitmap_max_bytes_store_in_fdb) {
+DeleteBitmapFileWriter file_writer(tablet.tablet_id(), 
rowset_id, storage_resource);
+RETURN_IF_ERROR(file_writer.init());
+RETURN_IF_ERROR(file_writer.write(delete_bitmap_pb));
+RETURN_IF_ERROR(file_writer.close());
+delete_bitmap_pb.Clear();
+delete_bitmap_storage.set_store_in_fdb(false);
+} else {
+delete_bitmap_storage.set_store_in_fdb(true);
+*(delete_bitmap_storage.mutable_delete_bitmap()) = 
std::move(delete_bitmap_pb);
+}
+LOG(INFO) << "handle one rowset delete bitmap for tablet_id: " << 
tablet.tablet_id()
+  << ", rowset_id: " << rowset_id
+  << ", delete_bitmap num: " << 
delete_bitmap_pb.rowset_ids_size()
+  << ", store_in_fdb=" << 
delete_bitmap_storage.store_in_fdb()
+  << ",  size: " << delete_bitmap_pb.ByteSizeLong() << ", 
keys=[" << ss.str()
+  << "]";
+req.add_v2_rowset_ids(rowset_id);
+*(req.add_delete_bitmap_storages()) = 
std::move(delete_bitmap_storage);
+return Status::OK();
+};
+req.set_store_version(2);
+LOG(INFO) << "update delete bitmap for tablet_id: " << 
tablet.tablet_id()

Review Comment:
   should be controlled by already existed mow_debug config.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service,

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2285525390


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -1384,15 +1690,108 @@ Status CloudMetaMgr::update_delete_bitmap(const 
CloudTablet& tablet, int64_t loc
 if (next_visible_version > 0) {
 req.set_next_visible_version(next_visible_version);
 }
-for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
-req.add_rowset_ids(std::get<0>(key).to_string());
-req.add_segment_ids(std::get<1>(key));
-req.add_versions(std::get<2>(key));
-// To save space, convert array and bitmap containers to run containers
-bitmap.runOptimize();
-std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
-bitmap.write(bitmap_data.data());
-*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+
+bool write_v1 =
+config::delete_bitmap_store_version == 1 || 
config::delete_bitmap_store_version == 3;
+bool write_v2 = config::delete_bitmap_store_version >= 2;
+// write v1 kvs
+if (write_v1) {
+for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
+req.add_rowset_ids(std::get<0>(key).to_string());
+req.add_segment_ids(std::get<1>(key));
+req.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+}
+}
+
+// write v2 kvs
+if (write_v2) {
+// TODO skip update if delete bitmap is empty
+//  it's ok for compaction and schema change, but for load, ms check 
partition version
+auto add_delete_bitmap = [](DeleteBitmapPB& delete_bitmap_pb,
+const DeleteBitmap::BitmapKey& key, 
roaring::Roaring& bitmap) {
+delete_bitmap_pb.add_rowset_ids(std::get<0>(key).to_string());
+delete_bitmap_pb.add_segment_ids(std::get<1>(key));
+delete_bitmap_pb.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(delete_bitmap_pb.add_segment_delete_bitmaps()) = 
std::move(bitmap_data);
+};
+auto handle_rowset_delete_bitmap = [&](std::string& rowset_id,
+   DeleteBitmapPB& 
delete_bitmap_pb) {
+std::stringstream ss;
+DeleteBitmapStoragePB delete_bitmap_storage;
+for (int i = 0; i < delete_bitmap_pb.rowset_ids_size(); i++) {
+ss << "{rid=" << delete_bitmap_pb.rowset_ids(i)
+   << ", sid=" << delete_bitmap_pb.segment_ids(i)
+   << ", ver=" << delete_bitmap_pb.versions(i) << "}, ";
+}
+if (config::delete_bitmap_max_bytes_store_in_fdb >= 0 &&
+delete_bitmap_pb.ByteSizeLong() > 
config::delete_bitmap_max_bytes_store_in_fdb) {
+DeleteBitmapFileWriter file_writer(tablet.tablet_id(), 
rowset_id, storage_resource);
+RETURN_IF_ERROR(file_writer.init());
+RETURN_IF_ERROR(file_writer.write(delete_bitmap_pb));
+RETURN_IF_ERROR(file_writer.close());
+delete_bitmap_pb.Clear();
+delete_bitmap_storage.set_store_in_fdb(false);
+} else {
+delete_bitmap_storage.set_store_in_fdb(true);
+*(delete_bitmap_storage.mutable_delete_bitmap()) = 
std::move(delete_bitmap_pb);
+}
+LOG(INFO) << "handle one rowset delete bitmap for tablet_id: " << 
tablet.tablet_id()
+  << ", rowset_id: " << rowset_id
+  << ", delete_bitmap num: " << 
delete_bitmap_pb.rowset_ids_size()
+  << ", store_in_fdb=" << 
delete_bitmap_storage.store_in_fdb()
+  << ",  size: " << delete_bitmap_pb.ByteSizeLong() << ", 
keys=[" << ss.str()
+  << "]";
+req.add_v2_rowset_ids(rowset_id);
+*(req.add_delete_bitmap_storages()) = 
std::move(delete_bitmap_storage);
+return Status::OK();
+};
+req.set_store_version(2);
+LOG(INFO) << "update delete bitmap for tablet_id: " << 
tablet.tablet_id()
+  << ", rowset_id: " << rowset_id
+  << ", delete_bitmap num: " << 
delete_bitmap_v2->delete_bitmap.size()
+  << ", lock_id=" << lock_id << ", initiator=" << initiator;
+if (rowset_id.empty()) {

Review Comment:
   Does only schema change run the path?



-- 
This is an automated

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2285522079


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -1384,15 +1690,108 @@ Status CloudMetaMgr::update_delete_bitmap(const 
CloudTablet& tablet, int64_t loc
 if (next_visible_version > 0) {
 req.set_next_visible_version(next_visible_version);
 }
-for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
-req.add_rowset_ids(std::get<0>(key).to_string());
-req.add_segment_ids(std::get<1>(key));
-req.add_versions(std::get<2>(key));
-// To save space, convert array and bitmap containers to run containers
-bitmap.runOptimize();
-std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
-bitmap.write(bitmap_data.data());
-*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+
+bool write_v1 =
+config::delete_bitmap_store_version == 1 || 
config::delete_bitmap_store_version == 3;
+bool write_v2 = config::delete_bitmap_store_version >= 2;
+// write v1 kvs
+if (write_v1) {
+for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
+req.add_rowset_ids(std::get<0>(key).to_string());
+req.add_segment_ids(std::get<1>(key));
+req.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+}
+}
+
+// write v2 kvs
+if (write_v2) {
+// TODO skip update if delete bitmap is empty
+//  it's ok for compaction and schema change, but for load, ms check 
partition version
+auto add_delete_bitmap = [](DeleteBitmapPB& delete_bitmap_pb,
+const DeleteBitmap::BitmapKey& key, 
roaring::Roaring& bitmap) {
+delete_bitmap_pb.add_rowset_ids(std::get<0>(key).to_string());
+delete_bitmap_pb.add_segment_ids(std::get<1>(key));
+delete_bitmap_pb.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(delete_bitmap_pb.add_segment_delete_bitmaps()) = 
std::move(bitmap_data);
+};
+auto handle_rowset_delete_bitmap = [&](std::string& rowset_id,
+   DeleteBitmapPB& 
delete_bitmap_pb) {
+std::stringstream ss;
+DeleteBitmapStoragePB delete_bitmap_storage;
+for (int i = 0; i < delete_bitmap_pb.rowset_ids_size(); i++) {
+ss << "{rid=" << delete_bitmap_pb.rowset_ids(i)
+   << ", sid=" << delete_bitmap_pb.segment_ids(i)
+   << ", ver=" << delete_bitmap_pb.versions(i) << "}, ";
+}
+if (config::delete_bitmap_max_bytes_store_in_fdb >= 0 &&
+delete_bitmap_pb.ByteSizeLong() > 
config::delete_bitmap_max_bytes_store_in_fdb) {
+DeleteBitmapFileWriter file_writer(tablet.tablet_id(), 
rowset_id, storage_resource);
+RETURN_IF_ERROR(file_writer.init());
+RETURN_IF_ERROR(file_writer.write(delete_bitmap_pb));
+RETURN_IF_ERROR(file_writer.close());
+delete_bitmap_pb.Clear();
+delete_bitmap_storage.set_store_in_fdb(false);
+} else {
+delete_bitmap_storage.set_store_in_fdb(true);
+*(delete_bitmap_storage.mutable_delete_bitmap()) = 
std::move(delete_bitmap_pb);
+}
+LOG(INFO) << "handle one rowset delete bitmap for tablet_id: " << 
tablet.tablet_id()
+  << ", rowset_id: " << rowset_id
+  << ", delete_bitmap num: " << 
delete_bitmap_pb.rowset_ids_size()
+  << ", store_in_fdb=" << 
delete_bitmap_storage.store_in_fdb()
+  << ",  size: " << delete_bitmap_pb.ByteSizeLong() << ", 
keys=[" << ss.str()
+  << "]";
+req.add_v2_rowset_ids(rowset_id);
+*(req.add_delete_bitmap_storages()) = 
std::move(delete_bitmap_storage);
+return Status::OK();
+};
+req.set_store_version(2);
+LOG(INFO) << "update delete bitmap for tablet_id: " << 
tablet.tablet_id()
+  << ", rowset_id: " << rowset_id
+  << ", delete_bitmap num: " << 
delete_bitmap_v2->delete_bitmap.size()
+  << ", lock_id=" << lock_id << ", initiator=" << initiator;
+if (rowset_id.empty()) {
+std::string pre_rowset_id = "";
+DeleteBitmapPB delete_bitmap_pb;

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2285518236


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -1384,15 +1690,108 @@ Status CloudMetaMgr::update_delete_bitmap(const 
CloudTablet& tablet, int64_t loc
 if (next_visible_version > 0) {
 req.set_next_visible_version(next_visible_version);
 }
-for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
-req.add_rowset_ids(std::get<0>(key).to_string());
-req.add_segment_ids(std::get<1>(key));
-req.add_versions(std::get<2>(key));
-// To save space, convert array and bitmap containers to run containers
-bitmap.runOptimize();
-std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
-bitmap.write(bitmap_data.data());
-*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+
+bool write_v1 =
+config::delete_bitmap_store_version == 1 || 
config::delete_bitmap_store_version == 3;
+bool write_v2 = config::delete_bitmap_store_version >= 2;
+// write v1 kvs
+if (write_v1) {
+for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
+req.add_rowset_ids(std::get<0>(key).to_string());
+req.add_segment_ids(std::get<1>(key));
+req.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+}
+}
+
+// write v2 kvs
+if (write_v2) {
+// TODO skip update if delete bitmap is empty
+//  it's ok for compaction and schema change, but for load, ms check 
partition version
+auto add_delete_bitmap = [](DeleteBitmapPB& delete_bitmap_pb,
+const DeleteBitmap::BitmapKey& key, 
roaring::Roaring& bitmap) {
+delete_bitmap_pb.add_rowset_ids(std::get<0>(key).to_string());
+delete_bitmap_pb.add_segment_ids(std::get<1>(key));
+delete_bitmap_pb.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(delete_bitmap_pb.add_segment_delete_bitmaps()) = 
std::move(bitmap_data);
+};
+auto handle_rowset_delete_bitmap = [&](std::string& rowset_id,

Review Comment:
   store_delete_bitmap is a better name than handle_rowset_delete_bitmap?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2285507975


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -1384,15 +1690,108 @@ Status CloudMetaMgr::update_delete_bitmap(const 
CloudTablet& tablet, int64_t loc
 if (next_visible_version > 0) {
 req.set_next_visible_version(next_visible_version);
 }
-for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
-req.add_rowset_ids(std::get<0>(key).to_string());
-req.add_segment_ids(std::get<1>(key));
-req.add_versions(std::get<2>(key));
-// To save space, convert array and bitmap containers to run containers
-bitmap.runOptimize();
-std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
-bitmap.write(bitmap_data.data());
-*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+
+bool write_v1 =
+config::delete_bitmap_store_version == 1 || 
config::delete_bitmap_store_version == 3;
+bool write_v2 = config::delete_bitmap_store_version >= 2;
+// write v1 kvs
+if (write_v1) {
+for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
+req.add_rowset_ids(std::get<0>(key).to_string());
+req.add_segment_ids(std::get<1>(key));
+req.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+}
+}
+
+// write v2 kvs
+if (write_v2) {
+// TODO skip update if delete bitmap is empty
+//  it's ok for compaction and schema change, but for load, ms check 
partition version
+auto add_delete_bitmap = [](DeleteBitmapPB& delete_bitmap_pb,
+const DeleteBitmap::BitmapKey& key, 
roaring::Roaring& bitmap) {
+delete_bitmap_pb.add_rowset_ids(std::get<0>(key).to_string());
+delete_bitmap_pb.add_segment_ids(std::get<1>(key));
+delete_bitmap_pb.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(delete_bitmap_pb.add_segment_delete_bitmaps()) = 
std::move(bitmap_data);
+};
+auto handle_rowset_delete_bitmap = [&](std::string& rowset_id,
+   DeleteBitmapPB& 
delete_bitmap_pb) {
+std::stringstream ss;
+DeleteBitmapStoragePB delete_bitmap_storage;
+for (int i = 0; i < delete_bitmap_pb.rowset_ids_size(); i++) {
+ss << "{rid=" << delete_bitmap_pb.rowset_ids(i)
+   << ", sid=" << delete_bitmap_pb.segment_ids(i)
+   << ", ver=" << delete_bitmap_pb.versions(i) << "}, ";
+}
+if (config::delete_bitmap_max_bytes_store_in_fdb >= 0 &&
+delete_bitmap_pb.ByteSizeLong() > 
config::delete_bitmap_max_bytes_store_in_fdb) {
+DeleteBitmapFileWriter file_writer(tablet.tablet_id(), 
rowset_id, storage_resource);
+RETURN_IF_ERROR(file_writer.init());
+RETURN_IF_ERROR(file_writer.write(delete_bitmap_pb));
+RETURN_IF_ERROR(file_writer.close());
+delete_bitmap_pb.Clear();
+delete_bitmap_storage.set_store_in_fdb(false);
+} else {
+delete_bitmap_storage.set_store_in_fdb(true);
+*(delete_bitmap_storage.mutable_delete_bitmap()) = 
std::move(delete_bitmap_pb);
+}
+LOG(INFO) << "handle one rowset delete bitmap for tablet_id: " << 
tablet.tablet_id()
+  << ", rowset_id: " << rowset_id
+  << ", delete_bitmap num: " << 
delete_bitmap_pb.rowset_ids_size()
+  << ", store_in_fdb=" << 
delete_bitmap_storage.store_in_fdb()
+  << ",  size: " << delete_bitmap_pb.ByteSizeLong() << ", 
keys=[" << ss.str()
+  << "]";
+req.add_v2_rowset_ids(rowset_id);
+*(req.add_delete_bitmap_storages()) = 
std::move(delete_bitmap_storage);
+return Status::OK();
+};
+req.set_store_version(2);
+LOG(INFO) << "update delete bitmap for tablet_id: " << 
tablet.tablet_id()
+  << ", rowset_id: " << rowset_id
+  << ", delete_bitmap num: " << 
delete_bitmap_v2->delete_bitmap.size()
+  << ", lock_id=" << lock_id << ", initiator=" << initiator;
+if (rowset_id.empty()) {
+std::string pre_rowset_id = "";
+DeleteBitmapPB delete_bitmap_pb;

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2285512790


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -1384,15 +1690,108 @@ Status CloudMetaMgr::update_delete_bitmap(const 
CloudTablet& tablet, int64_t loc
 if (next_visible_version > 0) {
 req.set_next_visible_version(next_visible_version);
 }
-for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
-req.add_rowset_ids(std::get<0>(key).to_string());
-req.add_segment_ids(std::get<1>(key));
-req.add_versions(std::get<2>(key));
-// To save space, convert array and bitmap containers to run containers
-bitmap.runOptimize();
-std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
-bitmap.write(bitmap_data.data());
-*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+
+bool write_v1 =
+config::delete_bitmap_store_version == 1 || 
config::delete_bitmap_store_version == 3;
+bool write_v2 = config::delete_bitmap_store_version >= 2;
+// write v1 kvs
+if (write_v1) {
+for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
+req.add_rowset_ids(std::get<0>(key).to_string());
+req.add_segment_ids(std::get<1>(key));
+req.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+}
+}
+
+// write v2 kvs
+if (write_v2) {
+// TODO skip update if delete bitmap is empty
+//  it's ok for compaction and schema change, but for load, ms check 
partition version
+auto add_delete_bitmap = [](DeleteBitmapPB& delete_bitmap_pb,
+const DeleteBitmap::BitmapKey& key, 
roaring::Roaring& bitmap) {
+delete_bitmap_pb.add_rowset_ids(std::get<0>(key).to_string());
+delete_bitmap_pb.add_segment_ids(std::get<1>(key));
+delete_bitmap_pb.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(delete_bitmap_pb.add_segment_delete_bitmaps()) = 
std::move(bitmap_data);
+};

Review Comment:
   duplicated code as 1700-1707.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2285507975


##
be/src/cloud/cloud_meta_mgr.cpp:
##
@@ -1384,15 +1690,108 @@ Status CloudMetaMgr::update_delete_bitmap(const 
CloudTablet& tablet, int64_t loc
 if (next_visible_version > 0) {
 req.set_next_visible_version(next_visible_version);
 }
-for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
-req.add_rowset_ids(std::get<0>(key).to_string());
-req.add_segment_ids(std::get<1>(key));
-req.add_versions(std::get<2>(key));
-// To save space, convert array and bitmap containers to run containers
-bitmap.runOptimize();
-std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
-bitmap.write(bitmap_data.data());
-*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+
+bool write_v1 =
+config::delete_bitmap_store_version == 1 || 
config::delete_bitmap_store_version == 3;
+bool write_v2 = config::delete_bitmap_store_version >= 2;
+// write v1 kvs
+if (write_v1) {
+for (auto& [key, bitmap] : delete_bitmap->delete_bitmap) {
+req.add_rowset_ids(std::get<0>(key).to_string());
+req.add_segment_ids(std::get<1>(key));
+req.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(req.add_segment_delete_bitmaps()) = std::move(bitmap_data);
+}
+}
+
+// write v2 kvs
+if (write_v2) {
+// TODO skip update if delete bitmap is empty
+//  it's ok for compaction and schema change, but for load, ms check 
partition version
+auto add_delete_bitmap = [](DeleteBitmapPB& delete_bitmap_pb,
+const DeleteBitmap::BitmapKey& key, 
roaring::Roaring& bitmap) {
+delete_bitmap_pb.add_rowset_ids(std::get<0>(key).to_string());
+delete_bitmap_pb.add_segment_ids(std::get<1>(key));
+delete_bitmap_pb.add_versions(std::get<2>(key));
+// To save space, convert array and bitmap containers to run 
containers
+bitmap.runOptimize();
+std::string bitmap_data(bitmap.getSizeInBytes(), '\0');
+bitmap.write(bitmap_data.data());
+*(delete_bitmap_pb.add_segment_delete_bitmaps()) = 
std::move(bitmap_data);
+};
+auto handle_rowset_delete_bitmap = [&](std::string& rowset_id,
+   DeleteBitmapPB& 
delete_bitmap_pb) {
+std::stringstream ss;
+DeleteBitmapStoragePB delete_bitmap_storage;
+for (int i = 0; i < delete_bitmap_pb.rowset_ids_size(); i++) {
+ss << "{rid=" << delete_bitmap_pb.rowset_ids(i)
+   << ", sid=" << delete_bitmap_pb.segment_ids(i)
+   << ", ver=" << delete_bitmap_pb.versions(i) << "}, ";
+}
+if (config::delete_bitmap_max_bytes_store_in_fdb >= 0 &&
+delete_bitmap_pb.ByteSizeLong() > 
config::delete_bitmap_max_bytes_store_in_fdb) {
+DeleteBitmapFileWriter file_writer(tablet.tablet_id(), 
rowset_id, storage_resource);
+RETURN_IF_ERROR(file_writer.init());
+RETURN_IF_ERROR(file_writer.write(delete_bitmap_pb));
+RETURN_IF_ERROR(file_writer.close());
+delete_bitmap_pb.Clear();
+delete_bitmap_storage.set_store_in_fdb(false);
+} else {
+delete_bitmap_storage.set_store_in_fdb(true);
+*(delete_bitmap_storage.mutable_delete_bitmap()) = 
std::move(delete_bitmap_pb);
+}
+LOG(INFO) << "handle one rowset delete bitmap for tablet_id: " << 
tablet.tablet_id()
+  << ", rowset_id: " << rowset_id
+  << ", delete_bitmap num: " << 
delete_bitmap_pb.rowset_ids_size()
+  << ", store_in_fdb=" << 
delete_bitmap_storage.store_in_fdb()
+  << ",  size: " << delete_bitmap_pb.ByteSizeLong() << ", 
keys=[" << ss.str()
+  << "]";
+req.add_v2_rowset_ids(rowset_id);
+*(req.add_delete_bitmap_storages()) = 
std::move(delete_bitmap_storage);
+return Status::OK();
+};
+req.set_store_version(2);
+LOG(INFO) << "update delete bitmap for tablet_id: " << 
tablet.tablet_id()
+  << ", rowset_id: " << rowset_id
+  << ", delete_bitmap num: " << 
delete_bitmap_v2->delete_bitmap.size()
+  << ", lock_id=" << lock_id << ", initiator=" << initiator;
+if (rowset_id.empty()) {
+std::string pre_rowset_id = "";
+DeleteBitmapPB delete_bitmap_pb;

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-19 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2285418551


##
be/src/cloud/cloud_tablet.cpp:
##
@@ -1213,14 +1214,38 @@ Status CloudTablet::calc_delete_bitmap_for_compaction(
 }
 
 // 3. store delete bitmap
-auto st = _engine.meta_mgr().update_delete_bitmap(*this, -1, initiator,
-  
output_rowset_delete_bitmap.get());
+DeleteBitmapPtr delete_bitmap_v2 = 
std::make_shared(*output_rowset_delete_bitmap);
+auto delete_bitmap_size = 
output_rowset_delete_bitmap->delete_bitmap.size();
+if (config::delete_bitmap_store_version >= 2) {
+std::vector> pre_rowsets_to_segment_num;

Review Comment:
   retained_rowset is a better name than pre_rowsets_to_segment_num.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-18 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2282350457


##
cloud/src/meta-service/meta_service.cpp:
##
@@ -3327,13 +3483,23 @@ void 
MetaServiceImpl::update_delete_bitmap(google::protobuf::RpcController* cont
 }
 
 // 4. store all pending delete bitmap for this txn
-PendingDeleteBitmapPB delete_bitmap_keys;
+std::vector delete_bitmap_keys_v1;
+std::vector delete_bitmap_keys_v2;
 for (size_t i = 0; i < request->rowset_ids_size(); ++i) {
+std::string key;
 MetaDeleteBitmapInfo key_info {instance_id, tablet_id, 
request->rowset_ids(i),
request->versions(i), 
request->segment_ids(i)};
-std::string key;
 meta_delete_bitmap_key(key_info, &key);
-delete_bitmap_keys.add_delete_bitmap_keys(key);
+delete_bitmap_keys_v1.emplace_back(key);
+}
+if (write_v2) {
+for (size_t i = 0; i < request->v2_rowset_ids_size(); ++i) {

Review Comment:
   v2_rowset_ids is similar to rowset_meta.rowset_id_v2(), so we'd better name 
them in other ways.
   
   e.g. v2_rowset_ids -> delta_rowset_ids indicates rowsets which introduce 
changes, while others base_rowset_ids.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-18 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2282302102


##
cloud/src/meta-service/meta_service.cpp:
##
@@ -3393,147 +3564,33 @@ void 
MetaServiceImpl::update_delete_bitmap(google::protobuf::RpcController* cont
   << " value_size: " << pending_val.size();
 }
 
-// 5. Update delete bitmap for curent txn
-size_t current_key_count = 0;
-size_t current_value_count = 0;
-size_t total_key_count = 0;
-size_t total_value_count = 0;
-size_t total_txn_put_keys = 0;
-size_t total_txn_put_bytes = 0;
-size_t total_txn_size = 0;
-size_t total_txn_count = 0;
+// 5. Update delete bitmap for current txn
+UpdateDeleteBitmapTxnStats txn_stats;
 std::set non_exist_rowset_ids;
-for (size_t i = 0; i < request->rowset_ids_size(); ++i) {
-auto& key = delete_bitmap_keys.delete_bitmap_keys(i);
+// write v1 kvs
+for (size_t i = 0; i < delete_bitmap_keys_v1.size(); ++i) {
+auto& key = delete_bitmap_keys_v1[i];
 auto& val = request->segment_delete_bitmaps(i);
-
-// Split into multiple fdb transactions, because the size of one fdb
-// transaction can't exceed 10MB.
-if (txn->approximate_bytes() + key.size() * 3 + val.size() > 
config::max_txn_commit_byte) {
-LOG(INFO) << "fdb txn size more than " << 
config::max_txn_commit_byte
-  << ", current size: " << txn->approximate_bytes()
-  << ", tablet_id: " << tablet_id << " lock_id=" << 
request->lock_id()
-  << " initiator=" << request->initiator() << ", need to 
commit";
-err = txn->commit();
-TEST_SYNC_POINT_CALLBACK("update_delete_bitmap:commit:err", 
request->initiator(), i,
- &err);
-total_txn_put_keys += txn->num_put_keys();
-total_txn_put_bytes += txn->put_bytes();
-total_txn_size += txn->approximate_bytes();
-total_txn_count++;
-if (err != TxnErrorCode::TXN_OK) {
-code = cast_as(err);
-ss << "failed to update delete bitmap, err=" << err << " 
tablet_id=" << tablet_id
-   << " lock_id=" << request->lock_id() << " initiator=" << 
request->initiator()
-   << " delete_bitmap_key=" << current_key_count
-   << " delete_bitmap_value=" << current_value_count
-   << " put_size=" << txn->put_bytes() << " num_put_keys=" << 
txn->num_put_keys()
-   << " txn_size=" << txn->approximate_bytes();
-msg = ss.str();
-g_bvar_update_delete_bitmap_fail_counter << 1;
-return;
-}
-stats.get_bytes += txn->get_bytes();
-stats.put_bytes += txn->put_bytes();
-stats.del_bytes += txn->delete_bytes();
-stats.get_counter += txn->num_get_keys();
-stats.put_counter += txn->num_put_keys();
-stats.del_counter += txn->num_del_keys();
-current_key_count = 0;
-current_value_count = 0;
-TxnErrorCode err = txn_kv_->create_txn(&txn);
-if (err != TxnErrorCode::TXN_OK) {
-code = cast_as(err);
-msg = "failed to init txn";
-return;
-}
-if (!without_lock) {
-std::string lock_key =
-meta_delete_bitmap_update_lock_key({instance_id, 
table_id, -1});
-DeleteBitmapUpdateLockPB lock_info;
-if (!check_delete_bitmap_lock(code, msg, ss, txn, instance_id, 
table_id,
-  request->lock_id(), 
request->initiator(), lock_key,
-  lock_info, use_version, log)) {
-LOG(WARNING) << "failed to check delete bitmap lock, 
table_id=" << table_id
- << " request lock_id=" << request->lock_id()
- << " request initiator=" << 
request->initiator() << " msg " << msg;
-return;
-}
-}
-}
-
-#ifdef ENABLE_INJECTION_POINT
-if (config::enable_update_delete_bitmap_kv_check) {
-if (!check_delete_bitmap_kv_exists(code, msg, txn, key, 
instance_id, tablet_id,
-   request->lock_id())) {
-return;
-}
-}
-#endif
-
-if (without_lock && request->has_pre_rowset_agg_end_version() &&
-request->pre_rowset_agg_end_version() > 0) {
-// check the rowset exists
-if (non_exist_rowset_ids.contains(request->rowset_ids(i))) {
-LOG(INFO) << "skip update delete bitmap, rowset_id=" << 
request->rowset_ids(i)
-  << " version=" << request->pre_rowset_versions(i)
-

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-18 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2282302102


##
cloud/src/meta-service/meta_service.cpp:
##
@@ -3393,147 +3564,33 @@ void 
MetaServiceImpl::update_delete_bitmap(google::protobuf::RpcController* cont
   << " value_size: " << pending_val.size();
 }
 
-// 5. Update delete bitmap for curent txn
-size_t current_key_count = 0;
-size_t current_value_count = 0;
-size_t total_key_count = 0;
-size_t total_value_count = 0;
-size_t total_txn_put_keys = 0;
-size_t total_txn_put_bytes = 0;
-size_t total_txn_size = 0;
-size_t total_txn_count = 0;
+// 5. Update delete bitmap for current txn
+UpdateDeleteBitmapTxnStats txn_stats;
 std::set non_exist_rowset_ids;
-for (size_t i = 0; i < request->rowset_ids_size(); ++i) {
-auto& key = delete_bitmap_keys.delete_bitmap_keys(i);
+// write v1 kvs
+for (size_t i = 0; i < delete_bitmap_keys_v1.size(); ++i) {
+auto& key = delete_bitmap_keys_v1[i];
 auto& val = request->segment_delete_bitmaps(i);
-
-// Split into multiple fdb transactions, because the size of one fdb
-// transaction can't exceed 10MB.
-if (txn->approximate_bytes() + key.size() * 3 + val.size() > 
config::max_txn_commit_byte) {
-LOG(INFO) << "fdb txn size more than " << 
config::max_txn_commit_byte
-  << ", current size: " << txn->approximate_bytes()
-  << ", tablet_id: " << tablet_id << " lock_id=" << 
request->lock_id()
-  << " initiator=" << request->initiator() << ", need to 
commit";
-err = txn->commit();
-TEST_SYNC_POINT_CALLBACK("update_delete_bitmap:commit:err", 
request->initiator(), i,
- &err);
-total_txn_put_keys += txn->num_put_keys();
-total_txn_put_bytes += txn->put_bytes();
-total_txn_size += txn->approximate_bytes();
-total_txn_count++;
-if (err != TxnErrorCode::TXN_OK) {
-code = cast_as(err);
-ss << "failed to update delete bitmap, err=" << err << " 
tablet_id=" << tablet_id
-   << " lock_id=" << request->lock_id() << " initiator=" << 
request->initiator()
-   << " delete_bitmap_key=" << current_key_count
-   << " delete_bitmap_value=" << current_value_count
-   << " put_size=" << txn->put_bytes() << " num_put_keys=" << 
txn->num_put_keys()
-   << " txn_size=" << txn->approximate_bytes();
-msg = ss.str();
-g_bvar_update_delete_bitmap_fail_counter << 1;
-return;
-}
-stats.get_bytes += txn->get_bytes();
-stats.put_bytes += txn->put_bytes();
-stats.del_bytes += txn->delete_bytes();
-stats.get_counter += txn->num_get_keys();
-stats.put_counter += txn->num_put_keys();
-stats.del_counter += txn->num_del_keys();
-current_key_count = 0;
-current_value_count = 0;
-TxnErrorCode err = txn_kv_->create_txn(&txn);
-if (err != TxnErrorCode::TXN_OK) {
-code = cast_as(err);
-msg = "failed to init txn";
-return;
-}
-if (!without_lock) {
-std::string lock_key =
-meta_delete_bitmap_update_lock_key({instance_id, 
table_id, -1});
-DeleteBitmapUpdateLockPB lock_info;
-if (!check_delete_bitmap_lock(code, msg, ss, txn, instance_id, 
table_id,
-  request->lock_id(), 
request->initiator(), lock_key,
-  lock_info, use_version, log)) {
-LOG(WARNING) << "failed to check delete bitmap lock, 
table_id=" << table_id
- << " request lock_id=" << request->lock_id()
- << " request initiator=" << 
request->initiator() << " msg " << msg;
-return;
-}
-}
-}
-
-#ifdef ENABLE_INJECTION_POINT
-if (config::enable_update_delete_bitmap_kv_check) {
-if (!check_delete_bitmap_kv_exists(code, msg, txn, key, 
instance_id, tablet_id,
-   request->lock_id())) {
-return;
-}
-}
-#endif
-
-if (without_lock && request->has_pre_rowset_agg_end_version() &&
-request->pre_rowset_agg_end_version() > 0) {
-// check the rowset exists
-if (non_exist_rowset_ids.contains(request->rowset_ids(i))) {
-LOG(INFO) << "skip update delete bitmap, rowset_id=" << 
request->rowset_ids(i)
-  << " version=" << request->pre_rowset_versions(i)
-

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-18 Thread via GitHub



dataroaring commented on code in PR #53908:
URL: https://github.com/apache/doris/pull/53908#discussion_r2281756329


##
gensrc/proto/cloud.proto:
##
@@ -1714,6 +1720,10 @@ message UpdateDeleteBitmapRequest {
 // when update delete_bitmap of pre rowsets, check the rowset exists
 repeated int64 pre_rowset_versions = 17;
 optional string request_ip = 18;
+optional int64 store_version = 19;

Review Comment:
   It's better to start from a larger id, e.g. 50. Because, we can reduce 
complexcity of add fields in release branches.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-13 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3183247101

   # BE Regression && UT Coverage Report
   Increment line coverage `66.28% (399/602)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_64a3960c608816bdde0e69efa2a56e40a5b11b16_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_64a3960c608816bdde0e69efa2a56e40a5b11b16_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 81.80% (22627/27662) |
   | Line Coverage | 74.40% (234638/315385) |
   | Region Coverage   | 61.60% (195167/316820) |
   | Branch Coverage   | 65.74% (84246/128155) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-13 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3182923566

   # BE UT Coverage Report
   Increment line coverage `2.17% (13/600)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/64a3960c608816bdde0e69efa2a56e40a5b11b16_64a3960c608816bdde0e69efa2a56e40a5b11b16/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/64a3960c608816bdde0e69efa2a56e40a5b11b16_64a3960c608816bdde0e69efa2a56e40a5b11b16/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 59.00% (16629/28185) |
   | Line Coverage | 47.83% (150841/315378) |
   | Region Coverage   | 36.62% (112991/308559) |
   | Branch Coverage   | 39.60% (50142/126609) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-13 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3182814638

   
   
   ClickBench: Total hot run time: 32.85 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 64a3960c608816bdde0e69efa2a56e40a5b11b16, 
data reload: false
   
   query1   0.040.030.04
   query2   0.080.040.04
   query3   0.240.070.07
   query4   1.630.110.11
   query5   0.450.430.42
   query6   1.160.640.66
   query7   0.020.020.02
   query8   0.040.040.03
   query9   0.610.520.51
   query10  0.560.570.57
   query11  0.160.120.11
   query12  0.140.120.12
   query13  0.630.610.60
   query14  0.820.830.86
   query15  0.900.860.85
   query16  0.410.410.39
   query17  1.061.041.04
   query18  0.210.200.19
   query19  1.971.821.83
   query20  0.020.010.01
   query21  15.39   0.930.56
   query22  0.761.140.72
   query23  14.89   1.380.63
   query24  6.661.800.82
   query25  0.540.230.17
   query26  0.610.160.14
   query27  0.060.050.04
   query28  10.56   0.940.43
   query29  12.53   4.013.28
   query30  3.103.033.00
   query31  2.820.590.39
   query32  3.240.570.47
   query33  3.143.163.09
   query34  16.04   5.444.88
   query35  4.964.975.01
   query36  0.680.530.51
   query37  0.090.080.07
   query38  0.050.040.04
   query39  0.030.030.03
   query40  0.180.140.14
   query41  0.070.030.02
   query42  0.030.020.02
   query43  0.040.030.03
   Total cold run time: 107.62 s
   Total hot run time: 32.85 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-13 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3182794432

   
   
   TPC-DS: Total hot run time: 185291 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 64a3960c608816bdde0e69efa2a56e40a5b11b16, 
data reload: false
   
   query1   978 403 431 403
   query2   6549170616901690
   query3   6743216 221 216
   query4   26373   23252   23486   23252
   query5   4351618 510 510
   query6   315 221 218 218
   query7   4635512 292 292
   query8   276 237 228 228
   query9   8604289228622862
   query10  485 321 286 286
   query11  15722   15444   14757   14757
   query12  173 119 114 114
   query13  1660538 419 419
   query14  9527572357115711
   query15  214 181 169 169
   query16  7597640 482 482
   query17  1210744 597 597
   query18  2026420 324 324
   query19  195 194 162 162
   query20  129 116 114 114
   query21  212 124 116 116
   query22  4105416540204020
   query23  34388   33424   33464   33424
   query24  8059233623692336
   query25  537 473 402 402
   query26  1243287 159 159
   query27  2699499 344 344
   query28  4281224622362236
   query29  738 560 443 443
   query30  284 223 227 223
   query31  900 819 709 709
   query32  82  76  80  76
   query33  541 370 338 338
   query34  785 832 501 501
   query35  802 845 740 740
   query36  940 1023913 913
   query37  125 106 88  88
   query38  4033403740244024
   query39  1537143514311431
   query40  222 125 116 116
   query41  66  61  61  61
   query42  124 111 112 111
   query43  505 511 477 477
   query44  1329850 848 848
   query45  180 170 162 162
   query46  855 1006636 636
   query47  1752183217181718
   query48  380 405 301 301
   query49  716 486 391 391
   query50  645 683 417 417
   query51  4141405541824055
   query52  121 108 103 103
   query53  232 267 202 202
   query54  593 576 508 508
   query55  94  89  110 89
   query56  303 311 316 311
   query57  1175119211231123
   query58  277 266 270 266
   query59  2715276325872587
   query60  349 330 329 329
   query61  127 123 127 123
   query62  822 706 658 658
   query63  228 209 185 185
   query64  43021069687 687
   query65  4363422541404140
   query66  1092422 331 331
   query67  15469   15427   15127   15127
   query68  8564910 562 562
   query69  468 320 291 291
   query70  1204113611321132
   query71  462 334 315 315
   query72  5716478050834780
   query73  772 689 353 353
   query74  9097902489768976
   query75  3960307226082608
   query76  37061128728 728
   query77  779 398 324 324
   query78  9657981988578857
   query79  2082799 603 603
   query80  626 546 477 477
   query81  474 257 223 223
   query82  410 132 150 132
   query83  271 253 237 237
   query84  242 110 84  84
   query85  783 369 342 342
   query86  358 337 304 304
   query87  4284421040684068
   query88  2986221421872187
   query89  393 321 292 292
   query90  1974230 224 224
   query91  146 148 114 114
   query92  89  73  69  69
   query93  11171009644 644
   query94  677 407 303 303
   query95  400 317 305 305
   query96  494 578 272 272
   query97  2644269325892589
   query98  244 218 254 218
   query99  1449141412871287
   Total cold run time: 273758 ms
   Total hot run time: 185291 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-13 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3182757419

   
   
   TPC-H: Total hot run time: 33823 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 64a3960c608816bdde0e69efa2a56e40a5b11b16, 
data reload: false
   
   -- Round 1 --
   q1   17221   525450715071
   q2   1923289 183 183
   q3   10092   1270701 701
   q4   10224   1019512 512
   q5   7549245923052305
   q6   178 160 129 129
   q7   911 762 634 634
   q8   9299128510021002
   q9   7138519351765176
   q10  7023239319631963
   q11  490 286 263 263
   q12  344 353 223 223
   q13  17793   368130273027
   q14  240 240 203 203
   q15  559 470 468 468
   q16  420 424 375 375
   q17  610 862 356 356
   q18  7524708271797082
   q19  1094982 578 578
   q20  353 335 216 216
   q21  3758319523822382
   q22  10551038974 974
   Total cold run time: 105798 ms
   Total hot run time: 33823 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5174509351415093
   q2   236 316 216 216
   q3   2213271023202320
   q4   1358180413621362
   q5   4226452746224527
   q6   220 170 127 127
   q7   2095196018421842
   q8   2667253625032503
   q9   7466727372297229
   q10  3120328729542954
   q11  568 510 510 510
   q12  697 842 650 650
   q13  3532420232353235
   q14  285 291 273 273
   q15  538 490 467 467
   q16  468 509 449 449
   q17  1278155114301430
   q18  8190773475327532
   q19  811 822 847 822
   q20  2056209219831983
   q21  4829430043524300
   q22  1127103210111011
   Total cold run time: 53154 ms
   Total hot run time: 50835 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-13 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3182559069

   # Cloud UT Coverage Report
   Increment line coverage `78.09% (310/397)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/64a3960c608816bdde0e69efa2a56e40a5b11b16_64a3960c608816bdde0e69efa2a56e40a5b11b16_cloud/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/64a3960c608816bdde0e69efa2a56e40a5b11b16_64a3960c608816bdde0e69efa2a56e40a5b11b16_cloud/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 83.80% (1438/1716) |
   | Line Coverage | 66.92% (24449/36532) |
   | Region Coverage   | 68.04% (12190/17915) |
   | Branch Coverage   | 57.68% (6414/11120) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-13 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3182454905

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-04 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3149852133

   # BE Regression && UT Coverage Report
   Increment line coverage `66.14% (422/638)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_9a25dedd480423262b2d1a06a2ae97d9048bd185_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_9a25dedd480423262b2d1a06a2ae97d9048bd185_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 80.95% (22154/27369) |
   | Line Coverage | 73.53% (228898/311307) |
   | Region Coverage   | 61.13% (191330/313013) |
   | Branch Coverage   | 64.98% (82281/126621) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-04 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3149581716

   run cloud_p0


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-04 Thread via GitHub



hello-stephen commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3149479219

   # BE Regression && UT Coverage Report
   Increment line coverage `66.14% (422/638)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_9a25dedd480423262b2d1a06a2ae97d9048bd185_merge/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/53908_9a25dedd480423262b2d1a06a2ae97d9048bd185_merge/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 80.95% (22154/27369) |
   | Line Coverage | 73.53% (228898/311307) |
   | Region Coverage   | 61.13% (191330/313013) |
   | Branch Coverage   | 64.98% (82281/126621) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-03 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3149247415

   # BE UT Coverage Report
   Increment line coverage `0.16% (1/638)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/9a25dedd480423262b2d1a06a2ae97d9048bd185_9a25dedd480423262b2d1a06a2ae97d9048bd185/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/9a25dedd480423262b2d1a06a2ae97d9048bd185_9a25dedd480423262b2d1a06a2ae97d9048bd185/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 57.95% (16156/27880) |
   | Line Coverage | 46.67% (145466/311668) |
   | Region Coverage   | 35.75% (109157/305301) |
   | Branch Coverage   | 38.53% (48262/125249) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-08-03 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3149043133

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-07-29 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3134792688

   # Cloud UT Coverage Report
   Increment line coverage `77.75% (304/391)` :tada:
   
   [Increment coverage 
report](http://coverage.selectdb-in.cc/coverage/dac10e6df5f4b97447dfd531af0e04ee50a41cf4_dac10e6df5f4b97447dfd531af0e04ee50a41cf4_cloud/increment_report/index.html)
   [Complete coverage 
report](http://coverage.selectdb-in.cc/coverage/dac10e6df5f4b97447dfd531af0e04ee50a41cf4_dac10e6df5f4b97447dfd531af0e04ee50a41cf4_cloud/report/index.html)
   | Category  | Coverage   |
   |---||
   | Function Coverage | 80.77% (1336/1654) |
   | Line Coverage | 65.64% (22758/34672) |
   | Region Coverage   | 67.11% (11380/16956) |
   | Branch Coverage   | 56.62% (5980/10562) |


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-07-29 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3132895979

   
   
   ClickBench: Total hot run time: 33.59 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit dac10e6df5f4b97447dfd531af0e04ee50a41cf4, 
data reload: false
   
   query1   0.040.030.04
   query2   0.070.040.05
   query3   0.240.070.08
   query4   1.620.110.11
   query5   0.430.440.44
   query6   1.200.680.66
   query7   0.030.020.02
   query8   0.050.040.04
   query9   0.550.480.46
   query10  0.530.530.52
   query11  0.160.100.10
   query12  0.140.110.11
   query13  0.660.660.64
   query14  0.921.131.09
   query15  0.930.880.89
   query16  0.390.400.38
   query17  1.091.061.09
   query18  0.210.210.22
   query19  2.121.961.92
   query20  0.010.010.01
   query21  15.38   0.840.56
   query22  0.761.280.83
   query23  14.78   1.180.60
   query24  6.661.181.04
   query25  0.480.140.24
   query26  0.540.150.13
   query27  0.050.050.06
   query28  9.910.830.43
   query29  12.71   3.803.30
   query30  3.042.942.95
   query31  2.820.560.39
   query32  3.240.570.50
   query33  3.043.173.26
   query34  16.07   5.374.87
   query35  4.885.004.96
   query36  0.710.510.51
   query37  0.100.070.08
   query38  0.060.050.05
   query39  0.030.030.03
   query40  0.170.140.13
   query41  0.090.030.03
   query42  0.040.030.03
   query43  0.040.040.04
   Total cold run time: 106.99 s
   Total hot run time: 33.59 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-07-29 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3132874376

   
   
   TPC-DS: Total hot run time: 174473 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit dac10e6df5f4b97447dfd531af0e04ee50a41cf4, 
data reload: false
   
   
   query1   1031397 412 397
   query2   6519172016611661
   query3   6740231 217 217
   query4   26866   23537   22985   22985
   query5   4377636 534 534
   query6   335 255 237 237
   query7   4641524 298 298
   query8   297 253 229 229
   query9   8601323132133213
   query10  472 338 303 303
   query11  15982   15005   15114   15005
   query12  183 140 136 136
   query13  1681572 412 412
   query14  9643787178637863
   query15  261 204 169 169
   query16  8026673 487 487
   query17  1670807 654 654
   query18  2088447 333 333
   query19  298 222 186 186
   query20  156 150 144 144
   query21  223 131 113 113
   query22  4002394338333833
   query23  34730   34228   34573   34228
   query24  8122239324192393
   query25  555 507 443 443
   query26  856 304 161 161
   query27  2616503 346 346
   query28  4240228322692269
   query29  692 600 498 498
   query30  306 235 220 220
   query31  885 805 716 716
   query32  88  76  79  76
   query33  552 442 384 384
   query34  819 849 522 522
   query35  810 843 769 769
   query36  999 1025954 954
   query37  140 112 93  93
   query38  4004407939573957
   query39  1469136714131367
   query40  241 141 129 129
   query41  66  57  53  53
   query42  141 124 125 124
   query43  500 516 473 473
   query44  1421892 887 887
   query45  204 193 180 180
   query46  949 1064670 670
   query47  1775187917521752
   query48  423 453 316 316
   query49  702 492 422 422
   query50  683 704 416 416
   query51  5566554253925392
   query52  127 131 126 126
   query53  267 290 203 203
   query54  647 649 558 558
   query55  94  89  88  88
   query56  355 370 342 342
   query57  1201121611391139
   query58  361 358 339 339
   query59  2531259926772599
   query60  409 400 390 390
   query61  156 122 134 122
   query62  774 743 665 665
   query63  255 226 219 219
   query64  34881087839 839
   query65  4438415241174117
   query66  939 445 332 332
   query67  query68 17911   883 970 883
   query69  1035284 286 284
   query70  1378109611531096
   query71  681 312 355 312
   query72  9191232322602260
   query73  4066668 354 354
   query74  9047903788088808
   query75  7470317426592659
   query76  85601221799 799
   query77  1165411 344 344
   query78  964011988   95729572
   query79  15313   650 607 607
   query80  3209576 481 481
   query81  581 306 240 240
   query82  524 154 121 121
   query83  359 282 265 265
   query84  296 84  83  83
   query85  929 358 327 327
   query86  343 329 309 309
   query87  4333426042804260
   query88  5576223322722233
   query89  492 318 329 318
   query90  2474229 227 227
   query91  146 135 109 109
   query92  91  67  67  67
   query93  6904945 695 695
   query94  1383399 286 286
   query95  434 338 322 322
   query96  504 602 290 290
   query97  2685270926532653
   query98  249 246 231 231
   query99  1453137712811281
   Total cold run time: 313110 ms
   Total hot run time: 174473 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-07-29 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3132826787

   
   
   TPC-H: Total hot run time: 34151 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit dac10e6df5f4b97447dfd531af0e04ee50a41cf4, 
data reload: false
   
   -- Round 1 --
   q1   17420   557656285576
   q2   1932288 199 199
   q3   10141   1281713 713
   q4   10219   1002525 525
   q5   7572235423282328
   q6   180 164 132 132
   q7   888 766 615 615
   q8   9311124110611061
   q9   6925520051775177
   q10  6972239019981998
   q11  466 277 263 263
   q12  355 374 238 238
   q13  17783   370729852985
   q14  227 242 219 219
   q15  539 477 464 464
   q16  432 434 374 374
   q17  572 830 351 351
   q18  7338705169046904
   q19  1358977 536 536
   q20  322 314 212 212
   q21  2853306123232323
   q22  10751029958 958
   Total cold run time: 104880 ms
   Total hot run time: 34151 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5877576757455745
   q2   234 314 216 216
   q3   2066256022442244
   q4   1344170113431343
   q5   4159464345294529
   q6   226 187 138 138
   q7   1987196918331833
   q8   2558246025062460
   q9   7565741275537412
   q10  3155333728342834
   q11  554 515 506 506
   q12  697 781 644 644
   q13  3461383532053205
   q14  309 314 296 296
   q15  494 459 454 454
   q16  471 503 467 467
   q17  1291156813441344
   q18  9443771376077607
   q19  865 847 767 767
   q20  1969208819041904
   q21  6790446642724272
   q22  10581054992 992
   Total cold run time: 56573 ms
   Total hot run time: 51212 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-07-29 Thread via GitHub



mymeiyi commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3132490137

   run buildall


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-07-29 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3132004186

   
   
   ClickBench: Total hot run time: 33.31 s
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
   ClickBench test result on commit 384fdc6a48f52c8c677e82f8b5c7d9d348ed14e4, 
data reload: false
   
   query1   0.040.040.03
   query2   0.080.040.04
   query3   0.250.070.07
   query4   1.620.100.10
   query5   0.430.420.44
   query6   1.170.700.69
   query7   0.020.020.02
   query8   0.040.030.04
   query9   0.580.480.47
   query10  0.540.530.51
   query11  0.150.100.10
   query12  0.150.110.12
   query13  0.660.640.66
   query14  0.941.150.97
   query15  0.920.890.89
   query16  0.400.410.39
   query17  1.031.091.10
   query18  0.230.200.20
   query19  2.021.791.99
   query20  0.010.010.01
   query21  15.37   0.830.54
   query22  0.781.240.66
   query23  14.94   1.140.64
   query24  6.741.111.05
   query25  0.450.320.08
   query26  0.540.160.13
   query27  0.070.060.05
   query28  9.520.860.45
   query29  12.57   3.823.36
   query30  3.093.012.96
   query31  2.820.570.39
   query32  3.240.570.50
   query33  3.053.243.23
   query34  15.97   5.374.91
   query35  4.834.974.94
   query36  0.720.520.50
   query37  0.100.070.07
   query38  0.060.050.04
   query39  0.030.030.03
   query40  0.170.130.14
   query41  0.080.030.03
   query42  0.040.030.02
   query43  0.050.030.03
   Total cold run time: 106.51 s
   Total hot run time: 33.31 s
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-07-29 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3131983605

   
   
   TPC-DS: Total hot run time: 173829 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
   TPC-DS sf100 test result on commit 384fdc6a48f52c8c677e82f8b5c7d9d348ed14e4, 
data reload: false
   
   
   query1   1028399 397 397
   query2   6529179016771677
   query3   6756220 226 220
   query4   27444   23718   23405   23405
   query5   4403615 519 519
   query6   324 245 224 224
   query7   4649520 298 298
   query8   287 247 225 225
   query9   8601318031543154
   query10  496 346 292 292
   query11  16166   14991   14815   14815
   query12  187 136 133 133
   query13  1663573 428 428
   query14  9574778976927692
   query15  221 200 176 176
   query16  7826638 468 468
   query17  1631804 655 655
   query18  2092465 329 329
   query19  306 219 187 187
   query20  154 149 145 145
   query21  224 129 114 114
   query22  3922410639273927
   query23  34419   34203   34076   34076
   query24  8059238724092387
   query25  537 508 436 436
   query26  933 292 159 159
   query27  2631520 357 357
   query28  4239229422812281
   query29  696 611 485 485
   query30  302 223 198 198
   query31  905 808 694 694
   query32  90  77  74  74
   query33  551 423 376 376
   query34  825 861 515 515
   query35  829 842 761 761
   query36  10301013917 917
   query37  134 111 88  88
   query38  4040401540764015
   query39  1428136513601360
   query40  240 141 137 137
   query41  63  54  54  54
   query42  143 122 129 122
   query43  519 515 454 454
   query44  1393866 868 866
   query45  211 187 181 181
   query46  933 1047665 665
   query47  1826188317771777
   query48  392 429 320 320
   query49  682 504 408 408
   query50  657 697 426 426
   query51  5525557456195574
   query52  126 136 119 119
   query53  254 287 211 211
   query54  669 675 565 565
   query55  90  88  86  86
   query56  354 368 356 356
   query57  1211123611581158
   query58  335 339 337 337
   query59  2597261325652565
   query60  409 393 394 393
   query61  124 122 125 122
   query62  839 724 675 675
   query63  254 215 221 215
   query64  34981121774 774
   query65  4297408140794079
   query66  924 438 326 326
   query67  query68 17610   623 588 588
   query69  987 319 306 306
   query70  1412111211161112
   query71  692 339 335 335
   query72  9111225823442258
   query73  3901634 360 360
   query74  8833897486008600
   query75  7511311827262726
   query76  85051215776 776
   query77  1162410 347 347
   query78  962910521   94809480
   query79  15288   604 570 570
   query80  3074549 495 495
   query81  555 271 249 249
   query82  549 150 118 118
   query83  391 289 267 267
   query84  362 96  92  92
   query85  928 359 335 335
   query86  354 340 294 294
   query87  4354431141954195
   query88  5595227222832272
   query89  509 375 323 323
   query90  2501227 225 225
   query91  144 136 109 109
   query92  90  72  68  68
   query93  6815979 654 654
   query94  1241403 284 284
   query95  430 337 333 333
   query96  503 600 281 281
   query97  2697279926602660
   query98  244 235 223 223
   query99  1477145712701270
   Total cold run time: 312150 ms
   Total hot run time: 173829 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL

Re: [PR] [fix](mow) modify delete bitmap storage format in ms [doris]

2025-07-29 Thread via GitHub



doris-robot commented on PR #53908:
URL: https://github.com/apache/doris/pull/53908#issuecomment-3131937779

   
   
   TPC-H: Total hot run time: 34117 ms
   
   ```
   machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
   scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
   Tpch sf100 test result on commit 384fdc6a48f52c8c677e82f8b5c7d9d348ed14e4, 
data reload: false
   
   -- Round 1 --
   q1   17761   567454495449
   q2   1964272 189 189
   q3   10632   1301733 733
   q4   10277   1002514 514
   q5   8660238522972297
   q6   184 162 130 130
   q7   909 764 625 625
   q8   9290128110971097
   q9   6809520851745174
   q10  6897235319941994
   q11  476 281 274 274
   q12  355 365 234 234
   q13  17793   349229892989
   q14  241 256 221 221
   q15  533 477 463 463
   q16  443 442 375 375
   q17  578 824 360 360
   q18  7608713569516951
   q19  2097990 543 543
   q20  326 319 216 216
   q21  3431248222802280
   q22  1051108310091009
   Total cold run time: 108315 ms
   Total hot run time: 34117 ms
   
   - Round 2, with runtime_filter_mode=off -
   q1   5939584257625762
   q2   223 312 207 207
   q3   2099259521602160
   q4   1325173913451345
   q5   4439454445114511
   q6   220 184 137 137
   q7   2019195517571757
   q8   2607251124532453
   q9   7465741071967196
   q10  3285335729292929
   q11  540 519 498 498
   q12  720 721 647 647
   q13  3547366134293429
   q14  303 289 340 289
   q15  522 474 473 473
   q16  454 562 502 502
   q17  1188150713741374
   q18  8006751875467518
   q19  4821922 920 920
   q20  2006205818511851
   q21  15078   429841864186
   q22  10541042996 996
   Total cold run time: 67860 ms
   Total hot run time: 51140 ms
   ```
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

1 2 3 >

1 - 100 of 230 matches

Mail list logo