Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4148570471 @morrySnow @zclllyybb -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4142110502 # BE Regression && UT Coverage Report Increment line coverage `100% (0/0)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_1584a6155932303cd0a4fc152aeca29492033f91_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_1584a6155932303cd0a4fc152aeca29492033f91_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 71.79% (26504/36919) | | Line Coverage | 54.70% (279795/511511) | | Region Coverage | 52.03% (232864/447534) | | Branch Coverage | 53.39% (100435/188122) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4141586330 # BE UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/1584a6155932303cd0a4fc152aeca29492033f91_1584a6155932303cd0a4fc152aeca29492033f91/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/1584a6155932303cd0a4fc152aeca29492033f91_1584a6155932303cd0a4fc152aeca29492033f91/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.91% (19946/37696) | | Line Coverage | 36.45% (186960/512991) | | Region Coverage | 32.70% (144961/443290) | | Branch Coverage | 33.89% (63545/187509) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4141575507 # FE UT Coverage Report Increment line coverage ` 84.09% (37/44)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_1584a6155932303cd0a4fc152aeca29492033f91/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_1584a6155932303cd0a4fc152aeca29492033f91/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4141365139 TPC-DS: Total hot run time: 169112 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 1584a6155932303cd0a4fc152aeca29492033f91, data reload: false query5 4336633 519 519 query6 341 224 201 201 query7 4202483 270 270 query8 344 242 224 224 query9 8713267526752675 query10 529 388 353 353 query11 6978508948904890 query12 182 126 156 126 query13 1283448 327 327 query14 5750366934723472 query14_12812276329212763 query15 204 190 175 175 query16 962 469 462 462 query17 915 702 614 614 query18 2438440 338 338 query19 209 203 197 197 query20 134 128 126 126 query21 217 134 109 109 query22 13273 14366 14672 14366 query23 16635 16294 16018 16018 query23_116444 16125 15845 15845 query24 7397163512281228 query24_11222122412171217 query25 565 507 441 441 query26 1240304 150 150 query27 2734478 294 294 query28 4504183818401838 query29 797 564 469 469 query30 296 225 192 192 query31 994 944 869 869 query32 82 72 71 71 query33 520 337 288 288 query34 898 884 519 519 query35 624 676 606 606 query36 10951148983 983 query37 138 90 79 79 query38 2913289829152898 query39 879 838 806 806 query39_1790 785 790 785 query40 246 156 136 136 query41 67 59 60 59 query42 258 254 253 253 query43 240 239 220 220 query44 query45 197 190 185 185 query46 868 973 600 600 query47 2137213420592059 query48 318 315 226 226 query49 636 464 378 378 query50 710 269 219 219 query51 4075409139783978 query52 262 265 255 255 query53 296 351 282 282 query54 302 279 277 277 query55 90 84 85 84 query56 302 323 312 312 query57 1897173315741574 query58 277 279 268 268 query59 2801295827562756 query60 343 330 328 328 query61 162 152 183 152 query62 628 576 526 526 query63 307 275 274 274 query64 4998131710111011 query65 query66 1466465 348 348 query67 24192 24326 24144 24144 query68 query69 418 319 283 283 query70 1001960 939 939 query71 337 312 294 294 query72 2794273825422542 query73 531 550 319 319 query74 9613959893899389 query75 2868279424712471 query76 22901024668 668 query77 397 396 313 313 query78 10904 11056 10463 10463 query79 3046732 575 575 query80 1796654 579 579 query81 569 260 230 230 query82 981 153 121 121 query83 336 265 251 251 query84 299 112 104 104 query85 928 497 459 459 query86 505 309 289 289 query87 3091311630403040 query88 3523262426322624 query89 423 369 349 349 query90 2105177 169 169 query91 175 170 139 139 query92 88 74 72 72 query93 2097838 493 493 query94 659 311 283 283 query95 583 340 337 337 query96 645 507 231 231 query97 2422248224092409 query98 242 218 218 218 query99 1006993 897 897 Total cold run time: 253975 ms Total hot run time: 169112 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4141269585 TPC-H: Total hot run time: 26795 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 1584a6155932303cd0a4fc152aeca29492033f91, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17637 452643274327 q2 q3 10645 770 517 517 q4 4686357 251 251 q5 7773121010101010 q6 183 179 147 147 q7 836 830 686 686 q8 10518 149613261326 q9 5886475747064706 q10 6326195016301630 q11 493 256 240 240 q12 738 577 459 459 q13 18048 271319741974 q14 231 244 210 210 q15 q16 745 744 658 658 q17 765 838 453 453 q18 5908541952905290 q19 1418984 640 640 q20 558 483 388 388 q21 4548196816071607 q22 397 337 276 276 Total cold run time: 98339 ms Total hot run time: 26795 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4878472946774677 q2 q3 3945438638573857 q4 893 1214760 760 q5 4077441343884388 q6 203 185 145 145 q7 1801171615641564 q8 2509269425802580 q9 7551742175087421 q10 3818398836323632 q11 505 444 419 419 q12 495 586 429 429 q13 2526304821202120 q14 288 310 415 310 q15 q16 891 778 733 733 q17 1200141413851385 q18 7189680466456645 q19 920 932 940 932 q20 2042213819911991 q21 3946356732783278 q22 515 425 366 366 Total cold run time: 50192 ms Total hot run time: 47632 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4141173422 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4140241839 # BE Regression && UT Coverage Report Increment line coverage `100% (0/0)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_4d33233b0738ba708fd3904628154c624af6c0ed_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_4d33233b0738ba708fd3904628154c624af6c0ed_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 71.75% (26468/36890) | | Line Coverage | 54.64% (279235/511012) | | Region Coverage | 51.97% (232383/447131) | | Branch Coverage | 53.33% (100198/187892) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4140225688 # BE Regression && UT Coverage Report Increment line coverage `100% (0/0)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_4d33233b0738ba708fd3904628154c624af6c0ed_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_4d33233b0738ba708fd3904628154c624af6c0ed_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 71.75% (26469/36890) | | Line Coverage | 54.64% (279229/511012) | | Region Coverage | 51.97% (232361/447131) | | Branch Coverage | 53.33% (100198/187892) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4139858917 run cloud_p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4139858258 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4133614028 # BE Regression && UT Coverage Report Increment line coverage `100% (0/0)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_4d33233b0738ba708fd3904628154c624af6c0ed_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_4d33233b0738ba708fd3904628154c624af6c0ed_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 71.75% (26469/36890) | | Line Coverage | 54.64% (279229/511012) | | Region Coverage | 51.97% (232361/447131) | | Branch Coverage | 53.33% (100198/187892) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4133582445 # BE UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/4d33233b0738ba708fd3904628154c624af6c0ed_4d33233b0738ba708fd3904628154c624af6c0ed/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/4d33233b0738ba708fd3904628154c624af6c0ed_4d33233b0738ba708fd3904628154c624af6c0ed/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.92% (19934/37665) | | Line Coverage | 36.44% (186797/512557) | | Region Coverage | 32.67% (144721/443006) | | Branch Coverage | 33.89% (63481/187326) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4132679584 TPC-DS: Total hot run time: 169776 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 4d33233b0738ba708fd3904628154c624af6c0ed, data reload: false query5 4340643 505 505 query6 327 228 206 206 query7 4209468 260 260 query8 340 235 253 235 query9 8762274827262726 query10 473 389 355 355 query11 6963508048604860 query12 184 127 126 126 query13 1264492 351 351 query14 5753374134863486 query14_12872279727972797 query15 199 190 173 173 query16 959 459 442 442 query17 871 705 597 597 query18 2435440 354 354 query19 211 206 177 177 query20 135 122 124 122 query21 209 133 107 107 query22 13253 13995 14624 13995 query23 16723 16342 16206 16206 query23_116075 16399 15731 15731 query24 7184161612151215 query24_11225124212391239 query25 537 468 405 405 query26 1247256 148 148 query27 2792482 297 297 query28 4506183718751837 query29 832 573 487 487 query30 303 226 188 188 query31 1000932 862 862 query32 85 76 76 76 query33 521 340 288 288 query34 886 892 518 518 query35 646 699 598 598 query36 10921158966 966 query37 132 91 86 86 query38 2912293529192919 query39 860 829 825 825 query39_1773 796 800 796 query40 230 148 133 133 query41 62 60 57 57 query42 263 257 255 255 query43 247 246 219 219 query44 query45 200 187 175 175 query46 884 997 608 608 query47 2089209920732073 query48 315 317 228 228 query49 637 473 403 403 query50 691 275 220 220 query51 4091405839813981 query52 260 265 256 256 query53 295 344 284 284 query54 306 291 267 267 query55 98 87 83 83 query56 312 323 309 309 query57 1927182718581827 query58 280 281 274 274 query59 2782295927812781 query60 360 365 354 354 query61 182 182 184 182 query62 625 594 554 554 query63 315 296 281 281 query64 5191138511261126 query65 query66 1478480 368 368 query67 24259 24290 24190 24190 query68 query69 414 333 308 308 query70 975 974 921 921 query71 334 316 305 305 query72 3051279625512551 query73 543 547 319 319 query74 9639966995289528 query75 2916280025182518 query76 22961089698 698 query77 371 401 316 316 query78 11063 11122 10564 10564 query79 2985780 576 576 query80 1719631 545 545 query81 580 272 229 229 query82 978 156 116 116 query83 337 267 249 249 query84 306 123 99 99 query85 919 518 468 468 query86 519 339 301 301 query87 3157313130093009 query88 3545264426502644 query89 431 371 346 346 query90 2170186 173 173 query91 177 157 135 135 query92 86 80 75 75 query93 1868826 494 494 query94 673 307 294 294 query95 589 336 381 336 query96 647 515 230 230 query97 2481247023812381 query98 237 225 219 219 query99 10221006903 903 Total cold run time: 253483 ms Total hot run time: 169776 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected].
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4132618727 TPC-H: Total hot run time: 26969 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 4d33233b0738ba708fd3904628154c624af6c0ed, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17652 42984298 q2 q3 10641 791 532 532 q4 4678366 245 245 q5 7717121810341034 q6 189 176 154 154 q7 818 891 682 682 q8 10276 154513771377 q9 5801483247804780 q10 6330195516371637 q11 451 246 242 242 q12 751 593 472 472 q13 18031 276319651965 q14 227 237 211 211 q15 q16 745 757 669 669 q17 739 883 436 436 q18 5976533552405240 q19 1126996 635 635 q20 531 516 395 395 q21 4612202316931693 q22 382 333 272 272 Total cold run time: 97673 ms Total hot run time: 26969 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4691464846484648 q2 q3 3901437638353835 q4 893 1230805 805 q5 4094447146354471 q6 204 198 161 161 q7 1845169615721572 q8 2527274726552655 q9 7598737473707370 q10 3779407837243724 q11 507 435 419 419 q12 502 597 438 438 q13 2526312621012101 q14 285 303 283 283 q15 q16 724 783 739 739 q17 1208144514331433 q18 7415696166306630 q19 950 933 928 928 q20 2070214320232023 q21 4022358433793379 q22 437 433 387 387 Total cold run time: 50178 ms Total hot run time: 48001 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4132570923 # FE UT Coverage Report Increment line coverage ` 84.09% (37/44)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_4d33233b0738ba708fd3904628154c624af6c0ed/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_4d33233b0738ba708fd3904628154c624af6c0ed/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4132113998 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4065828586 # BE Regression && UT Coverage Report Increment line coverage `100% (0/0)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_6e417c81df1c798c3990716599b2199890f82f10_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_6e417c81df1c798c3990716599b2199890f82f10_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 73.22% (26827/36638) | | Line Coverage | 56.61% (286862/506765) | | Region Coverage | 53.85% (238404/442755) | | Branch Coverage | 55.59% (103161/185578) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4065803656 # BE UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/6e417c81df1c798c3990716599b2199890f82f10_6e417c81df1c798c3990716599b2199890f82f10/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/6e417c81df1c798c3990716599b2199890f82f10_6e417c81df1c798c3990716599b2199890f82f10/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.66% (19713/37432) | | Line Coverage | 36.22% (184195/508478) | | Region Coverage | 32.38% (142067/438706) | | Branch Coverage | 33.57% (62130/185096) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4065276139 TPC-DS: Total hot run time: 168544 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 6e417c81df1c798c3990716599b2199890f82f10, data reload: false query5 4324634 496 496 query6 345 239 197 197 query7 4207468 266 266 query8 327 227 221 221 query9 8736268826772677 query10 525 375 328 328 query11 6939511048864886 query12 185 129 135 129 query13 1268459 373 373 query14 5708372434363436 query14_12802281327782778 query15 196 190 175 175 query16 964 453 464 453 query17 856 717 593 593 query18 2429438 340 340 query19 215 210 184 184 query20 135 127 125 125 query21 228 129 107 107 query22 13444 14037 14822 14037 query23 16072 15830 15563 15563 query23_115738 15980 15815 15815 query24 7164160812371237 query24_11252125212221222 query25 580 485 450 450 query26 1252268 152 152 query27 2776486 304 304 query28 4510184218621842 query29 864 582 502 502 query30 305 246 191 191 query31 1012944 874 874 query32 87 74 68 68 query33 521 355 309 309 query34 931 878 539 539 query35 661 676 604 604 query36 11101149999 999 query37 142 94 84 84 query38 2944298328652865 query39 863 828 824 824 query39_1806 788 791 788 query40 241 155 139 139 query41 68 65 65 65 query42 265 257 262 257 query43 247 249 221 221 query44 query45 200 191 191 191 query46 887 983 619 619 query47 2819219620772077 query48 333 323 239 239 query49 675 451 366 366 query50 674 278 211 211 query51 4067408540084008 query52 264 266 255 255 query53 286 331 280 280 query54 302 297 270 270 query55 95 93 81 81 query56 324 322 318 318 query57 1922189817251725 query58 284 267 270 267 query59 2807296327732773 query60 343 341 325 325 query61 153 151 149 149 query62 637 567 551 551 query63 311 277 274 274 query64 5081125810421042 query65 query66 1457459 365 365 query67 24190 24334 24138 24138 query68 query69 395 318 284 284 query70 1011986 898 898 query71 348 300 305 300 query72 2767264323902390 query73 542 553 341 341 query74 9612954194019401 query75 2868273824712471 query76 22751038670 670 query77 361 372 306 306 query78 10975 11218 10508 10508 query79 1094801 583 583 query80 1334622 527 527 query81 553 270 238 238 query82 995 156 120 120 query83 352 265 240 240 query84 305 120 100 100 query85 913 497 434 434 query86 443 314 319 314 query87 3158307931033079 query88 3562265726722657 query89 422 383 347 347 query90 2023180 176 176 query91 167 161 139 139 query92 81 72 75 72 query93 1024858 501 501 query94 634 301 291 291 query95 576 334 387 334 query96 656 512 230 230 query97 2488248624082408 query98 248 220 224 220 query99 10141014886 886 Total cold run time: 250173 ms Total hot run time: 168544 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4065239279 TPC-H: Total hot run time: 26911 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 6e417c81df1c798c3990716599b2199890f82f10, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17616 449742954295 q2 q3 10643 839 517 517 q4 4678362 254 254 q5 75501215998 998 q6 172 173 145 145 q7 804 864 659 659 q8 9658147513221322 q9 5255476047014701 q10 6326195016871687 q11 480 261 237 237 q12 751 571 460 460 q13 18037 294821812181 q14 232 245 211 211 q15 q16 757 758 687 687 q17 733 863 435 435 q18 5959539852855285 q19 1165983 608 608 q20 545 499 369 369 q21 4479181815671567 q22 478 379 293 293 Total cold run time: 96318 ms Total hot run time: 26911 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4781457945744574 q2 q3 4023443838653865 q4 872 1218808 808 q5 4125445543114311 q6 177 179 144 144 q7 1763165614691469 q8 2516274226272627 q9 7621732475447324 q10 3783400135463546 q11 522 444 420 420 q12 500 735 450 450 q13 2714319723432343 q14 296 330 290 290 q15 q16 722 783 733 733 q17 1194140914251409 q18 7553697166766676 q19 882 897 923 897 q20 2067226420912091 q21 4015354633653365 q22 471 432 384 384 Total cold run time: 50597 ms Total hot run time: 47726 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4065113901 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2938082534
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/SplitByString.java:
##
@@ -60,10 +71,25 @@ private SplitByString(ScalarFunctionParams functionParams) {
*/
@Override
public SplitByString withChildren(List children) {
-Preconditions.checkArgument(children.size() == 2);
+Preconditions.checkArgument(children.size() == 2 || children.size() ==
3);
return new SplitByString(getFunctionParams(children));
}
+@Override
+public void checkLegalityBeforeTypeCoercion() {
+checkLegalityAfterRewrite();
+}
+
+@Override
+public void checkLegalityAfterRewrite() {
Review Comment:
If the expression `split_by_string('one,two,three,', ',', '-1')` need to be
supported, checkLegalityBeforeTypeCoercion should be removed as you said.
For checkLegalityAfterRewrite, I'd like to keep it because at that point
'-1' has already been folded to IntegerLiteral(-1) and passes the check. But
column references like split_by_string(v1, v2, k1) would still be correctly
rejected since the BE extracts the limit once in open() and doesn't support
per-row values. This is consistent with Sha2's pattern. Does this approach work
for you?
And I'll add more tests about this case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
morrySnow commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2937948876
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/SplitByString.java:
##
@@ -60,10 +71,25 @@ private SplitByString(ScalarFunctionParams functionParams) {
*/
@Override
public SplitByString withChildren(List children) {
-Preconditions.checkArgument(children.size() == 2);
+Preconditions.checkArgument(children.size() == 2 || children.size() ==
3);
return new SplitByString(getFunctionParams(children));
}
+@Override
+public void checkLegalityBeforeTypeCoercion() {
+checkLegalityAfterRewrite();
+}
+
+@Override
+public void checkLegalityAfterRewrite() {
Review Comment:
i think both checkLegalityBeforeTypeCoercion and checkLegalityAfterRewrite
should be remove, we should support expression like
`split_by_string('one,two,three,', ',', '-1')`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4063590713 @zclllyybb @linrrzqqq need re-review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4063530041 # BE Regression && UT Coverage Report Increment line coverage `100% (0/0)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_7ffdfb36c486ee34f6e8849b1a96e1389fac0781_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_7ffdfb36c486ee34f6e8849b1a96e1389fac0781_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 71.55% (26231/36660) | | Line Coverage | 54.32% (275217/506672) | | Region Coverage | 51.45% (228068/443264) | | Branch Coverage | 52.94% (98294/185674) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4063363173 TPC-DS: Total hot run time: 168394 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 7ffdfb36c486ee34f6e8849b1a96e1389fac0781, data reload: false query5 4329612 516 516 query6 313 227 206 206 query7 4222467 275 275 query8 334 241 218 218 query9 8729271126912691 query10 517 409 346 346 query11 6974509548844884 query12 186 131 136 131 query13 1263455 359 359 query14 5629375335283528 query14_12835282328602823 query15 200 194 179 179 query16 983 483 446 446 query17 896 750 610 610 query18 2444458 349 349 query19 212 205 181 181 query20 131 128 127 127 query21 215 134 113 113 query22 13336 13992 14902 13992 query23 16143 15931 15680 15680 query23_115777 15802 15756 15756 query24 7360163012611261 query24_11264125312641253 query25 582 494 437 437 query26 1245268 208 208 query27 2711475 295 295 query28 4457184418531844 query29 857 564 469 469 query30 291 224 193 193 query31 1009968 856 856 query32 78 74 70 70 query33 503 325 277 277 query34 878 884 517 517 query35 628 681 587 587 query36 1112112110281028 query37 136 95 83 83 query38 2928288228502850 query39 877 841 801 801 query39_1798 785 788 785 query40 236 152 139 139 query41 62 59 59 59 query42 262 253 256 253 query43 241 261 215 215 query44 query45 194 184 179 179 query46 883 968 616 616 query47 2107214020602060 query48 317 355 241 241 query49 631 475 388 388 query50 669 281 207 207 query51 4212406240024002 query52 261 261 251 251 query53 293 333 285 285 query54 288 269 259 259 query55 94 86 88 86 query56 320 312 311 311 query57 1927173816481648 query58 279 273 266 266 query59 2774297727372737 query60 337 337 324 324 query61 152 151 148 148 query62 627 591 542 542 query63 305 279 277 277 query64 50921281987 987 query65 query66 1451452 357 357 query67 24295 24278 24177 24177 query68 query69 402 308 279 279 query70 943 974 1012974 query71 332 308 290 290 query72 2778269923772377 query73 543 548 315 315 query74 9608957194089408 query75 2840273124972497 query76 22971028685 685 query77 364 381 300 300 query78 10844 10984 10477 10477 query79 3097753 562 562 query80 1740641 543 543 query81 578 257 217 217 query82 926 149 119 119 query83 330 266 241 241 query84 299 118 99 99 query85 909 495 440 440 query86 454 283 291 283 query87 3114308630083008 query88 3520264226112611 query89 423 374 351 351 query90 2004178 175 175 query91 167 158 139 139 query92 76 78 70 70 query93 1406833 503 503 query94 648 316 279 279 query95 580 336 313 313 query96 642 505 234 234 query97 2486250324112411 query98 243 230 224 224 query99 10011004916 916 Total cold run time: 250868 ms Total hot run time: 168394 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected].
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4063332789 TPC-H: Total hot run time: 27062 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 7ffdfb36c486ee34f6e8849b1a96e1389fac0781, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17658 452542964296 q2 q3 10641 797 518 518 q4 4676371 257 257 q5 7546120110411041 q6 177 176 147 147 q7 783 837 676 676 q8 9537147812921292 q9 4948475047454745 q10 6330191716851685 q11 464 257 247 247 q12 743 563 466 466 q13 18057 291222042204 q14 229 227 220 220 q15 q16 734 752 671 671 q17 720 806 475 475 q18 5858542253715371 q19 1254977 616 616 q20 536 500 376 376 q21 4475184014901490 q22 509 350 269 269 Total cold run time: 95875 ms Total hot run time: 27062 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4795450446664504 q2 q3 3905430937963796 q4 873 1184821 821 q5 4062441543334333 q6 183 172 141 141 q7 1733163515291529 q8 2501269525952595 q9 7584751774077407 q10 3792406036593659 q11 495 441 430 430 q12 489 581 443 443 q13 2686328623402340 q14 286 295 274 274 q15 q16 710 775 718 718 q17 1194146913421342 q18 7234685466546654 q19 931 1030907 907 q20 2112217120202020 q21 3986352933143314 q22 445 447 374 374 Total cold run time: 49996 ms Total hot run time: 47601 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4063288029 run nonConcurrent -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4063279500 run performance -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4063004063 # BE Regression && UT Coverage Report Increment line coverage `100% (0/0)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_7ffdfb36c486ee34f6e8849b1a96e1389fac0781_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_7ffdfb36c486ee34f6e8849b1a96e1389fac0781_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 71.55% (26231/36660) | | Line Coverage | 54.32% (275239/506672) | | Region Coverage | 51.49% (228227/443264) | | Branch Coverage | 52.95% (98312/185674) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4062878069 # FE UT Coverage Report Increment line coverage ` 80.43% (37/46)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_7ffdfb36c486ee34f6e8849b1a96e1389fac0781/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_7ffdfb36c486ee34f6e8849b1a96e1389fac0781/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4062363890 run compile -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4062322792 # BE UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/7ffdfb36c486ee34f6e8849b1a96e1389fac0781_7ffdfb36c486ee34f6e8849b1a96e1389fac0781/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/7ffdfb36c486ee34f6e8849b1a96e1389fac0781_7ffdfb36c486ee34f6e8849b1a96e1389fac0781/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.65% (19719/37454) | | Line Coverage | 36.25% (184267/508385) | | Region Coverage | 32.36% (142111/439215) | | Branch Coverage | 33.57% (62164/185192) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2936289704
##
regression-test/suites/nereids_p0/sql_functions/string_functions/test_split_by_string_limit.groovy:
##
@@ -0,0 +1,88 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_split_by_string_limit") {
+// === Constant expression tests with limit ===
+
+// Basic limit functionality
+qt_limit1 "select split_by_string('one,two,three,', ',', 2);"
+qt_limit2 "select split_by_string('one,two,three,', ',', 3);"
+qt_limit3 "select split_by_string('one,two,three,', ',', 4);"
+qt_limit4 "select split_by_string('one,two,three,', ',', 10);"
+qt_limit5 "select split_by_string('one,two,three', ',', 1);"
+
+// limit = -1 (no limit, same as 2-arg)
+qt_limit6 "select split_by_string('one,two,three,', ',', -1);"
+
+// limit = 0 (no limit, same as 2-arg)
+qt_limit7 "select split_by_string('a,b,c', ',', 0);"
+
+// Empty source string + limit
+qt_limit8 "select split_by_string('', ',', 2);"
+
+// Empty delimiter + limit (split by character)
+qt_limit9 "select split_by_string('abcde', '', 3);"
+qt_limit10 "select split_by_string('abcde', '', 1);"
+qt_limit11 "select split_by_string('abcde', '', 10);"
+
+// Multi-char delimiter + limit
+qt_limit12 "select split_by_string('a::b::c::d', '::', 2);"
+qt_limit13 "select split_by_string('a::b::c::d', '::', 3);"
+qt_limit14 "select split_by_string('1,,2,3,,4,5,,abcde', ',,', 2);"
+
+// NULL handling
+qt_limit15 "select split_by_string(NULL, ',', 2);"
+
+// UTF-8 + limit
+qt_limit16 "select split_by_string('你a好b世c界', '', 3);"
+
+// Edge cases: consecutive delimiters + limit
+qt_limit17 "select split_by_string(',,,', ',', 2);"
+qt_limit18 "select split_by_string(',,a,b,c,', ',', 3);"
+
+// === Table data tests ===
+sql """DROP TABLE IF EXISTS test_split_limit"""
+sql """
+CREATE TABLE IF NOT EXISTS test_split_limit (
+`k1` int(11) NULL COMMENT "",
+`v1` varchar(50) NULL COMMENT "",
+`v2` varchar(10) NOT NULL COMMENT ""
+) ENGINE=OLAP
+DUPLICATE KEY(`k1`)
+DISTRIBUTED BY HASH(`k1`) BUCKETS 1
+PROPERTIES (
+"replication_allocation" = "tag.location.default: 1",
+"storage_format" = "V2"
+)
+"""
+sql """ INSERT INTO test_split_limit VALUES(1, 'a,b,c,d', ',') """
+sql """ INSERT INTO test_split_limit VALUES(2, 'x::y::z', '::') """
+sql """ INSERT INTO test_split_limit VALUES(3, 'hello', ',') """
+sql """ INSERT INTO test_split_limit VALUES(4, null, ',') """
+sql """ INSERT INTO test_split_limit VALUES(5, 'a,b,c,d,e', ',') """
+
+qt_table1 "SELECT k1, split_by_string(v1, v2, 2) FROM test_split_limit
ORDER BY k1"
Review Comment:
Having add more tests
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2936288925
##
be/src/exprs/function/function_string.h:
##
@@ -2163,44 +2158,143 @@ class FunctionSplitByString : public IFunction {
}
}
-void split_empty_delimiter(const StringRef& str_ref, ColumnString::Chars&
column_string_chars,
- ColumnString::Offsets& column_string_offsets,
- ColumnArray::Offset64& string_pos,
- ColumnArray::Offset64& dest_pos) const {
+static void split_empty_delimiter(const StringRef& str_ref,
+ ColumnString::Chars& column_string_chars,
+ ColumnString::Offsets&
column_string_offsets,
+ ColumnArray::Offset64& string_pos,
+ ColumnArray::Offset64& dest_pos, Int32
limit_value) {
const size_t old_size = column_string_chars.size();
const size_t new_size = old_size + str_ref.size;
column_string_chars.resize(new_size);
memcpy(column_string_chars.data() + old_size, str_ref.data,
str_ref.size);
-if (simd::VStringFunctions::is_ascii(str_ref)) {
-const auto size = str_ref.size;
-
-const auto nested_old_size = column_string_offsets.size();
-const auto nested_new_size = nested_old_size + size;
-column_string_offsets.resize(nested_new_size);
-std::iota(column_string_offsets.data() + nested_old_size,
- column_string_offsets.data() + nested_new_size,
string_pos + 1);
-
-string_pos += size;
-dest_pos += size;
-// The above code is equivalent to the code in the following
comment.
-// for (size_t i = 0; i < str_ref.size; i++) {
-// string_pos++;
-// column_string_offsets.push_back(string_pos);
-// (*dest_nested_null_map).push_back(false);
-// dest_pos++;
-// }
+
+if (limit_value > 0) {
+// With limit: split character by character up to limit-1, then
remainder
+Int32 split_count = 0;
+size_t i = 0;
+if (simd::VStringFunctions::is_ascii(str_ref)) {
+for (; i < str_ref.size; i++) {
+if (split_count == limit_value - 1) {
+// remainder
+string_pos += str_ref.size - i;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+return;
+}
+string_pos++;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+split_count++;
+}
+} else {
+for (size_t utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
+utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
+if (split_count == limit_value - 1) {
+// remainder
+string_pos += str_ref.size - i;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+return;
+}
+string_pos += utf8_char_len;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+split_count++;
+}
+}
} else {
-for (size_t i = 0, utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
-utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
+// No limit: original behavior
+if (simd::VStringFunctions::is_ascii(str_ref)) {
+const auto size = str_ref.size;
+
+const auto nested_old_size = column_string_offsets.size();
+const auto nested_new_size = nested_old_size + size;
+column_string_offsets.resize(nested_new_size);
+std::iota(column_string_offsets.data() + nested_old_size,
+ column_string_offsets.data() + nested_new_size,
string_pos + 1);
+
+string_pos += size;
+dest_pos += size;
+} else {
+for (size_t i = 0, utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
+utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
-string_pos += utf8_char_len;
-column_string_offsets.push_back(string_pos);
-dest_pos++;
+string_pos += utf8_char_len;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+}
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4062270453 run feut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4062218657 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4062203612 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4062193101 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4061489573 # BE Regression && UT Coverage Report Increment line coverage `100% (0/0)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_613dd9cce0835110702c94c44ed191b042bc413f_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_613dd9cce0835110702c94c44ed191b042bc413f_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 71.59% (26250/36669) | | Line Coverage | 54.36% (275459/506774) | | Region Coverage | 51.60% (228904/443640) | | Branch Coverage | 53.02% (98490/185765) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4061355311 # FE UT Coverage Report Increment line coverage ` 80.43% (37/46)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_613dd9cce0835110702c94c44ed191b042bc413f/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_613dd9cce0835110702c94c44ed191b042bc413f/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4061347915 # BE UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/613dd9cce0835110702c94c44ed191b042bc413f_613dd9cce0835110702c94c44ed191b042bc413f/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/613dd9cce0835110702c94c44ed191b042bc413f_613dd9cce0835110702c94c44ed191b042bc413f/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.67% (19734/37464) | | Line Coverage | 36.26% (184387/508496) | | Region Coverage | 32.36% (142257/439592) | | Branch Coverage | 33.57% (62201/185283) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4061274236 TPC-DS: Total hot run time: 168255 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 613dd9cce0835110702c94c44ed191b042bc413f, data reload: false query5 4318640 497 497 query6 343 234 209 209 query7 4205464 269 269 query8 340 239 227 227 query9 8704279327682768 query10 515 368 342 342 query11 6957511048544854 query12 179 124 123 123 query13 1244463 347 347 query14 5803368734723472 query14_12828278327962783 query15 203 192 182 182 query16 953 469 424 424 query17 851 700 586 586 query18 2433443 333 333 query19 208 201 180 180 query20 130 125 124 124 query21 211 139 109 109 query22 13215 14289 15133 14289 query23 16057 16023 15530 15530 query23_115663 15604 15637 15604 query24 7119162712131213 query24_11223122112071207 query25 542 463 398 398 query26 1231264 147 147 query27 2784492 300 300 query28 4481195319251925 query29 877 552 481 481 query30 295 227 187 187 query31 1002941 870 870 query32 86 69 67 67 query33 502 335 286 286 query34 908 909 530 530 query35 627 677 605 605 query36 10821127984 984 query37 139 93 82 82 query38 2942289229312892 query39 865 832 811 811 query39_1794 805 807 805 query40 237 152 136 136 query41 63 60 59 59 query42 258 254 249 249 query43 238 250 214 214 query44 query45 192 187 185 185 query46 889 987 621 621 query47 2102213020402040 query48 328 316 227 227 query49 633 452 386 386 query50 679 280 213 213 query51 4079409539923992 query52 260 266 257 257 query53 299 330 279 279 query54 299 269 269 269 query55 94 82 81 81 query56 297 313 311 311 query57 1932184915531553 query58 279 271 263 263 query59 2775294927412741 query60 358 348 339 339 query61 178 171 174 171 query62 639 590 525 525 query63 312 285 279 279 query64 5209135110881088 query65 query66 1479488 369 369 query67 24234 24324 24193 24193 query68 query69 414 308 286 286 query70 979 964 942 942 query71 350 315 299 299 query72 2985278223732373 query73 542 538 322 322 query74 9579953294159415 query75 2846274924782478 query76 22851024659 659 query77 355 361 312 312 query78 10870 10989 10440 10440 query79 3050826 585 585 query80 1736619 538 538 query81 575 266 224 224 query82 964 151 117 117 query83 340 259 240 240 query84 304 121 98 98 query85 908 484 435 435 query86 496 303 288 288 query87 3112314930233023 query88 3555268126592659 query89 439 370 337 337 query90 2253173 171 171 query91 162 165 131 131 query92 101 74 72 72 query93 2592859 499 499 query94 638 309 277 277 query95 576 334 316 316 query96 636 518 231 231 query97 2450246124142414 query98 233 220 217 217 query99 980 1009912 912 Total cold run time: 252833 ms Total hot run time: 168255 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected].
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4061251959 TPC-H: Total hot run time: 26930 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 613dd9cce0835110702c94c44ed191b042bc413f, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17677 448543184318 q2 q3 10640 793 534 534 q4 4680363 253 253 q5 7554118810381038 q6 177 179 147 147 q7 793 861 676 676 q8 9448145912841284 q9 5135472346904690 q10 6331190916381638 q11 478 247 234 234 q12 755 578 478 478 q13 18034 295121792179 q14 227 224 223 223 q15 q16 747 745 661 661 q17 695 855 415 415 q18 5929544853485348 q19 11741000655 655 q20 555 507 392 392 q21 4537184314811481 q22 462 364 286 286 Total cold run time: 96028 ms Total hot run time: 26930 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4767458447154584 q2 q3 3830435238053805 q4 924 1212837 837 q5 4069436943464346 q6 188 188 143 143 q7 1778169315961596 q8 2564270725192519 q9 7438715174447151 q10 3840406336033603 q11 499 426 412 412 q12 490 585 447 447 q13 2819307225912591 q14 290 304 282 282 q15 q16 724 775 726 726 q17 1161135513031303 q18 7140693967666766 q19 889 889 912 889 q20 2064217020082008 q21 3935349933263326 q22 462 436 373 373 Total cold run time: 49871 ms Total hot run time: 47707 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4061194951 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060968567 # BE Regression && UT Coverage Report Increment line coverage `100% (0/0)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_c3b59b6c07d411d25263683b0dcbcaaa1010b571_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_c3b59b6c07d411d25263683b0dcbcaaa1010b571_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 73.28% (26871/36669) | | Line Coverage | 56.57% (286697/506774) | | Region Coverage | 53.85% (238920/443640) | | Branch Coverage | 55.60% (103279/185765) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060786994 # BE UT Coverage Report Increment line coverage ` ` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/c3b59b6c07d411d25263683b0dcbcaaa1010b571_c3b59b6c07d411d25263683b0dcbcaaa1010b571/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/c3b59b6c07d411d25263683b0dcbcaaa1010b571_c3b59b6c07d411d25263683b0dcbcaaa1010b571/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.67% (19734/37464) | | Line Coverage | 36.26% (184386/508496) | | Region Coverage | 32.34% (142182/439592) | | Branch Coverage | 33.57% (62196/185283) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060780232 # FE UT Coverage Report Increment line coverage ` 80.43% (37/46)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_c3b59b6c07d411d25263683b0dcbcaaa1010b571/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_c3b59b6c07d411d25263683b0dcbcaaa1010b571/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060731987 TPC-DS: Total hot run time: 168229 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit c3b59b6c07d411d25263683b0dcbcaaa1010b571, data reload: false query5 4324636 488 488 query6 332 228 213 213 query7 4212485 267 267 query8 352 245 244 244 query9 8687277127632763 query10 522 368 341 341 query11 7027511648394839 query12 188 129 125 125 query13 1311471 358 358 query14 5766372534773477 query14_12873283228082808 query15 225 197 178 178 query16 975 459 443 443 query17 899 725 617 617 query18 2454448 352 352 query19 216 209 185 185 query20 135 125 125 125 query21 218 131 117 117 query22 13188 13886 14649 13886 query23 16220 15944 15844 15844 query23_115727 15672 15711 15672 query24 7186160112271227 query24_11253124412191219 query25 524 454 414 414 query26 1241266 147 147 query27 2815484 354 354 query28 4503188118761876 query29 839 556 471 471 query30 301 232 192 192 query31 992 959 863 863 query32 84 69 72 69 query33 520 329 282 282 query34 884 869 538 538 query35 659 668 584 584 query36 10921140996 996 query37 138 96 80 80 query38 2975290827902790 query39 860 836 808 808 query39_1784 806 819 806 query40 235 151 138 138 query41 63 59 59 59 query42 261 259 257 257 query43 238 248 219 219 query44 query45 203 190 185 185 query46 878 972 612 612 query47 3239213020612061 query48 318 323 258 258 query49 637 474 406 406 query50 677 280 219 219 query51 4046403740054005 query52 265 268 261 261 query53 291 339 288 288 query54 298 275 270 270 query55 93 89 83 83 query56 314 319 302 302 query57 1934183515521552 query58 280 271 278 271 query59 2773294627402740 query60 335 328 325 325 query61 154 153 150 150 query62 644 587 546 546 query63 308 283 280 280 query64 5011128410201020 query65 query66 1457465 351 351 query67 24242 24275 24124 24124 query68 query69 414 315 292 292 query70 944 989 992 989 query71 343 307 297 297 query72 2722263224122412 query73 534 547 320 320 query74 9586957294039403 query75 2862274824482448 query76 22821023677 677 query77 360 388 304 304 query78 10852 11038 10408 10408 query79 3028765 585 585 query80 1762621 528 528 query81 584 259 228 228 query82 1003152 119 119 query83 333 270 238 238 query84 307 111 103 103 query85 900 482 447 447 query86 482 312 293 293 query87 3187311429952995 query88 3562268726862686 query89 431 376 363 363 query90 1914174 175 174 query91 167 159 137 137 query92 95 77 67 67 query93 1437834 503 503 query94 641 309 281 281 query95 588 398 310 310 query96 631 510 226 226 query97 2445244223682368 query98 241 225 218 218 query99 1017984 935 935 Total cold run time: 252870 ms Total hot run time: 168229 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060715958 TPC-H: Total hot run time: 26615 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit c3b59b6c07d411d25263683b0dcbcaaa1010b571, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17664 446643084308 q2 q3 10647 783 524 524 q4 4682370 249 249 q5 7557118910041004 q6 173 174 145 145 q7 796 839 671 671 q8 9300145512821282 q9 4922471247204712 q10 6309190616471647 q11 462 245 224 224 q12 742 564 478 478 q13 18042 287221762176 q14 228 230 219 219 q15 q16 744 739 675 675 q17 713 840 438 438 q18 5795535052115211 q19 1133957 610 610 q20 551 479 385 385 q21 4471184113891389 q22 338 279 268 268 Total cold run time: 95269 ms Total hot run time: 26615 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4897466246314631 q2 q3 3985434738383838 q4 905 1200788 788 q5 4091437143714371 q6 176 176 138 138 q7 1734162515211521 q8 2472273725922592 q9 7538745474137413 q10 3773398335753575 q11 511 451 415 415 q12 471 599 447 447 q13 2723328222452245 q14 273 291 278 278 q15 q16 710 763 759 759 q17 1166147813401340 q18 7183671866296629 q19 959 878 934 878 q20 2108215420092009 q21 3873351032923292 q22 491 420 436 420 Total cold run time: 50039 ms Total hot run time: 47579 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060664178 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060606239 # FE UT Coverage Report Increment line coverage ` 80.43% (37/46)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_10b7fd05d4e379d2c9b3529bf2660d73f4cecb12/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_10b7fd05d4e379d2c9b3529bf2660d73f4cecb12/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060555329 TPC-DS: Total hot run time: 168514 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 10b7fd05d4e379d2c9b3529bf2660d73f4cecb12, data reload: false query5 4315626 499 499 query6 331 228 212 212 query7 4215463 267 267 query8 345 263 233 233 query9 8705278227752775 query10 491 367 354 354 query11 6932508148594859 query12 183 121 122 121 query13 1267456 358 358 query14 5682373234983498 query14_12888280527732773 query15 214 199 176 176 query16 991 460 440 440 query17 907 721 626 626 query18 2436452 354 354 query19 215 212 182 182 query20 139 131 127 127 query21 222 135 116 116 query22 13321 14014 14665 14014 query23 16128 15714 1 1 query23_116315 15795 15744 15744 query24 7248161212331233 query24_11248124712501247 query25 564 490 441 441 query26 1249268 152 152 query27 2783492 303 303 query28 4452193318841884 query29 827 572 468 468 query30 300 227 193 193 query31 1031942 873 873 query32 85 77 66 66 query33 506 331 278 278 query34 894 874 525 525 query35 672 668 581 581 query36 10691110978 978 query37 133 94 79 79 query38 2943292028942894 query39 864 834 812 812 query39_1822 802 805 802 query40 234 153 139 139 query41 62 58 62 58 query42 258 250 256 250 query43 245 245 225 225 query44 query45 197 184 185 184 query46 874 979 616 616 query47 2150212120182018 query48 324 314 230 230 query49 625 498 385 385 query50 686 280 216 216 query51 4118406440054005 query52 268 269 255 255 query53 292 338 286 286 query54 290 268 291 268 query55 91 84 81 81 query56 311 340 313 313 query57 1939180516741674 query58 282 282 275 275 query59 2758292927502750 query60 347 340 318 318 query61 155 151 153 151 query62 619 583 536 536 query63 314 283 271 271 query64 5059128510171017 query65 query66 1461462 356 356 query67 24400 24311 24207 24207 query68 query69 400 314 285 285 query70 993 956 965 956 query71 334 306 301 301 query72 2816269324222422 query73 543 562 323 323 query74 9614960893669366 query75 2856275024442444 query76 22621044668 668 query77 353 404 308 308 query78 10844 11055 10436 10436 query79 1143771 590 590 query80 1375615 546 546 query81 543 257 231 231 query82 996 156 119 119 query83 336 265 239 239 query84 301 124 101 101 query85 929 505 439 439 query86 413 303 293 293 query87 3126315530643064 query88 3602270027232700 query89 426 373 345 345 query90 2017183 174 174 query91 174 159 139 139 query92 74 73 68 68 query93 918 858 514 514 query94 635 327 308 308 query95 581 333 382 333 query96 653 541 241 241 query97 2487250323872387 query98 237 220 214 214 query99 988 985 927 927 Total cold run time: 249806 ms Total hot run time: 168514 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060539842 TPC-H: Total hot run time: 27067 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 10b7fd05d4e379d2c9b3529bf2660d73f4cecb12, data reload: false -- Round 1 -- orders Doris NULLNULL0 0 0 NULL0 NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 17599 455343584358 q2 q3 10647 768 512 512 q4 4682370 251 251 q5 7550120910281028 q6 177 179 148 148 q7 792 873 679 679 q8 9700147113511351 q9 5281476747584758 q10 6308189016651665 q11 458 263 241 241 q12 774 581 472 472 q13 18046 299021682168 q14 238 227 223 223 q15 q16 758 724 669 669 q17 711 863 426 426 q18 5996535452225222 q19 1158983 621 621 q20 533 495 382 382 q21 4442184516181618 q22 445 327 275 275 Total cold run time: 96295 ms Total hot run time: 27067 ms - Round 2, with runtime_filter_mode=off - orders Doris NULLNULL15000 42 6422171781 NULL22778155NULLNULL2023-12-26 18:27:23 2023-12-26 18:42:55 NULLutf-8 NULLNULL q1 4751463447244634 q2 q3 3904439438493849 q4 891 1206806 806 q5 4054441643434343 q6 190 186 144 144 q7 1786162615461546 q8 2499279126282628 q9 7568737172627262 q10 3722405535633563 q11 511 431 416 416 q12 484 588 444 444 q13 2763329723292329 q14 285 307 283 283 q15 q16 741 822 729 729 q17 1149158113551355 q18 7324686568096809 q19 884 870 879 870 q20 2074215019411941 q21 3983348533763376 q22 450 430 376 376 Total cold run time: 50013 ms Total hot run time: 47703 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4060500385 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2934838567
##
be/src/vec/functions/function_string.h:
##
@@ -2163,44 +2158,143 @@ class FunctionSplitByString : public IFunction {
}
}
-void split_empty_delimiter(const StringRef& str_ref, ColumnString::Chars&
column_string_chars,
- ColumnString::Offsets& column_string_offsets,
- ColumnArray::Offset64& string_pos,
- ColumnArray::Offset64& dest_pos) const {
+static void split_empty_delimiter(const StringRef& str_ref,
+ ColumnString::Chars& column_string_chars,
+ ColumnString::Offsets&
column_string_offsets,
+ ColumnArray::Offset64& string_pos,
+ ColumnArray::Offset64& dest_pos, Int32
limit_value) {
const size_t old_size = column_string_chars.size();
const size_t new_size = old_size + str_ref.size;
column_string_chars.resize(new_size);
memcpy(column_string_chars.data() + old_size, str_ref.data,
str_ref.size);
-if (simd::VStringFunctions::is_ascii(str_ref)) {
-const auto size = str_ref.size;
-
-const auto nested_old_size = column_string_offsets.size();
-const auto nested_new_size = nested_old_size + size;
-column_string_offsets.resize(nested_new_size);
-std::iota(column_string_offsets.data() + nested_old_size,
- column_string_offsets.data() + nested_new_size,
string_pos + 1);
-
-string_pos += size;
-dest_pos += size;
-// The above code is equivalent to the code in the following
comment.
-// for (size_t i = 0; i < str_ref.size; i++) {
-// string_pos++;
-// column_string_offsets.push_back(string_pos);
-// (*dest_nested_null_map).push_back(false);
-// dest_pos++;
-// }
+
+if (limit_value > 0) {
+// With limit: split character by character up to limit-1, then
remainder
+Int32 split_count = 0;
+size_t i = 0;
+if (simd::VStringFunctions::is_ascii(str_ref)) {
+for (; i < str_ref.size; i++) {
+if (split_count == limit_value - 1) {
+// remainder
+string_pos += str_ref.size - i;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+return;
+}
+string_pos++;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+split_count++;
+}
+} else {
+for (size_t utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
+utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
+if (split_count == limit_value - 1) {
+// remainder
+string_pos += str_ref.size - i;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+return;
+}
+string_pos += utf8_char_len;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+split_count++;
+}
+}
} else {
-for (size_t i = 0, utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
-utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
+// No limit: original behavior
+if (simd::VStringFunctions::is_ascii(str_ref)) {
+const auto size = str_ref.size;
+
+const auto nested_old_size = column_string_offsets.size();
+const auto nested_new_size = nested_old_size + size;
+column_string_offsets.resize(nested_new_size);
+std::iota(column_string_offsets.data() + nested_old_size,
+ column_string_offsets.data() + nested_new_size,
string_pos + 1);
+
+string_pos += size;
+dest_pos += size;
+} else {
+for (size_t i = 0, utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
+utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
-string_pos += utf8_char_len;
-column_string_offsets.push_back(string_pos);
-dest_pos++;
+string_pos += utf8_char_len;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+}
}
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
linrrzqqq commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2934828633
##
regression-test/suites/nereids_p0/sql_functions/string_functions/test_split_by_string_limit.groovy:
##
@@ -0,0 +1,88 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_split_by_string_limit") {
+// === Constant expression tests with limit ===
+
+// Basic limit functionality
+qt_limit1 "select split_by_string('one,two,three,', ',', 2);"
+qt_limit2 "select split_by_string('one,two,three,', ',', 3);"
+qt_limit3 "select split_by_string('one,two,three,', ',', 4);"
+qt_limit4 "select split_by_string('one,two,three,', ',', 10);"
+qt_limit5 "select split_by_string('one,two,three', ',', 1);"
+
+// limit = -1 (no limit, same as 2-arg)
+qt_limit6 "select split_by_string('one,two,three,', ',', -1);"
+
+// limit = 0 (no limit, same as 2-arg)
+qt_limit7 "select split_by_string('a,b,c', ',', 0);"
+
+// Empty source string + limit
+qt_limit8 "select split_by_string('', ',', 2);"
+
+// Empty delimiter + limit (split by character)
+qt_limit9 "select split_by_string('abcde', '', 3);"
+qt_limit10 "select split_by_string('abcde', '', 1);"
+qt_limit11 "select split_by_string('abcde', '', 10);"
+
+// Multi-char delimiter + limit
+qt_limit12 "select split_by_string('a::b::c::d', '::', 2);"
+qt_limit13 "select split_by_string('a::b::c::d', '::', 3);"
+qt_limit14 "select split_by_string('1,,2,3,,4,5,,abcde', ',,', 2);"
+
+// NULL handling
+qt_limit15 "select split_by_string(NULL, ',', 2);"
+
+// UTF-8 + limit
+qt_limit16 "select split_by_string('你a好b世c界', '', 3);"
+
+// Edge cases: consecutive delimiters + limit
+qt_limit17 "select split_by_string(',,,', ',', 2);"
+qt_limit18 "select split_by_string(',,a,b,c,', ',', 3);"
+
+// === Table data tests ===
+sql """DROP TABLE IF EXISTS test_split_limit"""
+sql """
+CREATE TABLE IF NOT EXISTS test_split_limit (
+`k1` int(11) NULL COMMENT "",
+`v1` varchar(50) NULL COMMENT "",
+`v2` varchar(10) NOT NULL COMMENT ""
+) ENGINE=OLAP
+DUPLICATE KEY(`k1`)
+DISTRIBUTED BY HASH(`k1`) BUCKETS 1
+PROPERTIES (
+"replication_allocation" = "tag.location.default: 1",
+"storage_format" = "V2"
+)
+"""
+sql """ INSERT INTO test_split_limit VALUES(1, 'a,b,c,d', ',') """
+sql """ INSERT INTO test_split_limit VALUES(2, 'x::y::z', '::') """
+sql """ INSERT INTO test_split_limit VALUES(3, 'hello', ',') """
+sql """ INSERT INTO test_split_limit VALUES(4, null, ',') """
+sql """ INSERT INTO test_split_limit VALUES(5, 'a,b,c,d,e', ',') """
+
+qt_table1 "SELECT k1, split_by_string(v1, v2, 2) FROM test_split_limit
ORDER BY k1"
Review Comment:
also add some cast wraped in `testFoldConst`
##
regression-test/suites/nereids_p0/sql_functions/string_functions/test_split_by_string_limit.groovy:
##
@@ -0,0 +1,88 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_split_by_string_limit") {
+// === Constant expression tests with limit ===
+
+// Basic limit functionality
+qt_limit1 "select split_by_string('one,two,three,', ',', 2);"
+qt_limit2 "select split_by_string('one,two,three,', ',', 3);"
+qt_limit3 "select split_by_string('one,two,three,', ',', 4);"
+qt_li
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
linrrzqqq commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2934827599
##
be/src/vec/functions/function_string.h:
##
@@ -2163,44 +2158,143 @@ class FunctionSplitByString : public IFunction {
}
}
-void split_empty_delimiter(const StringRef& str_ref, ColumnString::Chars&
column_string_chars,
- ColumnString::Offsets& column_string_offsets,
- ColumnArray::Offset64& string_pos,
- ColumnArray::Offset64& dest_pos) const {
+static void split_empty_delimiter(const StringRef& str_ref,
+ ColumnString::Chars& column_string_chars,
+ ColumnString::Offsets&
column_string_offsets,
+ ColumnArray::Offset64& string_pos,
+ ColumnArray::Offset64& dest_pos, Int32
limit_value) {
const size_t old_size = column_string_chars.size();
const size_t new_size = old_size + str_ref.size;
column_string_chars.resize(new_size);
memcpy(column_string_chars.data() + old_size, str_ref.data,
str_ref.size);
-if (simd::VStringFunctions::is_ascii(str_ref)) {
-const auto size = str_ref.size;
-
-const auto nested_old_size = column_string_offsets.size();
-const auto nested_new_size = nested_old_size + size;
-column_string_offsets.resize(nested_new_size);
-std::iota(column_string_offsets.data() + nested_old_size,
- column_string_offsets.data() + nested_new_size,
string_pos + 1);
-
-string_pos += size;
-dest_pos += size;
-// The above code is equivalent to the code in the following
comment.
-// for (size_t i = 0; i < str_ref.size; i++) {
-// string_pos++;
-// column_string_offsets.push_back(string_pos);
-// (*dest_nested_null_map).push_back(false);
-// dest_pos++;
-// }
+
+if (limit_value > 0) {
+// With limit: split character by character up to limit-1, then
remainder
+Int32 split_count = 0;
+size_t i = 0;
+if (simd::VStringFunctions::is_ascii(str_ref)) {
+for (; i < str_ref.size; i++) {
+if (split_count == limit_value - 1) {
+// remainder
+string_pos += str_ref.size - i;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+return;
+}
+string_pos++;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+split_count++;
+}
+} else {
+for (size_t utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
+utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
+if (split_count == limit_value - 1) {
+// remainder
+string_pos += str_ref.size - i;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+return;
+}
+string_pos += utf8_char_len;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+split_count++;
+}
+}
} else {
-for (size_t i = 0, utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
-utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
+// No limit: original behavior
+if (simd::VStringFunctions::is_ascii(str_ref)) {
+const auto size = str_ref.size;
+
+const auto nested_old_size = column_string_offsets.size();
+const auto nested_new_size = nested_old_size + size;
+column_string_offsets.resize(nested_new_size);
+std::iota(column_string_offsets.data() + nested_old_size,
+ column_string_offsets.data() + nested_new_size,
string_pos + 1);
+
+string_pos += size;
+dest_pos += size;
+} else {
+for (size_t i = 0, utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
+utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
-string_pos += utf8_char_len;
-column_string_offsets.push_back(string_pos);
-dest_pos++;
+string_pos += utf8_char_len;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+}
}
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
github-actions[bot] commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2931632741
##
be/src/vec/functions/function_string.h:
##
@@ -2163,44 +2158,143 @@ class FunctionSplitByString : public IFunction {
}
}
-void split_empty_delimiter(const StringRef& str_ref, ColumnString::Chars&
column_string_chars,
- ColumnString::Offsets& column_string_offsets,
- ColumnArray::Offset64& string_pos,
- ColumnArray::Offset64& dest_pos) const {
+static void split_empty_delimiter(const StringRef& str_ref,
+ ColumnString::Chars& column_string_chars,
+ ColumnString::Offsets&
column_string_offsets,
+ ColumnArray::Offset64& string_pos,
+ ColumnArray::Offset64& dest_pos, Int32
limit_value) {
const size_t old_size = column_string_chars.size();
const size_t new_size = old_size + str_ref.size;
column_string_chars.resize(new_size);
memcpy(column_string_chars.data() + old_size, str_ref.data,
str_ref.size);
-if (simd::VStringFunctions::is_ascii(str_ref)) {
-const auto size = str_ref.size;
-
-const auto nested_old_size = column_string_offsets.size();
-const auto nested_new_size = nested_old_size + size;
-column_string_offsets.resize(nested_new_size);
-std::iota(column_string_offsets.data() + nested_old_size,
- column_string_offsets.data() + nested_new_size,
string_pos + 1);
-
-string_pos += size;
-dest_pos += size;
-// The above code is equivalent to the code in the following
comment.
-// for (size_t i = 0; i < str_ref.size; i++) {
-// string_pos++;
-// column_string_offsets.push_back(string_pos);
-// (*dest_nested_null_map).push_back(false);
-// dest_pos++;
-// }
+
+if (limit_value > 0) {
+// With limit: split character by character up to limit-1, then
remainder
+Int32 split_count = 0;
+size_t i = 0;
+if (simd::VStringFunctions::is_ascii(str_ref)) {
+for (; i < str_ref.size; i++) {
+if (split_count == limit_value - 1) {
+// remainder
+string_pos += str_ref.size - i;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+return;
+}
+string_pos++;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+split_count++;
+}
+} else {
+for (size_t utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
+utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
+if (split_count == limit_value - 1) {
+// remainder
+string_pos += str_ref.size - i;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+return;
+}
+string_pos += utf8_char_len;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+split_count++;
+}
+}
} else {
-for (size_t i = 0, utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
-utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
+// No limit: original behavior
+if (simd::VStringFunctions::is_ascii(str_ref)) {
+const auto size = str_ref.size;
+
+const auto nested_old_size = column_string_offsets.size();
+const auto nested_new_size = nested_old_size + size;
+column_string_offsets.resize(nested_new_size);
+std::iota(column_string_offsets.data() + nested_old_size,
+ column_string_offsets.data() + nested_new_size,
string_pos + 1);
+
+string_pos += size;
+dest_pos += size;
+} else {
+for (size_t i = 0, utf8_char_len = 0; i < str_ref.size; i +=
utf8_char_len) {
+utf8_char_len = UTF8_BYTE_LENGTH[(unsigned
char)str_ref.data[i]];
-string_pos += utf8_char_len;
-column_string_offsets.push_back(string_pos);
-dest_pos++;
+string_pos += utf8_char_len;
+column_string_offsets.push_back(string_pos);
+dest_pos++;
+}
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
zclllyybb commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2931540870
##
regression-test/suites/nereids_p0/sql_functions/string_functions/test_split_by_string_limit.groovy:
##
@@ -0,0 +1,88 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_split_by_string_limit") {
+// === Constant expression tests with limit ===
+
+// Basic limit functionality
+qt_limit1 "select split_by_string('one,two,three,', ',', 2);"
+qt_limit2 "select split_by_string('one,two,three,', ',', 3);"
+qt_limit3 "select split_by_string('one,two,three,', ',', 4);"
+qt_limit4 "select split_by_string('one,two,three,', ',', 10);"
+qt_limit5 "select split_by_string('one,two,three', ',', 1);"
+
+// limit = -1 (no limit, same as 2-arg)
+qt_limit6 "select split_by_string('one,two,three,', ',', -1);"
+
+// limit = 0 (no limit, same as 2-arg)
+qt_limit7 "select split_by_string('a,b,c', ',', 0);"
+
+// Empty source string + limit
+qt_limit8 "select split_by_string('', ',', 2);"
+
+// Empty delimiter + limit (split by character)
+qt_limit9 "select split_by_string('abcde', '', 3);"
+qt_limit10 "select split_by_string('abcde', '', 1);"
+qt_limit11 "select split_by_string('abcde', '', 10);"
+
+// Multi-char delimiter + limit
+qt_limit12 "select split_by_string('a::b::c::d', '::', 2);"
+qt_limit13 "select split_by_string('a::b::c::d', '::', 3);"
+qt_limit14 "select split_by_string('1,,2,3,,4,5,,abcde', ',,', 2);"
+
+// NULL handling
+qt_limit15 "select split_by_string(NULL, ',', 2);"
+
+// UTF-8 + limit
+qt_limit16 "select split_by_string('你a好b世c界', '', 3);"
+
+// Edge cases: consecutive delimiters + limit
+qt_limit17 "select split_by_string(',,,', ',', 2);"
+qt_limit18 "select split_by_string(',,a,b,c,', ',', 3);"
+
+// === Table data tests ===
+sql """DROP TABLE IF EXISTS test_split_limit"""
+sql """
+CREATE TABLE IF NOT EXISTS test_split_limit (
+`k1` int(11) NULL COMMENT "",
+`v1` varchar(50) NULL COMMENT "",
+`v2` varchar(10) NOT NULL COMMENT ""
+) ENGINE=OLAP
+DUPLICATE KEY(`k1`)
+DISTRIBUTED BY HASH(`k1`) BUCKETS 1
+PROPERTIES (
+"replication_allocation" = "tag.location.default: 1",
+"storage_format" = "V2"
+)
+"""
+sql """ INSERT INTO test_split_limit VALUES(1, 'a,b,c,d', ',') """
+sql """ INSERT INTO test_split_limit VALUES(2, 'x::y::z', '::') """
+sql """ INSERT INTO test_split_limit VALUES(3, 'hello', ',') """
+sql """ INSERT INTO test_split_limit VALUES(4, null, ',') """
+sql """ INSERT INTO test_split_limit VALUES(5, 'a,b,c,d,e', ',') """
+
+qt_table1 "SELECT k1, split_by_string(v1, v2, 2) FROM test_split_limit
ORDER BY k1"
Review Comment:
please test more constancy combinations. like column, literal, column ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
zclllyybb commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4055498129 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4011636048 # BE Regression && UT Coverage Report Increment line coverage `92.86% (104/112)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_997de02c3b00e70bc9df4459879a6b19b92c2701_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_997de02c3b00e70bc9df4459879a6b19b92c2701_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 71.42% (26151/36614) | | Line Coverage | 54.22% (274088/505548) | | Region Coverage | 51.40% (227328/442246) | | Branch Coverage | 52.77% (97652/185050) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4011213711 # FE UT Coverage Report Increment line coverage ` 80.43% (37/46)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_997de02c3b00e70bc9df4459879a6b19b92c2701/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_997de02c3b00e70bc9df4459879a6b19b92c2701/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4011205873 # BE UT Coverage Report Increment line coverage `87.50% (98/112)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/997de02c3b00e70bc9df4459879a6b19b92c2701_997de02c3b00e70bc9df4459879a6b19b92c2701/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/997de02c3b00e70bc9df4459879a6b19b92c2701_997de02c3b00e70bc9df4459879a6b19b92c2701/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.58% (19660/37392) | | Line Coverage | 36.21% (183635/507087) | | Region Coverage | 32.52% (142462/438098) | | Branch Coverage | 33.49% (61780/184474) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4011047946 TPC-DS: Total hot run time: 152876 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 997de02c3b00e70bc9df4459879a6b19b92c2701, data reload: false query5 4337652 511 511 query6 317 225 217 217 query7 4226454 260 260 query8 353 246 229 229 query9 8680273527242724 query10 508 362 332 332 query11 7400589856345634 query12 189 134 125 125 query13 1263442 350 350 query14 5751375935213521 query14_12791282928042804 query15 212 196 177 177 query16 1000502 474 474 query17 948 677 590 590 query18 2422434 333 333 query19 204 207 173 173 query20 130 127 127 127 query21 223 143 125 125 query22 5027485547704770 query23 16713 16174 15730 15730 query23_115855 15584 15895 15584 query24 7694159412571257 query24_11229122412181218 query25 548 457 398 398 query26 1222258 141 141 query27 2802465 292 292 query28 4524186918591859 query29 827 566 466 466 query30 312 244 210 210 query31 1327126612221222 query32 80 71 74 71 query33 504 321 275 275 query34 925 899 569 569 query35 652 650 592 592 query36 1104114910021002 query37 127 89 83 83 query38 2925287928792879 query39 875 858 838 838 query39_1827 811 847 811 query40 231 154 138 138 query41 63 61 58 58 query42 344 303 300 300 query43 244 246 221 221 query44 query45 194 193 183 183 query46 872 972 602 602 query47 2114210620012001 query48 312 308 219 219 query49 615 455 396 396 query50 670 276 213 213 query51 4091412740474047 query52 288 300 279 279 query53 287 334 288 288 query54 289 288 262 262 query55 91 84 83 83 query56 306 317 307 307 query57 1368134912651265 query58 289 280 270 270 query59 1346147812981298 query60 343 342 317 317 query61 175 144 147 144 query62 614 575 538 538 query63 310 276 285 276 query64 5056126610041004 query65 query66 1479461 370 370 query67 16522 16347 16390 16347 query68 query69 409 324 303 303 query70 973 1012961 961 query71 351 314 305 305 query72 3030288726332633 query73 539 545 328 328 query74 10026 989597709770 query75 2886275824942494 query76 23221025693 693 query77 360 375 311 311 query78 11265 11409 10666 10666 query79 1124789 595 595 query80 1345614 533 533 query81 566 274 246 246 query82 989 153 116 116 query83 336 263 247 247 query84 287 113 99 99 query85 914 488 452 452 query86 412 311 300 300 query87 3173313229542954 query88 3528266326492649 query89 425 375 345 345 query90 2011177 170 170 query91 163 153 132 132 query92 74 76 69 69 query93 962 816 503 503 query94 638 309 290 290 query95 589 341 378 341 query96 636 516 227 227 query97 2511248724632463 query98 240 223 223 223 query99 10131027945 945 Total cold run time: 234322 ms Total hot run time: 152876 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4010997880 TPC-H: Total hot run time: 27799 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 997de02c3b00e70bc9df4459879a6b19b92c2701, data reload: false -- Round 1 -- q1 17629 463643174317 q2 q3 10646 817 516 516 q4 4682366 252 252 q5 7560121110271027 q6 178 176 148 148 q7 781 863 659 659 q8 9299145913101310 q9 4850478847444744 q10 6255190916601660 q11 466 247 243 243 q12 679 579 464 464 q13 18033 298221852185 q14 234 243 229 229 q15 966 789 804 789 q16 767 729 688 688 q17 723 859 425 425 q18 6074534353125312 q19 1113993 602 602 q20 495 486 407 407 q21 4461210615501550 q22 397 325 272 272 Total cold run time: 96288 ms Total hot run time: 27799 ms - Round 2, with runtime_filter_mode=off - q1 4715460745224522 q2 q3 3837433138463846 q4 975 1196779 779 q5 4049438843214321 q6 187 179 146 146 q7 1769166615511551 q8 2449272526902690 q9 7469734272897289 q10 3781423238023802 q11 513 438 411 411 q12 504 580 459 459 q13 2744332723292329 q14 287 302 278 278 q15 835 804 786 786 q16 715 755 715 715 q17 1131142014371420 q18 7275677566076607 q19 871 906 925 906 q20 2076216520132013 q21 3943348133073307 q22 483 458 395 395 Total cold run time: 50608 ms Total hot run time: 48572 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
github-actions[bot] commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4010881170 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4010849480 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2894942144
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/SplitByString.java:
##
@@ -60,10 +71,20 @@ private SplitByString(ScalarFunctionParams functionParams) {
*/
@Override
public SplitByString withChildren(List children) {
-Preconditions.checkArgument(children.size() == 2);
+Preconditions.checkArgument(children.size() == 2 || children.size() ==
3);
return new SplitByString(getFunctionParams(children));
}
+@Override
+public void checkLegalityBeforeTypeCoercion() {
Review Comment:
Done. Added checkLegalityAfterRewrite() following the same pattern as Sha2.
checkLegalityBeforeTypeCoercion() now delegates to it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
linrrzqqq commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2894839406
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/SplitByString.java:
##
@@ -60,10 +71,20 @@ private SplitByString(ScalarFunctionParams functionParams) {
*/
@Override
public SplitByString withChildren(List children) {
-Preconditions.checkArgument(children.size() == 2);
+Preconditions.checkArgument(children.size() == 2 || children.size() ==
3);
return new SplitByString(getFunctionParams(children));
}
+@Override
+public void checkLegalityBeforeTypeCoercion() {
Review Comment:
also check in `checkLegalityAfterRewrite`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-4010262431 @zclllyybb Just a gentle nudge on this PR—I’d appreciate your thoughts when you’re free. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3996641510 @zclllyybb I've revised the code the make PR-bot happy. Looks like everything is on track. Please reapprove the remaining check and merge the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3995797176 # BE Regression && UT Coverage Report Increment line coverage `92.86% (104/112)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_54bcef1bc5d12bb3a63d04ae1f3c91099b126282_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_54bcef1bc5d12bb3a63d04ae1f3c91099b126282_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 71.55% (26176/36586) | | Line Coverage | 54.31% (274369/505187) | | Region Coverage | 51.44% (227411/442050) | | Branch Coverage | 52.88% (97860/185071) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3995773489 # BE UT Coverage Report Increment line coverage `87.50% (98/112)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/54bcef1bc5d12bb3a63d04ae1f3c91099b126282_54bcef1bc5d12bb3a63d04ae1f3c91099b126282/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/54bcef1bc5d12bb3a63d04ae1f3c91099b126282_54bcef1bc5d12bb3a63d04ae1f3c91099b126282/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.60% (19652/37364) | | Line Coverage | 36.24% (183666/506740) | | Region Coverage | 32.54% (142501/437913) | | Branch Coverage | 33.47% (61755/184507) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3995742551 # FE Regression Coverage Report Increment line coverage ` 90.91% (40/44)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_54bcef1bc5d12bb3a63d04ae1f3c91099b126282_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_54bcef1bc5d12bb3a63d04ae1f3c91099b126282_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3995530916 run beut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3995173364 TPC-DS: Total hot run time: 184519 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 54bcef1bc5d12bb3a63d04ae1f3c91099b126282, data reload: false query5 4724653 524 524 query6 343 214 212 212 query7 4230479 278 278 query8 367 256 242 242 query9 8743278727812781 query10 522 403 351 351 query11 17024 17639 17283 17283 query12 220 139 173 139 query13 1523532 381 381 query14 7622342931173117 query14_12996295731162957 query15 208 198 186 186 query16 1134499 565 499 query17 2231747 660 660 query18 3022472 375 375 query19 240 242 191 191 query20 157 141 130 130 query21 287 141 123 123 query22 5187504047484748 query23 17323 16787 16670 16670 query23_116756 16883 16686 16686 query24 7009162212211221 query24_11240124512331233 query25 597 495 434 434 query26 1238272 154 154 query27 2756485 310 310 query28 4448188518801880 query29 796 561 473 473 query30 318 238 215 215 query31 871 740 635 635 query32 83 73 71 71 query33 509 365 277 277 query34 912 909 553 553 query35 623 674 648 648 query36 10861131991 991 query37 131 91 86 86 query38 2971290928882888 query39 886 869 837 837 query39_1826 823 830 823 query40 226 155 141 141 query41 64 61 59 59 query42 109 106 105 105 query43 376 405 349 349 query44 query45 199 191 188 188 query46 897 979 614 614 query47 2135215020512051 query48 332 324 231 231 query49 621 476 387 387 query50 681 282 224 224 query51 4091417240304030 query52 110 111 98 98 query53 301 335 292 292 query54 304 269 260 260 query55 94 84 86 84 query56 352 316 328 316 query57 1370133812351235 query58 296 291 287 287 query59 2682261225552555 query60 352 346 328 328 query61 158 148 151 148 query62 626 592 537 537 query63 307 282 278 278 query64 4834127210071007 query65 query66 1401466 357 357 query67 16432 16353 16304 16304 query68 query69 397 301 290 290 query70 988 954 982 954 query71 354 316 324 316 query72 2757274424252425 query73 543 554 326 326 query74 10009 992397489748 query75 2846277724892489 query76 23131034707 707 query77 370 376 298 298 query78 11172 11418 10665 10665 query79 1204819 600 600 query80 1370628 546 546 query81 557 288 246 246 query82 985 151 116 116 query83 366 271 244 244 query84 251 119 99 99 query85 892 496 457 457 query86 412 309 306 306 query87 3174311429912991 query88 3591266926662666 query89 435 373 348 348 query90 2001187 180 180 query91 168 161 138 138 query92 85 78 72 72 query93 960 827 529 529 query94 646 314 297 297 query95 601 410 330 330 query96 660 534 236 236 query97 2471251024222422 query98 231 222 216 216 query99 973 983 911 911 Total cold run time: 256706 ms Total hot run time: 184519 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3995142205 TPC-H: Total hot run time: 28800 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 54bcef1bc5d12bb3a63d04ae1f3c91099b126282, data reload: false -- Round 1 -- q1 17629 452843164316 q2 q3 10653 775 523 523 q4 4676354 268 268 q5 7542119210061006 q6 175 180 153 153 q7 785 853 675 675 q8 9296147413841384 q9 4860474047064706 q10 6838185716411641 q11 453 255 243 243 q12 729 564 473 473 q13 17771 421033963396 q14 229 225 206 206 q15 947 793 791 791 q16 778 729 676 676 q17 707 867 432 432 q18 6142526952225222 q19 1121990 631 631 q20 521 485 384 384 q21 4700187314301430 q22 344 287 244 244 Total cold run time: 96896 ms Total hot run time: 28800 ms - Round 2, with runtime_filter_mode=off - q1 4399433744314337 q2 q3 1768216617111711 q4 860 1165777 777 q5 4031434343214321 q6 178 175 140 140 q7 1715159314831483 q8 2421265325222522 q9 7561749173687368 q10 2678294124422442 q11 538 454 409 409 q12 532 582 467 467 q13 3955444236853685 q14 286 303 334 303 q15 900 828 827 827 q16 712 796 720 720 q17 1204159513711371 q18 7155686767186718 q19 969 966 933 933 q20 2086212519881988 q21 4011343335733433 q22 467 440 387 387 Total cold run time: 48426 ms Total hot run time: 46342 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3995036780 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3994878414 run beut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3994649324 run beut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3994642734 @zclllyybb I've revised code to make PR-bot happy. Looks like everything is on track. Please re-approve the remaining check and then merge the PR. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3993011636 run coverage -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3993008121 make coverage -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3992360494 TPC-DS: Total hot run time: 183786 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 997d9fdc279286c8ba223cc5d36563cc8e0144e9, data reload: false query5 4996633 524 524 query6 333 235 225 225 query7 4208467 273 273 query8 335 248 236 236 query9 8762274927552749 query10 553 389 352 352 query11 16960 17325 17105 17105 query12 205 138 123 123 query13 1322497 364 364 query14 7572333730503050 query14_12904293529922935 query15 248 209 190 190 query16 1029472 488 472 query17 1202792 705 705 query18 2747452 355 355 query19 217 210 184 184 query20 143 131 130 130 query21 216 140 117 117 query22 4845495947984798 query23 17285 16827 16646 16646 query23_116748 16726 16711 16711 query24 7097162212141214 query24_11212125912511251 query25 580 478 468 468 query26 1244253 154 154 query27 2760484 284 284 query28 4438186618941866 query29 790 557 465 465 query30 305 250 207 207 query31 863 729 654 654 query32 81 72 72 72 query33 532 338 286 286 query34 921 905 564 564 query35 621 682 641 641 query36 1045113510171017 query37 126 97 85 85 query38 2997290228812881 query39 883 884 846 846 query39_1836 839 839 839 query40 228 156 134 134 query41 64 61 58 58 query42 107 104 110 104 query43 388 381 354 354 query44 query45 199 192 182 182 query46 879 983 624 624 query47 2130211720742074 query48 310 331 234 234 query49 640 468 370 370 query50 690 273 211 211 query51 4058409840734073 query52 106 106 96 96 query53 290 336 299 299 query54 301 264 257 257 query55 88 88 79 79 query56 329 320 307 307 query57 1371134012811281 query58 288 278 273 273 query59 2628271824552455 query60 330 335 324 324 query61 154 150 149 149 query62 602 579 534 534 query63 313 272 272 272 query64 4856126310141014 query65 query66 1394451 348 348 query67 16392 16310 16162 16162 query68 query69 393 312 298 298 query70 10321016969 969 query71 343 297 302 297 query72 2855262224262426 query73 532 541 323 323 query74 10029 10004 97569756 query75 2833274624592459 query76 23211033686 686 query77 369 401 332 332 query78 11219 11352 10655 10655 query79 2800799 599 599 query80 1810620 561 561 query81 570 289 250 250 query82 986 152 116 116 query83 344 260 244 244 query84 247 124 93 93 query85 905 477 458 458 query86 437 301 288 288 query87 3104308430053005 query88 3550264226552642 query89 432 366 340 340 query90 1966173 173 173 query91 180 155 134 134 query92 77 74 71 71 query93 1586832 508 508 query94 657 316 305 305 query95 582 402 312 312 query96 636 509 229 229 query97 2504247424082408 query98 230 246 220 220 query99 10231017917 917 Total cold run time: 257173 ms Total hot run time: 183786 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected].
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3992288301 TPC-H: Total hot run time: 28806 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 997d9fdc279286c8ba223cc5d36563cc8e0144e9, data reload: false -- Round 1 -- q1 17634 448242894289 q2 q3 10646 800 518 518 q4 4683364 265 265 q5 7568121410161016 q6 172 174 147 147 q7 764 848 692 692 q8 9289145413411341 q9 4813473346884688 q10 6754186816421642 q11 456 267 243 243 q12 699 573 475 475 q13 17753 421834203420 q14 225 225 215 215 q15 916 809 784 784 q16 741 714 670 670 q17 723 850 422 422 q18 6049549952365236 q19 1108981 604 604 q20 522 511 385 385 q21 4845200915091509 q22 407 286 245 245 Total cold run time: 96767 ms Total hot run time: 28806 ms - Round 2, with runtime_filter_mode=off - q1 4642453744954495 q2 q3 1787224417421742 q4 880 1194804 804 q5 4050440943144314 q6 197 179 142 142 q7 1787165315431543 q8 2447286725672567 q9 7417738873647364 q10 2637279224162416 q11 501 436 429 429 q12 507 590 439 439 q13 3961438936413641 q14 298 309 360 309 q15 945 848 814 814 q16 739 759 738 738 q17 1192152212801280 q18 7173685864816481 q19 930 963 1026963 q20 2093222822082208 q21 3980359333363336 q22 477 422 371 371 Total cold run time: 48640 ms Total hot run time: 46396 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3992162682 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3992164000 # BE UT Coverage Report Increment line coverage `87.39% (97/111)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/1313c91421227cd1ac688f4839152ac9458b5cab_1313c91421227cd1ac688f4839152ac9458b5cab/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/1313c91421227cd1ac688f4839152ac9458b5cab_1313c91421227cd1ac688f4839152ac9458b5cab/report/index.html) | Category | Coverage | |---|| | Function Coverage | 52.58% (19647/37364) | | Line Coverage | 36.21% (183467/506738) | | Region Coverage | 32.47% (142191/437891) | | Branch Coverage | 33.44% (61699/184505) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3992060151 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3991959065 # FE Regression Coverage Report Increment line coverage ` 90.91% (40/44)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_1313c91421227cd1ac688f4839152ac9458b5cab_merge_fe/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_1313c91421227cd1ac688f4839152ac9458b5cab_merge_fe/report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
hello-stephen commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3991411945 # FE UT Coverage Report Increment line coverage ` 81.82% (36/44)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/60892_1313c91421227cd1ac688f4839152ac9458b5cab/fe_increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/60892_1313c91421227cd1ac688f4839152ac9458b5cab/fe_report/index.html) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3991144493 TPC-DS: Total hot run time: 18 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 1313c91421227cd1ac688f4839152ac9458b5cab, data reload: false query5 4809652 533 533 query6 340 230 216 216 query7 4212467 276 276 query8 350 285 238 238 query9 8767275127672751 query10 544 388 344 344 query11 16981 17688 17271 17271 query12 224 144 136 136 query13 1279509 374 374 query14 8114334330423042 query14_12881283930462839 query15 248 211 198 198 query16 1038502 486 486 query17 1163766 664 664 query18 2849442 352 352 query19 229 222 184 184 query20 139 133 133 133 query21 224 134 114 114 query22 5039500746974697 query23 17185 16750 16616 16616 query23_116732 16755 16584 16584 query24 6994161512251225 query24_11246125812051205 query25 547 523 408 408 query26 1233260 148 148 query27 2780467 277 277 query28 4487190018821882 query29 802 575 457 457 query30 310 239 212 212 query31 870 731 635 635 query32 84 73 68 68 query33 547 328 283 283 query34 916 914 565 565 query35 624 693 592 592 query36 10841137975 975 query37 131 100 86 86 query38 2937294928812881 query39 892 869 854 854 query39_1825 837 811 811 query40 229 151 134 134 query41 63 59 58 58 query42 107 106 105 105 query43 389 381 341 341 query44 query45 195 188 188 188 query46 894 971 607 607 query47 2112213920652065 query48 314 321 224 224 query49 617 453 376 376 query50 673 315 218 218 query51 4099408140504050 query52 106 104 94 94 query53 289 335 278 278 query54 297 260 263 260 query55 86 91 82 82 query56 316 306 308 306 query57 1358134312841284 query58 288 275 276 275 query59 2579266625002500 query60 332 329 330 329 query61 148 141 146 141 query62 634 588 532 532 query63 314 279 282 279 query64 48351279998 998 query65 query66 1380449 362 362 query67 16386 16316 16178 16178 query68 query69 391 296 281 281 query70 948 985 967 967 query71 333 302 291 291 query72 2773263624842484 query73 526 539 321 321 query74 9960997597769776 query75 2819276024832483 query76 22901041680 680 query77 354 369 306 306 query78 11223 11488 10645 10645 query79 1565793 615 615 query80 1401609 534 534 query81 571 273 265 265 query82 1012159 113 113 query83 364 257 240 240 query84 254 124 107 107 query85 876 461 427 427 query86 416 293 306 293 query87 3131307529922992 query88 3563269326602660 query89 430 377 346 346 query90 2030174 163 163 query91 165 150 133 133 query92 78 76 68 68 query93 1037837 513 513 query94 647 327 295 295 query95 594 396 312 312 query96 661 532 234 234 query97 2466245723852385 query98 232 224 219 219 query99 1000992 916 916 Total cold run time: 255125 ms Total hot run time: 18 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
doris-robot commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3991081551 TPC-H: Total hot run time: 29046 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 1313c91421227cd1ac688f4839152ac9458b5cab, data reload: false -- Round 1 -- q1 17616 454043734373 q2 q3 10635 794 527 527 q4 4680358 256 256 q5 7563122610101010 q6 174 175 146 146 q7 807 851 677 677 q8 9305145213241324 q9 4690473447384734 q10 6788186816541654 q11 475 252 253 252 q12 722 567 474 474 q13 17755 420334033403 q14 228 234 212 212 q15 907 791 789 789 q16 729 726 680 680 q17 731 872 423 423 q18 6002542952835283 q19 1273976 621 621 q20 512 492 396 396 q21 4885195315291529 q22 382 302 283 283 Total cold run time: 96859 ms Total hot run time: 29046 ms - Round 2, with runtime_filter_mode=off - q1 4704457645644564 q2 q3 1798219818041804 q4 875 1232782 782 q5 4074438642744274 q6 186 173 143 143 q7 1772165915231523 q8 2470285325122512 q9 7748734773817347 q10 2675289824092409 q11 567 447 420 420 q12 492 597 460 460 q13 3922438535683568 q14 283 305 282 282 q15 872 802 796 796 q16 721 767 713 713 q17 1180153612711271 q18 7154685166446644 q19 918 896 930 896 q20 2080222420092009 q21 4036355933053305 q22 456 435 379 379 Total cold run time: 48983 ms Total hot run time: 46101 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
github-actions[bot] commented on code in PR #60892:
URL: https://github.com/apache/doris/pull/60892#discussion_r2878195116
##
regression-test/suites/nereids_p0/sql_functions/string_functions/test_split_by_string_limit.groovy:
##
@@ -0,0 +1,93 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_split_by_string_limit") {
+sql "SET enable_nereids_planner=true"
+sql "SET enable_fallback_to_original_planner=false"
+
+// === Constant expression tests with limit ===
+
+// Basic limit functionality
+qt_limit1 "select split_by_string('one,two,three,', ',', 2);"
+qt_limit2 "select split_by_string('one,two,three,', ',', 3);"
+qt_limit3 "select split_by_string('one,two,three,', ',', 4);"
+qt_limit4 "select split_by_string('one,two,three,', ',', 10);"
+qt_limit5 "select split_by_string('one,two,three', ',', 1);"
+
+// limit = -1 (no limit, same as 2-arg)
+qt_limit6 "select split_by_string('one,two,three,', ',', -1);"
+
+// limit = 0 (no limit, same as 2-arg)
+qt_limit7 "select split_by_string('a,b,c', ',', 0);"
+
+// Empty source string + limit
+qt_limit8 "select split_by_string('', ',', 2);"
+
+// Empty delimiter + limit (split by character)
+qt_limit9 "select split_by_string('abcde', '', 3);"
+qt_limit10 "select split_by_string('abcde', '', 1);"
+qt_limit11 "select split_by_string('abcde', '', 10);"
+
+// Multi-char delimiter + limit
+qt_limit12 "select split_by_string('a::b::c::d', '::', 2);"
+qt_limit13 "select split_by_string('a::b::c::d', '::', 3);"
+qt_limit14 "select split_by_string('1,,2,3,,4,5,,abcde', ',,', 2);"
+
+// NULL handling
+qt_limit15 "select split_by_string(NULL, ',', 2);"
+
+// UTF-8 + limit
+qt_limit16 "select split_by_string('你a好b世c界', '', 3);"
+
+// Edge cases: consecutive delimiters + limit
+qt_limit17 "select split_by_string(',,,', ',', 2);"
+qt_limit18 "select split_by_string(',,a,b,c,', ',', 3);"
+
+// === Table data tests ===
+def tableName = "test_split_limit"
+
Review Comment:
**[Low]** Per coding standard: "For ordinary single test tables, do not use
`def tableName` form; instead hardcode your table name in all SQL." Please
replace `${tableName}` with the literal `test_split_limit` in all SQL
statements.
##
regression-test/suites/nereids_p0/sql_functions/string_functions/test_split_by_string_limit.groovy:
##
@@ -0,0 +1,93 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_split_by_string_limit") {
+sql "SET enable_nereids_planner=true"
+sql "SET enable_fallback_to_original_planner=false"
Review Comment:
**[Low]** Per coding standard: "Except for variables you explicitly need to
adjust for testing current functionality, other variables do not need extra
setup before testing. For example, nereids optimizer and pipeline engine
settings can use default states." These two SET statements are unnecessary
since nereids is the default planner. Please remove lines 19-20.
##
be/src/vec/functions/function_string.h:
##
@@ -2163,44 +2158,142 @@ class FunctionSplitByString : public IFunction {
}
}
-void split_empty_delimiter(const StringRef& str_ref, ColumnString::Chars&
column_string_chars,
- ColumnString::Offsets& column_string_offsets,
- ColumnArray::Offset64& string_pos,
-
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
zclllyybb commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3990869841 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
zclllyybb commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3990869186 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
zclllyybb closed pull request #60892: [feat](function) Add limit parameter support for SPLIT_BY_STRING URL: https://github.com/apache/doris/pull/60892 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
zclllyybb commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3977165644 > @zclllyybb please review ok, I'll review it in few days -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [feat](function) Add limit parameter support for SPLIT_BY_STRING [doris]
floation-cutie commented on PR #60892: URL: https://github.com/apache/doris/pull/60892#issuecomment-3977069653 @morningman please review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
