Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
jacktengg merged PR #67463: URL: https://github.com/apache/doris/pull/67463 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5559767380 # BE Regression && UT Coverage Report Increment line coverage `93.55% (29/31)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/67463_4952c782f6089078bdcedee9ebeb966c684e6366_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/67463_4952c782f6089078bdcedee9ebeb966c684e6366_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 65.73% (29856/45425) | | Line Coverage | 50.59% (322472/637410) | | Region Coverage | 46.50% (265713/571453) | | Branch Coverage | 47.63% (122335/256838) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5541853121 # BE Regression && UT Coverage Report Increment line coverage `93.55% (29/31)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/67463_4952c782f6089078bdcedee9ebeb966c684e6366_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/67463_4952c782f6089078bdcedee9ebeb966c684e6366_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 65.73% (29856/45425) | | Line Coverage | 50.59% (322472/637410) | | Region Coverage | 46.50% (265713/571453) | | Branch Coverage | 47.63% (122335/256838) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5539378822 ClickBench: Total hot run time: 14.53 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 4952c782f6089078bdcedee9ebeb966c684e6366, data reload: false query1 0.000.000.01 query2 0.080.040.04 query3 0.250.110.11 query4 1.600.100.11 query5 0.170.160.17 query6 1.250.680.66 query7 0.030.010.01 query8 0.050.030.03 query9 0.290.210.22 query10 0.350.340.34 query11 0.160.120.12 query12 0.150.120.12 query13 0.310.310.30 query14 0.450.460.44 query15 0.360.360.35 query16 0.240.230.23 query17 0.690.700.68 query18 0.190.160.17 query19 1.211.121.15 query20 0.010.010.01 query21 15.44 0.160.11 query22 5.060.040.04 query23 16.17 0.260.10 query24 2.990.340.25 query25 0.080.050.03 query26 0.810.160.11 query27 0.040.020.03 query28 3.660.560.27 query29 12.47 3.172.55 query30 0.250.100.12 query31 2.760.370.17 query32 3.510.330.23 query33 1.591.391.45 query34 15.37 2.181.76 query35 1.781.731.71 query36 0.470.300.28 query37 0.060.030.04 query38 0.050.030.03 query39 0.030.020.03 query40 0.110.080.07 query41 0.080.020.02 query42 0.020.020.02 query43 0.030.030.03 Total cold run time: 90.67 s Total hot run time: 14.53 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5539333647 TPC-DS: Total hot run time: 82053 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 4952c782f6089078bdcedee9ebeb966c684e6366, data reload: false query5 4257408 344 344 query6 376 144 146 144 query7 4926418 230 230 query8 284 126 117 117 query9 8674288328812881 query10 405 224 181 181 query11 53681036910 910 query12 118 72 72 72 query13 1194463 328 328 query14 6044220320872087 query14_11978197719501950 query15 174 119 112 112 query16 916 385 342 342 query17 792 467 368 368 query18 2337325 259 259 query19 175 142 113 113 query20 71 72 70 70 query21 200 103 88 88 query22 5477543653175317 query23 6602627861956195 query23_16096610662466106 query24 73081096776 776 query24_1778 789 777 777 query25 430 305 264 264 query26 1229220 132 132 query27 2791431 253 253 query28 4677151014991499 query29 964 432 339 339 query30 243 152 128 128 query31 800 396 319 319 query32 124 73 75 73 query33 450 207 160 160 query34 987 829 476 476 query35 393 394 339 339 query36 578 585 522 522 query37 120 80 71 71 query38 999 834 786 786 query39 486 457 481 457 query39_1469 465 473 465 query40 198 89 76 76 query41 54 52 53 52 query42 72 70 71 70 query43 241 242 206 206 query44 1037549 541 541 query45 108 101 101 101 query46 783 851 525 525 query47 767 749 718 718 query48 303 319 220 220 query49 545 238 187 187 query50 756 261 191 191 query51 7997798279437943 query52 66 67 66 66 query53 191 201 146 146 query54 215 169 162 162 query55 86 61 55 55 query56 219 169 166 166 query57 714 742 618 618 query58 201 176 151 151 query59 1228121410711071 query60 251 190 171 171 query61 115 121 125 121 query62 348 232 178 178 query63 167 138 142 138 query64 2691678 562 562 query65 1676157715531553 query66 1916278 210 210 query67 10193 973397179717 query68 2989776 776 query69 351 220 191 191 query70 677 628 608 608 query71 247 176 170 170 query72 2329168315421542 query73 628 570 346 346 query74 1986121211311131 query75 11821104962 962 query76 2385734 541 541 query77 261 262 210 210 query78 3900363632543254 query79 2861788 595 595 query80 1574329 279 279 query81 485 151 135 135 query82 617 128 100 100 query83 295 212 190 190 query84 288 110 91 91 query85 782 354 296 296 query86 383 173 174 173 query87 1007958 891 891 query88 2768209021052090 query89 285 193 176 176 query90 2030124 129 124 query91 137 122 101 101 query92 73 69 67 67 query93 14721055660 660 query94 651 257 220 220 query95 528 256 299 256 query96 769 599 271 271 query97 1082106410121012 query98 167 133 133 133 query99 421 339 307 307 Total cold run time: 178448 ms Total hot run time: 82053 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5539266528 TPC-H: Total hot run time: 16893 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 4952c782f6089078bdcedee9ebeb966c684e6366, data reload: false -- Round 1 -- q1 17569 297230012972 q2 2148270 220 220 q3 10196 922 522 522 q4 4672250 199 199 q5 7679563 375 375 q6 139 117 92 92 q7 530 493 390 390 q8 9245938 930 930 q9 3430239423572357 q10 6507846 752 752 q11 395 204 183 183 q12 614 253 201 201 q13 18118 150411491149 q14 156 150 137 137 q15 q16 432 397 367 367 q17 1388890 875 875 q18 3116229322642264 q19 1277908 771 771 q20 372 283 210 210 q21 5591170018631700 q22 338 267 227 227 Total cold run time: 93912 ms Total hot run time: 16893 ms - Round 2, with runtime_filter_mode=off - q1 3361328332823282 q2 498 413 366 366 q3 2198232721902190 q4 11821156886 886 q5 2171216021182118 q6 170 118 86 86 q7 1028927 857 857 q8 1589140313931393 q9 3113309830933093 q10 1860178916331633 q11 352 271 250 250 q12 450 436 334 334 q13 1488153911521152 q14 178 183 164 164 q15 q16 394 393 352 352 q17 3547330732623262 q18 4883441047584410 q19 841 867 870 867 q20 986 993 823 823 q21 3874314532503145 q22 390 341 328 328 Total cold run time: 34553 ms Total hot run time: 30991 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
jacktengg commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5538039632 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
jacktengg commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5538038559 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
HappenLee commented on PR #67463:
URL: https://github.com/apache/doris/pull/67463#issuecomment-5536575582
[P2] Consider preserving the safe int64 fast path
The previous implementation handled the common finite range [-2^63, 2^63)
with a direct, well-defined conversion to int64_t and returned immediately. The
new implementation sends every value through bit_cast, exponent/significand
decoding, wide-integer construction, and a multi-limb shift.
The binary64 decoder is necessary for values outside the built-in integer
range, but it may add avoidable work for ordinary small values, where a
hardware floating-point-to-int64 conversion is sufficient. Could we retain a
fast path similar to:
```cpp
constexpr double int64_min = -0x1p63;
constexpr double int64_limit = 0x1p63;
if (rhs >= int64_min && rhs < int64_limit) {
self = static_cast(rhs);
return;
}
```
and use the IEEE-754 decoder only for the large-value fallback? This keeps
the fix focused on the range that previously triggered undefined behavior while
preserving the inexpensive common path. If removing the fast path is
intentional, a focused microbenchmark covering typical small
Float/Double-to-wide-integer conversions would help demonstrate that the change
is neutral.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5527447013 TPC-H: Total hot run time: 16786 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 3b821d1be277664cdc9bb4fc7cfe25d1ce70e1e7, data reload: false -- Round 1 -- q1 17601 313730933093 q2 2113272 233 233 q3 10199 891 510 510 q4 4669251 200 200 q5 7681551 364 364 q6 135 118 91 91 q7 521 505 383 383 q8 9247922 936 922 q9 3414242223972397 q10 6514869 690 690 q11 391 193 181 181 q12 611 260 196 196 q13 18126 151811441144 q14 160 147 135 135 q15 q16 436 395 368 368 q17 1365963 804 804 q18 3133227322422242 q19 1289919 745 745 q20 381 290 198 198 q21 5626166517961665 q22 328 264 225 225 Total cold run time: 93940 ms Total hot run time: 16786 ms - Round 2, with runtime_filter_mode=off - q1 3452338333763376 q2 521 423 380 380 q3 2232229821432143 q4 11941153889 889 q5 2153210921232109 q6 170 118 87 87 q7 1025955 833 833 q8 1620145014391439 q9 3162314431343134 q10 1853182116241624 q11 370 274 260 260 q12 456 425 337 337 q13 1482154111601160 q14 176 169 169 169 q15 q16 391 399 354 354 q17 3563326532273227 q18 4846443647364436 q19 869 978 863 863 q20 10151001823 823 q21 3867325832073207 q22 407 357 319 319 Total cold run time: 34824 ms Total hot run time: 31169 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5527531033 TPC-DS: Total hot run time: 82719 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 3b821d1be277664cdc9bb4fc7cfe25d1ce70e1e7, data reload: false query5 4239408 341 341 query6 387 150 127 127 query7 4919436 236 236 query8 290 126 125 125 query9 8675292729182918 query10 394 226 183 183 query11 53801054918 918 query12 121 72 69 69 query13 1195435 324 324 query14 6149227821702170 query14_12062206220592059 query15 176 127 110 110 query16 918 369 362 362 query17 811 468 376 376 query18 2342344 251 251 query19 169 144 116 116 query20 71 69 72 69 query21 211 109 89 89 query22 5413550053325332 query23 6884626160386038 query23_16130625460056005 query24 72421112811 811 query24_1794 809 797 797 query25 460 306 268 268 query26 1221243 138 138 query27 2773385 265 265 query28 4690153615261526 query29 970 463 365 365 query30 257 167 132 132 query31 830 406 344 344 query32 148 77 81 77 query33 474 217 187 187 query34 1002816 494 494 query35 436 409 347 347 query36 572 572 528 528 query37 122 85 73 73 query38 1022858 815 815 query39 540 491 471 471 query39_1469 455 454 454 query40 223 95 81 81 query41 59 56 57 56 query42 77 74 74 74 query43 251 253 218 218 query44 1052554 548 548 query45 111 102 96 96 query46 765 864 532 532 query47 776 791 711 711 query48 324 309 233 233 query49 539 245 191 191 query50 772 265 192 192 query51 8356825982738259 query52 76 74 85 74 query53 228 201 148 148 query54 222 172 157 157 query55 76 59 54 54 query56 178 180 153 153 query57 657 676 646 646 query58 211 163 165 163 query59 1259127311091109 query60 239 192 180 180 query61 129 116 118 116 query62 373 209 175 175 query63 172 144 146 144 query64 2758723 573 573 query65 1662160116491601 query66 1783268 206 206 query67 9870959296529592 query68 29791211731 731 query69 342 233 202 202 query70 698 636 614 614 query71 255 181 164 164 query72 2394178215551555 query73 644 582 346 346 query74 2013125711721172 query75 11911107961 961 query76 2360779 543 543 query77 266 273 207 207 query78 4020375631723172 query79 2709798 598 598 query80 1588324 279 279 query81 492 157 133 133 query82 896 123 100 100 query83 279 212 195 195 query84 293 113 93 93 query85 808 446 313 313 query86 453 178 169 169 query87 1020962 887 887 query88 2799210421192104 query89 292 203 177 177 query90 2001135 139 135 query91 135 120 101 101 query92 79 71 67 67 query93 15551066723 723 query94 654 244 228 228 query95 526 247 299 247 query96 863 588 256 256 query97 1059105910341034 query98 165 134 139 134 query99 445 352 312 312 Total cold run time: 179894 ms Total hot run time: 82719 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5527587128 ClickBench: Total hot run time: 14.66 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 3b821d1be277664cdc9bb4fc7cfe25d1ce70e1e7, data reload: false query1 0.010.010.01 query2 0.070.040.03 query3 0.260.110.10 query4 1.600.090.10 query5 0.180.160.16 query6 1.260.700.70 query7 0.040.010.00 query8 0.040.030.03 query9 0.290.220.21 query10 0.350.340.34 query11 0.160.120.11 query12 0.150.120.12 query13 0.320.320.31 query14 0.450.440.44 query15 0.350.340.34 query16 0.230.220.22 query17 0.750.730.65 query18 0.180.160.17 query19 1.161.141.18 query20 0.020.010.01 query21 15.47 0.160.13 query22 5.070.040.04 query23 16.18 0.250.10 query24 3.000.310.27 query25 0.120.040.03 query26 0.820.170.12 query27 0.040.030.03 query28 3.660.570.29 query29 12.46 3.212.58 query30 0.250.120.12 query31 2.750.360.17 query32 3.560.330.22 query33 1.391.401.40 query34 15.33 2.151.74 query35 1.741.721.72 query36 0.450.290.29 query37 0.060.040.04 query38 0.050.030.03 query39 0.030.030.02 query40 0.120.070.08 query41 0.080.030.02 query42 0.030.030.03 query43 0.040.030.03 Total cold run time: 90.57 s Total hot run time: 14.66 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5526511127 # BE Regression && UT Coverage Report Increment line coverage `100.00% (26/26)` :tada: [Increment coverage report](http://coverage.selectdb-in.cc/coverage/67463_3b821d1be277664cdc9bb4fc7cfe25d1ce70e1e7_merge/increment_report/index.html) [Complete coverage report](http://coverage.selectdb-in.cc/coverage/67463_3b821d1be277664cdc9bb4fc7cfe25d1ce70e1e7_merge/report/index.html) | Category | Coverage | |---|| | Function Coverage | 76.16% (34590/45418) | | Line Coverage | 61.24% (390323/637325) | | Region Coverage | 57.35% (327699/571407) | | Branch Coverage | 58.24% (149565/256814) | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
jacktengg commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5524156258 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
jacktengg commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5523499464 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
jacktengg commented on code in PR #67463:
URL: https://github.com/apache/doris/pull/67463#discussion_r3921648373
##
be/src/core/wide_integer_impl.h:
##
@@ -313,91 +286,66 @@ struct integer::_impl {
}
}
-/**
- * N.B. t is constructed from double, so max(t) = max(double) ~ 2^310
- * the recursive call happens when t / 2^64 > 2^64, so there won't be more
than 5 of them.
- *
- * t = a1 * max_int + b1, a1 > max_int, b1 < max_int
- * a1 = a2 * max_int + b2, a2 > max_int, b2 < max_int
- * a_(n - 1) = a_n * max_int + b2, a_n <= max_int <- base case.
- */
-template
-CONSTEXPR_FROM_DOUBLE static void set_multiplier(integer&
self, T t) noexcept
-#if !DORIS_WIDE_FROM_DOUBLE_INLINE
-;
-#else
-{
-constexpr uint64_t max_int = std::numeric_limits::max();
-static_assert(std::is_same_v || std::is_same_v);
-/// Implementation specific behaviour on overflow (if we don't check
here, stack overflow will triggered in bigint_cast).
-if constexpr (std::is_same_v) {
-if (!std::isfinite(t)) {
-self = 0;
-return;
-}
-} else {
-if (!boost::math::isfinite(t)) {
-self = 0;
-return;
-}
+constexpr static void wide_integer_from_builtin(integer&
self,
+double rhs) noexcept {
+static_assert(sizeof(double) == sizeof(uint64_t));
+static_assert(std::numeric_limits::is_iec559);
+static_assert(std::numeric_limits::digits == 53);
+
+constexpr unsigned fraction_bits = std::numeric_limits::digits
- 1;
+constexpr uint64_t fraction_mask = (uint64_t(1) << fraction_bits) - 1;
+constexpr uint64_t exponent_mask = uint64_t(0x7ff) << fraction_bits;
+constexpr uint64_t sign_mask = uint64_t(1) << 63;
+constexpr int exponent_bias = 1023;
+
+const uint64_t bits = std::bit_cast(rhs);
+const uint64_t biased_exponent = (bits & exponent_mask) >>
fraction_bits;
+
+// Zero and subnormal values have an absolute value less than one.
Preserve the previous
+// behavior of mapping infinity and NaN to zero as well.
+if (biased_exponent == 0 || biased_exponent == 0x7ff) {
+self = 0;
+return;
}
-const T alpha = t / static_cast(max_int);
-
-/** Here we have to use strict comparison.
- * The max_int is 2^64 - 1.
- * When casted to floating point type, it will be rounded to the
closest representable number,
- * which is 2^64.
- * But 2^64 is not representable in uint64_t,
- * so the maximum representable number will be strictly less.
- */
-if (alpha < static_cast(max_int)) {
-self = static_cast(alpha);
-} else { // max(double) / 2^64 will surely contain less than 52
precision bits, so speed up computations.
-set_multiplier(self, static_cast(alpha));
+const int exponent = static_cast(biased_exponent) - exponent_bias;
+if (exponent < 0) {
+self = 0;
+return;
}
-self *= max_int;
-self += static_cast(t - floor(static_cast(alpha)) *
- static_cast(max_int)); //
+= b_i
-}
-#endif
+const uint64_t significand = (bits & fraction_mask) | (uint64_t(1) <<
fraction_bits);
+self = significand;
-CONSTEXPR_FROM_DOUBLE static void wide_integer_from_builtin(integer& self,
-double rhs)
noexcept
-#if !DORIS_WIDE_FROM_DOUBLE_INLINE
-;
-#else
-{
-constexpr int64_t max_int = std::numeric_limits::max();
-constexpr int64_t min_int = std::numeric_limits::lowest();
-
-/// There are values in int64 that have more than 53 significant bits
(in terms of double
-/// representation). Such values, being promoted to double, are
rounded up or down. If they are rounded up,
-/// the result may not fit in 64 bits.
-/// The example of such a number is 9.22337e+18.
-/// As to_Integral does a static_cast to int64_t, it may result in UB.
-/// The necessary check here is that FromDoubleIntermediateType has
enough significant (mantissa) bits to store the
-/// int64_t max value precisely.
-
-if (rhs > static_cast(min_int) &&
-rhs < static_cast(max_int)) {
-self = static_cast(rhs);
-return;
+const int shift = exponent - static_cast(fraction_bits);
+if (shift < 0) {
+self >>= -shift;
+} else {
+self <<= shift;
}
-const FromDoubleIntermediateType rhs_long_double =
-(static_cast(rhs) < 0)
-? -static_cast(rhs)
-
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
jacktengg commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5521429675 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
jacktengg commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5521428789 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
github-actions[bot] commented on code in PR #67463:
URL: https://github.com/apache/doris/pull/67463#discussion_r3920823230
##
be/test/core/wide_integer_test.cpp:
##
@@ -44,6 +47,40 @@ TEST(WideInteger, Conversions) {
1e72);
}
+TEST(WideInteger, FloatingPointConversions) {
+constexpr wide::Int256 constexpr_value(123.75);
+static_assert(constexpr_value.items[0] == 123);
Review Comment:
[P2] Avoid assuming the low limb is `items[0]`
On big-endian builds, `_impl::little(0)` is `item_count - 1`, and scalar
construction writes `123` there, leaving `items[0]` as the zero high limb. This
new `static_assert` therefore fails compilation on a platform the existing test
explicitly supports with its `__BYTE_ORDER__` branches. Please assert the value
representation-independently (for example, compare against `wide::Int256(123)`)
or index through `_impl::little(0)`.
##
be/src/core/wide_integer_impl.h:
##
@@ -313,91 +286,66 @@ struct integer::_impl {
}
}
-/**
- * N.B. t is constructed from double, so max(t) = max(double) ~ 2^310
- * the recursive call happens when t / 2^64 > 2^64, so there won't be more
than 5 of them.
- *
- * t = a1 * max_int + b1, a1 > max_int, b1 < max_int
- * a1 = a2 * max_int + b2, a2 > max_int, b2 < max_int
- * a_(n - 1) = a_n * max_int + b2, a_n <= max_int <- base case.
- */
-template
-CONSTEXPR_FROM_DOUBLE static void set_multiplier(integer&
self, T t) noexcept
-#if !DORIS_WIDE_FROM_DOUBLE_INLINE
-;
-#else
-{
-constexpr uint64_t max_int = std::numeric_limits::max();
-static_assert(std::is_same_v || std::is_same_v);
-/// Implementation specific behaviour on overflow (if we don't check
here, stack overflow will triggered in bigint_cast).
-if constexpr (std::is_same_v) {
-if (!std::isfinite(t)) {
-self = 0;
-return;
-}
-} else {
-if (!boost::math::isfinite(t)) {
-self = 0;
-return;
-}
+constexpr static void wide_integer_from_builtin(integer&
self,
+double rhs) noexcept {
+static_assert(sizeof(double) == sizeof(uint64_t));
+static_assert(std::numeric_limits::is_iec559);
+static_assert(std::numeric_limits::digits == 53);
+
+constexpr unsigned fraction_bits = std::numeric_limits::digits
- 1;
+constexpr uint64_t fraction_mask = (uint64_t(1) << fraction_bits) - 1;
+constexpr uint64_t exponent_mask = uint64_t(0x7ff) << fraction_bits;
+constexpr uint64_t sign_mask = uint64_t(1) << 63;
+constexpr int exponent_bias = 1023;
+
+const uint64_t bits = std::bit_cast(rhs);
+const uint64_t biased_exponent = (bits & exponent_mask) >>
fraction_bits;
+
+// Zero and subnormal values have an absolute value less than one.
Preserve the previous
+// behavior of mapping infinity and NaN to zero as well.
+if (biased_exponent == 0 || biased_exponent == 0x7ff) {
+self = 0;
+return;
}
-const T alpha = t / static_cast(max_int);
-
-/** Here we have to use strict comparison.
- * The max_int is 2^64 - 1.
- * When casted to floating point type, it will be rounded to the
closest representable number,
- * which is 2^64.
- * But 2^64 is not representable in uint64_t,
- * so the maximum representable number will be strictly less.
- */
-if (alpha < static_cast(max_int)) {
-self = static_cast(alpha);
-} else { // max(double) / 2^64 will surely contain less than 52
precision bits, so speed up computations.
-set_multiplier(self, static_cast(alpha));
+const int exponent = static_cast(biased_exponent) - exponent_bias;
+if (exponent < 0) {
+self = 0;
+return;
}
-self *= max_int;
-self += static_cast(t - floor(static_cast(alpha)) *
- static_cast(max_int)); //
+= b_i
-}
-#endif
+const uint64_t significand = (bits & fraction_mask) | (uint64_t(1) <<
fraction_bits);
+self = significand;
-CONSTEXPR_FROM_DOUBLE static void wide_integer_from_builtin(integer& self,
-double rhs)
noexcept
-#if !DORIS_WIDE_FROM_DOUBLE_INLINE
-;
-#else
-{
-constexpr int64_t max_int = std::numeric_limits::max();
-constexpr int64_t min_int = std::numeric_limits::lowest();
-
-/// There are values in int64 that have more than 53 significant bits
(in terms of double
-/// representation). Such values, being promoted to double, are
rounded up or down. If they are rounded up,
-/// the result may not fit in
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
jacktengg commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5519797157 /review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] [fix](be) Eliminate undefined float-to-wide-integer conversion [doris]
hello-stephen commented on PR #67463: URL: https://github.com/apache/doris/pull/67463#issuecomment-5519797928 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR). Please clearly describe your PR: 1. What problem was fixed (it's best to include specific error reporting information). How it was fixed. 2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be. 3. What features were added. Why was this function added? 4. Which code was refactored and why was this part of the code refactored? 5. Which functions were optimized and what is the difference before and after the optimization? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
