liutang123 commented on a change in pull request #2630: Support bitmap index
for more type
URL: https://github.com/apache/incubator-doris/pull/2630#discussion_r365669929
##########
File path: be/src/util/frame_of_reference_coding.cpp
##########
@@ -109,13 +131,20 @@ void ForEncoder<T>::bit_packing_one_frame_value(const T*
input) {
T max = input[0];
bool is_ascending = true;
uint8_t bit_width = 0;
+ T half_max_delta = numeric_limits_max() >> 1;
+ bool save_original_value = false;
+ // 1. make sure order_flag, save_original_value, and find max&min.
for (uint8_t i = 1; i < _buffered_values_num; ++i) {
if (is_ascending) {
if (input[i] < input[i - 1]) {
is_ascending = false;
} else {
- bit_width = std::max(bit_width, bits(input[i]- input[i - 1])) ;
+ if ((input[i] >> 1) - (input[i - 1] >> 1) > half_max_delta) {
// overflow
+ save_original_value = true;
Review comment:
answer for q1:
In my opinion, `save_original_value` and `is_ascending` are two independent
variables even though the difference in is_ascending only affects the encoding
format now. If break the loop when the half of delta value > half_max_delta, we
can not known the exact order.
answer for q2:
We can use subtraction result of `(max >> 1) - (min >> 1)` to determine
whether to save original value only when the order is **not** ascending. When
the order is ascending, condition `(max >> 1) - (min >> 1) > half_max_delta` is
insufficient to save original value.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
