zeroshade commented on code in PR #43957:
URL: https://github.com/apache/arrow/pull/43957#discussion_r1766941722
##########
cpp/src/arrow/util/decimal.cc:
##########
@@ -156,7 +162,9 @@ struct DecimalRealConversion : public
BaseDecimalRealConversion {
// NOTE: if `precision` is the full precision then the algorithm will
// lose the last digit. If `precision` is almost the full precision,
// there can be an off-by-one error due to rounding.
- const int mul_step = std::max(1, kMaxPrecision - precision);
+ constexpr int is_dec32_or_dec64 =
+ DecimalType::kByteWidth <= BasicDecimal64::kByteWidth;
+ const int mul_step = std::max(1, kMaxPrecision - precision -
is_dec32_or_dec64);
Review Comment:
With decimal32 and decimal64 I needed to reduce the `mul_step` by 1 in order
to eliminate off-by-one rounding errors. I didn't want to unconditionally add
extra operations to decimal128/256 by lowering their `mul_step` so I only did
it for decimal32/64
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
