sundapeng commented on code in PR #11308:
URL: https://github.com/apache/paimon/pull/11308#discussion_r4657491024


##########
paimon-core/src/main/java/org/apache/paimon/mergetree/compact/aggregate/FieldSumAgg.java:
##########
@@ -106,6 +115,15 @@ public Object retract(Object accumulator, Object 
inputField) {
                                 inFieldDD,
                                 mergeFieldDD.precision(),
                                 mergeFieldDD.scale());
+                if (sum == null) {
+                    throw new ArithmeticException(
+                            String.format(
+                                    "Decimal overflow: %s - %s (precision=%d, 
scale=%d)",
+                                    mergeFieldDD.toBigDecimal(),
+                                    inFieldDD.toBigDecimal(),
+                                    mergeFieldDD.precision(),
+                                    mergeFieldDD.scale()));
+                }

Review Comment:
   Good point — the unconditional throw ignored 
`fields.<field>.sum.fail-on-overflow` and could indeed put compaction / 
merge-on-read over committed files into an unrecoverable failure loop. 
35e753d1b7 makes the DECIMAL branch honor the option, mirroring the integral 
types:
   
   - `fail-on-overflow = true` → throw `ArithmeticException` (as before);
   - `fail-on-overflow = false` (the default) → the aggregated value becomes 
NULL, the same result `FieldProductAgg` already produces on a DECIMAL overflow, 
and the same "overflows are ignored by default" contract as 
TINYINT/SMALLINT/INTEGER/BIGINT.
   
   The default is still a strict improvement over master: with the 
`DecimalUtils` bounds check, both the compact and the non-compact path now 
return NULL, whereas the compact path used to return a `Decimal` whose unscaled 
value violated the declared precision and was written to the data files.
   
   Docs are updated in the same commit (`aggregation.mdx`: a DECIMAL precision 
overflow makes the aggregated value NULL by default), and 
`testFieldSumDecimalOverflowDisabled` pins the default mode for compact and 
non-compact, agg and retract.



##########
paimon-common/src/main/java/org/apache/paimon/utils/DecimalUtils.java:
##########
@@ -77,7 +79,9 @@ public static Decimal subtract(Decimal v1, Decimal v2, int 
precision, int scale)
                 long ls =
                         Math.subtractExact(
                                 v1.toUnscaledLong(), v2.toUnscaledLong()); // 
checks overflow
-                return fromUnscaledLong(ls, precision, scale);
+                if (ls < POW10[precision] && ls > -POW10[precision]) {
+                    return fromUnscaledLong(ls, precision, scale);

Review Comment:
   Applied in 35e753d1b7, with one refinement: the fast path now also requires 
`scale == v1.scale()` before returning null directly. Without that, a call 
whose target scale differs from the operands' scale — previously guarded only 
by `assert scale == v1.scale()`, i.e. silently wrong in production with asserts 
disabled — would return null instead of rescaling through `fromBigDecimal`. 
That case is now pinned by new assertions in 
`DecimalTest#testAddSubtractPrecisionOverflow` (`add(10, 15, 5, 2)` must be 
`25.00`, not `0.25`).
   
   With the condition in place, `ls` is the exact unscaled result at the target 
scale, so on a bound miss or an `addExact`/`subtractExact` overflow the 
BigDecimal path is guaranteed to return null (`|ls| >= 10^precision` implies 
`bd.precision() > precision`; a long overflow implies `|sum| >= 2^63 > 10^18 >= 
10^precision`) — returning null directly skips the three BigDecimal allocations.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to