dajiaohuang commented on PR #20059:
URL: https://github.com/apache/pinot/pull/20059#issuecomment-7033037706

   Pushed `fa8379795e`, which addresses both substantive findings from the 
review at `0c5d4dfa`. Replies are in the two existing threads; this is the 
summary.
   
   **The overshoot.** The tolerance was the bug. It grew with the magnitude of 
the bounds, so once a bound was spaced far wider than the increment it could 
reach half a step and promote a genuine fractional count. Your repro, 
`generateFloatArray(91321.76f, 91322.45f, 0.09f)`, has a step count of 7.7257 
and a tolerance of about 0.347, so `rint` took it to 8 and appended 91322.48, 
three float ulps past the end.
   
   I agree with your diagnosis that no tolerance value can fix this, and I took 
the first option you suggested rather than the "cap it below 0.5" one. Capping 
does not separate the cases: the decimal repros need a tolerance above roughly 
2e-9 and your float repro breaks at about 0.274, so a cap only moves which 
ranges are wrong. Instead the element that would actually be generated is 
checked, in the element type, and only a quotient that check cannot settle goes 
to the exact decimal quotient.
   
   All three repros now give the intended answer: 8 elements ending 91322.39f, 
`[1000000]`, and `[1e15, 1e15 + 1]`.
   
   **The float arithmetic.** The float path now promotes an operand before the 
subtraction, so both the subtraction and the division are in double precision, 
and the stale Javadoc wording is gone with the rewrite rather than reworded 
around.
   
   **Regression cases** are added for both types, in 
`GenerateArrayFunctionsTest`: the overshoot repro, ordinary float fractional 
steps, double at 1e15, and an end that is exactly on a step.
   
   Test run reported exactly as requested:
   
   ```
   ./mvnw -pl pinot-common -am -Dtest=GenerateArrayFunctionsTest 
-Dsurefire.failIfNoSpecifiedTests=false -Ppinot-fastdev test
   Tests run: 22, Failures: 0, Errors: 0, Skipped: 0
   BUILD SUCCESS
   ```
   (JDK 26.0.2; `checkstyle:check -pl pinot-common` also passes.)
   
   **Risk and mixed-version rollout.** You are right that this is HIGH risk, 
and the scope is wider than the length function: calls on literal arguments are 
folded into the array literal at planning time, so this changes results as well 
as runtime evaluation. During a rolling upgrade two brokers or segments on 
different versions can return different-length arrays for the same query, and a 
plan folded by a newer broker can differ from what an older one would have 
produced. In a gap-filling grid that is a different row count, not an error. 
There is no wire-format concern and nothing to gate, but it is worth a 
release-note line so a rolling upgrade is not mistaken for nondeterminism. I 
have added that to the PR description.
   
   Two limits I would rather state than leave implied. A step that is not 
representable at the magnitude of the bounds (0.0141 at 3.4e5, where one ulp is 
0.0151) has no unique count; the base, this revision and the previous one all 
differ there, and the count now follows the generated elements. And a range 
whose only in-bounds element repeats its predecessor still ends on the earlier 
one, as the base did.
   
   Beyond the suite I fuzzed the length computation standalone: 1.5M float and 
1.1M double ranges, checking that the last element never passes the end and 
that the end is never dropped, and 800k decimal-step ranges per type against 
the exact decimal count. No violations in any of those.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to