The inputs are decimals, and the correct answer is 1008618.49, also a decimal, and cannot be exactly represented as a binary floating point. I’m not sure why in this case you want a binary floating point.
> On Jan 30, 2024, at 2:46 PM, Mihai Budiu <mbu...@gmail.com> wrote: > > I am evaluating this expression: SELECT power(1004.3e0, 2e0) > The result in Java, or Postgres, when formatted as a string, is > 1008618.4899999999 > The result produced by the Calcite simplification code is 1008618.49 > The simplification code can produce RexLiterals - that's where this would be > useful. > This rounding error is not really necessary. > > Mihai > ________________________________ > From: Julian Hyde <jhyde.apa...@gmail.com> > Sent: Tuesday, January 30, 2024 2:40 PM > To: dev@calcite.apache.org <dev@calcite.apache.org> > Subject: Re: Storing RexLiteral as BigDecimal values > > Can you give a scenario where a RexLiteral should have a double value? > >> On Jan 30, 2024, at 2:36 PM, Mihai Budiu <mbu...@gmail.com> wrote: >> >> Hello, >> >> I have a question about the representation of RexLiteral values. >> Currently DOUBLE-valued literals are represented using a BigDecimal. >> This causes small rounding errors, introduced in the RexBuilder.clean() >> function. >> This causes FP expressions that are evaluated at compilation-time to produce >> results that are slightly off from the same expressions that may be >> evaluated at runtime, for no real reason. For example, I am running some >> Postgres tests for FP values, and they fail because of this small difference. >> >> I know that FP values cannot be compared for equality, and tests are >> supposed to have some slack, but I think that this particular rounding error >> is not necessary. >> >> Why can't RexLiteral actually store a Double value internally when the type >> is Double? >> Is this a bug or is there a deeper reason for this representation? >> If it's a bug I can file a JIRA issue and probably fix it. >> >> Thank you, >> Mihai >
