vbhanuchander-lang commented on issue #6111:
URL: https://github.com/apache/hop/issues/6111#issuecomment-5152073689
I've been digging into this and believe I've found the exact mechanism, so
sharing it here. I've opened #7744 with a fix.
The formatting layer isn't the culprit — both of Hop's default masks render
the value correctly:
```
DecimalFormat("####0.0#########") -> 55487400.0 (default
NUMBER mask)
DecimalFormat("######0.0###################") -> 55487400.0 (default
BIGNUMBER mask)
```
It comes from the Number to BigNumber conversion instead.
`BigDecimal.valueOf(double)` is specified as `new
BigDecimal(Double.toString(val))`, and `Double.toString(55487400.0)` returns
`"5.54874E7"`. That parses into an unscaled value of 554874 with a **scale of
-2**, and it is that negative scale which makes `BigDecimal.toString()` emit
`5.54874E+7`:
```java
BigDecimal bd = BigDecimal.valueOf(55487400.0);
bd.scale(); // -2
bd.unscaledValue(); // 554874
bd.toString(); // "5.54874E+7"
bd.toPlainString(); // "55487400"
```
The `+` sign is a useful giveaway that this is the BigDecimal path rather
than a bare `Double.toString()`, which would give `5.54874E7` without it.
In `ValueMetaBase` this happens in `getBigNumber()` for `TYPE_NUMBER` (all
three storage-type branches) and in `convertStringToBigNumber()`. The negative
scale then leaks out through every consumer that serializes with `toString()`:
- JDBC drivers that inline statement parameters — the Table Output case
reported here
- `writeBigNumber()`, used by the binary row serialization
- `getDataXml()`, used by the XML data serialization
One clarification on the downstream drift: at the Java level
`Double.parseDouble("5.54874E+7") == Double.parseDouble("55487400")` is `true`,
so the `55487400.00000001` you observed is coming from ClickHouse's own
string-to-Float64 parsing rather than from information lost in the string Hop
emits. That is consistent with what you found about ClickHouse PR #52791
affecting string-to-float conversion while INSERT statements behaved as before.
Hop still should not be emitting scientific notation here, so a Hop-side fix is
warranted regardless — it just means the fix removes the trigger rather than
the rounding itself.
The fix in #7744 routes those conversions through a helper that rescales to
zero when the scale would be negative. Rescaling upwards from a negative scale
is exact, so no rounding occurs and the numeric value is unchanged; values that
already convert to a non-negative scale keep their exact previous scale. The
full `core` module suite passes (1039 tests).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
