PDGGK opened a new issue, #39743:
URL: https://github.com/apache/beam/issues/39743
### What happened?
A DOUBLE field rejects any integral JSON literal outside `int` range, even
when a `double` holds it exactly:
```java
// RowJsonValueExtractors.java:138, inside doubleValueExtractor()'s validator
|| (jsonNode.isIntegralNumber()
&& jsonNode.canConvertToLong()
&& jsonNode.asLong() == (long) (double) jsonNode.asInt())
```
`asLong()` on the left, `asInt()` on the right. `asInt()` truncates anything
outside `int` range, so the two sides can never agree for a larger value and it
falls through to "out of range".
Epoch millis is the everyday case — this fails today:
```json
{"f_double": 1609459200000}
```
against a schema with `FieldType.DOUBLE`, even though `1609459200000.0d` is
exact.
The guard on the line above is `canConvertToLong()`, so the intent is
clearly "an integral value that fits in a long and survives the trip through
double". The `asInt()` is at odds with its own guard.
Anything under 2^53 is affected: epoch millis, IDs, byte counts. Values that
happen to fit in an `int` work, and a producer that emits `1.6094592e12` works,
which is presumably why this has gone unnoticed.
### Note for whoever fixes this
Swapping `asInt()` for `asLong()` is **not** the fix. `(double)
Long.MAX_VALUE` rounds up to 2^63, and narrowing 2^63 back to `long` saturates
at `Long.MAX_VALUE` rather than overflowing, so the round-trip appears to
succeed and the extractor stores `9223372036854775808` — trading an
over-rejection for silent corruption. `RowJsonTest`'s existing `LONG_STRING` is
2^63 − 2, which does not exercise that.
Comparing through `BigDecimal` is exact, and is already what the decimal
branch immediately below does.
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [x] Component: Java SDK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
