vogievetsky opened a new pull request, #20384:
URL: https://github.com/apache/druid/pull/20384
### Description
The web console parses every API response and query result with
`json-bigint-native`, which turns any integer with 16+ digits into a `BigInt`
(`parse.js`: `string.length > 15`). That threshold is digit-count based, so it
triggers at **10^15** — about 9x below `Number.MAX_SAFE_INTEGER`. Models
declare these fields as `number`, so nothing catches it at compile time.
A PB-scale datasource therefore crashes the Datasources view: in the `%
Compacted` column, `bytesCompacted + bytesAwaitingCompaction` mixes a BigInt
with a Number and throws `TypeError: Cannot mix BigInt and other types`, taking
down the whole table. When *both* operands cross the threshold nothing throws,
but BigInt division truncates and a nearly-compacted datasource renders as
`0.00%`.
This fixes that and the other instances of the same bug found while auditing
the rest of the console.
Three failure modes, all present:
1. **Throw** — mixed arithmetic, `Math.*`, or unary `+` (which is how d3's
`sum` coerces, so every `sum()` over API numerics is a throw site).
2. **Silently wrong** — BigInt-only division truncates toward zero.
3. **Type lies** — interfaces saying `number` where the parser can produce a
BigInt.
### Changes
**Datasources view** — `progress()` takes `NumberLike` and coerces; the `%
Compacted` accessor calls it instead of duplicating the expression inline.
Hoisted the `typeof` guard in the Availability cell above the arithmetic it
guards.
**Segment timeline** — `IntervalRow` holds raw `NumberLike` stats;
`aggregateSegmentStats` is the normalization boundary, so `TrimmedIntervalRow`
narrows back to `number` and everything downstream is compiler-checked. Fixes 5
throw sites plus a `size === 0` check that never fired against a BigInt.
**MSQ stages** — widened the raw report counters and made `simpleSum` the
single coercion point for all channel arrays. Added `NormalizedStorageCounter`
so the existing `normalizeStorageCounter` coercion is visible in the type.
**Explore view** — added `bigIntsToNumbers()` and applied it where query
rows become chart data, so scales and stacking are safe, not just tooltips.
**Services / supervisors / segments views** — coerced the d3 `sum` aggregate
footers; widened `CompactionStatus`, `SupervisorStatus`, `RowStatsCounter`
(with `AggregatedRowStatsCounter` for the summed form), and `segments-view`'s
`size`.
**Availability column sort (pre-existing bug)** — `sortMethod` read
`d1.num_available` / `d1.num_total`, which don't exist on that row type;
react-table hands `sortMethod` the accessor value, not the row, so every
comparison was `NaN` and the column never sorted. Moved the ratio into the
accessor and dropped the comparator.
Widening the models found four sites that inspection missed, including one
in an untyped `Cell({ original })`.
### Release note
Fix web console crash (`Cannot mix BigInt and other types`) on clusters
where byte or row counters exceed 10^15, and fix the Availability column not
sorting.
<hr>
##### Key changed/added classes in this PR
* `CompactionStatus`, `SupervisorStatus`, `RowStatsCounter` — numeric
fields widened to `NumberLike`
* `IntervalRow` / `TrimmedIntervalRow` — raw vs. normalized
segment-timeline stats
* `ChannelCounter`, `CpuCounter`, `StorageCounter` /
`NormalizedStorageCounter` — MSQ report counters
* `bigIntsToNumbers()` in `utils/general.tsx`
<hr>
This PR has:
- [x] been self-reviewed.
- [x] been tested in a test Druid cluster.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
