PDGGK opened a new pull request, #116:
URL: https://github.com/apache/iotdb-extras/pull/116
## Problem
The milliseconds aggregation path projects the numeric aggregate over a
`COALESCE` expression:
```sql
SELECT date_bin(<interval>ms, time, <startTs>) AS bucket_ts,
MAX(COALESCE(double_v, CAST(long_v AS DOUBLE))) AS agg_num,
...
FROM telemetry
WHERE <identity predicates> AND time >= <start> AND time < <end>
GROUP BY 1
```
IoTDB's grouped max accumulator seeds FLOAT/DOUBLE state with
`Float.MIN_VALUE` / `Double.MIN_VALUE` — the smallest *positive* values, not
the most negative ones — and only marks a group initialized when `value >=
state`. A bucket whose numeric maximum is zero or negative therefore never sets
the flag and comes back as `NULL` (apache/iotdb#18300, fixed upstream after
2.0.10).
`aggregatedEntry` reads a NULL numeric aggregate as "this bucket has no
value" and returns `null`, and `readMillisecondsAggregatedQuery` skips such
buckets. So a `MAX` downsampling query over telemetry that is always zero or
negative — a sub-zero temperature sensor, a gauge that sits at 0 — silently
returned *fewer points*, with no error raised anywhere.
Verified against the released `apache/iotdb:2.0.8` and `apache/iotdb:2.0.10`
standalone images: a bucket of `-5.0, -3.0` and a bucket of `0.0, 0.0` both
return `agg_num = NULL`, while a positive bucket is correct.
Scope of the problem, also verified on both images:
- `MIN`, `AVG`, `SUM` and `COUNT` are unaffected.
- The calendar path is unaffected — it issues one bounded aggregate per
bucket with no `GROUP BY`, and the non-grouped accumulator seeds correctly.
- A long-only bucket is unaffected: it reads the direct `MAX(long_v)`
channel, whose `LongBigArray` seeds with the true `Long.MIN_VALUE`.
- A mixed long+double bucket *is* affected, because it reads the numeric
channel.
## Fix
Project `-MIN(-x)` instead of `MAX(x)` for the numeric channel. The grouped
min accumulator seeds with `MAX_VALUE` and is not affected, and IEEE-754
negation is exact, so this returns the same value as `MAX(x)` on servers with
and without the upstream fix, with no extra query. The long and string channels
seed correctly and keep using `MAX`.
## Tests
Two integration tests against a real IoTDB container:
- `maxKeepsBucketsWhoseValuesAreAllNonPositiveAgainstRealIoTDB` — buckets of
all-negative and all-zero doubles keep their `MAX` (and `MIN` is unchanged).
- `maxOverNonPositiveLongOnlyAndMixedBucketsKeepsResultTypeAgainstRealIoTDB`
— covers the long-only channel and the mixed promotion, since they take
different paths.
Both fail on the previous projection with `bucket count expected: <2> but
was: <0>` and `<1>` respectively — the buckets disappear — and pass with the
fix. The existing aggregation coverage only ever used positive doubles, which
is why this was not caught earlier.
Full module build is green: 190 unit tests and 54 container integration
tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
