DaZuiZui commented on issue #18428:
URL: https://github.com/apache/iotdb/issues/18428#issuecomment-5278263388
## Scope-limited addendum: required implementation clarifications
This addendum only closes implementation ambiguities in the proposal above.
It is limited to the calendar-duration behavior requested by #18428 and is
pending maintainer confirmation. Where it conflicts with an earlier point, this
addendum supersedes only that point; the rest of the proposal remains unchanged.
### 1. CQ duration aliases
For the `EVERY` and `RANGE` duration positions only, support the following
case-insensitive calendar units:
- month: `mo`, `month`
- year: `y`, `year`
`1y` and `1year` normalize to 12 calendar months. Plural aliases, decimals,
signed components, and whitespace inside a duration remain unsupported. A
CQ-specific duration rule must consume and validate the complete value, so
adding `month` and `year` here does not change `GROUP BY TIME`, date
arithmetic, `FILL`, `SESSION`, or general identifier tokenization.
This replaces the earlier statement that this issue supports only the
abbreviated spellings.
### 2. Structured request and explicit BOUNDARY
Use a bounded structured value such as:
```text
CQDuration {
int64 monthPart
int64 fixedPart
}
```
`TCreateCQReq` adds optional structured values for `every`, `startOffset`,
and `endOffset`, plus an optional `boundaryExplicit` flag. For a new structured
request, all three effective durations and `boundaryExplicit` must be present;
a partial structured representation is rejected. All arithmetic and narrowing
conversions are checked.
The existing `i64` duration fields and field IDs remain for legacy
fixed-only requests. A request with no structured fields is interpreted as
legacy fixed-only. For a structured request, the structured values are
authoritative; the legacy `i64` fields must never be used by new code to
approximate a non-zero calendar month part. When every structured month part is
zero, the legacy and structured fixed values must agree or the request is
rejected.
`boundaryExplicit=false` means that ConfigNode applies the selected
omitted-BOUNDARY rule. `boundaryExplicit=true` preserves an explicitly written
`BOUNDARY 0` as the Unix epoch instant. The flag, normalized boundary,
durations, ZoneId, and progress index must pass through procedure/plan
serialization and CQ metadata snapshots.
### 3. Mixed-version creation policy
This issue does not claim support for creating a calendar CQ during a
rolling mixed-version window. Calendar CREATE is supported only after all
registered ConfigNodes and all DataNodes that can accept client SQL have been
upgraded to code that understands the structured representation. This condition
covers DataNodes because `CREATE CQ` is parsed there before `TCreateCQReq` is
built.
Before forwarding a structured calendar request, an upgraded SQL-ingress
DataNode must positively verify this condition using the existing cluster
node-version information; the upgraded ConfigNode repeats the check before
accepting the request. An unknown or unsupported registered version causes a
clear rejection. During an upgrade, old DataNodes must be removed from client
routing before calendar CREATE is used. Fixed-only CQ creation and historical
CQs retain their legacy behavior.
This is a narrow fail-closed compatibility check for #18428, not a new
feature-activation, membership-attestation, or upgrade framework.
### 4. Deterministic calendar and DST resolution
Let `B` be the exact anchor instant in persisted timestamp-precision ticks
and `Z` the persisted CQ `ZoneId`. For a duration vector `(M, F)`:
```text
anchor = instant(B).atZone(Z)
anchorLocal = anchor.toLocalDateTime()
anchorOffset = anchor.getOffset()
```
- If `M == 0`, `calendarApply(B, (M, F), Z)` is checked elapsed-tick
addition `B + F`. In particular, `calendarApply(B, ZERO, Z) == B`, and
fixed-only CQ behavior is unchanged.
- Otherwise, compute `targetLocal = anchorLocal.plusMonths(M)`, including
end-of-month clamping. Resolve it with the rules of `Z`, preferring
`anchorOffset` when that offset is valid at `targetLocal`, and then add `F` as
checked elapsed ticks.
- For a DST gap, shift the local time forward by the transition duration and
use the post-transition offset. For an overlap, use `anchorOffset` when valid;
otherwise use the earlier valid offset. This is the behavior of
`ZonedDateTime.ofLocal(targetLocal, Z, anchorOffset)` and must not depend on a
host default time zone.
An integer or explicitly-offset BOUNDARY is first parsed as the exact
instant `B`. A written offset selects that instant but does not replace the
persisted CQ zone `Z`; recurrence observes `B` in `Z`. An offset-less BOUNDARY
is resolved in `Z` using the same gap rule and the earlier offset for an
overlap when no preferred anchor offset exists.
This issue persists `ZoneId`, not a private copy of TZDB rules. Scheduling
nodes are expected to use compatible TZDB data; installing a newer TZDB may
legitimately affect future civil-time transitions.
### 5. Anchor-relative compound durations and progress
Duration arithmetic is component-wise and anchor-relative:
```text
E = (monthPart, fixedPart)
n * E = (n * monthPart, n * fixedPart)
executionTime(n) = calendarApply(B, n * E, Z)
```
Multiplication, vector addition/subtraction, and timestamp conversion are
checked. A compound duration is not repeatedly applied to the previous
occurrence. For example, with a UTC anchor of `2024-01-30 00:00` and `EVERY
1mo2d`, `n=1` is `2024-03-02 00:00` and `n=2` is `2024-04-03 00:00`. RANGE
vectors use the same component-wise arithmetic before one `calendarApply` call.
ConfigNode captures the CREATE reference instant `C` once. The initial index
is exactly:
```text
min { n >= 0 | executionTime(n) >= C }
```
Equality selects that occurrence. CQ metadata persists
`nextOccurrenceIndex`, defined as the first occurrence not yet durably
completed. Failure retries the same index; after success, progress is persisted
before the next occurrence is scheduled, and recovery resumes the stored index.
`BLOCKED` advances in order; `DISCARD` uses the same lower-bound rule against
one captured callback time when skipping missed indices. This preserves the
existing at-least-once execution boundary and does not introduce an
exactly-once protocol.
### 6. Timeout and focused tests
`TExecuteCQ.timeout` remains milliseconds. For occurrence `n`, derive it
from the actual adjacent scheduled instants:
```text
deltaTicks = executionTime(n + 1) - executionTime(n)
timeoutMs = ceil(deltaTicks / ticksPerMillisecond)
```
The subtraction and conversion are checked, and a positive sub-millisecond
interval becomes 1 ms. Timestamp-precision ticks must not be passed directly as
milliseconds, and no 30-day approximation is used.
In addition to the tests already listed, focused tests will cover:
- all four accepted aliases and rejection outside CQ `EVERY`/`RANGE`;
- partial/conflicting structured fields and `boundaryExplicit` round trips;
- mixed-version/unknown-version rejection at both CREATE ingress and
ConfigNode acceptance;
- `calendarApply(B, ZERO, Z) == B`, DST gaps, both overlap offsets, and
explicit-offset BOUNDARY with a different CQ ZoneId;
- compound EVERY/RANGE schedules and checked vector overflow;
- CREATE exactly on a boundary, persisted `nextOccurrenceIndex`, and
recovery without an off-by-one shift;
- timeout conversion under ms/us/ns precision.
Explicitly outside this addendum are new V2 RPC families, PREPARE/ACTIVATE
commands, reader-floor markers, node attestation, general mutation exact-retry
protocols, execution-admission redesign, and a custom ZoneRules snapshot format.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
