Caideyipi opened a new pull request, #18280:
URL: https://github.com/apache/iotdb/pull/18280
## Description
### Motivation and operator contract
`pipe_datanode_remaining_event_count == 0` does not cover data that was
still in a working TsFile processor when the value was sampled. Likewise, a
realtime source's TsFile/log mode is an extraction strategy, not an end-to-end
completion acknowledgement. This makes either signal unsafe as a test oracle by
itself.
This PR adds the fail-closed metric:
```text
pipe_datanode_completion_ready{name,creation_time}
```
It reports `1` only when all local DataRegion tasks of the pipe have passed
their latest successful full-FLUSH barriers through source assignment,
processing, the sink queue, sink acknowledgement, and ordered commit, while the
existing remaining-event count is zero. Any unknown or changing state reports
`0`, so a transient false negative is allowed but a false positive is not.
The intended test protocol is:
```text
join/drain writers
-> run a successful full FLUSH covering all relevant DataRegions and both
seq/unseq processors
-> freshly scrape this metric directly from every relevant sender DataNode
-> accept completion only when every expected series is 1
```
When Prometheus is used, callers must also verify that every sample
timestamp is later than that FLUSH, every expected sender target has `up == 1`,
and the returned series cardinality equals the expected sender DataNode count.
The gauge values `0` and `1` are represented exactly; stale samples and missing
targets, rather than floating-point precision, are the relevant risks.
The metric deliberately covers DataRegion/DML data-point completion only. It
does not prove SchemaRegion/DDL completion and is not added to the Grafana
dashboards.
### Barrier and concurrency design
- A full FLUSH first invalidates the old completion token, closes and waits
for the entry snapshot of all working/closing seq and unseq TsFile processors,
and publishes a barrier only after the close succeeds.
- Inserts invalidate the current token. This also covers inserts ignored by
a source mode and mixed TsFile/log-mode regions.
- The barrier follows the normal ordered Pipe path. Completion is recorded
only by the current committer after the sink has acknowledged the event.
- A full FLUSH overlapping an ordinary asynchronous flush waits for that
flush and then closes the processor. It tracks the processors present at FLUSH
entry rather than chasing processors created by later writes.
- Two topology/state observations reject concurrent replacement or mutation.
Source/assigner/committer replacement, Pipe drop, runtime/task exceptions,
degradation, publication failure, stale callbacks, unsupported tasks, or
nonzero remaining events all fail closed to `0`.
The supported configuration is a running USER Pipe with
`iotdb-extractor`/`iotdb-source`, `do-nothing-processor`, a built-in IoTDB
Thrift sink/connector alias, and synchronous TsFile loading. Unsupported
configurations report `0`.
The existing remaining-event metric keeps its current meaning.
### Validation
- 15 focused Pipe completion/barrier tests: 15 passed, 0 failed/skipped.
- 3 focused full-FLUSH/ordinary-flush concurrency tests: 3 passed, 0
failed/skipped.
- DataNode Checkstyle: 0 violations.
- DataNode Spotless check: 0 files need changes.
- English and Chinese-locale reactor test compilation passed during
implementation.
<hr>
This PR has:
- [x] been self-reviewed.
- [x] concurrent read
- [x] concurrent write
- [x] concurrent read and write
- [x] added Javadocs for the new completion operator and non-trivial barrier
methods.
- [x] added comments explaining concurrency intent and fail-closed behavior.
- [x] added or updated unit tests for the new code paths.
<hr>
##### Key changed/added classes (or packages if there are too many classes)
in this PR
- `PipeDataNodeCompletionOperator`: computes the fail-closed per-DataNode
gauge.
- `PipeDataNodeSinglePipeMetrics`: registers the metric and connects
lifecycle/barrier callbacks.
- `PipeDataRegionAssigner` and `PipeHeartbeatEvent`: generation-bound
completion barrier publication.
- `StorageEngine`, `DataRegion`, and `TsFileProcessor`: successful
full-FLUSH boundary and ordinary-flush overlap handling.
- Pipe source/processor/sink commit path: carries the barrier to ordered
sink acknowledgement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
