luoluoyuyu commented on PR #19073: URL: https://github.com/apache/iotdb/pull/19073#issuecomment-6573282705
Thanks — turning a silent partial load into a failure is the right direction. A few concerns before merge: 1. **The checkpoint is a per-series time watermark.** `recordProgress` advances *every* measurement in a tablet to the tablet's last row timestamp, and `readDeviceMeasurementsInOrder` treats `emitted >= statistics.getEndTime()` as "fully loaded". That only holds if each series' timestamps are non-decreasing across chunk groups, which TsFile writers don't enforce. If a later group holds an earlier timestamp, those points are skipped without recording a failure → `SUCCESS` + `on-success=delete` still deletes the source, i.e. the same bug class. Suggest advancing only to the last row where that measurement really has a value (nulls in aligned tablets over-advance it), plus a test with a non-monotonic file. 2. **No best-effort path.** One unreadable measurement now makes the whole file permanently unloadable, even when the corrupt region contains no data. Worth an explicit decision (or an opt-in flag). 3. **Upstream retry heuristics key off message text** (`ActiveLoadFailedMessageHandler.isExceptionMessageShouldRetry`). The surfaced message is the generic `"Failed to iterate tablets while loading TsFile."`, so even transient IO errors get moved to the fail dir. Including device/measurement and the cause (via a `DataNodeQueryMessages` constant) would make this diagnosable and retryable. 4. **Task count**: one `QueryTask` per distinct checkpoint means one `TsFileReader` open per task; wide devices can turn fallback into thousands of file opens, and the split path can emit O(M) WARNs each printing the full measurement list. 5. Minor: a split task whose halves are all filtered out by `emitted >= endTime` loses the failure entirely; only the first failure is reported; and `waitUntilClusterAnswers()` returning `true` unconditionally weakens the IT readiness check (unrelated to this fix). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
