Caideyipi commented on PR #17871: URL: https://github.com/apache/iotdb/pull/17871#issuecomment-6848124071
Independent re-review completed: one full round of the 86-file diff and relevant implementation, API, and test context. - Reviewed head: `111fd493b7a4ab63437394e50beaa09ae72b7c94` - Comparison base: `cca1dbbde6d0dffa38495c92c92a48b2626d2f99` - Final local commit, confirmed source branch head, and confirmed PR head: `32f25d9cedd8af43529c3c0ea26487b2b7344c6a` Two behavioral issues were found and fixed in this round. Locations below refer to the reviewed head. 1. **[P2] DTW raw and continued input did not invalidate cached matches when timestamps went backwards or an earlier value was replaced.** [DTWState.java:108](https://github.com/Caideyipi/iotdb/blob/111fd493b7a4ab63437394e50beaa09ae72b7c94/library-udf/src/main/java/org/apache/iotdb/library/match/model/DTWState.java#L108). With pattern `1,2,3` and threshold `0`, feeding timestamps `6,5,4,3,2,1` with corresponding values `3,2,1,3,2,1` missed both chronological matches. Likewise, after merging and observing times `1,3,4`, inserting time `2` did not rebuild the historical windows. Updating an earlier value could leave an obsolete match. Non-increasing timestamps now mark the state for the same deferred chronological rebuild used by merges. Regressions cover batches, late continuation after output, historical replacement, serialization before rebuilding, and repeated output. 2. **[P2] Moving-average compensation could itself lose small values permanently.** [UDTFMvAvg.java:97](https://github.com/Caideyipi/iotdb/blob/111fd493b7a4ab63437394e50beaa09ae72b7c94/library-udf/src/main/java/org/apache/iotdb/library/dprofile/UDTFMvAvg.java#L97). For `window=3` and input `[1e300,1e150,1,1,1]`, the final window `[1,1,1]` returned `0` instead of `1`; all sums remained finite, so overflow recovery did not help. The fix detects lost compensation bits or overflowing additions, reconstructs the active window sum exactly once, and then updates that exact sum incrementally. The regression covers both signs, subsequent windows, and resetting the function for another run. The review also covered MAD null handling and INT64 extremes, sampling Row reuse and typed INT64 output, DWT levels, Histogram normal/subnormal/extreme ranges, MinMax extremes, Spline timestamps/coordinates/endpoints/duplicate timestamps and fixtures, AR/DTW validation, LOF valid-window coverage, and DTW history, cross-partition matches, interleaved/reversed/multilevel merges, serialization, continuation, and repeated output. Validation: - All three new regressions failed against the reviewed production implementation, then passed after repair. - Local offline `mvn -o clean test -pl library-udf -am`: **142 tests, zero failures/errors/skips**. - Local offline Chinese `clean test-compile -pl library-udf -am -P with-zh-locale -DskipTests`: passed. - Spotless with `with-integration-tests`, lifecycle Checkstyle (zero violations), and `git diff --check`: passed. - Required IoTDB dependencies were rebuilt from clean source; both clean reactors disabled local and remote build caches. All newly added tests were run locally. The local offline fallback retained the previously established remote Maven plugin-download limitation; no new external dependencies were introduced. - No full-project build or performance benchmark was run. CI was neither waited on nor polled. Resource assessment: DTW still retains O(N + R) state and serialized data for N effective input points and R matches. A pending final rebuild costs O(N * L^2) for pattern length L, with O(L) DTW scratch space. Combines insert incoming points without recalculating all historical DTW windows; serialization and multilevel transfer still traverse retained input. This is an algorithmic assessment, not a measured performance result. The moving-average exact fallback retains the active window and an incremental sum, with a one-time window reconstruction. The source branch was checked immediately before pushing and had no concurrent update. The source branch and PR head were both confirmed at the final SHA above, and the PR remains **Draft** because this round found behavioral issues. A fresh independent full-diff re-review is required before Ready. No second review round was started after these repairs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
