pavan51 commented on PR #57346: URL: https://github.com/apache/spark/pull/57346#issuecomment-5448994582
Thanks @peter-toth for the continued thorough review! Here is what was addressed in this round: **Finding 25 (RANGE + spill, blocking):** Added `SPARK-58201: Range-based moving frame: MIN/MAX with spill` to `MonotonicDequeWindowFunctionSuite`. It uses `rangeBetween(-2, 2)` with `WINDOW_EXEC_BUFFER_IN_MEMORY_THRESHOLD=8` / `WINDOW_EXEC_BUFFER_SPILL_THRESHOLD=16` to force `ExternalAppendOnlyUnsafeRowArray` into its `SpillableArrayIterator` path, pinning the two-concurrent-iterator invariant introduced by e02bbe4. **Finding 24 (SegmentTreeWindowMetricsSuite conf pin, non-blocking):** Added `WINDOW_MONOTONIC_DEQUE_ENABLED.key -> "false"` to all 7 `withSQLConf` blocks in `SegmentTreeWindowMetricsSuite`. Without this pin, flipping the conf default to `true` would silently break all 6 counter-checking tests — the same landmine that was already defused in `WindowSegmentTreeAllowlistSuite`. **Scalastyle / linter errors (from CI):** Fixed the two lines exceeding 100 characters and corrected the `QueryTest` import ordering in `WindowSegmentTreeAllowlistSuite`. **Finding 2 (benchmark provenance, non-blocking):** Noted. The measured delta between the current code and pre-e02bbe4 is within run-to-run noise in both directions (your own measurements confirm this). The results file stands. As suggested, I'll make the benchmark regeneration the last commit before merge. Let me know if anything else needs attention! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
