ulysses-you commented on PR #57742: URL: https://github.com/apache/spark/pull/57742#issuecomment-5262036510
@peter-toth Thanks for re-checking through `cb5b6a9`. All three items from that round are addressed: **1 (blocking)** Fixed in `77a8aad0997`. The drain now runs in a loop, so the whole frozen map comes out right after the trigger row, before the fan-out batch continues (previously one map row per bypassed row). Your `lit(6L)` reproduction passes. Two new tests cover the wider shape: `a wide fan-out drains the whole frozen map before streaming the batch` (four-row-wide batch, two collisions) and `a frozen map larger than the fan-out batch preserves the merge order` (compares the generated and interpreted paths per cell, since the forced-fallback cells straddle the `Final` own spill). Full suite: 51 tests green. **5 (non-blocking)** PR description updated: the eligibility paragraph now notes the `session_window` exclusion, the suite count is corrected to 51, and the "generated and interpreted paths always emit the same order" sentence holds again now that the drain order is fixed. The suite class doc and the tuning-doc "first bypassed row" sentence share that claim and were re-checked against the same fix. **19 (minor)** Fixed in `b039db81c79`. The `createNewAggregationBuffer` comment now lists `passThroughAggregationBuffer` as the third caller. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
