Ma77Ball opened a new pull request, #8920: URL: https://github.com/apache/texera/pull/8920
### What changes were proposed in this PR? - Add `IndexedShufflePartitioner` (new `indexed_shuffle_partitioner.py`), a base that owns the shared shuffle machinery: the `__init__` (batch size + per-worker receiver slots), `add_tuple_to_batch`, `flush`, and `flush_state`. - Reduce `RoundRobinPartitioner`, `HashBasedShufflePartitioner`, and `RangeBasedShufflePartitioner` to just their routing: an abstract `_route(tuple_) -> int`, an `_advance()` hook (round-robin bumps its index), and a `_clear_batch_on_flush` flag (round-robin drains a flushed slot; hash/range do not). Each subclass shrinks by roughly 50 lines. - The two round-robin-only quirks (index advance, clearing a flushed slot) are made explicit via the hook and flag rather than changed, so behavior is preserved. `OneToOnePartitioner` and `BroadcastPartitioner` keep their distinct single-batch structure and are untouched. ### Any related issues, documentation, discussions? Closes: #8919 ### How was this PR tested? - Run the partitioner and output-manager suites from `amber/`: `python -m pytest src/test/python/core/architecture/sendsemantics/test_partitioners.py src/test/python/core/architecture/packaging/test_output_manager.py`. Expect all green: the 74 existing specs (which assert exact emitted tuples, residual slot state, `round_robin_index`, and `get_receiver_index` outputs) pass unchanged, serving as the behavior-preservation anchor. - Two tests were added to pin the one genuine divergence: round-robin clears a flushed slot's batch, while hash/range leave it intact. ### Was this PR authored or co-authored using generative AI tooling? Co-authored with Claude Opus 4.8 in compliance with ASF -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
