carloea2 opened a new issue, #8169: URL: https://github.com/apache/texera/issues/8169
### What happened? The Python range partitioner deduplicates downstream receivers when it builds its batches, but calculates the range width from the original channel count. When multiple channels point to the same worker, the calculated receiver index can exceed the deduplicated receiver list. Before: channels A, A, B over range 0 through 8 route value 8 to index 2 in a two-receiver list and raise `IndexError`. After: the range width uses the two distinct receivers, so value 8 routes to receiver B at index 1. The Scala range partitioner already calculates its range width from distinct receivers. ### How to reproduce? 1. Construct a Python `RangeBasedShufflePartitioner` with channels to A, A, and B. 2. Set the range to 0 through 8. 3. Add a tuple whose range key is 8. 4. Observe the out-of-range receiver lookup. ### Version or branch 1.3.0-incubating-SNAPSHOT on main ### Commit hash 70c21145887920528d7d5540e3fb790b43e8b759 ### Relevant log output IndexError: list index out of range -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
