carloea2 opened a new pull request, #8190:
URL: https://github.com/apache/texera/pull/8190
### What changes were proposed in this PR?
Track which broadcast receivers have consumed the shared pending batch
during a targeted control-boundary flush. Keep that batch until every
configured receiver has received it, suppress repeated delivery to one
receiver, and preserve it when an unknown receiver is requested.
Before: the first receiver got pending tuples and later receivers got only
the control message.
After: every configured receiver gets pending tuples exactly once before the
batch is cleared.
### Any related issues, documentation, discussions?
Closes #8189
### How was this PR tested?
Regression test first: the full partitioner suite reported 33 passed and 2
failed before the source change. The failures reproduced tuple loss for the
second receiver and data loss on an unknown receiver.
After the fix:
$env:PYTHONDONTWRITEBYTECODE='1';
C:\Users\carlo\texera\texera\.venv312\Scripts\python.exe -c "import sys,pytest;
sys.path[:0]=[r'C:\Users\carlo\texera\texera-worktrees\fix-pyamber-broadcast-flush-all\amber\src\main\python',r'C:\Users\carlo\texera\texera\amber\src\main\python'];
raise
SystemExit(pytest.main([r'amber\src\test\python\core\architecture\sendsemantics\test_partitioners.py','-q','-p','no:cacheprovider']))"
Result: 35 passed.
C:\Users\carlo\texera\texera\.venv312\Scripts\ruff.exe check
amber/src/main/python amber/src/test/python
C:\Users\carlo\texera\texera\.venv312\Scripts\ruff.exe format --check
amber/src/main/python amber/src/test/python
Result: all checks passed and 213 files were already formatted.
A live production-class probe flushed A, repeated A, flushed B, and tried
unknown Z. A and B each received one tuple batch, repeated A received only the
marker, Z received nothing, and the pending batch survived Z.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: OpenAI Codex, GPT-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
