aglinxinyuan opened a new issue, #6925:
URL: https://github.com/apache/texera/issues/6925
### What happened?
In the Python worker's `LinkedBlockingMultiQueue`, `add_sub_queue` appends
new priority groups in **registration order** instead of inserting them by
priority, so control-priority ordering breaks whenever a data channel happens
to register before a control channel.
There is already an `xfail` pinning this in
`amber/src/test/python/core/models/test_internal_queue.py`:
```python
@pytest.mark.xfail(
reason=(
"LinkedBlockingMultiQueue.add_sub_queue does not currently insert
new "
"priority groups ahead of lower-priority ones, so registering data
before "
"control can break control-priority ordering."
)
)
def
test_control_elements_dequeue_before_data_even_if_data_channel_registered_first(...):
queue.put(data) # registers the data channel first
queue.put(dcm) # registers the control channel later
assert queue.get() is dcm # FAILS today: data dequeues first
```
Effects:
- Control messages (pause, EndWorker, stats queries) can queue **behind
data** on a worker whose first-ever received element was data — delaying
control responsiveness in exactly the situations control priority exists for.
- `InternalQueue.peek()` (added in #6916 for EndWorker diagnostics) reports
the wrong head in the same situation — a cosmetic consequence, the
drained/not-drained decision does not depend on it.
The fix is in `add_sub_queue` / the priority-group bookkeeping in
`amber/src/main/python/core/util/customized_queue/linked_blocking_multi_queue.py`
(~:415-431): insert the new priority group at its sorted position (and update
the sub-queue selector's iteration state) instead of appending. Fix should flip
the existing xfail to a passing test (test-first).
### How to reproduce?
Run the xfail test above without the marker:
```bash
cd amber && python -m pytest
"src/test/python/core/models/test_internal_queue.py::TestInternalQueue::test_control_elements_dequeue_before_data_even_if_data_channel_registered_first"
--runxfail
```
### Version/Branch
main (observed at 429be110a7).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
