aglinxinyuan opened a new pull request, #6544:
URL: https://github.com/apache/texera/pull/6544
### What changes were proposed in this PR?
Adds two regression tests to
`amber/src/test/python/core/runnables/test_main_loop.py`
covering a Python-UDF worker whose input port receives **zero tuples**
across a
`StartChannel` → `EndChannel` bracket — e.g. the untaken branch of an `If`
operator read
through an `InputPortMaterializationReaderRunnable`, or a filter that drops
everything on a
materialized edge:
- **`test_zero_tuple_channel_completes_worker`** — `StartChannel` then
`EndChannel` with no
data tuples must still complete both ports and report
`PortCompleted(input)`,
`PortCompleted(output)`, and `WorkerExecutionCompleted`, forwarding both
channel ECMs
downstream.
- **`test_empty_on_finish_after_tuples_completes_worker`** — an operator
whose `on_finish`
yields nothing (like `BatchOperator` when the input size is an exact
multiple of
`BATCH_SIZE`) must still complete cleanly after processing tuples, since
the empty
`on_finish` is exhausted within a single MainLoop↔DataProcessor hand-off.
Both run the MainLoop on a **daemon thread** and detect completion by
**deadline-polling
`WorkerState.COMPLETED`**, so a regression that deadlocks the handshake
fails the test
cleanly instead of hanging the whole pytest process.
This is **test-only** — no production changes. It pins behavior the
MainLoop↔DataProcessor
handshake already handles on `main` (the internal marker is dispatched in
`_process_ecm`
without an intervening context switch; the worker state graph permits `READY
→ COMPLETED`,
which a zero-tuple worker needs because it never enters `RUNNING`) so it
can't silently
regress.
### Any related issues, documentation, discussions?
Closes #6543
### How was this PR tested?
```
cd amber && python -m pytest
src/test/python/core/runnables/test_main_loop.py \
-k "zero_tuple_channel or empty_on_finish"
# 2 passed, 21 deselected
cd amber && python -m pytest src/test/python/core/runnables/test_main_loop.py
# 23 passed
```
- Both new tests pass on `main`; the whole `test_main_loop.py` stays green
(23 passed).
- `ruff check` → all checks passed; `ruff format --check` → already
formatted.
- Guard-verified that each test actually catches a regression (mutation
reverted after each):
- reintroducing the `_process_ecm` pre-marker `self._switch_context()` →
`test_zero_tuple_channel_completes_worker` deadlocks (deadline-fail);
- removing `COMPLETED` from the `READY` set in the worker state graph →
`test_zero_tuple_channel_completes_worker` fails with
`InvalidTransitionException`;
- gating `finished_current.set()` behind an "`on_finish` produced ≥1
output" flag →
`test_empty_on_finish_after_tuples_completes_worker` hangs
(deadline-fail), while
Echo-based tests stay green.
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Opus 4.8 [1M context])
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
