DanielLeens commented on PR #11338:
URL: https://github.com/apache/seatunnel/pull/11338#issuecomment-4914114938
Thanks for the update. I re-reviewed the latest head `6070c76036` from the
factory-returned multi-table wrapper down to the Java/Python worker protocol.
The current head does close two earlier gaps from my previous rounds:
- the schema-change path now invalidates and rebuilds the cached worker
instead of keeping stale schema metadata alive
- the outer multi-table wrapper now closes the inner transforms, so the
direct worker cleanup path is no longer lost behind the wrapper
Current runtime path rechecked:
```text
Job config uses transform = Python
-> PythonTransformFactory.createTransform(...)
-> PythonMultiCatalogTransform
-> PythonTransform.open()
-> PythonProcessWorker.open()
-> initializeRemoteContext()
-> write init payload
-> stdoutReader.readLine()
-> PythonTransform.getOutputFieldValues(...)
-> PythonProcessWorker.processRow(...)
-> write row payload
-> stdoutReader.readLine()
```
I still see one production-path blocker on the current revision:
1. Python worker initialization and per-row execution can still block
forever without a timeout.
- Location:
`seatunnel-transforms-v2/src/main/java/org/apache/seatunnel/transform/python/PythonProcessWorker.java`
- Why this is still real: both `initializeRemoteContext(...)` and
`processRow(...)` still perform a direct `stdoutReader.readLine()` after
writing to the Python worker. If initialization hangs, or a user function never
responds for one row, the transform thread blocks indefinitely on the main data
path.
- Risk: a single stuck Python call can stall task progress, cancellation,
and troubleshooting with no explicit SeaTunnel timeout boundary.
- Best fix: add explicit initialization and per-row response timeouts,
ideally through a dedicated response-reading thread or bounded queue, and fail
fast with recent stderr context when the timeout is hit.
The current tests still do not cover the no-response / hanging-worker path,
which is the exact lifecycle boundary that remains open.
### Merge conclusion: fix before merge
Blocking items:
1. Add a clear timeout / cancellation boundary for Python worker
initialization and per-row responses.
Once that timeout boundary is closed on the latest head, I’m happy to
re-review again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
