DanielLeens commented on PR #11337:
URL: https://github.com/apache/seatunnel/pull/11337#issuecomment-4914114769
Thanks for the update. I re-reviewed the latest head `b9f53b50b0` from the
connector registration path down to the Python reader lifecycle.
The good news first: the unrelated MySQL CDC E2E disabling change from my
previous round is no longer part of the current diff, so that earlier blocker
is closed on the latest head.
Current runtime path rechecked:
```text
Job config uses source = PythonSource
-> PythonSourceFactory.createSource(...)
-> PythonSource.createReader(...)
-> PythonSourceReader.open()
-> start Python process
-> write `python.script.config` to stdin
-> PythonSourceReader.internalPollNext(...)
-> stdoutReader.readLine() loop
-> deserialize each line to SeaTunnelRow
-> waitForProcessExit()
```
I still see one blocking item on the current revision:
1. `PythonSourceReader.internalPollNext(...)` can still block indefinitely
until the Python process closes stdout.
- Location:
`seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java`
- Why this is still real: the reader still drains the whole stdout stream
inside a single `pollNext` call and then waits for process exit. If the user
script hangs, waits on an external service, writes slowly without closing
stdout, or simply never exits, the SeaTunnel reader thread stays blocked in the
main source path.
- Risk: the job can appear hung instead of failing with a clear
timeout/cancellation boundary, and one stuck script can pin the source task
indefinitely.
- Best fix: add an explicit timeout / cancellation boundary around the
bounded Python execution contract, or move stdout draining to an async pump
plus bounded queue so `pollNext` can periodically return and observe task
cancellation/process health.
Tests still do not cover the long-running-script / stdout-not-closed case,
which is the key lifecycle boundary here.
### Merge conclusion: fix before merge
Blocking items:
1. Close the indefinite blocking gap in `internalPollNext(...)`.
Once that lifecycle boundary is fixed on the latest head, I’m happy to
re-review again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
