rmannibucau opened a new issue, #9400:
URL: https://github.com/apache/hop/issues/9400

   ## Problem
   Follow-up audit to the local pipeline shutdown deadlock found 
completion-accounting and cancellation problems in several transforms that can 
strand completion/admission waits or leak outstanding-job counts:
   
   - Html2Text, DetectLanguage and StanfordSimpleNlp: failed jobs can leak 
outstanding-job counts and strand EOF/admission waits.
   - LanguageModelChat: submission accounting, borrowed common-pool vs owned 
private executor distinction, and executor shut-down on stop were incorrect in 
places.
   - Execute Process: stop() waited indefinitely for the execution thread's 
latch; a descendant can keep a pipe open after the direct child exits. Stop now 
sends SIGKILL immediately, giving scripts that trap SIGTERM no chance to clean 
up (must be graceful destroy() then 30s wait then destroyForcibly()).
   
   ## Fix
   See PR (transform lifecycle split out from #9374 per review) for the 
detailed changes:
   - Reserve work before submission; balance accounting and notify waiters on 
worker completion and rejected submission.
   - Make asynchronous failures observable instead of logging them and silently 
losing rows; preserve per-row error routing and serialized output.
   - Drain accepted work before disposal so workers do not publish after the 
transform has been disposed.
   - LanguageModelChat: correct submission accounting; distinguish borrowed 
common-pool execution from an owned private executor; shut down owned executors 
without waiting for a worker from a stop callback.
   - Execute Process: make stopRunning() a non-waiting cancellation request; 
observe cancellation during launch/process-wait/output-drain; graceful process 
kill on the cleanup thread.
   
   Originally part of PR #9374; split out per review.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to