tkaymak commented on PR #40103: URL: https://github.com/apache/beam/pull/40103#issuecomment-5662081056
Reworked in fcbd007 along your proposal. - `cancel()` stops the evaluation, cancels the job group and returns. `waitUntilFinish()` reports CANCELLED once the cancelled execution ends. - `SparkSessionFactory` counts pipelines per session it created, the last one to release stops it on the execution thread. Sessions it did not create are never stopped. A pipeline starting while another holds the session shares it and logs that its conf is not applied. Nothing waits for another pipeline. The one lock covers the map and the stop itself, so a pipeline that starts during a stop creates a new session instead of adopting a stopping one. - Tests: two unit, three live, one of them starts a pipeline right after `cancel()` without waiting. Rationale comment above the map as requested. `getOrCreateSession` kept as a deprecated delegate. - Diff is +296 −55 against master. State after an asynchronous cancel is #40120. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
