vgkowski opened a new pull request, #74870:
URL: https://github.com/apache/airflow/pull/74870
Add `EmrServerlessStopSessionOperator`, which terminates an EMR Serverless
interactive
(Spark Connect) session and can wait until it reaches `TERMINATED`.
Why:
- Interactive sessions keep their Spark drivers and executors running until
they are
terminated or hit their idle timeout, so Dags need an explicit way to end
a session
once their work is done.
Changes:
- The operator calls the `TerminateSession` boto3 API directly, after the
same
`botocore >= 1.43.0` runtime check the other session operators use, so the
provider's
dependency floor is unchanged.
- `EmrServerlessStopSessionTrigger` and the `serverless_session_terminated`
waiter support
synchronous and deferrable waiting. As in
`EmrServerlessStartSessionOperator`,
`wait_for_completion` decides whether the task waits and `deferrable` only
picks how,
and the trigger polls with the task's region, SSL verification and
botocore settings.
- Docs (howto section), a system-test example step, and unit tests.
This completes the interactive session lifecycle started in #70763 (start)
and #70766
(get endpoint). It replaces #70768, which the stale bot closed before it was
reviewed.
* related: #69207
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Kiro (Claude Opus 5.5)
Generated-by: Kiro (Claude Opus 5.5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
