seanghaeli opened a new pull request, #75698: URL: https://github.com/apache/airflow/pull/75698
The Amazon provider tags every AWS call's user agent with `Caller/<ClassName>`. The ECS, Batch and Lambda executors get their own name there, but AwsEksExecutor doesn't. Its only AWS call is `eks:DescribeCluster`, made when its client factory builds a Kubernetes client, and that call says `Caller/KubernetesExecutor`. From the pod watcher process it says `Caller/AirflowKubernetesScheduler`. The stack walk finds the KubernetesExecutor first, and the pod watcher has no executor on its stack at all. The fix is a small private `EksHook` subclass in the client factory that always reports `Caller/AwsEksExecutor`. Every client the executor builds goes through that one function: the scheduler's client, per-team clients, the async client, the pod watcher and live log reads in the API server. So they all get the right name now. related: #75449 **Testing** - New unit test: builds a client through both the sync and async factories and checks the EKS client's user agent says `Caller/AwsEksExecutor`. It fails on main (`Caller/Unknown`) and passes with the fix. - EKS executor tests pass in DB and non-DB mode, and so do the rest of the amazon executor tests and `test_base_aws.py`. - Real `User-Agent` header of the `DescribeCluster` request, captured with a botocore `before-send` hook against moto, from each place the client gets built: Before (main): ``` AwsEksExecutor.start() (scheduler) Caller/KubernetesExecutor get_streaming_task_log() (api-server) Caller/KubernetesExecutor _get_eks_async_kube_client() (direct call) Caller/Unknown KubernetesJobWatcher.run() (forked child) Caller/AirflowKubernetesScheduler plain function (pod watcher stand-in) Caller/Unknown User-Agent: Boto3/1.43.106 ... Botocore/1.43.106 Airflow/3.4.0 AmPP/9.39.0 Caller/KubernetesExecutor DagRunKey/00000000-0000-0000-0000-000000000000 MultiTeam/False TeamNameKey/00000000-0000-0000-0000-000000000000 ``` After (this PR): ``` AwsEksExecutor.start() (scheduler) Caller/AwsEksExecutor get_streaming_task_log() (api-server) Caller/AwsEksExecutor _get_eks_async_kube_client() (direct call) Caller/AwsEksExecutor KubernetesJobWatcher.run() (forked child) Caller/AwsEksExecutor plain function (pod watcher stand-in) Caller/AwsEksExecutor User-Agent: Boto3/1.43.106 ... Botocore/1.43.106 Airflow/3.4.0 AmPP/9.39.0 Caller/AwsEksExecutor DagRunKey/00000000-0000-0000-0000-000000000000 MultiTeam/False TeamNameKey/00000000-0000-0000-0000-000000000000 ``` --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Code (Opus 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
