viiccwen opened a new pull request, #54582:
URL: https://github.com/apache/airflow/pull/54582
## Summary
This PR introduces two related fixes to stabilize ORM initialization and
session handling in AWS-related utilities.
## Changes
1. **Add `_ensure_db_session` in `eks_get_token`**
* Introduced `_ensure_db_session()` to explicitly initialize the ORM when
`engine` or `Session` are `None`.
* Ensures that running `eks_get_token` as a standalone CLI properly
initializes Airflow settings before creating the `EksHook`.
* Added corresponding unit test `test_ensure_db_session_initializes_orm`
in `test_eks_get_token.py` to validate this behavior.
2. **Refactor config initialization in `base_aws`**
* Moved configuration creation logic inside the `get_session` method.
* This ensures lazy evaluation and avoids potential issues when `Session`
is not yet available at import time.
* Improves reliability when invoking AWS hooks in contexts where Airflow
ORM is not pre-initialized.
## Files Changed
* `airflow/providers/amazon/aws/utils/eks_get_token.py`
* `airflow/providers/amazon/aws/utils/tests/test_eks_get_token.py`
* `airflow/providers/amazon/aws/hooks/base_aws.py`
## Motivation
Previously, invoking `eks_get_token` directly from CLI could fail because
Airflow’s ORM (`settings.engine`, `settings.Session`) was not yet initialized.
This PR ensures ORM initialization is explicit and reliable.
Additionally, deferring configuration creation inside `get_session` removes
the risk of creating invalid or stale sessions during import, aligning with
lazy evaluation best practices.
## Testing
* Added new unit test for `_ensure_db_session` to validate ORM
initialization when `engine` and `Session` are `None`.
## Related Issues
resolves #53578
## Reproduced Step
```
# 1) initialization
# airflow db migrate
# airflow connections add val_aws_assume_test \
--conn-uri 'aws://FAKE_KEY:FAKE_SECRET@/?region_name=eu-central-1'
# 2) reproduced
# export AWS_EC2_METADATA_DISABLED=true
# export PYTHON_OPERATORS_VIRTUAL_ENV_MODE=1
# python -m airflow.providers.amazon.aws.utils.eks_get_token \
--cluster-name dummy --region-name eu-central-1
```
in `main` branch:
```
[2025-08-16T16:13:59.043+0000] {base_aws.py:621} WARNING - Unable to find
AWS Connection ID 'val_aws_assume_test', switching to empty.
[2025-08-16T16:13:59.044+0000] {base_aws.py:197} INFO - No connection ID
provided. Fallback on boto3 credential strategy
(region_name='\x1b[1meu-central-1\x1b[22m'). See:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html
[2025-08-16T16:13:59.048+0000] {base_aws.py:197} INFO - No connection ID
provided. Fallback on boto3 credential strategy
(region_name='\x1b[1meu-central-1\x1b[22m'). See:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html
[2025-08-16T16:13:59.619+0000] {base_aws.py:621} WARNING - Unable to find
AWS Connection ID 'aws_default', switching to empty.
[2025-08-16T16:13:59.619+0000] {base_aws.py:197} INFO - No connection ID
provided. Fallback on boto3 credential strategy
(region_name='\x1b[1meu-central-1\x1b[22m'). See:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html
Traceback (most recent call last):
File "<frozen runpy>", line 198, in _run_module_as_main
File "<frozen runpy>", line 88, in _run_code
File
"/opt/airflow/providers/amazon/src/airflow/providers/amazon/aws/utils/eks_get_token.py",
line 64, in <module>
main()
File
"/opt/airflow/providers/amazon/src/airflow/providers/amazon/aws/utils/eks_get_token.py",
line 58, in main
access_token = eks_hook.fetch_access_token_for_cluster(args.cluster_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/airflow/providers/amazon/src/airflow/providers/amazon/aws/hooks/eks.py",
line 648, in fetch_access_token_for_cluster
signed_url = signer.generate_presigned_url(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/botocore/signers.py", line
355, in generate_presigned_url
self.sign(
File "/usr/local/lib/python3.12/site-packages/botocore/signers.py", line
200, in sign
auth.add_auth(request)
File "/usr/local/lib/python3.12/site-packages/botocore/auth.py", line 421,
in add_auth
raise NoCredentialsError()
botocore.exceptions.NoCredentialsError: Unable to locate credentials
```
in `fix-aws-orm-initialization`:
```
[2025-08-17T05:17:18.561+0000] {connection_wrapper.py:334} INFO - AWS
Connection (conn_id='val_aws_assume_test', conn_type='aws') credentials
retrieved from login and password.
[2025-08-17T05:17:19.043+0000] {base_aws.py:621} WARNING - Unable to find
AWS Connection ID 'aws_default', switching to empty.
[2025-08-17T05:17:19.043+0000] {base_aws.py:197} INFO - No connection ID
provided. Fallback on boto3 credential strategy
(region_name='\x1b[1meu-central-1\x1b[22m'). See:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html
expirationTimestamp: 2025-08-17T05:31:19Z, token:
k8s-aws-v1.aHR0cHM6Ly9zdHMuZXUtY2VudHJhbC0xLmFtYXpvbmF3cy5jb20vP0FjdGlvbj1HZXRDYWxsZXJJZGVudGl0eSZWZXJzaW9uPTIwMTEtMDYtMTUmWC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1GQUtFX0tFWSUyRjIwMjUwODE3JTJGZXUtY2VudHJhbC0xJTJGc3RzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTA4MTdUMDUxNzE5WiZYLUFtei1FeHBpcmVzPTYwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCUzQngtazhzLWF3cy1pZCZYLUFtei1TaWduYXR1cmU9NjI2NmNhMzRjOTAyMTE4ZjdhOTA5YWU1MGI3NzkzMzg4ZDBkNGZiYmZlYzhmZjkwN2NiMzJkODE5NWMzZjU0Yg
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
