yaningz commented on issue #43717:
URL: https://github.com/apache/airflow/issues/43717#issuecomment-2711690887
Hi @o-nikolas
> Are you using MWAA version 2.10.1?
Yes, I verified the version of the environments in the AWS console just now.
> What are the requirements of the custom-internal-package dependency here?
Can you share those as well?
The full package dependency is: `airflow-provider-om1` (our internal Airflow
provider code) -> internal package CLI -> internal package.
`airflow-provider-om1`
```
[tool.poetry.dependencies]
python = ">=3.11,<3.12"
apache-airflow = "2.10.1"
apache-airflow-providers-amazon = "8.28.0"
apache-airflow-providers-databricks = "6.9.0"
apache-airflow-providers-docker = "3.13.0"
apache-airflow-providers-snowflake = {extras = ["common-compat"], version =
"5.7.0"}
boto3 = "1.35.7"
pendulum = "^3"
boto3-stubs = {extras = ["ecs", "resourcegroupstaggingapi"], version =
"1.35.7"}
<internal-package-cli> = {version = "^0.2.43", source = "codeartifact"}
deepmerge = ">=2.0"
```
Internal package CLI
```
[tool.poetry.dependencies]
boto3 = "^1.34"
boto3-stubs = {extras = ["essential"], version = "^1.34"}
cement = "^3.0"
certifi = "^2024.07.04"
colorlog = "^6.7.0"
#docker = "^6.0.1"
jinja2 = "^3.1.2"
jsonschema = "^4.17.1"
<internal-package> = {path = "../parser"}
pyhocon = "^0.3.59"
python = "^3.11"
python-terraform = "^0.10.1"
pyyaml = "^6.0.0"
urllib3 = "^2.0"
```
Internal package (parser)
```
[tool.poetry.dependencies]
boto3 = "^1.34"
boto3-stubs = {extras = ["essential"], version = "^1.34"}
certifi = "^2024.07.04"
jsonschema = "^4.17.1"
pyhocon = "^0.3.59"
python = "^3.9"
```
> Otherwise, the only other difference I can guess is something to do with
your AWS networking/compute configuration.
I'm not sure how to debug this, but I will share this issue with our
infrastructure engineers who would be more equipped to answer particular
questions about potential failure points of our networking configuration.
> Your issue seems to be different from @yaningz, they are missing the tail
of their logs, and you are missing the head. I'm still unable to reproduce
either case.
Additionally, regarding your question to @mjordyn , Jordyn is a member of a
different team at our organization facing a similar but distinct issue (I
manage the team tasked with managing the platform and our provider code,
Jordyn's team is a user team implementing DAGs). Her issue was discovered in
developing a separate task to run within our MWAA environments. All
environments are on `apache-airflow = "2.10.1"`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
