o-nikolas commented on issue #43717: URL: https://github.com/apache/airflow/issues/43717#issuecomment-2711376426
Thanks @yaningz, more questions in-line: > [@o-nikolas](https://github.com/o-nikolas) here's our requirements.txt (including 1 custom internal provider). We are still using `apache-airflow = "2.10.1"`, but have observed this log issue since an upgrade from 2.5 -> 2.7 over a year ago (but it wasn't discovered for several months). Are you using MWAA version 2.10.1? > > ``` > --constraint "/usr/local/airflow/dags/constraints.txt" > > airflow-provider-hex==0.1.9 > airflow-provider-om1==2.10.0 > apache-airflow-providers-airbyte==3.9.0 > apache-airflow-providers-amazon==8.28.0 > apache-airflow-providers-databricks==6.9.0 > apache-airflow-providers-docker==3.13.0 > apache-airflow-providers-slack==8.9.0 > apache-airflow-providers-snowflake==5.7.0 > opencv-python==4.10.0.84 > pillow==11.0.0 > ``` So `airflow-provider-om1` is your package that is installed here that has the below requirements? > > I can't share our internal provider, but here are the specific dependencies of that package. There's a separate internal package embedded, but we began observing the log issues prior to that package being introduced to our Airflow environments. > > ``` > [tool.poetry.dependencies] > python = ">=3.11,<3.12" > apache-airflow = "2.10.1" > apache-airflow-providers-amazon = "8.28.0" > apache-airflow-providers-databricks = "6.9.0" > apache-airflow-providers-docker = "3.13.0" > apache-airflow-providers-snowflake = {extras = ["common-compat"], version = "5.7.0"} > boto3 = "1.35.7" > pendulum = "^3" > boto3-stubs = {extras = ["ecs", "resourcegroupstaggingapi"], version = "1.35.7"} > <custom-internal-package> = {version = "^0.2.43", source = "codeartifact"} > deepmerge = ">=2.0" > ``` > > Please let me know if there's anything else I can provide. Thank you for all the help! What are the requirements of the custom-internal-package dependency here? Can you share those as well? My next most feasible guess is that something in this dependency closure is downgrading the provider version, but I think you would have seen that. Otherwise, the only other difference I can guess is something to do with your AWS networking/compute configuration. So far you're the only user complaining of your particular issue of logs still missing from tail after the fix from @vincbeck. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
