potiuk edited a comment on issue #14994: URL: https://github.com/apache/airflow/issues/14994#issuecomment-808481700
We are using constraints to limit the version of dill that is installed, however we know apache beam has a number of problems with dependencies. The reason we are not failing in CI is simple: https://github.com/apache/airflow/blob/fe2a70c713f8d12199579bfca26eb916035ab206/setup.py#L750 ``` # Those packages are excluded because they break tests and they are not needed to run our test suite. # This can be removed as soon as we get non-conflicting # requirements for the apache-beam as well. # # Currently Apache Beam has very narrow and old dependencies for 'dill' and 'mock' packages which # are required by our tests (but only for tests). # PACKAGES_EXCLUDED_FOR_CI = [ 'apache-beam', ] ``` The `dill` failing the build was a different reason entirely. This has been fixed in https://github.com/apache/airflow/pull/14998. The problem was that a different bug caused the CI image to rebuild in an environment ( providers) where we do not care about pip check where we want to just check all providers importability. And this docker image build (which was unnecessary and harmful) was run in the way that did not exclude `apache-beam` and failed the `pip check`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
