[
https://issues.apache.org/jira/browse/AIRFLOW-2978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Davies updated AIRFLOW-2978:
--------------------------------
Description:
I have a Python package that depends on Airflow that forms part of our DAG
build out process.
This pipeline installs Airflow 1.10.0 and then the build fails with:
{code:java}
Installed /python-packages/pkg/pkg/.eggs/pytzdata-2018.5-py2.7.egg
Traceback (most recent call last):
File "setup.py", line 39, in <module>
test_suite="tests"
File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line
131, in setup
return distutils.core.setup(**attrs)
File "/usr/local/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/site-packages/setuptools/command/test.py",
line 216, in run
installed_dists = self.install_dists(self.distribution)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/test.py",
line 207, in install_dists
ir_d = dist.fetch_build_eggs(dist.install_requires)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 514,
in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line
782, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (tzlocal 2.0.0b1
(/python-packages/pkg/pkg/.eggs/tzlocal-2.0.0b1-py2.7.egg),
Requirement.parse('tzlocal<2.0.0.0,>=1.5.0.0'), set(['pendulum']))
{code}
It would seem that Airflow pins ancient versions here:
-
https://github.com/apache/incubator-airflow/blob/314232cc49ebe75cf12e58af9588123441976647/setup.py#L316
For example:
psutil is at 5.4.7
pygments is at 2.2.0
pendulum is at 2.0.3
All of these install fine with their latest versions together in a fresh 2.7.15
Docker container.
was:
I have a Python package that depends on Airflow that forms part of our DAG
build out process.
This pipeline installs Airflow 1.10.0 and then the build fails with:
{code:java}
Installed /python-packages/pkg/pkg/.eggs/pytzdata-2018.5-py2.7.egg
Traceback (most recent call last):
File "setup.py", line 39, in <module>
test_suite="tests"
File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line
131, in setup
return distutils.core.setup(**attrs)
File "/usr/local/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/site-packages/setuptools/command/test.py",
line 216, in run
installed_dists = self.install_dists(self.distribution)
File "/usr/local/lib/python2.7/site-packages/setuptools/command/test.py",
line 207, in install_dists
ir_d = dist.fetch_build_eggs(dist.install_requires)
File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 514,
in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line
782, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (tzlocal 2.0.0b1
(/python-packages/pkg/pkg/.eggs/tzlocal-2.0.0b1-py2.7.egg),
Requirement.parse('tzlocal<2.0.0.0,>=1.5.0.0'), set(['pendulum']))
{code}
It would seem that Airflow pins ancient versions here:
-
https://github.com/apache/incubator-airflow/blob/314232cc49ebe75cf12e58af9588123441976647/setup.py#L316
psutil is at 5.4.7
pygments is at 2.2.0
pendulum is at 2.0.3
All of these install fine with their latest versions together in a fresh 2.7.15
Docker container.
> Airflow deps breaking on ancient Python packages
> ------------------------------------------------
>
> Key: AIRFLOW-2978
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2978
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Jon Davies
> Priority: Major
>
> I have a Python package that depends on Airflow that forms part of our DAG
> build out process.
> This pipeline installs Airflow 1.10.0 and then the build fails with:
> {code:java}
> Installed /python-packages/pkg/pkg/.eggs/pytzdata-2018.5-py2.7.egg
> Traceback (most recent call last):
> File "setup.py", line 39, in <module>
> test_suite="tests"
> File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line
> 131, in setup
> return distutils.core.setup(**attrs)
> File "/usr/local/lib/python2.7/distutils/core.py", line 151, in setup
> dist.run_commands()
> File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
> self.run_command(cmd)
> File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
> cmd_obj.run()
> File "/usr/local/lib/python2.7/site-packages/setuptools/command/test.py",
> line 216, in run
> installed_dists = self.install_dists(self.distribution)
> File "/usr/local/lib/python2.7/site-packages/setuptools/command/test.py",
> line 207, in install_dists
> ir_d = dist.fetch_build_eggs(dist.install_requires)
> File "/usr/local/lib/python2.7/site-packages/setuptools/dist.py", line 514,
> in fetch_build_eggs
> replace_conflicting=True,
> File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py",
> line 782, in resolve
> raise VersionConflict(dist, req).with_context(dependent_req)
> pkg_resources.ContextualVersionConflict: (tzlocal 2.0.0b1
> (/python-packages/pkg/pkg/.eggs/tzlocal-2.0.0b1-py2.7.egg),
> Requirement.parse('tzlocal<2.0.0.0,>=1.5.0.0'), set(['pendulum']))
> {code}
> It would seem that Airflow pins ancient versions here:
> -
> https://github.com/apache/incubator-airflow/blob/314232cc49ebe75cf12e58af9588123441976647/setup.py#L316
> For example:
> psutil is at 5.4.7
> pygments is at 2.2.0
> pendulum is at 2.0.3
> All of these install fine with their latest versions together in a fresh
> 2.7.15 Docker container.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
