[ https://issues.apache.org/jira/browse/AIRFLOW-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16854548#comment-16854548 ]
Jarek Potiuk commented on AIRFLOW-4098: --------------------------------------- Fixed by AIRFLOW-4546 > gcp_api dependency conflicts > ---------------------------- > > Key: AIRFLOW-4098 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4098 > Project: Apache Airflow > Issue Type: Bug > Components: gcp > Affects Versions: 1.10.2 > Environment: Ubuntu 16.04.6 LTS > Python 2.7 / Python 3.5 / Python 3.6 > Reporter: Manuel Rodríguez Guimeráns > Priority: Major > Fix For: 2.0.0 > > > The {{gcp_api}} extra contains inconsistencies in its subdependencies. When > installed with {{pip}} it completes with a warning, but {{pipenv}} and > {{poetry}} are not so forgiving and it cannot be installed at all. > Note that on the master branch as of 419e30 (where {{gcp_api}} is now called > just {{gcp}}) there are still dependency errors as well. > h3. pip with python 2.7 > {code:java} > $ mkvirtualenv -p /usr/bin/python2.7 airflow_pip > ... > $ pip --version > pip 19.0.3 from > /home/manu/.local/share/virtualenvs/airflow_pip/local/lib/python2.7/site-packages/pip > (python 2.7) > $ SLUGIFY_USES_TEXT_UNIDECODE=yes pip install > 'apache-airflow[gcp_api]==1.10.2' > ... > google-cloud-bigquery 1.10.0 has requirement > google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1 > which is incompatible. > google-cloud-spanner 1.8.0 has requirement > google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1 > which is incompatible. > ... > {code} > h3. pipenv with python 3.6 > {code:java} > $ mkdir airflow_pipenv > $ cd airflow_pipenv > $ pipenv --version > pipenv, version 2018.11.26 > $ pipenv --python 3.6 > Creating a virtualenv for this project... > Pipfile: /tmp/airflow_pipenv/Pipfile > Using /home/manu/.pyenv/versions/3.6.6/bin/python3.6 (3.6.6) to create > virtualenv... > ⠋ Creating virtual environment...Already using interpreter > /home/manu/.pyenv/versions/3.6.6/bin/python3.6 > Using base prefix '/home/manu/.pyenv/versions/3.6.6' > New python executable in > /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python3.6 > Also creating executable in > /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python > Installing setuptools, pip, wheel...done. > ✔ Successfully created virtual environment! > Virtualenv location: > /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO > $ airflow_pipenv pipenv install 'apache-airflow[gcp_api]==1.10.2' > Installing apache-airflow[gcp_api]==1.10.2... > Adding apache-airflow to Pipfile's [packages]... > ✔ Installation Succeeded > Pipfile.lock (e3de4b) out of date, updating to (ca72e7)... > Locking [dev-packages] dependencies... > Locking [packages] dependencies... > ✘ Locking Failed! > [pipenv.exceptions.ResolutionFailure]: File > "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/resolver.py", > line 69, in resolve > [pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir > [pipenv.exceptions.ResolutionFailure]: File > "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", > line 726, in resolve_deps > [pipenv.exceptions.ResolutionFailure]: req_dir=req_dir, > [pipenv.exceptions.ResolutionFailure]: File > "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", > line 480, in actually_resolve_deps > [pipenv.exceptions.ResolutionFailure]: resolved_tree = > resolver.resolve() > [pipenv.exceptions.ResolutionFailure]: File > "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", > line 395, in resolve > [pipenv.exceptions.ResolutionFailure]: raise > ResolutionFailure(message=str(e)) > [pipenv.exceptions.ResolutionFailure]: > pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version > that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0 > [pipenv.exceptions.ResolutionFailure]: Tried: 0.20.0, 0.20.0, 0.21.0, > 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1, > 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0, > 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0, > 0.29.1, 0.29.1 > [pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not > be resolved. You likely have a mismatch in your sub-dependencies. > First try clearing your dependency cache with $ pipenv lock --clear, then > try the original command again. > Alternatively, you can use $ pipenv install --skip-lock to bypass this > mechanism, then run $ pipenv graph to inspect the situation. > Hint: try $ pipenv lock --pre if it is a pre-release dependency. > ERROR: ERROR: Could not find a version that matches > google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0 > Tried: 0.20.0, 0.20.0, 0.21.0, 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, > 0.23.0, 0.23.0, 0.23.1, 0.23.1, 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, > 0.25.0, 0.26.0, 0.26.0, 0.27.0, 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, > 0.28.1, 0.28.1, 0.29.0, 0.29.0, 0.29.1, 0.29.1 > There are incompatible versions in the resolved dependencies. > [pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir > [pipenv.exceptions.ResolutionFailure]: File > "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", > line 726, in resolve_deps > [pipenv.exceptions.ResolutionFailure]: req_dir=req_dir, > [pipenv.exceptions.ResolutionFailure]: File > "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", > line 480, in actually_resolve_deps > [pipenv.exceptions.ResolutionFailure]: resolved_tree = > resolver.resolve() > [pipenv.exceptions.ResolutionFailure]: File > "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py", > line 395, in resolve > [pipenv.exceptions.ResolutionFailure]: raise > ResolutionFailure(message=str(e)) > [pipenv.exceptions.ResolutionFailure]: > pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version > that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0 > [pipenv.exceptions.ResolutionFailure]: Tried: 0.20.0, 0.20.0, 0.21.0, > 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1, > 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0, > 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0, > 0.29.1, 0.29.1 > [pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not > be resolved. You likely have a mismatch in your sub-dependencies. > First try clearing your dependency cache with $ pipenv lock --clear, then > try the original command again. > Alternatively, you can use $ pipenv install --skip-lock to bypass this > mechanism, then run $ pipenv graph to inspect the situation. > Hint: try $ pipenv lock --pre if it is a pre-release dependency. > ERROR: ERROR: Could not find a version that matches > google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0 > Tried: 0.20.0, 0.20.0, 0.21.0, 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, > 0.23.0, 0.23.0, 0.23.1, 0.23.1, 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, > 0.25.0, 0.26.0, 0.26.0, 0.27.0, 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, > 0.28.1, 0.28.1, 0.29.0, 0.29.0, 0.29.1, 0.29.1 > There are incompatible versions in the resolved dependencies. > {code} > h3. poetry with python 3.5 > {code:java} > $ pyenv shell 3.5.5 > $ poetry --version > Poetry 0.12.11 > $ poetry new airflow_poetry > Created package airflow_poetry in airflow_poetry > $ cd airflow_poetry > $ poetry add apache-airflow==1.10.2 -E gcp_api > Creating virtualenv airflow-poetry-py3.5 in > /home/manu/.cache/pypoetry/virtualenvs > Updating dependencies > Resolving dependencies... (3.0s) > > > > [SolverProblemError] > > > Because no versions of google-cloud-spanner match >1.7.1,<1.8.0 || >1.8.0 > > > and google-cloud-spanner (1.7.1) depends on google-cloud-core > (>=0.29.0,<0.30dev), google-cloud-spanner (>=1.7.1,<1.8.0 || >1.8.0) requires > google-cloud-core (>=0.29.0,<0.30dev). > And because google-cloud-spanner (1.8.0) depends on google-cloud-core > (>=0.29.0,<0.30dev), google-cloud-spanner (>=1.7.1) requires > google-cloud-core (>=0.29.0,<0.30dev). > > And because apache-airflow (1.10.2) depends on google-cloud-bigtable (0.31.0) > which depends on google-cloud-core (>=0.28.0,<0.29dev), google-cloud-spanner > (>=1.7.1) is incompatible with apache-airflow (1.10.2 > ). > > > So, because airflow-poetry depends on apache-airflow (=1.10.2) which depends > on google-cloud-spanner (>=1.7.1), version solving failed. > > > > > add [-D|--dev] [--git GIT] [--path PATH] [-E|--extras EXTRAS] [--optional] > [--python PYTHON] [--platform PLATFORM] [--allow-prereleases] [--dry-run] > [--] <name> (<name>)... > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)