[jira] [Commented] (AIRFLOW-4098) gcp_api dependency conflicts
[
https://issues.apache.org/jira/browse/AIRFLOW-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16854548#comment-16854548
]
Jarek Potiuk commented on AIRFLOW-4098:
---
Fixed by AIRFLOW-4546
> gcp_api dependency conflicts
>
>
> Key: AIRFLOW-4098
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4098
> Project: Apache Airflow
> Issue Type: Bug
> Components: gcp
>Affects Versions: 1.10.2
> Environment: Ubuntu 16.04.6 LTS
> Python 2.7 / Python 3.5 / Python 3.6
>Reporter: Manuel Rodríguez Guimeráns
>Priority: Major
> Fix For: 2.0.0
>
>
> The {{gcp_api}} extra contains inconsistencies in its subdependencies. When
> installed with {{pip}} it completes with a warning, but {{pipenv}} and
> {{poetry}} are not so forgiving and it cannot be installed at all.
> Note that on the master branch as of 419e30 (where {{gcp_api}} is now called
> just {{gcp}}) there are still dependency errors as well.
> h3. pip with python 2.7
> {code:java}
> $ mkvirtualenv -p /usr/bin/python2.7 airflow_pip
> ...
> $ pip --version
> pip 19.0.3 from
> /home/manu/.local/share/virtualenvs/airflow_pip/local/lib/python2.7/site-packages/pip
> (python 2.7)
> $ SLUGIFY_USES_TEXT_UNIDECODE=yes pip install
> 'apache-airflow[gcp_api]==1.10.2'
> ...
> google-cloud-bigquery 1.10.0 has requirement
> google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1
> which is incompatible.
> google-cloud-spanner 1.8.0 has requirement
> google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1
> which is incompatible.
> ...
> {code}
> h3. pipenv with python 3.6
> {code:java}
> $ mkdir airflow_pipenv
> $ cd airflow_pipenv
> $ pipenv --version
> pipenv, version 2018.11.26
> $ pipenv --python 3.6
> Creating a virtualenv for this project...
> Pipfile: /tmp/airflow_pipenv/Pipfile
> Using /home/manu/.pyenv/versions/3.6.6/bin/python3.6 (3.6.6) to create
> virtualenv...
> ⠋ Creating virtual environment...Already using interpreter
> /home/manu/.pyenv/versions/3.6.6/bin/python3.6
> Using base prefix '/home/manu/.pyenv/versions/3.6.6'
> New python executable in
> /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python3.6
> Also creating executable in
> /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python
> Installing setuptools, pip, wheel...done.
> ✔ Successfully created virtual environment!
> Virtualenv location:
> /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO
> $ airflow_pipenv pipenv install 'apache-airflow[gcp_api]==1.10.2'
> Installing apache-airflow[gcp_api]==1.10.2...
> Adding apache-airflow to Pipfile's [packages]...
> ✔ Installation Succeeded
> Pipfile.lock (e3de4b) out of date, updating to (ca72e7)...
> Locking [dev-packages] dependencies...
> Locking [packages] dependencies...
> ✘ Locking Failed!
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/resolver.py",
> line 69, in resolve
> [pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py",
> line 726, in resolve_deps
> [pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py",
> line 480, in actually_resolve_deps
> [pipenv.exceptions.ResolutionFailure]: resolved_tree =
> resolver.resolve()
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py",
> line 395, in resolve
> [pipenv.exceptions.ResolutionFailure]: raise
> ResolutionFailure(message=str(e))
> [pipenv.exceptions.ResolutionFailure]:
> pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version
> that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0
> [pipenv.exceptions.ResolutionFailure]: Tried: 0.20.0, 0.20.0, 0.21.0,
> 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1,
> 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0,
> 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0,
> 0.29.1, 0.29.1
> [pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not
> be resolved. You likely have a mismatch in your sub-dependencies.
> First try clearing your dependency cache with $ pipenv lock --clear, then
> try the original command again.
> Alternatively, you can use $ pipenv install --skip-lock to bypass this
> mechanism, then run $ pipenv graph to inspect the situation.
> Hint: try $ pipenv lock --pre if it is a pre-release dependency.
[jira] [Commented] (AIRFLOW-4098) gcp_api dependency conflicts
[
https://issues.apache.org/jira/browse/AIRFLOW-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16836518#comment-16836518
]
Zixun Zhang commented on AIRFLOW-4098:
--
Experience the same issue on 1.10.2
ERROR: google-cloud-spanner 1.8.0 has requirement
google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1
which is incompatible.
ERROR: google-cloud-bigquery 1.11.2 has requirement
google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1
which is incompatible.
> gcp_api dependency conflicts
>
>
> Key: AIRFLOW-4098
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4098
> Project: Apache Airflow
> Issue Type: Bug
> Components: gcp
>Affects Versions: 1.10.2
> Environment: Ubuntu 16.04.6 LTS
> Python 2.7 / Python 3.5 / Python 3.6
>Reporter: Manuel Rodríguez Guimeráns
>Priority: Major
>
> The {{gcp_api}} extra contains inconsistencies in its subdependencies. When
> installed with {{pip}} it completes with a warning, but {{pipenv}} and
> {{poetry}} are not so forgiving and it cannot be installed at all.
> Note that on the master branch as of 419e30 (where {{gcp_api}} is now called
> just {{gcp}}) there are still dependency errors as well.
> h3. pip with python 2.7
> {code:java}
> $ mkvirtualenv -p /usr/bin/python2.7 airflow_pip
> ...
> $ pip --version
> pip 19.0.3 from
> /home/manu/.local/share/virtualenvs/airflow_pip/local/lib/python2.7/site-packages/pip
> (python 2.7)
> $ SLUGIFY_USES_TEXT_UNIDECODE=yes pip install
> 'apache-airflow[gcp_api]==1.10.2'
> ...
> google-cloud-bigquery 1.10.0 has requirement
> google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1
> which is incompatible.
> google-cloud-spanner 1.8.0 has requirement
> google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1
> which is incompatible.
> ...
> {code}
> h3. pipenv with python 3.6
> {code:java}
> $ mkdir airflow_pipenv
> $ cd airflow_pipenv
> $ pipenv --version
> pipenv, version 2018.11.26
> $ pipenv --python 3.6
> Creating a virtualenv for this project...
> Pipfile: /tmp/airflow_pipenv/Pipfile
> Using /home/manu/.pyenv/versions/3.6.6/bin/python3.6 (3.6.6) to create
> virtualenv...
> ⠋ Creating virtual environment...Already using interpreter
> /home/manu/.pyenv/versions/3.6.6/bin/python3.6
> Using base prefix '/home/manu/.pyenv/versions/3.6.6'
> New python executable in
> /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python3.6
> Also creating executable in
> /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python
> Installing setuptools, pip, wheel...done.
> ✔ Successfully created virtual environment!
> Virtualenv location:
> /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO
> $ airflow_pipenv pipenv install 'apache-airflow[gcp_api]==1.10.2'
> Installing apache-airflow[gcp_api]==1.10.2...
> Adding apache-airflow to Pipfile's [packages]...
> ✔ Installation Succeeded
> Pipfile.lock (e3de4b) out of date, updating to (ca72e7)...
> Locking [dev-packages] dependencies...
> Locking [packages] dependencies...
> ✘ Locking Failed!
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/resolver.py",
> line 69, in resolve
> [pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py",
> line 726, in resolve_deps
> [pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py",
> line 480, in actually_resolve_deps
> [pipenv.exceptions.ResolutionFailure]: resolved_tree =
> resolver.resolve()
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py",
> line 395, in resolve
> [pipenv.exceptions.ResolutionFailure]: raise
> ResolutionFailure(message=str(e))
> [pipenv.exceptions.ResolutionFailure]:
> pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version
> that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0
> [pipenv.exceptions.ResolutionFailure]: Tried: 0.20.0, 0.20.0, 0.21.0,
> 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1,
> 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0,
> 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0,
> 0.29.1, 0.29.1
> [pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not
> be resolved. You likely have a mismatch in your sub-dependencies.
> First try clearing your
[jira] [Commented] (AIRFLOW-4098) gcp_api dependency conflicts
[
https://issues.apache.org/jira/browse/AIRFLOW-4098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16827941#comment-16827941
]
jack commented on AIRFLOW-4098:
---
[~kaxilnaik] have you encountered this?
> gcp_api dependency conflicts
>
>
> Key: AIRFLOW-4098
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4098
> Project: Apache Airflow
> Issue Type: Bug
> Components: gcp
>Affects Versions: 1.10.2
> Environment: Ubuntu 16.04.6 LTS
> Python 2.7 / Python 3.5 / Python 3.6
>Reporter: Manuel Rodríguez Guimeráns
>Priority: Major
>
> The {{gcp_api}} extra contains inconsistencies in its subdependencies. When
> installed with {{pip}} it completes with a warning, but {{pipenv}} and
> {{poetry}} are not so forgiving and it cannot be installed at all.
> Note that on the master branch as of 419e30 (where {{gcp_api}} is now called
> just {{gcp}}) there are still dependency errors as well.
> h3. pip with python 2.7
> {code:java}
> $ mkvirtualenv -p /usr/bin/python2.7 airflow_pip
> ...
> $ pip --version
> pip 19.0.3 from
> /home/manu/.local/share/virtualenvs/airflow_pip/local/lib/python2.7/site-packages/pip
> (python 2.7)
> $ SLUGIFY_USES_TEXT_UNIDECODE=yes pip install
> 'apache-airflow[gcp_api]==1.10.2'
> ...
> google-cloud-bigquery 1.10.0 has requirement
> google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1
> which is incompatible.
> google-cloud-spanner 1.8.0 has requirement
> google-cloud-core<0.30dev,>=0.29.0, but you'll have google-cloud-core 0.28.1
> which is incompatible.
> ...
> {code}
> h3. pipenv with python 3.6
> {code:java}
> $ mkdir airflow_pipenv
> $ cd airflow_pipenv
> $ pipenv --version
> pipenv, version 2018.11.26
> $ pipenv --python 3.6
> Creating a virtualenv for this project...
> Pipfile: /tmp/airflow_pipenv/Pipfile
> Using /home/manu/.pyenv/versions/3.6.6/bin/python3.6 (3.6.6) to create
> virtualenv...
> ⠋ Creating virtual environment...Already using interpreter
> /home/manu/.pyenv/versions/3.6.6/bin/python3.6
> Using base prefix '/home/manu/.pyenv/versions/3.6.6'
> New python executable in
> /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python3.6
> Also creating executable in
> /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO/bin/python
> Installing setuptools, pip, wheel...done.
> ✔ Successfully created virtual environment!
> Virtualenv location:
> /home/manu/.local/share/virtualenvs/airflow_pipenv-ePWbOumO
> $ airflow_pipenv pipenv install 'apache-airflow[gcp_api]==1.10.2'
> Installing apache-airflow[gcp_api]==1.10.2...
> Adding apache-airflow to Pipfile's [packages]...
> ✔ Installation Succeeded
> Pipfile.lock (e3de4b) out of date, updating to (ca72e7)...
> Locking [dev-packages] dependencies...
> Locking [packages] dependencies...
> ✘ Locking Failed!
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/resolver.py",
> line 69, in resolve
> [pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py",
> line 726, in resolve_deps
> [pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py",
> line 480, in actually_resolve_deps
> [pipenv.exceptions.ResolutionFailure]: resolved_tree =
> resolver.resolve()
> [pipenv.exceptions.ResolutionFailure]: File
> "/home/manu/.pyenv/versions/3.6.6/lib/python3.6/site-packages/pipenv/utils.py",
> line 395, in resolve
> [pipenv.exceptions.ResolutionFailure]: raise
> ResolutionFailure(message=str(e))
> [pipenv.exceptions.ResolutionFailure]:
> pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version
> that matches google-cloud-core<0.29dev,<0.30dev,>=0.28.0,>=0.29.0
> [pipenv.exceptions.ResolutionFailure]: Tried: 0.20.0, 0.20.0, 0.21.0,
> 0.21.0, 0.22.0, 0.22.0, 0.22.1, 0.22.1, 0.23.0, 0.23.0, 0.23.1, 0.23.1,
> 0.24.0, 0.24.0, 0.24.1, 0.24.1, 0.25.0, 0.25.0, 0.26.0, 0.26.0, 0.27.0,
> 0.27.0, 0.27.1, 0.27.1, 0.28.0, 0.28.0, 0.28.1, 0.28.1, 0.29.0, 0.29.0,
> 0.29.1, 0.29.1
> [pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not
> be resolved. You likely have a mismatch in your sub-dependencies.
> First try clearing your dependency cache with $ pipenv lock --clear, then
> try the original command again.
> Alternatively, you can use $ pipenv install --skip-lock to bypass this
> mechanism, then run $ pipenv graph to inspect the situation.
> Hint: try $ pipenv lock --pre if it is a pre-release dependency.
> ERROR: ERROR: Could not
