amoghrajesh commented on code in PR #43604:
URL: https://github.com/apache/airflow/pull/43604#discussion_r1826559014
##########
contributing-docs/07_local_virtualenv.rst:
##########
@@ -388,8 +314,8 @@ run the command above and commit the changes to
``pyproject.toml``. Then running
install the dependencies automatically when you create or switch to a
development environment.
-Installing recommended version of dependencies
-----------------------------------------------
+Installing "golden" version of dependencies
+-------------------------------------------
Whatever virtualenv solution you use, when you want to make sure you are using
the same
version of dependencies as in main, you can install recommended version of the
dependencies by using
Review Comment:
```suggestion
version of dependencies as in main, you can install recommended version of
the dependencies by using pip:
```
##########
hatch_build.py:
##########
@@ -101,7 +101,7 @@
"python-ldap>=3.4.4",
],
"leveldb": [
- "plyvel>=1.5.1",
+ "plyvel>=1.5.1; sys_platform != 'darwin'",
Review Comment:
Not so sure why this is needed
##########
contributing-docs/07_local_virtualenv.rst:
##########
@@ -61,26 +62,114 @@ of required packages.
released wheel packages.
-Installing Airflow
-..................
+Creating and maintaining local virtualenv with uv
+-------------------------------------------------
+
+As of November 2024 we are recommending to use ``uv`` for local virtualenv
management for Airflow development.
+Th e``uv`` is a build frontend tool that is designed to manage python,
virtualenvs and workspaces for development
+and testing of Python projects. It is a modern tool that is designed to work
with PEP 517/518 compliant projects
+and it is much faster than "reference" ``pip`` tool. It has extensive support
to not only create development
+environment but also to manage python versions, development environments,
workspaces and Python tools used
+to develop Airflow (via ``uv tool`` command - such as ``pre-commit`` and
others, you can also use ``uv tool``
+to install ``breeze`` - containerized development environment that we use to
be able to reproduce the
+CI environment locally and to run release-management and certain development
tasks.
+
+You can read more about ``uv`` in `UV Getting started
<https://docs.astral.sh/uv/getting-started/>`_ but
+below you will find a few typical steps to get you started with ``uv``.
+
+Installing uv
+.............
+
+You can follow the `installation instructions
<https://docs.astral.sh/uv/getting-started/installation/>`_ to install
+``uv`` on your system. Once you have ``uv`` installed, you can do all the
environment preparation tasks using
+``uv`` commands.
+
+Installing Python versions
+..........................
+
+You can install Python versions using ``uv python install`` command. For
example, to install Python 3.9.7, you can run:
+
+.. code:: bash
+
+ uv python install 3.9.7
+
+This is optional step - UV will automatically install the Python version you
need when you create a virtualenv.
+
+Creating virtualenvs with uv
+............................
+
+.. code:: bash
+
+ uv venv
+
+This will create a default venv in your project's ``.venv`` directory. You can
also create a venv
+with a specific Python version by running:
+
+.. code:: bash
+
+ uv venv --python 3.9.7
-The simplest way to install Airflow in local virtualenv is to use ``pip``:
+You can also create a venv with a different folder name by running:
.. code:: bash
- pip install -e ".[devel,<OTHER EXTRAS>]" # for example: pip install -e
".[devel,google,postgres]"
+ uv venv .my-venv
+
+However UV creation/re-creation of venvs is so fast that you can easily create
and delete venvs as needed.
+So usually you do not need to have more than one venv and recreate it as
needed - for example when you
+need to change python version
+
+Syncing project (including providers) with uv
+.............................................
+
+In a project like airflow it's important to have a consistent set of
dependencies across all developers.
+You can use ``uv sync`` to install dependencies from ``pyproject.toml`` file.
This will install all dependencies
+from the ``pyproject.toml`` file in the current directory.
+
+.. code:: bash
+
+ uv sync
+
+If you also need to install development and provider dependencies you can
specify extras for that providers:
+
+.. code:: bash
+
+ uv sync --extra devel --extra devel-tests --extra google
+
+This will synchronize all extras that you need for development and testing of
Airflow and google provider
+dependencies - including their runtime dependencies.
+
+.. code:: bash
+
+ uv sync --all-extras
Review Comment:
So its graphviz + pkg-conf
##########
contributing-docs/07_local_virtualenv.rst:
##########
@@ -61,26 +62,114 @@ of required packages.
released wheel packages.
-Installing Airflow
-..................
+Creating and maintaining local virtualenv with uv
+-------------------------------------------------
+
+As of November 2024 we are recommending to use ``uv`` for local virtualenv
management for Airflow development.
+Th e``uv`` is a build frontend tool that is designed to manage python,
virtualenvs and workspaces for development
+and testing of Python projects. It is a modern tool that is designed to work
with PEP 517/518 compliant projects
+and it is much faster than "reference" ``pip`` tool. It has extensive support
to not only create development
+environment but also to manage python versions, development environments,
workspaces and Python tools used
+to develop Airflow (via ``uv tool`` command - such as ``pre-commit`` and
others, you can also use ``uv tool``
+to install ``breeze`` - containerized development environment that we use to
be able to reproduce the
+CI environment locally and to run release-management and certain development
tasks.
+
+You can read more about ``uv`` in `UV Getting started
<https://docs.astral.sh/uv/getting-started/>`_ but
+below you will find a few typical steps to get you started with ``uv``.
+
+Installing uv
+.............
+
+You can follow the `installation instructions
<https://docs.astral.sh/uv/getting-started/installation/>`_ to install
+``uv`` on your system. Once you have ``uv`` installed, you can do all the
environment preparation tasks using
+``uv`` commands.
+
+Installing Python versions
+..........................
+
+You can install Python versions using ``uv python install`` command. For
example, to install Python 3.9.7, you can run:
+
+.. code:: bash
+
+ uv python install 3.9.7
+
+This is optional step - UV will automatically install the Python version you
need when you create a virtualenv.
+
+Creating virtualenvs with uv
+............................
+
+.. code:: bash
+
+ uv venv
Review Comment:
How does it otherwise detect the version?
##########
contributing-docs/07_local_virtualenv.rst:
##########
@@ -61,26 +62,114 @@ of required packages.
released wheel packages.
-Installing Airflow
-..................
+Creating and maintaining local virtualenv with uv
+-------------------------------------------------
+
+As of November 2024 we are recommending to use ``uv`` for local virtualenv
management for Airflow development.
+Th e``uv`` is a build frontend tool that is designed to manage python,
virtualenvs and workspaces for development
Review Comment:
```suggestion
The ``uv`` utility is a build frontend tool that is designed to manage
python, virtualenvs and workspaces for development
```
##########
contributing-docs/07_local_virtualenv.rst:
##########
@@ -61,26 +62,114 @@ of required packages.
released wheel packages.
-Installing Airflow
-..................
+Creating and maintaining local virtualenv with uv
+-------------------------------------------------
+
+As of November 2024 we are recommending to use ``uv`` for local virtualenv
management for Airflow development.
+Th e``uv`` is a build frontend tool that is designed to manage python,
virtualenvs and workspaces for development
+and testing of Python projects. It is a modern tool that is designed to work
with PEP 517/518 compliant projects
+and it is much faster than "reference" ``pip`` tool. It has extensive support
to not only create development
+environment but also to manage python versions, development environments,
workspaces and Python tools used
+to develop Airflow (via ``uv tool`` command - such as ``pre-commit`` and
others, you can also use ``uv tool``
+to install ``breeze`` - containerized development environment that we use to
be able to reproduce the
Review Comment:
```suggestion
to install ``breeze`` - containerized development environment for Airflow
that we use to reproduce the
```
##########
contributing-docs/07_local_virtualenv.rst:
##########
@@ -61,26 +62,114 @@ of required packages.
released wheel packages.
-Installing Airflow
-..................
+Creating and maintaining local virtualenv with uv
+-------------------------------------------------
+
+As of November 2024 we are recommending to use ``uv`` for local virtualenv
management for Airflow development.
+Th e``uv`` is a build frontend tool that is designed to manage python,
virtualenvs and workspaces for development
+and testing of Python projects. It is a modern tool that is designed to work
with PEP 517/518 compliant projects
+and it is much faster than "reference" ``pip`` tool. It has extensive support
to not only create development
+environment but also to manage python versions, development environments,
workspaces and Python tools used
+to develop Airflow (via ``uv tool`` command - such as ``pre-commit`` and
others, you can also use ``uv tool``
+to install ``breeze`` - containerized development environment that we use to
be able to reproduce the
+CI environment locally and to run release-management and certain development
tasks.
+
+You can read more about ``uv`` in `UV Getting started
<https://docs.astral.sh/uv/getting-started/>`_ but
+below you will find a few typical steps to get you started with ``uv``.
+
+Installing uv
+.............
+
+You can follow the `installation instructions
<https://docs.astral.sh/uv/getting-started/installation/>`_ to install
+``uv`` on your system. Once you have ``uv`` installed, you can do all the
environment preparation tasks using
+``uv`` commands.
+
+Installing Python versions
+..........................
+
+You can install Python versions using ``uv python install`` command. For
example, to install Python 3.9.7, you can run:
+
+.. code:: bash
+
+ uv python install 3.9.7
+
+This is optional step - UV will automatically install the Python version you
need when you create a virtualenv.
+
+Creating virtualenvs with uv
+............................
+
+.. code:: bash
+
+ uv venv
+
+This will create a default venv in your project's ``.venv`` directory. You can
also create a venv
+with a specific Python version by running:
+
+.. code:: bash
+
+ uv venv --python 3.9.7
-The simplest way to install Airflow in local virtualenv is to use ``pip``:
+You can also create a venv with a different folder name by running:
Review Comment:
```suggestion
You can also create a venv with a different venv directory name by running:
```
##########
contributing-docs/07_local_virtualenv.rst:
##########
@@ -61,26 +62,114 @@ of required packages.
released wheel packages.
-Installing Airflow
-..................
+Creating and maintaining local virtualenv with uv
+-------------------------------------------------
+
+As of November 2024 we are recommending to use ``uv`` for local virtualenv
management for Airflow development.
+Th e``uv`` is a build frontend tool that is designed to manage python,
virtualenvs and workspaces for development
+and testing of Python projects. It is a modern tool that is designed to work
with PEP 517/518 compliant projects
+and it is much faster than "reference" ``pip`` tool. It has extensive support
to not only create development
+environment but also to manage python versions, development environments,
workspaces and Python tools used
+to develop Airflow (via ``uv tool`` command - such as ``pre-commit`` and
others, you can also use ``uv tool``
+to install ``breeze`` - containerized development environment that we use to
be able to reproduce the
+CI environment locally and to run release-management and certain development
tasks.
+
+You can read more about ``uv`` in `UV Getting started
<https://docs.astral.sh/uv/getting-started/>`_ but
+below you will find a few typical steps to get you started with ``uv``.
+
+Installing uv
+.............
+
+You can follow the `installation instructions
<https://docs.astral.sh/uv/getting-started/installation/>`_ to install
+``uv`` on your system. Once you have ``uv`` installed, you can do all the
environment preparation tasks using
+``uv`` commands.
+
+Installing Python versions
+..........................
+
+You can install Python versions using ``uv python install`` command. For
example, to install Python 3.9.7, you can run:
+
+.. code:: bash
+
+ uv python install 3.9.7
+
+This is optional step - UV will automatically install the Python version you
need when you create a virtualenv.
+
+Creating virtualenvs with uv
+............................
+
+.. code:: bash
+
+ uv venv
+
+This will create a default venv in your project's ``.venv`` directory. You can
also create a venv
+with a specific Python version by running:
+
+.. code:: bash
+
+ uv venv --python 3.9.7
-The simplest way to install Airflow in local virtualenv is to use ``pip``:
+You can also create a venv with a different folder name by running:
.. code:: bash
- pip install -e ".[devel,<OTHER EXTRAS>]" # for example: pip install -e
".[devel,google,postgres]"
+ uv venv .my-venv
+
+However UV creation/re-creation of venvs is so fast that you can easily create
and delete venvs as needed.
+So usually you do not need to have more than one venv and recreate it as
needed - for example when you
+need to change python version
+
+Syncing project (including providers) with uv
+.............................................
+
+In a project like airflow it's important to have a consistent set of
dependencies across all developers.
+You can use ``uv sync`` to install dependencies from ``pyproject.toml`` file.
This will install all dependencies
+from the ``pyproject.toml`` file in the current directory.
+
+.. code:: bash
+
+ uv sync
+
+If you also need to install development and provider dependencies you can
specify extras for that providers:
+
+.. code:: bash
+
+ uv sync --extra devel --extra devel-tests --extra google
+
+This will synchronize all extras that you need for development and testing of
Airflow and google provider
+dependencies - including their runtime dependencies.
+
+.. code:: bash
+
+ uv sync --all-extras
Review Comment:
Tried this and only pygraphviz fails:
```
[stderr]
/Users/amoghdesai/.cache/uv/builds-v0/.tmpScysKG/lib/python3.9/site-packages/setuptools/_distutils/dist.py:261:
UserWarning: Unknown distribution option: 'tests_require'
warnings.warn(msg)
warning: no files found matching '*.png' under directory 'doc'
warning: no files found matching '*.txt' under directory 'doc'
warning: no files found matching '*.css' under directory 'doc'
warning: no previously-included files matching '*~' found anywhere in
distribution
warning: no previously-included files matching '*.pyc' found anywhere in
distribution
warning: no previously-included files matching '.svn' found anywhere in
distribution
no previously-included directories found matching 'doc/build'
pygraphviz/graphviz_wrap.c:3020:10: fatal error: 'graphviz/cgraph.h' file
not found
#include "graphviz/cgraph.h"
^~~~~~~~~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit code 1
Caused by: This error likely indicates that you need to install a library
that provides "graphviz/cgraph.h" for [email protected]
```
We should add that graphviz will be needed here
##########
contributing-docs/07_local_virtualenv.rst:
##########
@@ -61,26 +62,114 @@ of required packages.
released wheel packages.
-Installing Airflow
-..................
+Creating and maintaining local virtualenv with uv
+-------------------------------------------------
+
+As of November 2024 we are recommending to use ``uv`` for local virtualenv
management for Airflow development.
+Th e``uv`` is a build frontend tool that is designed to manage python,
virtualenvs and workspaces for development
+and testing of Python projects. It is a modern tool that is designed to work
with PEP 517/518 compliant projects
+and it is much faster than "reference" ``pip`` tool. It has extensive support
to not only create development
+environment but also to manage python versions, development environments,
workspaces and Python tools used
+to develop Airflow (via ``uv tool`` command - such as ``pre-commit`` and
others, you can also use ``uv tool``
+to install ``breeze`` - containerized development environment that we use to
be able to reproduce the
+CI environment locally and to run release-management and certain development
tasks.
+
+You can read more about ``uv`` in `UV Getting started
<https://docs.astral.sh/uv/getting-started/>`_ but
+below you will find a few typical steps to get you started with ``uv``.
+
+Installing uv
+.............
+
+You can follow the `installation instructions
<https://docs.astral.sh/uv/getting-started/installation/>`_ to install
+``uv`` on your system. Once you have ``uv`` installed, you can do all the
environment preparation tasks using
+``uv`` commands.
+
+Installing Python versions
+..........................
+
+You can install Python versions using ``uv python install`` command. For
example, to install Python 3.9.7, you can run:
+
+.. code:: bash
+
+ uv python install 3.9.7
+
+This is optional step - UV will automatically install the Python version you
need when you create a virtualenv.
+
+Creating virtualenvs with uv
+............................
+
+.. code:: bash
+
+ uv venv
+
+This will create a default venv in your project's ``.venv`` directory. You can
also create a venv
+with a specific Python version by running:
+
+.. code:: bash
+
+ uv venv --python 3.9.7
-The simplest way to install Airflow in local virtualenv is to use ``pip``:
+You can also create a venv with a different folder name by running:
.. code:: bash
- pip install -e ".[devel,<OTHER EXTRAS>]" # for example: pip install -e
".[devel,google,postgres]"
+ uv venv .my-venv
+
+However UV creation/re-creation of venvs is so fast that you can easily create
and delete venvs as needed.
+So usually you do not need to have more than one venv and recreate it as
needed - for example when you
+need to change python version
Review Comment:
```suggestion
need to change the python version.
```
##########
contributing-docs/07_local_virtualenv.rst:
##########
@@ -61,26 +62,114 @@ of required packages.
released wheel packages.
-Installing Airflow
-..................
+Creating and maintaining local virtualenv with uv
+-------------------------------------------------
+
+As of November 2024 we are recommending to use ``uv`` for local virtualenv
management for Airflow development.
+Th e``uv`` is a build frontend tool that is designed to manage python,
virtualenvs and workspaces for development
+and testing of Python projects. It is a modern tool that is designed to work
with PEP 517/518 compliant projects
+and it is much faster than "reference" ``pip`` tool. It has extensive support
to not only create development
+environment but also to manage python versions, development environments,
workspaces and Python tools used
+to develop Airflow (via ``uv tool`` command - such as ``pre-commit`` and
others, you can also use ``uv tool``
+to install ``breeze`` - containerized development environment that we use to
be able to reproduce the
+CI environment locally and to run release-management and certain development
tasks.
+
+You can read more about ``uv`` in `UV Getting started
<https://docs.astral.sh/uv/getting-started/>`_ but
+below you will find a few typical steps to get you started with ``uv``.
+
+Installing uv
+.............
+
+You can follow the `installation instructions
<https://docs.astral.sh/uv/getting-started/installation/>`_ to install
+``uv`` on your system. Once you have ``uv`` installed, you can do all the
environment preparation tasks using
+``uv`` commands.
+
+Installing Python versions
+..........................
+
+You can install Python versions using ``uv python install`` command. For
example, to install Python 3.9.7, you can run:
+
+.. code:: bash
+
+ uv python install 3.9.7
+
+This is optional step - UV will automatically install the Python version you
need when you create a virtualenv.
+
+Creating virtualenvs with uv
+............................
+
+.. code:: bash
+
+ uv venv
+
+This will create a default venv in your project's ``.venv`` directory. You can
also create a venv
+with a specific Python version by running:
+
+.. code:: bash
+
+ uv venv --python 3.9.7
-The simplest way to install Airflow in local virtualenv is to use ``pip``:
+You can also create a venv with a different folder name by running:
.. code:: bash
- pip install -e ".[devel,<OTHER EXTRAS>]" # for example: pip install -e
".[devel,google,postgres]"
+ uv venv .my-venv
+
+However UV creation/re-creation of venvs is so fast that you can easily create
and delete venvs as needed.
+So usually you do not need to have more than one venv and recreate it as
needed - for example when you
+need to change python version
+
+Syncing project (including providers) with uv
+.............................................
+
+In a project like airflow it's important to have a consistent set of
dependencies across all developers.
+You can use ``uv sync`` to install dependencies from ``pyproject.toml`` file.
This will install all dependencies
+from the ``pyproject.toml`` file in the current directory.
+
+.. code:: bash
+
+ uv sync
+
+If you also need to install development and provider dependencies you can
specify extras for that providers:
+
+.. code:: bash
+
+ uv sync --extra devel --extra devel-tests --extra google
+
+This will synchronize all extras that you need for development and testing of
Airflow and google provider
+dependencies - including their runtime dependencies.
+
+.. code:: bash
+
+ uv sync --all-extras
Review Comment:
Okay pkg-config also:
```
[stdout]
Trying pkg-config --exists mysqlclient
Command 'pkg-config --exists mysqlclient' returned non-zero exit status 127.
Trying pkg-config --exists mariadb
Command 'pkg-config --exists mariadb' returned non-zero exit status 127.
Trying pkg-config --exists libmariadb
Command 'pkg-config --exists libmariadb' returned non-zero exit status 127.
Trying pkg-config --exists perconaserverclient
Command 'pkg-config --exists perconaserverclient' returned non-zero exit
status 127.
[stderr]
/bin/sh: pkg-config: command not found
/bin/sh: pkg-config: command not found
/bin/sh: pkg-config: command not found
/bin/sh: pkg-config: command not found
Traceback (most recent call last):
File "<string>", line 14, in <module>
File
"/Users/amoghdesai/.cache/uv/builds-v0/.tmpeV7Qf0/lib/python3.9/site-packages/setuptools/build_meta.py",
line 333, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=[])
File
"/Users/amoghdesai/.cache/uv/builds-v0/.tmpeV7Qf0/lib/python3.9/site-packages/setuptools/build_meta.py",
line 303, in _get_build_requires
self.run_setup()
File
"/Users/amoghdesai/.cache/uv/builds-v0/.tmpeV7Qf0/lib/python3.9/site-packages/setuptools/build_meta.py",
line 319, in run_setup
exec(code, locals())
File "<string>", line 155, in <module>
File "<string>", line 49, in get_config_posix
File "<string>", line 28, in find_package_name
Exception: Can not find valid pkg-config name.
Specify MYSQLCLIENT_CFLAGS and MYSQLCLIENT_LDFLAGS env vars manually
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
