xinbinhuang commented on a change in pull request #6942: [AIRFLOW-XXXX] Fix development packages installtion instructions URL: https://github.com/apache/airflow/pull/6942#discussion_r361909277
########## File path: LOCAL_VIRTUALENV.rst ########## @@ -91,26 +91,47 @@ To use your IDE for Airflow development and testing, you need to configure a vir environment. Ideally you should set up virtualenv for all Python versions that Airflow supports (3.5, 3.6). -Consider using one of the following utilities to create virtual environments and easily -switch between them with the ``workon`` command: +To create and initialize the local virtualenv: -- `pyenv <https://github.com/pyenv/pyenv>`_ -- `pyenv-virtualenv <https://github.com/pyenv/pyenv-virtualenv>`_ -- `virtualenvwrapper <https://virtualenvwrapper.readthedocs.io/en/latest/>`_ +1. Create an environment with one of the two options: -To create and initialize the local virtualenv: + - Option 1: consider using one of the following utilities to create virtual environments and easily switch between them with the ``workon`` command: + + - `pyenv <https://github.com/pyenv/pyenv>`_ + - `pyenv-virtualenv <https://github.com/pyenv/pyenv-virtualenv>`_ + - `virtualenvwrapper <https://virtualenvwrapper.readthedocs.io/en/latest/>`_ + + ``mkvirtualenv <ENV_NAME> --python=python<VERSION>`` + + - Option 2: create a local virtualenv with Conda -1. Create an environment as follows: + - install `miniconda3 <https://docs.conda.io/en/latest/miniconda.html>`_ - ``mkvirtualenv <ENV_NAME> --python=python<VERSION>`` + .. code-block:: bash + + conda create -n airflow python=3.6 + conda activate airflow 2. Install Python PIP requirements: + + .. code-block:: bash + + pip install -U -e ".[devel,<OTHER EXTRAS>]" # for example: pip install -U -e ".[devel,gcp,postgres]" + +Note: when you first initialize database (the next step), airflow by default will try to load in examples dags where +some of them requires dependencies ``gcp`` and ``postgres``. You can solve the problem by - ``pip install -e ".[devel]"`` +- installing the extras i.e. ``[devel,gcp,postgres]`` or +- disable the example dags with environment variable: ``export AIRFLOW__CORE__LOAD_EXAMPLES=False`` or +- simply ignore the error messages and proceed Review comment: Actually, the problem I am referring to here is not related to the migration: there are some example dags (i.e. [example_gcs_to_bq.py](https://github.com/apache/airflow/blob/master/airflow/example_dags/example_gcs_to_bq.py) or [example_postgres_to_gcs.py](https://github.com/apache/airflow/blob/master/airflow/example_dags/example_postgres_to_gcs.py) ) require the `gcp`/`postgres` extra to be installed. When we do `airflow db init`, by default it will try to load in the example dags, but it will raise `ModuleNotFoundError` when parsing these example dags. This also happens when we do `airflow webserver/scheduler`. The note here is to notify this behaviors to contributors that are not familiar with Airflow. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
