potiuk commented on a change in pull request #6285: [AIRFLOW-XXX] Updates to
Breeze documentation from GSOD
URL: https://github.com/apache/airflow/pull/6285#discussion_r333987222
##########
File path: BREEZE.rst
##########
@@ -290,104 +275,85 @@ It is as easy as copy&pasting this line into your code:
import ipdb; ipdb.set_trace()
-Once you hit the line you will be dropped into interactive ipdb debugger
where you have colors
-and auto-completion to guide your debugging. This works from the console where
you started your program.
-Note that in case of ``nosetest`` you need to provide ``--nocapture`` flag to
avoid nosetests
+Once you hit the line, you will be dropped into an interactive ``ipdb``
debugger where you have colors
+and autocompletion to guide your debugging. This works from the console where
you started your program.
+Note that in case of ``nosetest`` you need to provide the ``--nocapture`` flag
to avoid nosetests
capturing the stdout of your process.
-Airflow directory structure inside Docker
------------------------------------------
-
-When you are in the container note that following directories are used:
-.. code-block:: text
-
- /opt/airflow - here sources of Airflow are mounted from the host
(AIRFLOW_SOURCES)
- /root/airflow - all the "dynamic" Airflow files are created here:
(AIRFLOW_HOME)
- airflow.db - sqlite database in case sqlite is used
- dags - folder where non-test dags are stored (test dags are in
/opt/airflow/tests/dags)
- logs - logs from airflow executions are created there
- unittest.cfg - unit test configuration generated when entering the
environment
- webserver_config.py - webserver configuration generated when running
airflow in the container
-
-Note that when run in your local environment ``/root/airflow/logs`` folder is
actually mounted from your
-``logs`` directory in airflow sources, so all logs created in the container
are automatically visible in the host
-as well. Every time you enter the container the logs directory is cleaned so
that logs do not accumulate.
-
-Port forwarding
+Port Forwarding
---------------
When you run Airflow Breeze, the following ports are automatically forwarded:
-* 28080 -> forwarded to airflow webserver -> airflow-testing:8080
-* 25433 -> forwarded to postgres database -> postgres:5432
-* 23306 -> forwarded to mysql database -> mysql:3306
+* 28080 -> forwarded to Airflow webserver -> airflow-testing:8080
+* 25433 -> forwarded to Postgres database -> postgres:5432
+* 23306 -> forwarded to Mysql database -> mysql:3306
-You can connect to those ports/databases using:
+You can connect to these ports/databases using:
* Webserver: ``http://127.0.0.1:28080``
* Postgres:
``jdbc:postgresql://127.0.0.1:25433/airflow?user=postgres&password=airflow``
* Mysql: ``jdbc:mysql://localhost:23306/airflow?user=root``
-Note that you need to start the webserver manually with ``airflow webserver``
command if you want to connect
-to the webserver (you can use ``tmux`` to multiply terminals).
+Start the webserver manually with the ``airflow webserver`` command if you
want to connect
+to the webserver. You can use ``tmux`` to multiply terminals.
-For databases you need to run ``airflow db reset`` at least once (or run some
tests) after you started
-Airflow Breeze to get the database/tables created. You can connect to databases
-with IDE or any other Database client:
+For databases, you need to run ``airflow db reset`` at least once (or run some
tests) after you started
+Airflow Breeze to get the database/tables created. You can connect to
databases with IDE or any other database client:
.. image:: images/database_view.png
:align: center
:alt: Database view
-You can change host port numbers used by setting appropriate environment
variables:
+You can change the used host port numbers by setting appropriate environment
variables:
* ``WEBSERVER_HOST_PORT``
* ``POSTGRES_HOST_PORT``
* ``MYSQL_HOST_PORT``
-When you set those variables, next time when you enter the environment the new
ports should be in effect.
+If you set these variables, next time when you enter the environment the new
ports should be in effect.
+
+Cleaning Up the Environment
+---------------------------
+
+You may need to clean up your Docker environment occasionally. The images are
quite big
+(1.5GB for both images needed for static code analysis and CI tests) and, if
you often rebuild/update
+them, you may end up with some unused image data.
+
+To clean up the Docker environment:
-Cleaning up the images
-----------------------
+1. `Stop Breeze <#stopping-breeze>`_ with ``./breeze --stop-environment``.
-You might need to cleanup your Docker environment occasionally. The images are
quite big
-(1.5GB for both images needed for static code analysis and CI tests). And if
you often rebuild/update
-images you might end up with some unused image data.
+2. Run the ``docker system prune`` command.
-Cleanup can be performed with ``docker system prune`` command.
-Make sure to `Stop Breeze <#stopping-breeze>`_ first with ``./breeze
--stop-environment``.
+3. Run ``docker images --all`` and ``docker ps --all`` to verify that your
Docker is clean.
-If you run into disk space errors, we recommend you prune your docker images
using the
-``docker system prune --all`` command. You might need to restart the docker
-engine before running this command.
+ Both commands should return an empty list of images and containers
respectively.
-You can check if your docker is clean by running ``docker images --all`` and
``docker ps --all`` - both
-should return an empty list of images and containers respectively.
+If you run into disk space errors, consider pruning your Docker images with
the ``docker system prune --all`` command. You may need to restart the Docker
Engine before running this command.
Review comment:
usually when you run out of disk space in Docker I'd say it's safer to prune
all - or even reinstall docker. It might get into some weird state if that
happens.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
