potiuk commented on a change in pull request #9129:
URL: https://github.com/apache/airflow/pull/9129#discussion_r435076765
##########
File path: BREEZE.rst
##########
@@ -231,6 +232,41 @@ from your ``logs`` directory in the Airflow sources, so
all logs created in the
visible in the host as well. Every time you enter the container, the ``logs``
directory is
cleaned so that logs do not accumulate.
+CLI for cloud providers
+=======================
+
+For development convenience we installed simple wrappers for the most common
cloud providers CLIs. Those
+CLIs are not installed when you build or pull the image - they will be
downloaded as docker images
+the first time you attempt to use them. It is downloaded and executed in your
host's docker engine so once
+it is downloaded, it will stay until you remove the downloaded images from
your host container.
+
+For each of those CLI you need to provide credentials. You can do it in one of
the ways:
+
+* You can forward credentials from your host machine when starting breeze with
+ ``--forward-credentials`` flag added when you start ``Breeze``. Just a
watch-out - this makes
+ your host credentials available inside the image, so use it with care.
+
+* If you do not want to forward credentials, you can use traditional
authentication methods provided by
+ each of the tools (for example ``gcloud auth login``) - the credentials are
stored in a configuration
+ directory specific for each CLI tool until you stop breeze session. You will
need to re-authenticate
+ when you restart ``Breeze``.
+
+Those are currently installed CLIs (they are available as aliases to the
docker commands):
+
++-----------------------+----------+-------------------------------------------------+-------------------+
+| Cloud Provider | CLI tool | Docker image
| Configuration dir |
++=======================+==========+=================================================+===================+
+| Amazon Web Services | aws | amazon/aws-cli:latest
| .aws |
++-----------------------+----------+-------------------------------------------------+-------------------+
+| Google Cloud Platform | gcloud | mcr.microsoft.com/azure-cli:latest
| .config/gcloud |
++-----------------------+----------+-------------------------------------------------+-------------------+
+| Microsoft Azure | az |
gcr.io/google.com/cloudsdktool/cloud-sdk:latest | .azure |
++-----------------------+----------+-------------------------------------------------+-------------------+
+
+For each of the CLIs we have also an accompanying ``*-update`` alias (for
example aws-update) which
+will pull the latest image for the tool.
Review comment:
Ah yeah. That's really good this way. And also as a side effect if you
have 1.10 and master on the same docker engine, they will share those :)
##########
File path: Dockerfile.ci
##########
@@ -338,6 +308,24 @@ RUN if [[ -n "${ADDITIONAL_PYTHON_DEPS}" ]]; then \
pip install ${ADDITIONAL_PYTHON_DEPS}; \
fi
+ARG AWSCLI_IMAGE="amazon/aws-cli:latest"
+ENV AWSCLI_IMAGE=${AWSCLI_IMAGE}
+
+ARG AZURECLI_IMAGE="mcr.microsoft.com/azure-cli:latest"
+ENV AZURECLI_IMAGE=${AZURECLI_IMAGE}
+
+ARG GCLOUD_IMAGE="gcr.io/google.com/cloudsdktool/cloud-sdk:latest"
+ENV GCLOUD_IMAGE=${GCLOUD_IMAGE}
Review comment:
It's not necessary. Agree. Just got used to add those every time. But I
can clean it up :)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
