This is an automated email from the ASF dual-hosted git repository.
dimberman pushed a commit to branch v1-10-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v1-10-test by this push:
new 118c37d Update AWS connection example to show how to set from env var
(#9191)
118c37d is described below
commit 118c37d70ddfd746d7f9aa011994567ff4301100
Author: Ash Berlin-Taylor <[email protected]>
AuthorDate: Tue Jun 9 11:42:51 2020 +0100
Update AWS connection example to show how to set from env var (#9191)
The trailing `@` wasn't obvious/documented anywhere (and took me some
trial and error to work out) so to save time for the next person let's
add it to the docs
(cherry picked from commit d8e54908d9b4bed6c98468300244ce7a7936878e)
---
docs/howto/connection/aws.rst | 53 ++++++++++++++++++++++++++++++++++++++++---
1 file changed, 50 insertions(+), 3 deletions(-)
diff --git a/docs/howto/connection/aws.rst b/docs/howto/connection/aws.rst
index f55c580..6a969be 100644
--- a/docs/howto/connection/aws.rst
+++ b/docs/howto/connection/aws.rst
@@ -28,7 +28,8 @@ Authenticating to AWS
Authentication may be performed using any of the `boto3 options
<https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#configuring-credentials>`_.
Alternatively, one can pass credentials in as a Connection initialisation
parameter.
-To use IAM instance profile, create an "empty" connection (i.e. one with no
Login or Password specified).
+To use IAM instance profile, create an "empty" connection (i.e. one with no
Login or Password specified, or
+``aws://``).
Default Connection IDs
-----------------------
@@ -49,7 +50,9 @@ Password (optional)
Extra (optional)
Specify the extra parameters (as json dictionary) that can be used in AWS
- connection. The following parameters are supported:
+ connection. The following parameters are all optional:
+
+ * ``aws_session_token``: AWS session token used for the initial connection
if you use external credentials. You are responsible for renewing these.
* ``aws_account_id``: AWS account ID for the connection
* ``aws_iam_role``: AWS IAM role for the connection
@@ -59,7 +62,51 @@ Extra (optional)
* ``role_arn``: AWS role ARN for the connection
* ``aws_session_token``: AWS session token if you use external
credentials. You are responsible for renewing these.
- Example "extras" field:
+ * ``host``: Endpoint URL for the connection.
+ * ``region_name``: AWS region for the connection.
+ * ``external_id``: AWS external ID for the connection (deprecated, rather
use ``assume_role_kwargs``).
+
+ * ``config_kwargs``: Additional ``kwargs`` used to construct a
``botocore.config.Config`` passed to *boto3.client* and *boto3.resource*.
+ * ``session_kwargs``: Additional ``kwargs`` passed to
*boto3.session.Session*.
+
+If you are configuing the connection via a URI, ensure that all components of
the URI are URL-encoded.
+
+Examples
+--------
+
+**Using instance profile**:
+ .. code-block:: bash
+
+ export AIRFLOW_CONN_AWS_DEFAULT=aws://
+
+ This will use boto's default credential look-up chain (the profile named
"default" from the ~/.boto/ config files, and instance profile when running
inside AWS)
+
+**With a AWS IAM key pair**:
+ .. code-block:: bash
+
+ export
AIRFLOW_CONN_AWS_DEFAULT=aws://AKIAIOSFODNN7EXAMPLE:wJalrXUtnFEMI%2FK7MDENG%2FbPxRfiCYEXAMPLEKEY@
+
+ Note here, that the secret access key has been URL-encoded (changing ``/``
to ``%2F``), and also the
+ trailing ``@`` (without which, it is treated as ``<host>:<port>`` and will
not work)
+
+
+Examples for the **Extra** field
+--------------------------------
+
+1. Using *~/.aws/credentials* and *~/.aws/config* file, with a profile.
+
+This assumes all other Connection fields eg **Login** are empty.
+
+.. code-block:: json
+
+ {
+ "session_kwargs": {
+ "profile_name": "my_profile"
+ }
+ }
+
+
+2. Specifying a role_arn to assume and a region_name
.. code-block:: json
