This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-6-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit 06a88536db84edd0b4e61e3ef7caa1e796370405 Author: Jarek Potiuk <[email protected]> AuthorDate: Sat Jul 1 23:56:33 2023 +0200 Update security model of Airflow (#32098) * Update security model of Airflow This change updates the security model of Airflow to better explain what are the capabilities of various kinds of users in Airflow deployments and give both users and security researchers a way to understand what security measures they can take and whether they can qualify potential security issues in Airflow properly - taking into account that various users of Airflow have various capabilities and behaviours considered by some of the users as security vulnerabilities, are standard capabilities of the users. It also splits the security information of ours in two separate pages: * .github/SECURITY.md where we explain how to report the issues to Apache Airflow security team by the researchers * documentation security/index.html which is available via Airflow Website where we explain what our security model is and the different kinds of users we have. Both serve slightly different purpose and both contain cross-reference links to each other in order to be able to redirect people who read about the security model to find out how they can report the issues but also to guide security researchers who want to assess whether their findings are real vulnerabilities, or rather normal behaviours following the Airflow Security model. Security has been also moved to be a top level topic, so that it is easier to find and navigate to. Old links have been redirected to the new locations. Also chapters were added explaining Airflow vs. Providers security releases, what is the relation between Airflow and Providers security issues and how users should treat security announcements in providers. * Update .github/SECURITY.md Co-authored-by: Pankaj Koti <[email protected]> * Apply Niko's suggestions from code review Co-authored-by: Niko Oliveira <[email protected]> * fixup! Apply Niko's suggestions from code review --------- Co-authored-by: Pankaj Koti <[email protected]> Co-authored-by: Niko Oliveira <[email protected]> (cherry picked from commit 4efbcdc8fc48e929f18fbe75c61a5125072f71fc) --- .github/{SECURITY.rst => SECURITY.md} | 115 +++++++-------- .github/boring-cyborg.yml | 2 +- .pre-commit-config.yaml | 2 +- .../core-extensions/auth-backends.rst | 2 +- .../core-extensions/secrets-backends.rst | 2 +- .../administration-and-deployment/index.rst | 1 - .../security/index.rst | 28 ---- docs/apache-airflow/howto/connection.rst | 6 +- docs/apache-airflow/howto/variable.rst | 2 +- docs/apache-airflow/index.rst | 1 + .../installation/installing-from-pypi.rst | 1 + docs/apache-airflow/integration.rst | 8 +- docs/apache-airflow/public-airflow-interface.rst | 2 +- docs/apache-airflow/redirects.txt | 35 +++-- .../security/access-control.rst | 0 .../security/api.rst | 2 +- .../security/audit_logs.rst | 0 .../security/flower.rst | 0 docs/apache-airflow/security/index.rst | 164 +++++++++++++++++++++ .../security/kerberos.rst | 0 .../security/secrets/fernet.rst | 0 .../security/secrets/index.rst | 0 .../security/secrets/mask-sensitive-values.rst | 0 .../security/secrets/secrets-backend/index.rst | 0 .../local-filesystem-secrets-backend.rst | 0 .../security/webserver.rst | 2 +- .../security/workload.rst | 0 27 files changed, 254 insertions(+), 121 deletions(-) diff --git a/.github/SECURITY.rst b/.github/SECURITY.md similarity index 53% rename from .github/SECURITY.rst rename to .github/SECURITY.md index e7f5a40366..950ab7f445 100644 --- a/.github/SECURITY.rst +++ b/.github/SECURITY.md @@ -1,37 +1,28 @@ - .. Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - .. http://www.apache.org/licenses/LICENSE-2.0 - - .. Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. - -Security Model --------------- - -In the Airflow security model, the system administrators are fully trusted. -They are the only ones who can upload new DAGs, which gives them the ability -to execute any code on the server. - -Authenticated web interface and API users with Admin/Op permissions are trusted, -but to a lesser extent: they can configure the DAGs which gives them some control, -but not arbitrary code execution. - -Authenticated Web interface and API users with 'regular' permissions are trusted -to the point where they can impact resource consumption and pause/unpause configured DAGs, -but not otherwise influence their functionality. - -Reporting Vulnerabilities -------------------------- +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +This document contains information on how to report security vulnerabilities in Apache Airflow and +how the security issues reported to Apache Airflow security team are handled. If you would like +to learn about the security model of Airflow head to +[Airflow Security](https://airflow.apache.org/docs/apache-airflow/stable/security/) + +## Reporting Vulnerabilities **⚠️ Please do not file GitHub issues for security vulnerabilities as they are public! ⚠️** @@ -39,9 +30,9 @@ The Apache Software Foundation takes security issues very seriously. Apache Airflow specifically offers security features and is responsive to issues around its features. If you have any concern around Airflow Security or believe you have uncovered a vulnerability, we suggest that you get in touch via the -e-mail address [email protected]. In the message, try to provide a -description of the issue and ideally a way of reproducing it. The security team -will get back to you after assessing the description. +e-mail address [[email protected]](mailto:[email protected]). +In the message, try to provide a description of the issue and ideally a way of +reproducing it. The security team will get back to you after assessing the report. Note that this security address should be used only for undisclosed vulnerabilities. Dealing with fixed issues or general questions on how to use @@ -49,13 +40,30 @@ the security features should be handled regularly via the user and the dev lists. Please report any security problems to the project security address before disclosing it publicly. -The `ASF Security team's page <https://www.apache.org/security/>`_ describes -how vulnerability reports are handled, and includes PGP keys if you wish to use -that. +Before reporting vulnerabilities, please make sure to read and understand the +[security model](https://airflow.apache.org/docs/apache-airflow/stable/security/) of Airflow, because +some of the potential security vulnerabilities that are valid for projects that are publicly accessible +from the Internet, are not valid for Airflow. Airflow is not designed to be used by untrusted users, and some +trusted users are trusted enough to do a variety of operations that could be considered as vulnerabilities +in other products/circumstances. Therefore, some potential security vulnerabilities do not +apply to Airflow, or have a different severity than some generic scoring systems (for example `CVSS`) +calculation suggests. +The [ASF Security team's page](https://www.apache.org/security/) describes +how vulnerability reports are handled in general by all ASF projects, and includes PGP keys if +you wish to use them when you report the issues. -Handling security issues in Airflow ------------------------------------ +## Security vulnerabilities in Airflow and Airflow community managed providers + +Airflow core package is released separately from provider packages. While Airflow comes with ``constraints`` +which describe which version of providers have been tested when the version of Airflow was released, the +users of Airflow are advised to install providers independently from Airflow core when they want to apply +security fixes found and released in providers. Therefore, the issues found and fixed in providers do +not apply to the Airflow core package. There are also Airflow providers released by 3rd-parties, but the +Airflow community is not responsible for releasing and announcing security vulnerabilities in them, this +is handled entirely by the 3rd-parties that release their own providers. + +## Handling security issues in Airflow The security issues in Airflow are handled by the Airflow Security Team. The team consists of selected PMC members that are interested in looking at, discussing about and fixing the @@ -80,7 +88,7 @@ There are certain expectations from the members of the security team: experts that are available through Airflow stakeholders. The intent about involving 3rd parties has to be discussed and agreed up at [email protected]. -* They have to have an `ICLA <https://www.apache.org/licenses/contributor-agreements.html>`_ signed with +* They have to have an [ICLA](https://www.apache.org/licenses/contributor-agreements.html) signed with Apache Software Foundation. * The security team members might inform 3rd parties about fixes, for example in order to assess if the fix @@ -92,7 +100,7 @@ There are certain expectations from the members of the security team: with the intent of minimizing the time between the fix being available and the fix being released. In this case the PR might be sent to review and comment to the PMC members on private list, in order to request an expedited voting on the release. The voting for such release might be done on the - ``[email protected]`` mailing list and should be made public at the ``[email protected]`` + `[email protected]` mailing list and should be made public at the `[email protected]` mailing list as soon as the release is ready to be announced. * The security team members working on the fix might be mentioned as remediation developers in the CVE @@ -104,22 +112,5 @@ There are certain expectations from the members of the security team: release process. This is facilitated by the security tool provided by the Apache Software Foundation. * Severity of the issue is determined based on the criteria described in the - `Severity Rating blog post <https://security.apache.org/blog/severityrating/>`_ by the Apache Software + [Severity Rating blog post](https://security.apache.org/blog/severityrating/) by the Apache Software Foundation Security team - -Releasing Airflow with security patches ---------------------------------------- - -Apache Airflow uses strict `SemVer <https://semver.org>`_ versioning policy, which means that we strive for -any release of a given ``MAJOR`` Version (version "2" currently) to be backwards compatible. When we -release ``MINOR`` version, the development continues in the ``main`` branch where we prepare the next -``MINOR`` version, but we release ``PATCHLEVEL`` releases with selected bugfixes (including security -bugfixes) cherry-picked to the latest released ``MINOR`` line of Apache Airflow. At the moment, when we -release a new ``MINOR`` version, we stop releasing ``PATCHLEVEL`` releases for the previous ``MINOR`` version. - -For example, when we released ``2.6.0`` version on April 30, 2023, until we release ``2.7.0`` version, -all the security patches will be cherry-picked and released in ``2.6.*`` versions only. There will be no -``2.5.*`` versions released after ``2.6.0`` has been released. - -This means that in order to apply security fixes with Apache Airflow software released by us, you -MUST upgrade to the latest ``MINOR`` version of Airflow. diff --git a/.github/boring-cyborg.yml b/.github/boring-cyborg.yml index cf2b0aa8e1..59acb91ecd 100644 --- a/.github/boring-cyborg.yml +++ b/.github/boring-cyborg.yml @@ -265,7 +265,7 @@ labelPRBasedOnFilePath: - airflow/providers/**/secrets/* - tests/secrets/**/* - tests/providers/**/secrets/* - - docs/apache-airflow/administration-and-deployment/security/secrets/**/* + - docs/apache-airflow/security/secrets/**/* area:Serialization: - airflow/serialization/**/* diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index bc203399dc..a1b9773f0a 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -472,7 +472,7 @@ repos: ^airflow/providers/microsoft/winrm/hooks/winrm.py$| ^airflow/www/fab_security/manager.py$| ^docs/.*commits.rst$| - ^docs/apache-airflow/administration-and-deployment/security/webserver.rst$| + ^docs/apache-airflow/security/webserver.rst$| ^docs/apache-airflow-providers-apache-cassandra/connections/cassandra.rst$| ^airflow/providers/microsoft/winrm/operators/winrm.py$| ^airflow/providers/opsgenie/hooks/opsgenie.py$| diff --git a/docs/apache-airflow-providers/core-extensions/auth-backends.rst b/docs/apache-airflow-providers/core-extensions/auth-backends.rst index edb38bb882..325b0c2819 100644 --- a/docs/apache-airflow-providers/core-extensions/auth-backends.rst +++ b/docs/apache-airflow-providers/core-extensions/auth-backends.rst @@ -26,7 +26,7 @@ capabilities. You can read more about those in `FAB security docs <https://flask-appbuilder.readthedocs.io/en/latest/security.html>`_. You can also -take a look at Auth backends available in the core Airflow in :doc:`apache-airflow:administration-and-deployment/security/webserver` +take a look at Auth backends available in the core Airflow in :doc:`apache-airflow:security/webserver` or see those provided by the community-managed providers: .. airflow-auth-backends:: diff --git a/docs/apache-airflow-providers/core-extensions/secrets-backends.rst b/docs/apache-airflow-providers/core-extensions/secrets-backends.rst index 22fae4b8a6..26ee3ce882 100644 --- a/docs/apache-airflow-providers/core-extensions/secrets-backends.rst +++ b/docs/apache-airflow-providers/core-extensions/secrets-backends.rst @@ -28,7 +28,7 @@ via providers that implement secrets backends for services Airflow integrates wi You can also take a look at Secret backends available in the core Airflow in -:doc:`apache-airflow:administration-and-deployment/security/secrets/secrets-backend/index` and here you can see the ones +:doc:`apache-airflow:security/secrets/secrets-backend/index` and here you can see the ones provided by the community-managed providers: .. airflow-secrets-backends:: diff --git a/docs/apache-airflow/administration-and-deployment/index.rst b/docs/apache-airflow/administration-and-deployment/index.rst index 1acb9d8e3a..5ba79a5439 100644 --- a/docs/apache-airflow/administration-and-deployment/index.rst +++ b/docs/apache-airflow/administration-and-deployment/index.rst @@ -24,7 +24,6 @@ This section contains information about deploying DAGs into production and the a :maxdepth: 2 production-deployment - security/index logging-monitoring/index kubernetes lineage diff --git a/docs/apache-airflow/administration-and-deployment/security/index.rst b/docs/apache-airflow/administration-and-deployment/security/index.rst deleted file mode 100644 index 65d02f7f53..0000000000 --- a/docs/apache-airflow/administration-and-deployment/security/index.rst +++ /dev/null @@ -1,28 +0,0 @@ - .. Licensed to the Apache Software Foundation (ASF) under one - or more contributor license agreements. See the NOTICE file - distributed with this work for additional information - regarding copyright ownership. The ASF licenses this file - to you under the Apache License, Version 2.0 (the - "License"); you may not use this file except in compliance - with the License. You may obtain a copy of the License at - - .. http://www.apache.org/licenses/LICENSE-2.0 - - .. Unless required by applicable law or agreed to in writing, - software distributed under the License is distributed on an - "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY - KIND, either express or implied. See the License for the - specific language governing permissions and limitations - under the License. - -Security -======== - -.. toctree:: - :maxdepth: 1 - :glob: - - * - secrets/index - -.. include:: /../../.github/SECURITY.rst diff --git a/docs/apache-airflow/howto/connection.rst b/docs/apache-airflow/howto/connection.rst index b43094a88b..0db100fd29 100644 --- a/docs/apache-airflow/howto/connection.rst +++ b/docs/apache-airflow/howto/connection.rst @@ -27,7 +27,7 @@ Airflow's :class:`~airflow.models.connection.Connection` object is used for stor Connections may be defined in the following ways: - in :ref:`environment variables <environment_variables_connections>` - - in an external :doc:`/administration-and-deployment/security/secrets/secrets-backend/index` + - in an external :doc:`/security/secrets/secrets-backend/index` - in the :ref:`Airflow metadata database <connections-in-database>` (using the :ref:`CLI <connection/cli>` or :ref:`web UI <creating_connection_ui>`) @@ -86,7 +86,7 @@ See :ref:`Connection URI format <connection-uri-format>` for more details on how Storing connections in a Secrets Backend ---------------------------------------- -You can store Airflow connections in external secrets backends like HashiCorp Vault, AWS SSM Parameter Store, and other such services. For more details see :doc:`/administration-and-deployment/security/secrets/secrets-backend/index`. +You can store Airflow connections in external secrets backends like HashiCorp Vault, AWS SSM Parameter Store, and other such services. For more details see :doc:`/security/secrets/secrets-backend/index`. .. _connections-in-database: @@ -94,7 +94,7 @@ Storing connections in the database ----------------------------------- .. seealso:: - Connections can alternatively be stored in :ref:`environment variables <environment_variables_connections>` or an :doc:`external secrets backend </administration-and-deployment/security/secrets/secrets-backend/index>` such as HashiCorp Vault, AWS SSM Parameter Store, etc. + Connections can alternatively be stored in :ref:`environment variables <environment_variables_connections>` or an :doc:`external secrets backend </security/secrets/secrets-backend/index>` such as HashiCorp Vault, AWS SSM Parameter Store, etc. When storing connections in the database, you may manage them using either the web UI or the Airflow CLI. diff --git a/docs/apache-airflow/howto/variable.rst b/docs/apache-airflow/howto/variable.rst index a9e0d82023..b4b395dd63 100644 --- a/docs/apache-airflow/howto/variable.rst +++ b/docs/apache-airflow/howto/variable.rst @@ -73,4 +73,4 @@ It guarantees that without the encryption password, content cannot be manipulate without the key. For information on configuring Fernet, look at :ref:`security/fernet`. In addition to retrieving variables from environment variables or the metastore database, you can enable -a secrets backend to retrieve variables. For more details see :doc:`/administration-and-deployment/security/secrets/secrets-backend/index`. +a secrets backend to retrieve variables. For more details see :doc:`/security/secrets/secrets-backend/index`. diff --git a/docs/apache-airflow/index.rst b/docs/apache-airflow/index.rst index 548ced7b07..feaf2d388b 100644 --- a/docs/apache-airflow/index.rst +++ b/docs/apache-airflow/index.rst @@ -133,6 +133,7 @@ so coding will always be required. Overview <self> start installation/index + security/index tutorial/index howto/index ui diff --git a/docs/apache-airflow/installation/installing-from-pypi.rst b/docs/apache-airflow/installation/installing-from-pypi.rst index 921f917040..93aa99b668 100644 --- a/docs/apache-airflow/installation/installing-from-pypi.rst +++ b/docs/apache-airflow/installation/installing-from-pypi.rst @@ -265,6 +265,7 @@ released and tested together when the version of Airflow you are installing was CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"; pip install "apache-airflow[postgres,google]==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}" +.. _installing-from-pypi-managing-providers-separately-from-airflow-core: Managing providers separately from Airflow core =============================================== diff --git a/docs/apache-airflow/integration.rst b/docs/apache-airflow/integration.rst index 3f47791104..2e9c450f1f 100644 --- a/docs/apache-airflow/integration.rst +++ b/docs/apache-airflow/integration.rst @@ -20,18 +20,18 @@ Integration Airflow has a mechanism that allows you to expand its functionality and integrate with other systems. -* :doc:`API Authentication backends </administration-and-deployment/security/api>` +* :doc:`API Authentication backends </security/api>` * :doc:`Email backends </howto/email-config>` * :doc:`Executor </core-concepts/executor/index>` -* :doc:`Kerberos </administration-and-deployment/security/kerberos>` +* :doc:`Kerberos </security/kerberos>` * :doc:`Logging </administration-and-deployment/logging-monitoring/logging-tasks>` * :doc:`Metrics (statsd) </administration-and-deployment/logging-monitoring/metrics>` * :doc:`Operators and hooks </operators-and-hooks-ref>` * :doc:`Plugins </authoring-and-scheduling/plugins>` * :doc:`Listeners </administration-and-deployment/listeners>` -* :doc:`Secrets backends </administration-and-deployment/security/secrets/secrets-backend/index>` +* :doc:`Secrets backends </security/secrets/secrets-backend/index>` * :doc:`Tracking systems </administration-and-deployment/logging-monitoring/tracking-user-activity>` -* :doc:`Web UI Authentication backends </administration-and-deployment/security/api>` +* :doc:`Web UI Authentication backends </security/api>` * :doc:`Serialization </authoring-and-scheduling/serializers>` It also has integration with :doc:`Sentry </administration-and-deployment/logging-monitoring/errors>` service for error tracking. Other applications can also integrate using diff --git a/docs/apache-airflow/public-airflow-interface.rst b/docs/apache-airflow/public-airflow-interface.rst index 74d8657244..5f0f511638 100644 --- a/docs/apache-airflow/public-airflow-interface.rst +++ b/docs/apache-airflow/public-airflow-interface.rst @@ -339,7 +339,7 @@ All Secrets Backend implementations are public. You can extend their functionali _api/airflow/secrets/index -You can read more about Secret Backends in :doc:`administration-and-deployment/security/secrets/secrets-backend/index`. +You can read more about Secret Backends in :doc:`security/secrets/secrets-backend/index`. You can also find all the available Secrets Backends implemented in community providers in :doc:`apache-airflow-providers:core-extensions/secrets-backends`. diff --git a/docs/apache-airflow/redirects.txt b/docs/apache-airflow/redirects.txt index b99a5b253c..9d8a5527fb 100644 --- a/docs/apache-airflow/redirects.txt +++ b/docs/apache-airflow/redirects.txt @@ -15,9 +15,26 @@ # specific language governing permissions and limitations # under the License. +# Administration and deployment security -> security + +administration-and-deployment/security/index.rst security/index.rst +administration-and-deployment/security/kerberos.rst security/kerberos.rst +administration-and-deployment/security/access-control.rst security/access-control.rst +administration-and-deployment/security/access-control/index.rst security/access-control.rst +administration-and-deployment/security/api.rst security/api.rst +administration-and-deployment/security/audit_logs.rst security/audit_logs.rst +administration-and-deployment/security/flower.rst security/flower.rst +administration-and-deployment/security/webserver.rst security/webserver.rst +administration-and-deployment/security/workload.rst security/workload.rst +administration-and-deployment/security/secrets/secrets-backends/index.rst security/secrets/secrets-backends/index.rst +administration-and-deployment/security/secrets/secrets-backends/local-filesystem-secrets-backend.rst security/secrets/secrets-backends/local-filesystem-secrets-backend.rst +administration-and-deployment/security/secrets/fernet.rst security/secrets/fernet.rst +administration-and-deployment/security/secrets/index.rst security/secrets/index.rst +administration-and-deployment/security/secrets/mask-sensitive-values.rst security/secrets/mask-sensitive-values.rst + # Security -howto/use-alternative-secrets-backend.rst administration-and-deployment/security/secrets/secrets-backend/index.rst -security.rst administration-and-deployment/security/index.rst +howto/use-alternative-secrets-backend.rst security/secrets/secrets-backend/index.rst +security.rst security/index.rst # Operators guides howto/operator/external.rst howto/operator/external_task_sensor.rst @@ -27,7 +44,7 @@ howto/customize-dag-ui-page-instance-name.rst howto/customize-ui.rst#customizing howto/customize-state-colors-ui.rst howto/customize-ui.rst#customizing-state-colours # Web UI -howto/add-new-role.rst administration-and-deployment/security/access-control.rst +howto/add-new-role.rst security/access-control.rst # Set up a database howto/initialize-database.rst howto/set-up-database.rst @@ -68,9 +85,6 @@ tutorial_taskflow_api.rst tutorial/taskflow.rst ## Docs Structure Refactor # indexes -security/index.rst administration-and-deployment/security/index.rst -security/secrets/index.rst administration-and-deployment/security/secrets/index.rst -security/secrets/secrets-backend/index.rst administration-and-deployment/security/secrets/secrets-backend/index.rst logging-monitoring/index.rst administration-and-deployment/logging-monitoring/index.rst concepts/index.rst core-concepts/index.rst executor/index.rst core-concepts/executor/index.rst @@ -88,15 +102,6 @@ usage-cli.rst howto/usage-cli.rst lineage.rst administration-and-deployment/lineage.rst dag-run.rst core-concepts/dag-run.rst concepts/taskflow.rst core-concepts/taskflow.rst -security/kerberos.rst administration-and-deployment/security/kerberos.rst -security/workload.rst administration-and-deployment/security/workload.rst -security/flower.rst administration-and-deployment/security/flower.rst -security/webserver.rst administration-and-deployment/security/webserver.rst -security/api.rst administration-and-deployment/security/api.rst -security/access-control.rst administration-and-deployment/security/access-control.rst -security/secrets/fernet.rst administration-and-deployment/security/secrets/fernet.rst -security/secrets/mask-sensitive-values.rst administration-and-deployment/security/secrets/mask-sensitive-values.rst -security/secrets/secrets-backend/local-filesystem-secrets-backend.rst administration-and-deployment/security/secrets/secrets-backend/local-filesystem-secrets-backend.rst logging-monitoring/tracking-user-activity.rst administration-and-deployment/logging-monitoring/tracking-user-activity.rst logging-monitoring/errors.rst administration-and-deployment/logging-monitoring/errors.rst logging-monitoring/logging-architecture.rst administration-and-deployment/logging-monitoring/logging-architecture.rst diff --git a/docs/apache-airflow/administration-and-deployment/security/access-control.rst b/docs/apache-airflow/security/access-control.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/access-control.rst rename to docs/apache-airflow/security/access-control.rst diff --git a/docs/apache-airflow/administration-and-deployment/security/api.rst b/docs/apache-airflow/security/api.rst similarity index 98% rename from docs/apache-airflow/administration-and-deployment/security/api.rst rename to docs/apache-airflow/security/api.rst index 6551ef928f..61319e21c6 100644 --- a/docs/apache-airflow/administration-and-deployment/security/api.rst +++ b/docs/apache-airflow/security/api.rst @@ -63,7 +63,7 @@ If you wish to have the experimental API work, and aware of the risks of enablin You can only disable authentication for experimental API, not the stable REST API. -See :doc:`../modules_management` for details on how Python and Airflow manage modules. +See :doc:`../administration-and-deployment/modules_management` for details on how Python and Airflow manage modules. Kerberos authentication ''''''''''''''''''''''' diff --git a/docs/apache-airflow/administration-and-deployment/security/audit_logs.rst b/docs/apache-airflow/security/audit_logs.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/audit_logs.rst rename to docs/apache-airflow/security/audit_logs.rst diff --git a/docs/apache-airflow/administration-and-deployment/security/flower.rst b/docs/apache-airflow/security/flower.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/flower.rst rename to docs/apache-airflow/security/flower.rst diff --git a/docs/apache-airflow/security/index.rst b/docs/apache-airflow/security/index.rst new file mode 100644 index 0000000000..3573348f10 --- /dev/null +++ b/docs/apache-airflow/security/index.rst @@ -0,0 +1,164 @@ + .. Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + .. http://www.apache.org/licenses/LICENSE-2.0 + + .. Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + +Security +======== + +.. toctree:: + :maxdepth: 1 + :glob: + + * + secrets/index + + +This document describes Airflow's security model from the perspective of +the Airflow user. It is intended to help users understand the security +model and make informed decisions about how to deploy and manage Airflow. +If you would like to know how to report security vulnerabilities and how +security reports are handled by the security team of Airflow, head to +`Airflow's Security Policy <https://github.com/apache/airflow/security/policy>`_. + +Airflow security model - user types +----------------------------------- + +The Airflow security model involves different types of users with +varying access and capabilities: + +1. **Deployment Managers**: They have the highest level of access and + control. They install and configure Airflow, make decisions about + technologies and permissions. They can potentially delete the entire + installation and have access to all credentials. Deployment Managers + can also decide to keep audits, backups and copies of information + outside of Airflow, which are not covered by Airflow's security + model. + +2. **DAG Authors**: They can upload, modify, and delete DAG files. The + code in DAG files is executed on workers. Therefore, DAG authors can create + and change code executed on workers and potentially access the credentials + that DAG code uses to access external systems. DAG Authors have full access + to the metadata database and internal audit logs. + +3. **Authenticated UI users**: They have access to the UI and API. Admin + users can manage permissions and execute code on workers. Connection + configuration users can configure connections and execute code on + workers. Operations users have access to DAG execution status. Trust + is crucial to prevent abuse and Denial of Service attacks. + +4. **Non-authenticated UI users**: Airflow doesn't support + unauthenticated users by default. If allowed, vulnerabilities must be + addressed by the Deployment Manager. + +Capabilities of authenticated UI users +-------------------------------------- + +The capabilities of **Authenticated UI users** can vary depending on +what roles have been configured by the Deployment Manager or Admin users as well as what permissions those roles have. Permissions on roles can be scoped as tightly as a single DAG, for example, or as broad as Admin. Below are three general categories to help conceptualize some of the capabilities authenticated users may have: + +1. **Admin users**: They manage and grant permissions to other users, + with full access to all UI capabilities. They can potentially execute + code on workers by configuring connections and need to be trusted not + to abuse these privileges. They have access to sensitive credentials + and can modify them. By default, they don't have access to + system-level configuration. They should be trusted not to misuse + sensitive information accessible through connection configuration. + They also have the ability to create a Webserver Denial of Service + situation and should be trusted not to misuse this capability. + +2. **Connection configuration users**: They configure connections and + potentially execute code on workers during DAG execution. Trust is + required to prevent misuse of these privileges. They have full access + to sensitive credentials stored in connections and can modify them. + Access to sensitive information through connection configuration + should be trusted not to be abused. They also have the ability to + create a Webserver Denial of Service situation and should be trusted + not to misuse this capability. + +3. **Operations users**: They have access to DAG execution status via + the UI. Currently, Airflow lacks full protection for accessing groups + of DAGs' history and execution. They can perform actions such as + clearing, re-running, triggering DAGs, and changing parameters. + Depending on access restrictions, they may also have access to + editing variables and viewing Airflow configuration. They should not + have access to sensitive system-level information or connections, and + they should not be able to access sensitive task information unless + deliberately exposed in logs by DAG authors. They should be trusted + not to abuse their privileges, as they can potentially overload the + server and cause Denial of Service situations. + +Responsibilities of Deployment Managers +--------------------------------------- + +Deployment Managers determine access levels and understand the potential +damage users can cause. Some Deployment Managers may further limit +access through fine-grained privileges for the **Authenticated UI +users**. However, these limitations are outside the basic Airflow's +security model and are at the discretion of Deployment Managers. +Examples of fine-grained access control include (but are not limited +to): + +- Limiting login permissions: Restricting the accounts that users can + log in with, allowing only specific accounts or roles belonging to + access the Airflow system. + +- Access restrictions to views or DAGs: Controlling user access to + certain views or specific DAGs, ensuring that users can only view or + interact with authorized components. + +- Implementing static code analysis and code review: Introducing + processes such as static code analysis and code review as part of the + DAG submission pipeline. This helps enforce code quality standards, + security checks, and adherence to best practices before DAGs are + deployed. + +These examples showcase ways in which administrators can refine and +limit user privileges within Airflow, providing tighter control and +ensuring that users have access only to the necessary components and +functionalities based on their roles and responsibilities, however the +fine-grained access control does not provide full isolation and +separation of access allowing isolation of different user groups in a +multi-tenant fashion yet. In the future versions of Airflow some of the +fine-grained access features might become part of the Airflow security +model. The Airflow community is working on a multi-tenant model that might +address some of the fine-grained access components. + + +Releasing Airflow with security patches +--------------------------------------- + +Apache Airflow uses a strict [SemVer](https://semver.org) versioning policy, which means that we strive for +any release of a given ``MAJOR`` Version (version "2" currently) to be backwards compatible. When we +release a ``MINOR`` version, the development continues in the ``main`` branch where we prepare the next +``MINOR`` version, but we release ``PATCHLEVEL`` releases with selected bugfixes (including security +bugfixes) cherry-picked to the latest released ``MINOR`` line of Apache Airflow. At the moment, when we +release a new ``MINOR`` version, we stop releasing ``PATCHLEVEL`` releases for the previous ``MINOR`` version. + +For example, once we released ``2.6.0`` version on April 30, 2023 all the security patches will be cherry-picked and released in ``2.6.*`` versions until we release ``2.7.0`` version. There will be no +``2.5.*`` versions released after ``2.6.0`` has been released. + +This means that in order to apply security fixes with Apache Airflow software released by us, you +MUST upgrade to the latest ``MINOR`` version of Airflow. + +Releasing Airflow providers with security patches +------------------------------------------------- + +Similarly to Airflow, providers use strict [SemVer](https://semver.org) versioning policy, and the same +policies apply for providers as for Airflow itself. This means that you need to upgrade to the latest +``MINOR`` version of the provider to get the latest security fixes. +Airflow providers are released independently from Airflow itself and the information about vulnerabilities +is published separately. You can upgrade providers independently from Airflow itself, following the +instructions found in :ref:`installing-from-pypi-managing-providers-separately-from-airflow-core`. diff --git a/docs/apache-airflow/administration-and-deployment/security/kerberos.rst b/docs/apache-airflow/security/kerberos.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/kerberos.rst rename to docs/apache-airflow/security/kerberos.rst diff --git a/docs/apache-airflow/administration-and-deployment/security/secrets/fernet.rst b/docs/apache-airflow/security/secrets/fernet.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/secrets/fernet.rst rename to docs/apache-airflow/security/secrets/fernet.rst diff --git a/docs/apache-airflow/administration-and-deployment/security/secrets/index.rst b/docs/apache-airflow/security/secrets/index.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/secrets/index.rst rename to docs/apache-airflow/security/secrets/index.rst diff --git a/docs/apache-airflow/administration-and-deployment/security/secrets/mask-sensitive-values.rst b/docs/apache-airflow/security/secrets/mask-sensitive-values.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/secrets/mask-sensitive-values.rst rename to docs/apache-airflow/security/secrets/mask-sensitive-values.rst diff --git a/docs/apache-airflow/administration-and-deployment/security/secrets/secrets-backend/index.rst b/docs/apache-airflow/security/secrets/secrets-backend/index.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/secrets/secrets-backend/index.rst rename to docs/apache-airflow/security/secrets/secrets-backend/index.rst diff --git a/docs/apache-airflow/administration-and-deployment/security/secrets/secrets-backend/local-filesystem-secrets-backend.rst b/docs/apache-airflow/security/secrets/secrets-backend/local-filesystem-secrets-backend.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/secrets/secrets-backend/local-filesystem-secrets-backend.rst rename to docs/apache-airflow/security/secrets/secrets-backend/local-filesystem-secrets-backend.rst diff --git a/docs/apache-airflow/administration-and-deployment/security/webserver.rst b/docs/apache-airflow/security/webserver.rst similarity index 99% rename from docs/apache-airflow/administration-and-deployment/security/webserver.rst rename to docs/apache-airflow/security/webserver.rst index d5e551f8b1..71ad85fb3a 100644 --- a/docs/apache-airflow/administration-and-deployment/security/webserver.rst +++ b/docs/apache-airflow/security/webserver.rst @@ -74,7 +74,7 @@ user will have by default: AUTH_ROLE_PUBLIC = 'Admin' -Be sure to checkout :doc:`/administration-and-deployment/security/api` for securing the API. +Be sure to checkout :doc:`/security/api` for securing the API. .. note:: diff --git a/docs/apache-airflow/administration-and-deployment/security/workload.rst b/docs/apache-airflow/security/workload.rst similarity index 100% rename from docs/apache-airflow/administration-and-deployment/security/workload.rst rename to docs/apache-airflow/security/workload.rst
