This is an automated email from the ASF dual-hosted git repository.
ephraimanierobi pushed a change to branch v2-8-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
from ba3213bb93 Upgrade to just released `pip` 23.3.2 (#36271)
new 4f05fe6583 Fix Python-based decorators templating (#36103)
new 373af529cc Deferrable Operators Docs Edits (#33620)
new 6e31c0f2d7 Use UTC explicitly in timetable tests (#36082)
new fcd235777e Create latest log dir symlink as relative link (#36019)
new 3b5a460d66 Remove Redshift mention from the list of managed Postgres
backends (#36217)
new 1180255fcc enable mark task as failed/success always (#36254)
new 8d76538d6e Check DAG read permission before accessing DAG code (#36257)
new 5422a3612b Replace deprecated get_accessible_dag_ids and use
get_readable_dags in get_dag_warnings (#36256)
new 2c4c5bc604 Stop deserializing pickle when enable_xcom_pickling is
False (#36255)
new e104aeb830 Fix code typo Docs ObjectStorage Extrenal Integrations
example (#36261)
new fef77f5a2c Bugfix/logging for pausing (#36182)
new a45e149bcb Improve audit_logs.rst (#36213)
new b915ed8871 Use `batch_is_authorized_dag` to check if user has
permission to read DAGs (#36279)
new 48afea83e6 Straighten typing in workday timetable (#36296)
new 185c9ff1b8 Increase width of execution_date input in trigger.html
(#36278)
new a6a305f0dc Kubernetes executor running slots leak fix (#36240)
new ca1aa4a5dc Update admonitions in Python operator doc to reflect
sentiment (#36340)
new 3c7032513e Allow PythonVirtualenvOperator.skip_on_exit_code to be zero
(#36361)
new d80d6a1a01 Fix AirflowSkipException message raised by BashOperator
(#36354)
new 8317580ccf Fix PythonVirtualenvOperator tests (#36367)
new 0dba06d660 fix datetime reference in DAG.is_fixed_time_schedule
(#36370)
new a51278d585 Link to Apache Software Foundation Privacy Policy. (#36394)
new 6624adeb01 Fix the required access for get_variable endpoint (#36396)
new 5dad650b52 Bugfix: Webserver returns 500 for POST requests to
api/dag/*/dagrun from anonymous user (#36275)
new e2357c26c1 Update metrics.rst for param dagrun.schedule_delay (#36404)
new bbf6ebc703 Remove redundant `docker` decorator type annotations
(#36406)
new 4d27170e3c Make `kubernetes` decorator type annotation consistent with
operator (#36405)
new 830fcb02b1 Use cards when describing priority weighting methods
(#36411)
new 56ecbffc40 Stop serializing timezone-naive datetime to timezone-aware
dateime with UTC tz (#36379)
new 4535686a98 Follow BaseHook connection fields method signature in
FSHook (#36444)
new 808ed02476 Fix get_leaves calculation for teardown in nested group
(#36456)
new ca1534b9ef fix: Repair defferable operator list in docs (#36476)
new 29dbafde99 Add further details to replacement documentation (#36485)
new e9b3201d58 Add branching based on mapped task group example to
dynamic-task-mapping.rst (#36480)
new 2c2c6a3129 Add description on the ways how users should approach DB
monitoring (#36483)
new 986f6d8ea2 Add few tests on the mapped task group. (#36149)
new 842fd3b7fa Refactor _manage_executor_state by refreshing TIs in batch
(#36418)
new 1dc0b2ec84 Add flask config: `MAX_CONTENT_LENGTH` (#36401)
new 525e229074 Revert "Refactor _manage_executor_state by refreshing TIs
in batch (#36418)" (#36500)
new 81b23317dd Refactor _manage_executor_state by refreshing TIs in batch
(#36502)
new 357e81051d Fix multiple issues in release notes does not have links
(#36503)
new c93209bf28 Update installation prerequisites after upgrading to Debian
Bookworm (#36521)
new 21b58ecac3 Raise error when ``DagRun`` fails while running ``dag
test`` (#36517)
new 1718b2bc7c Fix details tab not showing when using dynamic task mapping
(#36522)
new 96e0149119 Update dags.rst with information on DAG pausing (#36540)
new 07e30c9f9c Fix configuration of mypy plugins to point to paths not
modules (#36563)
new 80b19d7951 Fix copy&paste victim for installation instructions (#36572)
new b9447b1e08 Improve the error message displayed when there is a
webserver error (#36570)
new 408e2d2fe7 Ignore MyPy error introduced by the new Apache Beam (#36607)
new e98fed3e89 Remove option ot set a task instance to running state in UI
(#36518)
new b7dab91346 Improve and add more complete description in the
architecture diagrams (#36513)
new 81d1fec83f Fix run type icon alignment with run type text (#36616)
new bbf1fbdc79 Replace numpy example with practical exercise demonstrating
top-level code (#35097)
new b71d3882e9 Avoid using dict as default value in call_regular_interval
(#36608)
new 7b5395bb64 Redirect to index when user does not have permission to
access a page (#36623)
new ce43d4abea Change detection mechanism for Breeze self-upgrade (#36635)
new 718f802ffa Add section about live-upgrading Airflow (#36637)
new 79e54c9948 Add Github social media preview image (#36653)
new 1a92e92f4d In docs use logical_date instead of deprecated
execution_date (#36654)
new 171981fce8 Fix the type hint for tis_query in _process_executor_events
(#36655)
new 63c4546bb7 Bump min version of amazon-provider related dependencies
(#36660)
new 4c613f827a Provide the logger_name param to base hook in order to
override the logger name (#36674)
new 975ddce2ef Get rid of pyarrow-hotfix for CVE-2023-47248 (#36697)
new 5deebca173 Update mask-sensitive-values.rst (#36699)
new cbab5e4df3 rename concurrency label to max active tasks (#36691)
new 998dd5c84c Bump follow-redirects from 1.15.3 to 1.15.4 in /airflow/www
(#36700)
new 874227c2cf Set min pandas dependency to 1.2.5 for all providers and
airflow (#36698)
new 4314571c31 Support encryption for triggers parameters (#36492)
new 61567c8296 Do not let EventsTimetable schedule past events if
catchup=False (#36134)
new 3685d3ec57 Remove dot value (#36712)
new 0916f7ff31 Use `mariadb` by default when build final prod image
(#36716)
new 66143482ad Add back FAB constant in legacy security manager (#36719)
new cbc9aaf986 Make sure `multiple_outputs` is inferred correctly even
when using `TypedDict` (#36652)
The 73 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.asf.yaml | 6 +
Dockerfile | 2 +-
.../api_connexion/endpoints/dag_source_endpoint.py | 30 +++-
.../endpoints/dag_warning_endpoint.py | 5 +-
.../api_connexion/endpoints/variable_endpoint.py | 2 +-
airflow/api_connexion/security.py | 4 +-
airflow/auth/managers/base_auth_manager.py | 6 +-
airflow/cli/commands/dag_command.py | 5 +-
airflow/config_templates/config.yml | 7 +
airflow/decorators/__init__.pyi | 87 ++++++++---
airflow/decorators/base.py | 3 +-
airflow/decorators/branch_external_python.py | 1 +
airflow/decorators/branch_python.py | 1 +
airflow/decorators/branch_virtualenv.py | 1 +
airflow/decorators/external_python.py | 1 +
airflow/decorators/python_virtualenv.py | 1 +
airflow/decorators/short_circuit.py | 1 +
airflow/example_dags/plugins/workday.py | 29 ++--
airflow/hooks/filesystem.py | 12 +-
airflow/hooks/package_index.py | 4 +-
airflow/hooks/subprocess.py | 4 +-
airflow/jobs/backfill_job_runner.py | 42 +++++-
airflow/jobs/scheduler_job_runner.py | 4 +-
airflow/jobs/triggerer_job_runner.py | 27 +++-
airflow/models/abstractoperator.py | 1 -
airflow/models/dag.py | 4 +-
airflow/models/dagcode.py | 5 +-
airflow/models/dagrun.py | 4 +-
airflow/models/trigger.py | 13 +-
airflow/models/xcom.py | 6 +-
airflow/operators/bash.py | 6 +-
airflow/operators/python.py | 5 +-
airflow/providers/amazon/provider.yaml | 20 +--
airflow/providers/apache/beam/provider.yaml | 4 +-
airflow/providers/apache/hive/provider.yaml | 2 +-
.../kubernetes/executors/kubernetes_executor.py | 16 ++-
.../executors/kubernetes_executor_types.py | 7 +-
.../executors/kubernetes_executor_utils.py | 9 +-
airflow/providers/common/sql/provider.yaml | 2 +-
airflow/providers/exasol/provider.yaml | 2 +-
airflow/providers/google/provider.yaml | 2 +-
airflow/providers/presto/provider.yaml | 2 +-
airflow/providers/salesforce/provider.yaml | 2 +-
airflow/providers/trino/provider.yaml | 2 +-
airflow/providers/weaviate/provider.yaml | 2 +-
airflow/serialization/serializers/datetime.py | 11 +-
airflow/timetables/_cron.py | 5 +-
airflow/timetables/events.py | 41 ++++--
airflow/utils/event_scheduler.py | 4 +-
airflow/utils/log/file_processor_handler.py | 7 +-
airflow/utils/task_group.py | 13 +-
airflow/www/app.py | 2 +
airflow/www/auth.py | 6 +-
airflow/www/decorators.py | 4 +-
airflow/www/fab_security/manager.py | 3 +
airflow/www/static/css/main.css | 1 +
airflow/www/static/js/components/RunTypeIcon.tsx | 13 +-
airflow/www/static/js/dag/details/index.tsx | 2 +-
.../taskInstance/taskActions/MarkInstanceAs.tsx | 10 +-
airflow/www/templates/airflow/dag_details.html | 2 +-
airflow/www/templates/airflow/traceback.html | 17 ++-
airflow/www/views.py | 10 --
airflow/www/yarn.lock | 6 +-
dev/breeze/src/airflow_breeze/utils/path_utils.py | 23 +--
dev/breeze/tests/test_find_airflow_directory.py | 2 +-
docker_tests/test_prod_image.py | 3 +-
.../logging-monitoring/metrics.rst | 2 +-
.../priority-weight.rst | 55 +++----
.../production-deployment.rst | 68 ++++++++-
.../authoring-and-scheduling/deferring.rst | 139 ++++++++++--------
.../dynamic-task-mapping.rst | 44 +++++-
docs/apache-airflow/best-practices.rst | 74 +++++++---
docs/apache-airflow/core-concepts/dags.rst | 5 +-
.../apache-airflow/core-concepts/objectstorage.rst | 2 +-
docs/apache-airflow/core-concepts/overview.rst | 159 ++++++++++++++++++---
docs/apache-airflow/core-concepts/tasks.rst | 4 +-
docs/apache-airflow/howto/operator/python.rst | 20 +--
docs/apache-airflow/howto/set-up-database.rst | 98 ++++++++++++-
.../img/diagram_basic_airflow_architecture.md5sum | 2 +-
.../img/diagram_basic_airflow_architecture.png | Bin 87096 -> 117106 bytes
.../img/diagram_basic_airflow_architecture.py | 64 +++++----
...agram_dag_processor_airflow_architecture.md5sum | 2 +-
.../diagram_dag_processor_airflow_architecture.png | Bin 106642 -> 238002 bytes
.../diagram_dag_processor_airflow_architecture.py | 56 ++++++--
...diagram_distributed_airflow_architecture.md5sum | 1 +
.../diagram_distributed_airflow_architecture.png | Bin 0 -> 192331 bytes
...=> diagram_distributed_airflow_architecture.py} | 71 ++++++---
.../img/logos/github_repository_social_image.png | Bin 0 -> 58203 bytes
docs/apache-airflow/installation/dependencies.rst | 46 ++++--
docs/apache-airflow/installation/prerequisites.rst | 5 +-
docs/apache-airflow/privacy_notice.rst | 47 +-----
docs/apache-airflow/security/audit_logs.rst | 23 +--
.../security/secrets/mask-sensitive-values.rst | 2 +-
docs/apache-airflow/static/gh-jira-links.js | 6 +-
docs/apache-airflow/templates-ref.rst | 14 +-
docs/apache-airflow/tutorial/taskflow.rst | 4 +-
docs/diagrams/database.png | Bin 0 -> 41772 bytes
docs/diagrams/multiple_files.png | Bin 0 -> 69430 bytes
docs/diagrams/packages.png | Bin 0 -> 205387 bytes
docs/docker-stack/changelog.rst | 6 +
docs/exts/operators_and_hooks_ref.py | 8 +-
.../templates/deferrable_operators_list.rst.jinja2 | 7 +-
generated/provider_dependencies.json | 23 +--
kubernetes_tests/test_kubernetes_pod_operator.py | 4 +-
setup.cfg | 11 +-
setup.py | 14 +-
.../endpoints/test_dag_run_endpoint.py | 20 +++
.../endpoints/test_dag_source_endpoint.py | 67 +++++++--
.../endpoints/test_variable_endpoint.py | 36 ++++-
tests/api_connexion/schemas/test_xcom_schema.py | 3 +
tests/api_connexion/test_parameters.py | 5 +-
tests/cli/commands/test_dag_command.py | 11 ++
tests/decorators/test_branch_virtualenv.py | 9 +-
tests/decorators/test_external_python.py | 14 ++
tests/decorators/test_python.py | 13 ++
tests/decorators/test_python_virtualenv.py | 26 ++++
tests/models/test_mappedoperator.py | 59 ++++++++
tests/models/test_trigger.py | 49 ++++++-
tests/models/test_xcom.py | 18 +--
tests/operators/test_bash.py | 22 ++-
tests/operators/test_python.py | 26 ++--
tests/plugins/workday.py | 90 +-----------
.../executors/test_kubernetes_executor.py | 49 ++++++-
.../cncf/kubernetes/utils/test_pod_manager.py | 17 +--
tests/providers/openlineage/plugins/test_utils.py | 6 +-
.../serialization/serializers/test_serializers.py | 5 +
tests/serialization/test_serialized_objects.py | 4 +-
tests/timetables/test_events_timetable.py | 92 +++++++++---
tests/timetables/test_interval_timetable.py | 150 +++++++++----------
tests/timetables/test_trigger_timetable.py | 65 +++++----
tests/timetables/test_workday_timetable.py | 20 +--
tests/utils/log/test_file_processor_handler.py | 4 +-
tests/utils/test_task_group.py | 47 ++++++
tests/www/test_auth.py | 18 ++-
tests/www/views/test_views_paused.py | 50 +++++++
tests/www/views/test_views_tasks.py | 4 +-
136 files changed, 1874 insertions(+), 797 deletions(-)
create mode 100644
docs/apache-airflow/img/diagram_distributed_airflow_architecture.md5sum
create mode 100644
docs/apache-airflow/img/diagram_distributed_airflow_architecture.png
copy docs/apache-airflow/img/{diagram_basic_airflow_architecture.py =>
diagram_distributed_airflow_architecture.py} (51%)
create mode 100644
docs/apache-airflow/img/logos/github_repository_social_image.png
create mode 100644 docs/diagrams/database.png
create mode 100644 docs/diagrams/multiple_files.png
create mode 100644 docs/diagrams/packages.png
create mode 100644 tests/www/views/test_views_paused.py
