This is an automated email from the ASF dual-hosted git repository.
weilee pushed a change to branch render-json-logs-new-ui
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit 0cbedc283f7 fixup! Render structured logs in the new UI rather than
showing raw JSON
omit 58e52f034ba fixup! fixup! Render structured logs in the new UI rather
than showing raw JSON
omit c2abc93770e Fix typescript useLogs
omit ce00aea1822 fixup! Render structured logs in the new UI rather than
showing raw JSON
omit 76b9e8a05ad Render structured logs in the new UI rather than showing
raw JSON
add e740e1e7f21 Bump dompurify in
/providers/fab/src/airflow/providers/fab/www (#46798)
add b5d6a391e51 Add operators for Entry resource (#46647)
add 2db5dbefab3 Upgrade ubuntu runner to 22.04 as per deprecation notice
(#46830)
add cd8d564e860 AIP-84 Add asset_dependencies endpoint (#46770)
add b4d27074f60 Remove remnants of old provider's structure (#46829)
add cd0c32bb537 Upgrade installers to latest versions (#46834)
add 4c786c42dd6 Deprecate old bigquery operators and introduce
CreateTableOperator (#46673)
add 165ea9ed0c4 [EDGE]CLI allows to retrieve status of worker (#46818)
add fece9c7dfc5 Create operators for working with Clusters for GCP Apache
Kafka (#45795)
add c4091ab02e4 Remove tests from example_dags folder (#46819)
add fdbd68f02e8 TriggerDagRunOperator by defaults set logical date as null
(#46633)
add 8fbdf75dc7e Excluding system tests from dag serialisation tests
(#46850)
add 18396ed8083 Rework the TriggererJobRunner to run triggers in a process
without DB access (#46677)
add 8c374a89c7c Handle None in DagRunAssetReference (#46847)
add b715909ef1c fix test using time travel (#46859)
add d0dbf4c0cbf AIP-66 Refactor DagRun to DagVersion association (#46626)
add 3b5a503a26e [EDGE]Fix maintenance states during shut down (#46861)
add b3701292ce3 fix rst with code block (#46864)
add d5ffb467d1b Allow description to be passed in description file for
variables_import command (#46799)
add c33299d4325 Fixing log typos in Local & Celery Executors (#46866)
add 647a2d1cbec Move Macros to Task SDK (#46867)
add 4833b53705a Refactor result_processor and event_handler signatures in
MSGraphAsyncOperator (#46637)
add 3e6c6a109ea Fix wrong operator name in doc string of @task.kubernetes
(#46873)
add 3e13c2fb35c Remove deprecated get_link signature support (#46415)
add 6a7203a2da3 Use run_id to implement ti clear in API when given (#46883)
add d1bf9c5dd19 Fix MultipleCronTriggerTimetable deserialization (#46886)
add f0abca1e8e7 Import the right BaseOperator in baseoperatorlinks (#46895)
add 08c54b6baa4 Render structured logs in the new UI rather than showing
raw JSON
add ee20889bde9 Fix typescript useLogs
add 9de2af27747 style: group metadata pop
add 7c22edc8649 style: reduce if-else and directly use bool for assigning
metadata["download_logs"]
add a729667146c style: improve type annotation
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0cbedc283f7)
\
N -- N -- N refs/heads/render-json-logs-new-ui (a729667146c)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/actions/install-pre-commit/action.yml | 2 +-
.github/boring-cyborg.yml | 1 -
.github/workflows/news-fragment.yml | 2 +-
Dockerfile | 8 +-
Dockerfile.ci | 8 +-
airflow/api/common/trigger_dag.py | 2 +-
airflow/api_fastapi/core_api/datamodels/assets.py | 6 +-
.../datamodels/{dag_stats.py => ui/common.py} | 32 +-
.../core_api/datamodels/ui/structure.py | 19 +-
.../api_fastapi/core_api/openapi/v1-generated.yaml | 139 +-
airflow/api_fastapi/core_api/routes/public/log.py | 5 +-
.../core_api/routes/public/task_instances.py | 6 +-
airflow/api_fastapi/core_api/routes/ui/__init__.py | 2 +
.../api_fastapi/core_api/routes/ui/dependencies.py | 72 +
airflow/api_fastapi/core_api/routes/ui/grid.py | 10 +-
.../commands/remote_commands/variable_command.py | 14 +-
airflow/dag_processing/processor.py | 4 +-
airflow/example_dags/plugins/event_listener.py | 2 +-
airflow/executors/local_executor.py | 6 +-
airflow/executors/workloads.py | 34 +-
airflow/jobs/scheduler_job_runner.py | 18 +-
airflow/jobs/triggerer_job_runner.py | 953 +++---
airflow/macros/__init__.py | 71 +-
.../versions/0047_3_0_0_add_dag_versioning.py | 10 -
airflow/models/baseoperatorlink.py | 3 -
airflow/models/dag.py | 6 +-
airflow/models/dag_version.py | 1 -
airflow/models/dagrun.py | 85 +-
airflow/models/taskinstance.py | 5 +-
airflow/models/trigger.py | 97 +-
airflow/models/xcom.py | 3 -
airflow/serialization/serialized_objects.py | 8 +-
airflow/timetables/trigger.py | 2 +-
airflow/triggers/base.py | 143 +-
airflow/ui/openapi-gen/queries/common.ts | 13 +
airflow/ui/openapi-gen/queries/prefetch.ts | 12 +
airflow/ui/openapi-gen/queries/queries.ts | 20 +
airflow/ui/openapi-gen/queries/suspense.ts | 20 +
airflow/ui/openapi-gen/requests/schemas.gen.ts | 137 +-
airflow/ui/openapi-gen/requests/services.gen.ts | 16 +
airflow/ui/openapi-gen/requests/types.gen.ts | 73 +-
airflow/utils/log/file_task_handler.py | 9 +-
airflow/utils/log/log_reader.py | 9 +-
airflow/utils/log/trigger_handler.py | 140 -
contributing-docs/11_provider_packages.rst | 12 +-
dev/breeze/doc/ci/02_images.md | 4 +-
.../commands/release_management_commands.py | 4 +-
.../airflow_breeze/commands/testing_commands.py | 6 -
dev/breeze/src/airflow_breeze/global_constants.py | 4 +-
docs/apache-airflow/img/airflow_erd.sha256 | 2 +-
docs/apache-airflow/img/airflow_erd.svg | 3535 ++++++++++----------
docs/build_docs.py | 7 +-
docs/exts/docs_build/lint_checks.py | 11 +-
docs/spelling_wordlist.txt | 4 +
generated/provider_dependencies.json | 1 +
hatch_build.py | 3 +-
newsfragments/46415.significant.rst | 25 +
newsfragments/46613.significant.rst | 43 +-
.../tests/unit/amazon/aws/triggers/test_base.py | 1 +
.../providers/celery/executors/celery_executor.py | 21 +-
.../celery/executors/celery_executor_utils.py | 8 +-
.../cncf/kubernetes/decorators/kubernetes.py | 2 +-
providers/edge/README.rst | 6 +-
providers/edge/docs/changelog.rst | 18 +
providers/edge/provider.yaml | 2 +-
providers/edge/pyproject.toml | 6 +-
.../edge/src/airflow/providers/edge/__init__.py | 2 +-
.../src/airflow/providers/edge/cli/edge_command.py | 137 +-
.../airflow/providers/edge/get_provider_info.py | 2 +-
.../providers/edge/worker_api/routes/worker.py | 1 +
.../edge/tests/unit/edge/cli/test_edge_command.py | 44 +-
.../airflow/providers/fab/www/package-lock.json | 7 +-
providers/google/README.rst | 1 +
providers/google/docs/operators/cloud/bigquery.rst | 67 +-
providers/google/docs/operators/cloud/dataplex.rst | 112 +
providers/google/docs/operators/cloud/looker.rst | 4 +-
.../google/docs/operators/cloud/managed_kafka.rst | 78 +
.../operators/marketing_platform/display_video.rst | 20 +-
.../operators/transfer/facebook_ads_to_gcs.rst | 2 +-
.../docs/operators/transfer/presto_to_gcs.rst | 14 +-
.../docs/operators/transfer/salesforce_to_gcs.rst | 2 +-
.../google/docs/sensors/google-cloud-tasks.rst | 2 +-
providers/google/provider.yaml | 14 +
providers/google/pyproject.toml | 1 +
.../providers/google/cloud/hooks/bigquery.py | 76 +-
.../providers/google/cloud/hooks/dataplex.py | 419 ++-
.../providers/google/cloud/hooks/managed_kafka.py | 288 ++
.../providers/google/cloud/links/dataplex.py | 25 +
.../cloud/links/{datastore.py => managed_kafka.py} | 36 +-
.../google/cloud/log/stackdriver_task_handler.py | 7 +-
.../providers/google/cloud/operators/bigquery.py | 188 +-
.../providers/google/cloud/operators/dataplex.py | 721 +++-
.../google/cloud/operators/managed_kafka.py | 451 +++
.../airflow/providers/google/get_provider_info.py | 18 +
.../marketing_platform/example_dags/__init__.py | 16 -
.../google/cloud/bigquery/example_bigquery_dts.py | 21 +-
.../google/cloud/bigquery/example_bigquery_jobs.py | 8 +-
.../cloud/bigquery/example_bigquery_operations.py | 73 +-
.../example_bigquery_operations_location.py | 90 -
.../cloud/bigquery/example_bigquery_queries.py | 14 +-
.../bigquery/example_bigquery_queries_async.py | 9 +-
.../bigquery/example_bigquery_queries_location.py | 237 --
.../cloud/bigquery/example_bigquery_sensors.py | 14 +-
.../cloud/bigquery/example_bigquery_tables.py | 51 +-
.../cloud/bigquery/example_bigquery_to_bigquery.py | 24 +-
.../cloud/bigquery/example_bigquery_to_gcs.py | 23 +-
.../bigquery/example_bigquery_to_gcs_async.py | 116 -
.../cloud/bigquery/example_bigquery_to_mssql.py | 15 +-
.../cloud/bigquery/example_bigquery_to_mysql.py | 15 +-
.../cloud/bigquery/example_bigquery_to_postgres.py | 16 +-
.../cloud/bigquery/example_bigquery_transfer.py | 24 +-
.../cloud/bigquery/example_bigquery_value_check.py | 9 +-
.../google/cloud/dataflow/example_dataflow_yaml.py | 17 +-
.../cloud/dataplex/example_dataplex_catalog.py | 96 +-
.../google/cloud/dataplex/example_dataplex_dp.py | 14 +-
.../google/cloud/dataplex/example_dataplex_dq.py | 14 +-
.../system/google/cloud/gcs/example_firestore.py | 12 +-
.../google/cloud/gcs}/example_presto_to_gcs.py | 72 +-
.../google/cloud/gcs}/example_salesforce_to_gcs.py | 92 +-
.../google/cloud/gcs/example_trino_to_gcs.py | 35 +-
.../tests/system/google/cloud/looker}/__init__.py | 0
.../system/google/cloud/looker}/example_looker.py | 33 +-
.../system/google/cloud/managed_kafka}/__init__.py | 0
.../managed_kafka/example_managed_kafka_cluster.py | 140 +
.../system/google/cloud/tasks/example_queue.py | 13 +-
.../transfers}/example_facebook_ads_to_gcs.py | 77 +-
.../marketing_platform}/example_display_video.py | 61 +-
.../tests/unit/google/cloud/hooks/test_bigquery.py | 167 +-
.../tests/unit/google/cloud/hooks/test_dataplex.py | 174 +
.../unit/google/cloud/hooks/test_managed_kafka.py | 291 ++
.../tests/unit/google/cloud/links/test_dataplex.py | 28 +
.../unit/google/cloud/links/test_managed_kafka.py | 89 +
.../unit/google/cloud/operators/test_bigquery.py | 462 ++-
.../unit/google/cloud/operators/test_dataplex.py | 243 ++
.../google/cloud/operators/test_managed_kafka.py | 223 ++
.../transfers/test_facebook_ads_to_gcs_system.py | 76 -
.../transfers/test_salesforce_to_gcs_system.py | 48 -
.../operators/test_display_video_system.py | 68 -
.../providers/microsoft/azure/hooks/msgraph.py | 2 +-
.../providers/microsoft/azure/operators/msgraph.py | 53 +-
.../providers/microsoft/azure/sensors/msgraph.py | 21 +-
.../unit/microsoft/azure/hooks/test_msgraph.py | 21 +
.../unit/microsoft/azure/operators/test_msgraph.py | 99 +-
.../unit/microsoft/azure/sensors/test_msgraph.py | 47 +-
.../unit/openlineage/plugins/test_execution.py | 2 -
.../providers/standard/operators/trigger_dagrun.py | 26 +-
.../providers/standard/triggers/external_task.py | 87 +-
.../unit/standard/triggers/test_external_task.py | 25 +-
scripts/ci/install_breeze.sh | 4 +-
.../ci/pre_commit/update_providers_dependencies.py | 39 +-
scripts/docker/install_airflow.sh | 4 +-
.../in_container/run_provider_yaml_files_check.py | 11 +-
scripts/tools/setup_breeze | 2 +-
task_sdk/pyproject.toml | 3 +
task_sdk/src/airflow/sdk/api/client.py | 4 +-
task_sdk/src/airflow/sdk/definitions/macros.py | 56 +
.../src/airflow/sdk/execution_time/supervisor.py | 4 +-
task_sdk/src/airflow/sdk/log.py | 2 +
task_sdk/tests/definitions/test_macros.py | 71 +
tests/always/test_project_structure.py | 3 +
.../endpoints/test_extra_link_endpoint.py | 11 +-
tests/api_fastapi/common/test_exceptions.py | 6 +-
.../core_api/routes/public/test_task_instances.py | 48 +-
.../api_fastapi/core_api/routes/ui/test_assets.py | 9 +-
.../core_api/routes/ui/test_dependencies.py | 130 +
tests/api_fastapi/core_api/routes/ui/test_grid.py | 4 +-
.../remote_commands/test_variable_command.py | 44 +
tests/jobs/test_scheduler_job.py | 102 +-
tests/jobs/test_triggerer_job.py | 671 ++--
tests/jobs/test_triggerer_job_logging.py | 435 ---
tests/macros/test_macros.py | 93 -
tests/models/test_dag.py | 2 +-
tests/models/test_dagrun.py | 169 +-
tests/models/test_taskinstance.py | 2 +-
tests/operators/test_trigger_dagrun.py | 69 +-
tests/serialization/test_dag_serialization.py | 2 +-
tests/timetables/test_trigger_timetable.py | 25 +
tests/www/views/test_views_tasks.py | 15 +-
tests_common/pytest_plugin.py | 97 +-
179 files changed, 9298 insertions(+), 5234 deletions(-)
copy airflow/api_fastapi/core_api/datamodels/{dag_stats.py => ui/common.py}
(59%)
create mode 100644 airflow/api_fastapi/core_api/routes/ui/dependencies.py
delete mode 100644 airflow/utils/log/trigger_handler.py
create mode 100644 newsfragments/46415.significant.rst
create mode 100644 providers/google/docs/operators/cloud/managed_kafka.rst
create mode 100644
providers/google/src/airflow/providers/google/cloud/hooks/managed_kafka.py
copy providers/google/src/airflow/providers/google/cloud/links/{datastore.py
=> managed_kafka.py} (62%)
create mode 100644
providers/google/src/airflow/providers/google/cloud/operators/managed_kafka.py
delete mode 100644
providers/google/src/airflow/providers/google/marketing_platform/example_dags/__init__.py
delete mode 100644
providers/google/tests/system/google/cloud/bigquery/example_bigquery_operations_location.py
delete mode 100644
providers/google/tests/system/google/cloud/bigquery/example_bigquery_queries_location.py
delete mode 100644
providers/google/tests/system/google/cloud/bigquery/example_bigquery_to_gcs_async.py
rename providers/google/{src/airflow/providers/google/cloud/example_dags =>
tests/system/google/cloud/gcs}/example_presto_to_gcs.py (73%)
rename providers/google/{src/airflow/providers/google/cloud/example_dags =>
tests/system/google/cloud/gcs}/example_salesforce_to_gcs.py (56%)
copy {airflow/api_connexion =>
providers/google/tests/system/google/cloud/looker}/__init__.py (100%)
rename providers/google/{src/airflow/providers/google/cloud/example_dags =>
tests/system/google/cloud/looker}/example_looker.py (69%)
copy {airflow/api_connexion =>
providers/google/tests/system/google/cloud/managed_kafka}/__init__.py (100%)
create mode 100644
providers/google/tests/system/google/cloud/managed_kafka/example_managed_kafka_cluster.py
rename providers/google/{src/airflow/providers/google/cloud/example_dags =>
tests/system/google/cloud/transfers}/example_facebook_ads_to_gcs.py (63%)
rename
providers/google/{src/airflow/providers/google/marketing_platform/example_dags
=> tests/system/google/marketing_platform}/example_display_video.py (85%)
create mode 100644
providers/google/tests/unit/google/cloud/hooks/test_managed_kafka.py
create mode 100644
providers/google/tests/unit/google/cloud/links/test_managed_kafka.py
create mode 100644
providers/google/tests/unit/google/cloud/operators/test_managed_kafka.py
delete mode 100644
providers/google/tests/unit/google/cloud/transfers/test_facebook_ads_to_gcs_system.py
delete mode 100644
providers/google/tests/unit/google/cloud/transfers/test_salesforce_to_gcs_system.py
delete mode 100644
providers/google/tests/unit/google/marketing_platform/operators/test_display_video_system.py
create mode 100644 tests/api_fastapi/core_api/routes/ui/test_dependencies.py
delete mode 100644 tests/jobs/test_triggerer_job_logging.py
delete mode 100644 tests/macros/test_macros.py