This is an automated email from the ASF dual-hosted git repository.
o-nikolas pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 8004f48312d Remove hard-coded deferrable crawler run from example_glue
system test (#70206)
8004f48312d is described below
commit 8004f48312d5521d65ec2ae733a2b9dd6ece20c2
Author: Sean Ghaeli <[email protected]>
AuthorDate: Thu Jul 23 14:16:33 2026 -0700
Remove hard-coded deferrable crawler run from example_glue system test
(#70206)
The run_crawler_deferrable task added in #69930 hard-codes deferrable=True,
which hangs forever under dag.test(use_executor=True) (no triggerer runs in
that path), timing out executor-based system test harnesses. The non-deferrable
run_crawler task covers the operator; deferrable behavior is exercised by unit
tests and the harness-level deferrable mode
(AIRFLOW__OPERATORS__DEFAULT_DEFERRABLE).
---
providers/amazon/docs/operators/glue.rst | 6 ------
providers/amazon/tests/system/amazon/aws/example_glue.py | 10 ----------
2 files changed, 16 deletions(-)
diff --git a/providers/amazon/docs/operators/glue.rst
b/providers/amazon/docs/operators/glue.rst
index e4a9466ba22..ce97e693f3a 100644
--- a/providers/amazon/docs/operators/glue.rst
+++ b/providers/amazon/docs/operators/glue.rst
@@ -88,12 +88,6 @@ To run an existing crawler and wait for it to complete, use
The operator waits for completion by default. Set ``deferrable=True`` to
perform the wait without
occupying a worker slot.
-.. exampleinclude:: /../../amazon/tests/system/amazon/aws/example_glue.py
- :language: python
- :dedent: 4
- :start-after: [START howto_operator_glue_crawler_run_deferrable]
- :end-before: [END howto_operator_glue_crawler_run_deferrable]
-
.. _howto/operator:GlueCrawlerDeleteOperator:
Delete an AWS Glue crawler
diff --git a/providers/amazon/tests/system/amazon/aws/example_glue.py
b/providers/amazon/tests/system/amazon/aws/example_glue.py
index b6881a5683e..30d03c29d15 100644
--- a/providers/amazon/tests/system/amazon/aws/example_glue.py
+++ b/providers/amazon/tests/system/amazon/aws/example_glue.py
@@ -161,19 +161,10 @@ with DAG(
verify_crawler_update = verify_crawler_description(glue_crawler_name,
updated_crawler_description)
- # [START howto_operator_glue_crawler_run_deferrable]
- run_crawler_deferrable = GlueCrawlerRunOperator(
- task_id="run_crawler_deferrable",
- crawler_name=glue_crawler_name,
- deferrable=True,
- )
- # [END howto_operator_glue_crawler_run_deferrable]
-
# [START howto_operator_glue_crawler_run]
run_crawler = GlueCrawlerRunOperator(
task_id="run_crawler",
crawler_name=glue_crawler_name,
- deferrable=False,
)
# [END howto_operator_glue_crawler_run]
@@ -255,7 +246,6 @@ with DAG(
create_crawler,
update_crawler,
verify_crawler_update,
- run_crawler_deferrable,
run_crawler,
wait_for_crawl,
wait_for_catalog_partition,
