josh-fell opened a new pull request #16876:
URL: https://github.com/apache/airflow/pull/16876
Closes #10285
- Updated example DAG files such that `xcom_pull()` calls use an operator's
`.output` property
- Added comments to which task dependencies, if any, are handled and/or
created via `XComArgs` for transparency
- Removed or refactored the `default_args` pattern where necessary as
requested by @ashb (i.e. removed a separated `default_args` declaration for
deference for declaration as part of the `DAG` object)
- Other miscellaneous updates based on `.output` refactoring
>**Note:** There are several instances where the `xcom_pull()` call was not
updated. These instances involve accessing a specific value within the `XCom`
or calling user-defined macros with an `XCom` value. Reference #16618 for an
open issue to enhance the `XComArg` functionality to provide similar behavior
as the classic `xcom_pull()` method.
> **Note:** Not all DAGs were tested functionally (i.e. with hard
integrations to source systems and executed), however each DAG was tested to
compile and generate a DAG graph as expected locally.
An detailed summary of all changes made as part of this PR can be found
below:
| DAG File | Converted `xcom_pull()`? | Other Updates? | Comments |
| ---------| ------------------------ | -------------- | -------- |
| airflow/providers/google/leveldb/example_dags/example_leveldb.py | No |
Yes | Removed unneeded `default_args` pattern. |
|
airflow/providers/google/marketing_platform/example_dags/example_campaign_manager.py
| No | No | Current `XComArg` object does not support accessing specific
values of an iterable `XCom` value elegantly. Did not update any occurrences in
this DAG. |
|
airflow/providers/google/marketing_platform/example_dags/example_display_video.py
| Yes ** | Yes | ** Current `XComArg` object does not support accessing
specific values of an iterable `XCom` value elegantly. Did not update the
following occurrence: <br/>`'{{
task_instance.xcom_pull("create_sdf_download_task")["name"] }}'`<br/><br/>All
other `xcom_pull()` calls have been updated. <br/><br/> Removed explicit task
dependencies that are created via `XComArgs`. </br></br>Updated this instance
to reference the correct task rather than an non-existant one:</br>`'{{
task_instance.xcom_pull("upload_sdf_to_bigquery")}}'`|
|
airflow/providers/google/marketing_platform/example_dags/example_search_ads.py
| Yes | Yes | Removed explicit task dependencies that are created via
`XComArgs`. |
| airflow/providers/google/suite/example_dags/example_gcs_to_sheets.py | No
| No | Current `XComArg` object does not support accessing specific values of
an iterable `XCom` value elegantly. Did not update any occurrences in this DAG.
|
| airflow/providers/google/suite/example_dags/example_sheets.py | Yes ** |
No | ** Current `XComArg` object does not support accessing specific values of
an iterable `XCom` value elegantly. Did not update the following occurrences:
<br/>`"{{ task_instance.xcom_pull('upload_sheet_to_gcs')[0] }}"`<br/><br/>All
other `xcom_pull()` calls have been updated. |
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**
for more information.
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in
[UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
