TJaniF opened a new pull request, #29842:
URL: https://github.com/apache/airflow/pull/29842
When using a templated `tag_name` in the current GithubTagSensor, the
templated argument will not correctly be passed to the call to the
GithubSensor.poke method to be used in the `tag_checker` method that is passed
in as the `result_processor` function.
The result is that the sensor is looking for a tag of the format of the
string template as shown in the logs:
```text
[2023-03-01, 15:44:20 UTC] {custom_githubtagsensor.py:191} INFO - Poking for
tag: sync-metadata/1 in repository: TJaniF/airflow-fivetran-tutorial
[2023-03-01, 15:44:20 UTC] {base.py:73} INFO - Using connection ID
'github_conn' for task execution.
[2023-03-01, 15:44:21 UTC] {custom_githubtagsensor.py:210} INFO - Tag {{
ti.xcom_pull(task_ids='generate_tag_to_await', key='return_value') }} doesn't
exists in TJaniF/airflow-fivetran-tutorial repository yet.
```
This PR adds an `allow_templates_in_result_processor` parameter to the
GithubSensor, False by default to allow for backwards compatibility for users
who have built custom operators on top of the GithubSensor. If
`allow_templates_in_result_processor` is set to True as the GithubTagSensor now
does by default, a dict containing the values of all templated arguments can be
passed to the call to `GithubSensor.poke` and within that method to
`self.result_processor`.
Within the GithubTagSensor the `tag_checker` method now pulls the `tag_name`
from this passed dictionary instead of using self.tag_name, which was
retrieving the template-string before.
I also added `(BaseGithubRepositorySensor can't be used directly to create
Airflow tasks)` to the AirflowException raised when using the
BaseGithubRepositorySensor directly in a DAG, because the `Override me.`
message alone might be confusing for new Airflow users accidentally using this
sensor directly.
I'm of course very happy to take suggestions on how to better solve this
(and better naming 😅 ). I did not make any changes to the test running on the
GitHubTagSensor yet because I am not sure how to mock templating in a unittest.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
