uranusjr commented on code in PR #68778:
URL: https://github.com/apache/airflow/pull/68778#discussion_r3509160785
##########
airflow-core/src/airflow/jobs/scheduler_job_runner.py:
##########
@@ -2277,23 +2278,32 @@ def _create_dagruns_for_partitioned_asset_dags(self,
session: Session) -> set[st
source_key_by_asset = source_key_by_asset_per_apdr[apdr.id]
timetable = dag.timetable
- statuses: dict[SerializedAssetUniqueKey, bool] = {}
- for asset_id, (name, uri) in asset_info_per_apdr[apdr.id].items():
- key = SerializedAssetUniqueKey(name=name, uri=uri)
- if timetable.partitioned:
- statuses[key] = self._resolve_asset_partition_status(
- session=session,
- asset_id=asset_id,
- name=name,
- uri=uri,
- apdr=apdr,
- timetable=timetable,
- actual_by_asset=source_key_by_asset,
- )
- else:
- statuses[key] = True
- if not evaluator.run(timetable.asset_condition, statuses=statuses):
- continue
+ contributing_assets = asset_info_per_apdr[apdr.id]
+ if RerunPolicy(apdr.rerun_policy).fires_immediately:
+ # A REFRESH-stamped APDR supersedes an already-fired window:
the
+ # rest of the window is still materialized, so fire immediately
+ # with the re-arrived events rather than re-satisfying the wait
+ # policy. Still require at least one active contributing asset.
+ if not contributing_assets:
+ continue
Review Comment:
Does this need to evaluate the downstream’s asset_condition is satisfied?
This would unconditionally re-fire everything. Maybe a note in docs if this is
intended?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
