uranusjr commented on PR #70225: URL: https://github.com/apache/airflow/pull/70225#issuecomment-6438642790
Since this has been waiting for a while, I’ve pushed changes to finish it off, addressing the open review threads. The remaining design question is the run-relative window (see https://github.com/apache/airflow/pull/70225#pullrequestreview-5634588382). The sensor isn’t that useful to me without a run-relative window. I see three ways to implement this: 1. `since=` and `lookback=` as proposed. Explicit, but each new anchor brings a new enum value and branch, and it needs combination rules with `after`. 2. Let `after` and `before` accept a callable `(context) -> datetime | str`. Smaller, covers any anchor, but a lambda in the Dag file is less readable for the common case. It’s also less friendly for non-Python usages, which Airflow has been moving toward. 3. A few ready-made bounds e.g. `after=RunStart(lookback=timedelta(hours=1))` or `after=DataIntervalEnd()`, each just an object that resolves itself against the context. The argument would still accept a callable as a last resort. That reads similarly for the common anchors and stays pluggable for everything else. One thing I’d keep out of this PR is “since the last event this sensor consumed”. It needs state across runs, and the only tool for that (the task state store) isn’t available on 3.2, so it probably deserves its own issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
