vincbeck commented on PR #74293: URL: https://github.com/apache/airflow/pull/74293#issuecomment-6342008453
> Sorry, one more thing, I'm realizing it now that I'm spending more time thinking about the triggerer. There's a failure case in the situation of one trigger serving multiple assets. Now the trigger is going to retrieve the ETag from the state store. But the state store has a separate ETag per asset, so which ETag should it retrieve when there are multiple assets? I tested it and it errors out in this case with `ValueError`. We should catch that error, and refer to the last ETag in memory > > Kinesis has dealt with the same idea in the following way, we could copy it: > > https://github.com/apache/airflow/blob/38193762bcfe38b1ba492b66688de906b59989b5/providers/amazon/src/airflow/providers/amazon/aws/triggers/kinesis.py#L160-L171 Good catch! However, I think this is an issue we should fix in Airflow, we should not ask each trigger to handle this. This feels wrong to me. Thanks for cutting #75352, we should work on that issue, get it fixed and then get back to this PR. I'll park it in the meantime -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
