potiuk commented on issue #31819: URL: https://github.com/apache/airflow/issues/31819#issuecomment-1586123860
> That is true but even if we have 1, then the question of what is the difference is legit and we should try to clarify the mystery. Maybe we are talking about the same :). I do not think we argue here about the need, it's more on the fact that currently we simply do not have any standards there. I believe currently when "defferable=true" is just an internal convention in some of the sensors - GCS/S3/RedshiftCluster, the documentation should be there I think, because this is not a pattern we even have as a "common one". So far it just "happened" that we are using similar approach. I am not arguing we the need of documentation, I just think the place where it should be documented (if we do not make it a common approach) should be in those operators. However, what I see from this discussion is that we DO have an emerging pattern, that is worth explaining, but I think we should propose a PR describing why/how other priovider can apply it as well. Maybe this is exactly what you proposed, and we are just phrasing it differently @eladkal. I read the idea here is "let's discuss and agree what pattern we can apply for current and future sensors" rather than "let's improve the docs". The point here is that we have no patterns here (yet) because we've never discussed it. The pattern is clearly emerging and it's a good thing, and what I think here is that it should be a proposal - maybe just discussion here + summarising it to the devlist after we get some clear conclusions and proposed PR describing it is enough. I already see a few good things: @shubham22 explaining why sometimes you choose to do Operator and why sometimes you want to do a Sensor that can have a deferrable mode. So what I think here, we need to introduce it as a pattern, and have a document that: * describes when you might consider to implement a Sensor with deferrable=parameter and when you want to stick with deferrable Operators * agree that this is a common convention and pattern that is typically used (possibly with some examples) * explain the difference and use cases for those So maybe we are talking about the same. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
