I love this idea. I definitely think it can improve a lot the knowledge sharing across Airflow. Given the history and the number of components in Airflow, it is hard to keep up with everything, so having these ADRs would help a lot I think!
On 2023/12/03 23:57:11 Jarek Potiuk wrote: > Hey everyone, > > I think we had a bit of clash in > https://github.com/apache/airflow/pull/32319 where both "ideas" > (serialization and common.sql) had not been sufficiently > discussed/explained and I hope we can address it by adding (a bit) more > "whys" to our (developer) documentation. > > I think a number of our past decisions and reasoning behind them are often > staying in the heads of the people who were discussing them, and even if it > is captured in past discussions, PRs, it's difficult to do "archeology" on > them and re-process them and understand what we wanted to achieve and why. > Some of those are big enough to have impact on future PRs etc. while not > big enough to get to Airflow Improvement Proposals and I think we miss > a bit of persistent "decision records" for them. > > Two cases in question: Serialization and Common.sql API - both of which > have not been understood well by people involved in one, but not the other > in the past. > > With the "common.sql" PR (https://github.com/apache/airflow/pull/36015) - > my proposal is to add it in the form of ADR ("Architecture Decision > Records' - which is a very simple and lightweight way of storing the > decisions we made - and evolving them. > > ADRs are pretty popular and adopted in mature organisations/projects and > I've used them in `breeze` > https://github.com/apache/airflow/tree/main/dev/breeze/doc/adr and I think > they are perfect for capturing, context, decisions and putting down the > consequences of some decisions. > > They are usually kept close to the code the decision is about, they are > usually short and describe a single aspect of architectural decision, and > they are aimed to be read whenever in the future, people who were not > involved in those decisions can easily recover why the decisions are made > and what are the consequences of it. > > I am not saying - of course - we should do it for all or even most changes > - I am talking about decisions that have potential impact on others - in > the future. I.e. when we tell (this is how our approach should look in the > future for "general" behaviour. > > Both - serialization and common.sql are good examples of such decisions - > that I believe deserve to be captured "why" we are doing them and what we > wanted to achieve. > > WDYT? > > J. > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
