So a number of ASF projects have been called on the reduce CI load (including Airflow and Spark). There's been a number of e-mails about this, one of the airflow e-mails is https://lists.apache.org/thread.html/0cs9lo9lqxcdbzpqt1yhzvzx5pn1h7dt?find_parent=true
On Mon, Oct 26, 2026 at 12:20 PM Ash Berlin-Taylor <[email protected]> wrote: > First off: If a proposal is big enough to need an AIP, definitionally, it > is not appropriate for lazy consensus. AIP’s are for big changes. Big > changes get actively voted upon. And since it _does_ have an AIP, you > already acknowledged at some level it is a big change. > > AIP-120 starts out by saying "Apache Airflow receives more pull requests > than its maintainers can review”. That framing is very different to “ASF > org asked us to reduce our CI usage”. > > Where in the AIP document or previous discussion thread was the “we need > to reduce our CI usage” called out as a goal? Ci time is mentioned, but > never “we need to reduce this because the ASF org has asked us to”. I too > would like my CI to jobs to run quicker. Who wouldn’t? > > Where have we been called upon to reduce CI load? It was on this list? I > can’t see it. And if it isn’t on list it essentially didn’t happen. Another > ASF maxim, of which you are also painfully aware. > > I feel you are taking liberties here and making big changes to the project > almost unilaterally. And I think you should apologise for doing it in a > tone deaf way without true consensus. > > > > > On 26 Oct 2026, at 18:29, Jarek Potiuk <[email protected]> wrote: > > > > BTW. Airflow has been called upon many times last week by other projects > as > > *biggest resource hog" many times, people were angry, people asked Infra > > to limit Airflow's resource usage. They literally called Infra to shut us > > down during the release process. > > > > The only way I contained that was by telling people we are already > working > > on it and we have a clear plan to follow. This required a lot of "being > > firm". One comment I received was that after 3 weeks of effort, I finally > > refined the design enough that it raised no objections, allowing me to > > follow through and help other projects—including helping them reduce > their > > usage - which benefits everyone. > > > > I added this page > https://infra.apache.org/github-actions-optimization.html > > and added a feature in Magpie that allows people to optimize their > > workflows practically automatically.. > > > > All that following all the rules we agreed to. > > > > I will leave it at that. > > > > J. > > > > > > On Mon, Oct 26, 2026 at 7:22 PM Jarek Potiuk <[email protected]> wrote: > > > >> Sorry. We reduced our resource usage by 2/3 > >> > >> On Mon, Oct 26, 2026 at 7:20 PM Jarek Potiuk <[email protected]> wrote: > >> > >>> Ash, > >>> > >>> * The original discussion where I posted my last response from me to > you > >>> on the change was Sep 22 and waited for your answer. That was 1.5 > months > >>> ago. > >>> https://lists.apache.org/thread.html/w74655o8y1jzrzcnygntyyovocf7zmzj > - > >>> I have not seen you respond to it and I believe I answered your > concerns > >>> > >>> *.I implemented PR limits as the first step (with your consent) of > >>> improvements and announced my plan for the next steps > >>> https://lists.apache.org/thread.html/y0vzry5s55k2gy17rc93632zp0d5733o > >>> > >>> * I adapted design, respondsed to your concerns and explained what I am > >>> going to do on this thread - where I started DISCUSSION on the updated > >>> design - > >>> https://lists.apache.org/thread.html/ky392mrrcmj312y1r276r30p584n804c > - > >>> that was 21 days ago. - it did not receive any objection from you > >>> > >>> * the discussion run until - 10 days ago > >>> https://lists.apache.org/thread.html/ky392mrrcmj312y1r276r30p584n804c > - > >>> where no-one raised significant objections, this discussion had links > to > >>> design, AIP, Draft PRs and my clear intentions to run LAZY CONSENSUS > >>> > >>> * I raised Lazy consensus > >>> https://lists.apache.org/thread.html/nfno4r1jqx5dj75v867prjjgg6o77odx > - > >>> 7 days ago > >>> > >>> * I announced result on Friday: - 4 days after it has been waiting > >>> > >>> Yes. AIP-120 is approved. I cannot account for someone not paying > >>> attention for 1.5 months, sorry. You are just one of the PMC members > here - > >>> nobody else does not object. You are not more important than others > and we > >>> are not going to wait with decisions, that also help the whole ASF deal > >>> with the difficult situation of CI resources being exhausted by > >>> AI—especially since we gave ample time to respond to two discussions > and > >>> the lazy consensus. > >>> > >>> Get over that, Ash. If you would like to change somethign in the way > how > >>> AIP-120 is configured feel free to start discussiosn (you already did > in > >>> #contributors and made some suggestions - which is good). > >>> > >>> It is implemented as is now. We can disable it any time if we do not > want > >>> to lose our contributors. We have many configuration options, and we > >>> already reduced our resource usage by one-third, unblocking not only > our > >>> queue but also other projects' queues. > >>> > >>> Let's discuss what we want to change and how to configure it, simply. > >>> Let's work toward the common goal of faster and better CI for Airflow. > >>> > >>> J. > >>> > >>> > >>> On Mon, Oct 26, 2026 at 6:44 PM Ash Berlin-Taylor <[email protected]> > wrote: > >>> > >>>> No. > >>>> > >>>> Jarek: this behaviour _was not okay_. You knew when creating the lazy > >>>> consensus that there was not consensus. > >>>> > >>>> This was never a valid use case of a Lazy Consensus. This discussion > >>>> already had strong disagreements. I was the most vocal, but I wasn’t > the > >>>> only person to mention disagreement. And yet you really thought lazy > >>>> consensus was appropriate tool here? How would you feel if I passed a > >>>> “Remove Breeze” via lazy consensus when I knew you were on holiday? > >>>> > >>>> As per ASF’s own guidance, which I know you already know. > >>>> > >>>>> Sometimes lazy consensus is not appropriate. In such cases it is > >>>> necessary to make a proposal to the email list and discuss options > >>>> > >>>> And since the discussion _already_ had pushback, lazy consensus was > >>>> categorically not appropriate. > >>>> > >>>> This is not an appropriate response Jarek. Not even remotely. If you > >>>> think it is you need to re-evaluate your priorities around community > >>>> building. > >>>> > >>>> I repeat. There is no consensus on AIP-120. > >>>> > >>>>> On 26 Oct 2026, at 14:59, Jarek Potiuk <[email protected]> wrote: > >>>>> > >>>>>> There is no consensus reached here. > >>>>> > >>>>> Consensus has been reached. We've been through that before; there > was a > >>>>> long discussion followed by arguments from the other side. Your only > >>>>> opposition was that it is aimed only to save CI time of shared > runners > >>>>> (which also it does). > >>>>> And as the last time we discussed - you can always call to revert > that > >>>>> decision. AIP-120 was approved 3 days ago. > >>>>> > >>>>>> Calling a lazy consensus on this AIP given how strongly I voiced my > >>>>> opposition to it is appropriate. > >>>>> > >>>>> You can always call a change to remove it. In the meantime, we > proceed > >>>> with > >>>>> testing it. Also, an alternative is that we would likely have to cut > >>>> down > >>>>> our usage of shared ASF CI because we are taking nearly 30% of it. > >>>> Some of > >>>>> the changes and optimizations Shahar and I implemented over the > weekend > >>>>> reduced our usage to one-third (also Arrow and Datafusion also made > >>>> similar > >>>>> optimizations with room for more) - which already had an effect on > >>>>> significant decrease of queue time (which was also goal of AIP-120). > >>>>> > >>>>> But yes - feel free to start a new discussion on reverting it. This > >>>> is the > >>>>> way how decisions are being made here. > >>>>> > >>>>> J > >>>>> > >>>>> > >>>>> On Mon, Oct 26, 2026 at 2:23 PM Ash Berlin-Taylor <[email protected]> > >>>> wrote: > >>>>> > >>>>>> I’m angry and making typos. Is _not appropriate_. > >>>>>> > >>>>>> > >>>>>> > >>>>>>> On 26 Oct 2026, at 13:22, Ash Berlin-Taylor <[email protected]> > wrote: > >>>>>>> > >>>>>>> Calling a lazy consensus on this AIP given how strongly I voiced my > >>>>>> opposition to it is appropriate. > >>>>>>> > >>>>>>> This should have never a candidate for lazy consensus, and was > opened > >>>>>> and closed when many of us were busy working on the 3.4 release. > >>>> There is > >>>>>> no consensus reached here. > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>>> On 23 Oct 2026, at 21:52, Jarek Potiuk <[email protected]> wrote: > >>>>>>>> > >>>>>>>> Hello everyone, > >>>>>>>> > >>>>>>>> The lazy consensus on AIP-120 [1] ended on 2026-10-22 12:00 UTC > with > >>>>>>>> no objections, so AIP-120 is accepted. > >>>>>>>> > >>>>>>>> Next steps, as described in the proposal: merge the design PR and > >>>> the > >>>>>>>> stacked implementation PRs (everything ships switched off), run > the > >>>>>>>> opt-in pilot, then switch it on for all external contributors in a > >>>>>>>> separate one-line change. > >>>>>>>> > >>>>>>>> Reviews on the stack are very welcome: #73378, #74819, #74820, > >>>> #74822, > >>>>>>>> #74823, #74825. > >>>>>>>> > >>>>>>>> [1] > >>>> https://lists.apache.org/thread/nfno4r1jqx5dj75v867prjjgg6o77odx > >>>>>>>> > >>>>>>>> J. > >>>>>>>> > >>>>>>>> > >>>> --------------------------------------------------------------------- > >>>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>>> For additional commands, e-mail: [email protected] > >>>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> > --------------------------------------------------------------------- > >>>>>>> To unsubscribe, e-mail: [email protected] > >>>>>>> For additional commands, e-mail: [email protected] > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > --------------------------------------------------------------------- > >>>>>> To unsubscribe, e-mail: [email protected] > >>>>>> For additional commands, e-mail: [email protected] > >>>>>> > >>>>>> > >>>> > >>>> > >>>> --------------------------------------------------------------------- > >>>> To unsubscribe, e-mail: [email protected] > >>>> For additional commands, e-mail: [email protected] > >>>> > >>>> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Twitter: https://twitter.com/holdenkarau Fight Health Insurance: https://www.fighthealthinsurance.com/ <https://www.fighthealthinsurance.com/?q=hk_email> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9> YouTube Live Streams: https://www.youtube.com/user/holdenkarau Pronouns: she/her
