Hi Matthias, Thanks for your comments! Please see my reply inline.
> What do we do if we have enough VMs? Do we still allow companies to add more VMs to the pool even though it's not adding any value? The ASF policy[1] makes it very clear: "Project Thanks pages are to show appreciation for goods that the project truly needs, not just for goods that someone wants to donate." Therefore, the community should reject new VMs if it is enough. > The community lacks the openly accessible tools to monitor the VM usage independently as far as I know (the Azure Pipelines project is owned by Ververica right now). The Azure pipeline account is sponsored by Ververica, and is managed by the community. AFAIK, Chesnay and Robert both have admin permissions [2] to the Azure pipeline project. Others can contact the managers to get access to the environment. > I figured that there could be a chance for us to rely on Apache-provided infrastructure entirely with our current workload when switching over from Azure Pipelines. That sounds great. We can return back the VMs and mark the donations as historical on the Thank Page once the new GitHub Actions CI is ready. > I am fine with creating a Thank You page to acknowledge the financial contributions from Alibaba and Ververica in the past (since Apache allows historical donations) considering that the contributions of the two companies go way back in time and are quite significant in my opinion. I suggest focusing on the past for now because of the option to migrate to Apache infrastructure midterm. Sorry, do you mean we only mention past donations for now? IIUC, the new GitHub Actions might be ready after the end of v1.20, which probably be in half a year. I'm worried that if we say the sponsorship is ongoing until now (but it's not), it will confuse people and disrespect the sponsor. Besides, I'm not sure whether the new GitHub Actions CI will replace the machines for running flink-ci mirrors [3] and the flink benchmarks [4]. If not, I think it's inappropriate to say they are historical donations. Furthermore, we are collecting all kinds of donations. I just noticed that AWS donated [5] service costs for flink-connector-aws tests that hit real AWS services. This is an ongoing donation and I think it's not good to mark it as a historical donation. (Thanks for the donation, AWS, @Danny Cranmer <dannycran...@apache.org> @HongTeoh! We should add it to the Thank Page!) Best, Jark [1]: https://www.apache.org/foundation/marks/linking#projectthanks [2]: https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration#ContinuousIntegration-Contacts [3]: https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration#ContinuousIntegration-Repositories [4]: https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 [5]: https://issues.apache.org/jira/browse/INFRA-24474 On Wed, 6 Mar 2024 at 17:58, Matthias Pohl <map...@apache.org> wrote: > Thanks for starting this discussion. I see the value of such a page if we > want to encourage companies to sponsor CI infrastructure in case we need > this infrastructure (as Yun Tang pointed out). The question is, though: Do > we need more VMs? The amount of commits to master is constantly decreasing > since its peak in 2019/2020 [1]. Did we observe shortage of CI runners in > the past years? What do we do if we have enough VMs? Do we still allow > companies to add more VMs to the pool even though it's not adding any > value? Then it becomes a marketing tool for companies. The community lacks > the openly accessible tools to monitor the VM usage independently as far as > I know (the Azure Pipelines project is owned by Ververica right now). My > concern is (which goes towards what Max is saying) that this can be a > source of friction in the community (even if it's not about individuals but > companies). I'm not sure whether the need for additional infrastructure > out-weights the risk for friction. > > On another note: After monitoring the GitHub Action workflows (FLIP-396 > [2]) for the past weeks, I figured that there could be a chance for us to > rely on Apache-provided infrastructure entirely with our current workload > when switching over from Azure Pipelines. But that might be a premature > judgement because the monitoring started after the feature freeze of Flink > 1.19. We should wait with a final conclusion till the end of the 1.20 > release cycle. Apache Infra increased the amount of VMs they are offering > since 2018 (when the Apache Flink community decided to go for Azure > Pipelines and custom VMs as far as I know). That's based on a conversation > I had with the Apache Infra folks at one of their roundtable meetings [3]. > This and the fact that the amount of commits is decreasing in recent years > [1] (which correlates with the number of CI runs) could be indications that > additional VMs are not necessary (and with that, the need to have a Thank > You page as well). > > But I acknowledge that Alibaba and Ververica would like to be recognized > for their financial contributions to the community in the past. Therefore, > I am fine with creating a Thank You page to acknowledge the financial > contributions from Alibaba and Ververica in the past (since Apache allows > historical donations) considering that the contributions of the two > companies go way back in time and are quite significant in my opinion. I > suggest focusing on the past for now because of the option to migrate to > Apache infrastructure midterm. > > Best, > Matthias > > [1] https://github.com/apache/flink/graphs/contributors > [2] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-396%3A+Trial+to+test+GitHub+Actions+as+an+alternative+for+Flink%27s+current+Azure+CI+infrastructure > [3] > > https://cwiki.apache.org/confluence/display/INFRA/Infra+Roundtable+2023-12-06%2C+17%3A00+UTC > > On Wed, Mar 6, 2024 at 7:06 AM tison <wander4...@gmail.com> wrote: > > > > a rare way different than > > > individuals (few individuals can donate such resources) > > > > Theoretically, if an individual donates so, we list list him/her as well. > > > > I've seen such donations in The Perl Foundation like [1]. But since a > > PMC doesn't have a fundraising office, we may not accept raw money > > anyway; it's already out of the thread :D > > > > Best, > > tison. > > > > [1] https://news.perlfoundation.org/post/announcement_of_the_ian_hague > > > > Yun Tang <myas...@live.com> 于2024年3月6日周三 13:58写道： > > > > > > Thanks for Jark's proposal, and I'm +1 for adding such a page. > > > > > > The CI infrastructure helps the Apache Flink project to run well. I > > cannot imagine how insufficient CI machines would impact the development > > progress, especially when the feature freeze date is close. And I believe > > that most guys who contributed to the community would not know Alibaba > and > > Ververica had ever donated several machines to make the community work > > smoothly for years. > > > > > > > > > Best > > > Yun Tang > > > ________________________________ > > > From: Jark Wu <imj...@gmail.com> > > > Sent: Wednesday, March 6, 2024 11:35 > > > To: dev@flink.apache.org <dev@flink.apache.org> > > > Subject: Re: [DISCUSS] Add "Special Thanks" Page on the Flink Website > > > > > > Hi Max, > > > > > > Thank you for your input. > > > > > > According to ASF policy[1], the Thank Page is intended to thank third > > > parties > > > that provide physical resources like machines, services, and software > > that > > > the committers > > > or the project truly needs. I agree with Tison, such donation is > > countable > > > and that's why > > > I started this discussion to collect the full list. The thank Page is > not > > > intended to thank working > > > hours or contributions from individual volunteers which I think > > > is recognized in other ways > > > (e.g., credit of committer and PMC member). > > > > > > Best, > > > Jark > > > > > > [1]: https://www.apache.org/foundation/marks/linking#projectthanks > > > > > > On Wed, 6 Mar 2024 at 01:14, tison <wander4...@gmail.com> wrote: > > > > > > > Hi Max, > > > > > > > > Thanks for sharing your concerns :D > > > > > > > > I'd elaborate a bit on this topic with an example, that Apache > Airflow > > > > has a small section for its special sponsor who offers machines for > CI > > > > also [1]. > > > > > > > > In my understanding, companies employ developers to invest time in > the > > > > development of Flink and that is large, vague, and hard to be fair to > > > > list all of the companies. > > > > > > > > However, physical resources like CI machines are countable and they > > > > help the sustainability of our project in a rare way different than > > > > individuals (few individuals can donate such resources). We can > > > > maintain such a section or page for those sponsors so that it also > > > > decreases the friction when the company asks "what we can gain" (for > > > > explicit credits, at least, and easy understanding). > > > > > > > > Any entity is welcome to add themselves as long as it's valid. > > > > > > > > For the fair part, I'm not an employee of both companies listed on > the > > > > demo page and I don't feel uncomfortable. Those companies do invest a > > > > lot on our project and I'd regard it as a chance to encourage other > > > > companies to follow. > > > > > > > > Best, > > > > tison. > > > > > > > > [1] https://github.com/apache/airflow?tab=readme-ov-file#sponsors > > > > > > > > Maximilian Michels <m...@apache.org> 于2024年3月6日周三 00:49写道： > > > > > > > > > > I'm a bit torn on this idea. On the one hand, it makes sense to > thank > > > > > sponsors and entities who have supported Flink in the past. On > other > > > > > hand, this list is bound to be incomplete and maybe also biased, > even > > > > > if not intended to be so. I think the power of open-source comes > from > > > > > the unconditional donation of code and knowledge. Infrastructure > > costs > > > > > are a reality and donations in that area are meaningful, but they > are > > > > > just one piece of the total sum which consists of many volunteers > and > > > > > working hours. In my eyes, a Thank You page would have to display > > each > > > > > entity fairly which is going to be hard to achieve. > > > > > > > > > > -Max > > > > > > > > > > On Tue, Mar 5, 2024 at 2:30 PM Jingsong Li <jingsongl...@gmail.com > > > > > > wrote: > > > > > > > > > > > > +1 for setting up > > > > > > > > > > > > On Tue, Mar 5, 2024 at 5:39 PM Jing Ge > <j...@ververica.com.invalid > > > > > > > wrote: > > > > > > > > > > > > > > +1 and thanks for the proposal! > > > > > > > > > > > > > > Best regards, > > > > > > > Jing > > > > > > > > > > > > > > On Tue, Mar 5, 2024 at 10:26 AM tison <wander4...@gmail.com> > > wrote: > > > > > > > > > > > > > > > I like this idea, so +1 for setting up. > > > > > > > > > > > > > > > > For anyone who have the access, this is a related thread > about > > > > > > > > project-wise sponsor in the foundation level [1]. > > > > > > > > > > > > > > > > Best, > > > > > > > > tison. > > > > > > > > > > > > > > > > [1] > > > > https://lists.apache.org/thread/2nv0x9gfk9lfnpb2315xgywyx84y97v6 > > > > > > > > > > > > > > > > Jark Wu <imj...@gmail.com> 于2024年3月5日周二 17:17写道： > > > > > > > > > > > > > > > > > > Sorry, I posted the wrong [7] link. The Flink benchmark ML > > link > > > > is: > > > > > > > > > > > https://lists.apache.org/thread/bkw6ozoflgltwfwmzjtgx522hyssfko6 > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, 5 Mar 2024 at 16:56, Jark Wu <imj...@gmail.com> > > wrote: > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I want to propose adding a "Special Thanks" page to our > > Apache > > > > Flink > > > > > > > > website [1] > > > > > > > > > > > > > > > > > > > > to honor and appreciate the > > > > > > > > > > companies and organizations that have sponsored > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > machines or services for our project. The establishment > of > > > > such a page > > > > > > > > serves as > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > a public acknowledgment of our sponsors' contributions > and > > > > > > > > simultaneously acts > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > as a positive encouragement for other entities to > consider > > > > supporting > > > > > > > > our project. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Adding Per-Project Thanks pages is allowed by ASF > > policy[2], > > > > which > > > > > > > > says "PMCs > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > may wish to provide recognition for third parties that > > provide > > > > > > > > software or services > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > to the project's committers to further the goals of the > > > > project. These > > > > > > > > are typically > > > > > > > > > > > > > > > > > > > > called Per-Project Thanks pages". Many Apache projects > > have > > > > added such > > > > > > > > > > > > > > > > > > > > pages, for example, Apache HBase[3] and Apache Mina[4]. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > To initiate this idea, I have drafted a preliminary page > > under > > > > the > > > > > > > > > > "About" menu > > > > > > > > > > > > > > > > > > > > on the > > > > > > > > > > Flink website to specifically thank Alibaba and > Ververica, > > by > > > > following > > > > > > > > > > > > > > > > > > > > the ASF guidelines and the Apache Mina project. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > page image: > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/flink/assets/5378924/e51aaffe-565e-46d1-90af-3900904afcc0 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Below companies are on the thanks list for their donation > > to > > > > Flink > > > > > > > > testing infrastructure: > > > > > > > > > > > > > > > > > > > > - Alibaba donated 8 machines (32vCPU,64GB) for running > > Flink > > > > CI builds > > > > > > > > [5]. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > - Ververica donated 2 machines for hosting flink-ci > > > > repositories [6] > > > > > > > > and running Flink benchmarks [7]. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I may miss some other donations or companies, please add > > them > > > > if you > > > > > > > > know. > > > > > > > > > > > > > > > > > > > > Looking forward to your feedback about this proposal! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Best, > > > > > > > > > > > > > > > > > > > > Jark > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1]: https://flink.apache.org/ > > > > > > > > > > > > > > > > > > > > [2]: > > > > https://www.apache.org/foundation/marks/linking#projectthanks > > > > > > > > > > > > > > > > > > > > [3]: https://hbase.apache.org/sponsors.html > > > > > > > > > > > > > > > > > > > > [4]: https://mina.apache.org/special-thanks.html > > > > > > > > > > > > > > > > > > > > [5]: > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Azure+Pipelines#AzurePipelines-AvailableCustomBuildMachines > > > > > > > > > > > > > > > > > > > > [6]: > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/Continuous+Integration > > > > > > > > > > > > > > > > > > > > [7]: > > > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/41a68c775753a7841896690c75438e0a497634102e676db880f30225@%3Cdev.flink.apache.org%3E > > > > > > > > > > > > > > > > > > > > > > > > >
