I think it is a good idea to make the benchmarks available to the community via a repo under the Apache project and to make updating it part of the release process. Hence +1 for the proposal.
Cheers, Till On Thu, Apr 9, 2020 at 4:01 PM Piotr Nowojski <pi...@ververica.com> wrote: > Hi Yun Tang, > > Thanks for proposing the idea. Since we can not include benchmarks in the > Flink repository what you are proposing is the second best option. > > +1 from my side for the proposal. > > I think benchmarks have proven their value to justify this. > > Piotrek > > > On 9 Apr 2020, at 08:56, Yun Tang <myas...@live.com> wrote: > > > > Hi Flink devs, > > > > As Flink develops rapidly with more and more features added, how to > ensure no performance regression existed has become more and more > important. And we would like to create a new repo under apache project to > host previous flink-benchmarks [1] repo, which is inspired when we discuss > under FLINK-16850 [2] > > > > Some background context on flink-benchmarks, for those who are not > familiar with the project yet: > > > > - Current flink-benchmarks does not align with the Flink release, which > lead developers not easy to verify > > performance at specific Flink version because current flink-benchmarks > always depends on the latest interfaces. > > - Above problem could be solved well if we could ensure flink-benchmarks > also create release branch when we > > releasing Flink. However, current flink-benchmarks repo is hosted > under dataArtisans (the former name of > > ververica) project, which is not involved in Flink release manual [3]. > We propose to promote this repo under > > apache project so that release manager could have the right to release > on flink-benchmarks. > > - The reason why we not involve flink-benchmarks into the apache/flink > repo is because it heavily depends on > > JMH [4], which is under GPLv2 license. > > > > What do you think? > > > > Best, > > Yun Tang > > > > [1] https://github.com/dataArtisans/flink-benchmarks > > [2] https://issues.apache.org/jira/browse/FLINK-16850 > > [3] > https://cwiki.apache.org/confluence/display/FLINK/Creating+a+Flink+Release > > [4] https://openjdk.java.net/projects/code-tools/jmh/ > > > > > >
