Watching the benchmark data for days and indeed it's normalized for the time being. However, the result seems to be unstable. I also tried the benchmark locally and observed obvious wave even with the same commit...
I guess we may need to improve it such as increasing the RECORDS_PER_INVOCATION to generate a reproducible result. IMHO a stable micro benchmark is important to verify perf-related improvements (and I think the benchmark and website are already great ones but just need some love). Let me mark this as one of my backlog and will open a JIRA when prepared. Anyway good to know it's not a regression, and thanks for the efforts spent on checking it over! @Gordon @Chesnay Best Regards, Yu On Fri, 15 Mar 2019 at 19:20, Chesnay Schepler <ches...@apache.org> wrote: > The regressions is already normalizing again. I'd observer it further > before doing anything. > > The same applies to the benchmarkCount which tanked even more in that > same run. > > On 15.03.2019 06:02, Tzu-Li (Gordon) Tai wrote: > > @Yu > > Thanks for reporting that Yu, great that this was noticed. > > > > The serializerAvro case seems to only be testing on-wire serialization. > > I checked the changes to the `AvroSerializer`, and it seems like > > FLINK-11436 [1] with commit 479ebd59 was the only change that may have > > affected that. > > That commit wasn't introduced exactly around the time when the indicated > > performance regression occurred, but was still before the regression. > > The commit introduced some instanceof type checks / type casting in the > > readObject of the AvroSerializer, which may have caused this. > > > > Currently investigating further. > > > > Cheers, > > Gordon > > > > On Fri, Mar 15, 2019 at 11:45 AM Yu Li <car...@gmail.com> wrote: > > > >> Hi Aljoscha and all, > >> > >> From our performance benchmark web site ( > >> http://codespeed.dak8s.net:8000/changes/) I observed a noticeable > >> regression (-6.92%) on the serializerAvro case comparing the latest 100 > >> revisions, which may need some attention. Thanks. > >> > >> Best Regards, > >> Yu > >> > >> > >> On Thu, 14 Mar 2019 at 20:42, Aljoscha Krettek <aljos...@apache.org> > >> wrote: > >> > >>> Hi everyone, > >>> Please review and vote on the release candidate 2 for Flink 1.8.0, as > >>> follows: > >>> [ ] +1, Approve the release > >>> [ ] -1, Do not approve the release (please provide specific comments) > >>> > >>> > >>> The complete staging area is available for your review, which includes: > >>> * JIRA release notes [1], > >>> * the official Apache source release and binary convenience releases to > >> be > >>> deployed to dist.apache.org <http://dist.apache.org/> [2], which are > >>> signed with the key with fingerprint > >>> F2A67A8047499BBB3908D17AA8F4FD97121D7293 [3], > >>> * all artifacts to be deployed to the Maven Central Repository [4], > >>> * source code tag "release-1.8.0-rc2" [5], > >>> * website pull request listing the new release [6] > >>> * website pull request adding announcement blog post [7]. > >>> > >>> The vote will be open for at least 72 hours. It is adopted by majority > >>> approval, with at least 3 PMC affirmative votes. > >>> > >>> Thanks, > >>> Aljoscha > >>> > >>> [1] > >>> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344274 > >>> < > >>> > >> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12315522&version=12344274 > >>> [2] https://dist.apache.org/repos/dist/dev/flink/flink-1.8.0-rc2/ < > >>> https://dist.apache.org/repos/dist/dev/flink/flink-1.8.0-rc2/> > >>> [3] https://dist.apache.org/repos/dist/release/flink/KEYS < > >>> https://dist.apache.org/repos/dist/release/flink/KEYS> > >>> [4] > >> https://repository.apache.org/content/repositories/orgapacheflink-1213 > >>> < > https://repository.apache.org/content/repositories/orgapacheflink-1210/ > >>> > >>> [5] > >>> > >> > https://gitbox.apache.org/repos/asf?p=flink.git;a=tag;h=c77a329b71e3068bfde965ae91921ad5c47246dd > >>> < > >>> > >> > https://gitbox.apache.org/repos/asf?p=flink.git;a=tag;h=2d00b1c26d7b4554707063ab0d1d6cc236cfe8a5 > >>> [6] https://github.com/apache/flink-web/pull/180 < > >>> https://github.com/apache/flink-web/pull/180> > >>> [7] https://github.com/apache/flink-web/pull/179 < > >>> https://github.com/apache/flink-web/pull/179> > >>> > >>> P.S. The difference to the previous RC1 is very small, you can fetch > the > >>> two tags and do a "git log release-1.8.0-rc1..release-1.8.0-rc2” to see > >> the > >>> difference in commits. Its fixes for the issues that led to the > >>> cancellation of the previous RC plus smaller fixes. Most > >>> verification/testing that was carried out should apply as is to this > RC. > > >
