Blue is 0.12.2 with some minor backports not perf related. Red is from the 0.11.x series. This is effectively a bucketed PDF of the query times for a live cluster with Timeseries queries as self-reported by historical nodes. I mentioned elsewhere I'm not convinced query/time is a good proxy for user experience, but it does provide a good baseline for comparisons between versions. Low query times are suspected due to some aggressive caching or complete node misses (node very little data for that time range for that datasource). And high query time outliers are often the result of bad GC.
On our side there is a new java version going out with the 0.12.2 deployment so it is unclear how much is attributed to the new java version and how much is attributed to the druid jars or other config changes. Overall things seem to consistently display a small % improvement in the mean with our internal 0.12.2 release. This is good! Cheers, Charles Allen [image: Screen Shot 2018-08-08 at 4.01.24 PM.png] On Wed, Aug 8, 2018 at 3:11 PM David Lim <david...@apache.org> wrote: > +1, thank you! > > On Wed, Aug 8, 2018 at 3:16 PM Jonathan Wei <jon...@apache.org> wrote: > > > +1, thanks Jihoon! > > > > On Wed, Aug 8, 2018 at 1:18 PM, Jihoon Son <jihoon...@apache.org> wrote: > > > > > Awesome! Thanks Charles! > > > > > > Jihoon > > > > > > On Wed, Aug 8, 2018 at 1:16 PM Gian Merlino <g...@apache.org> wrote: > > > > > > > Thanks, it will be nice to see! > > > > > > > > On Wed, Aug 8, 2018 at 1:15 PM Charles Allen <charles.al...@snap.com > > > > .invalid> > > > > wrote: > > > > > > > > > I don't think it should be a blocker to release, but I have to run > > perf > > > > > tests for rollouts anyways so I figured I'd publish what I find :-P > > > > > > > > > > Cheers, > > > > > Charles Allen > > > > > > > > > > > > > > > On Wed, Aug 8, 2018 at 12:33 PM Gian Merlino <g...@apache.org> > > wrote: > > > > > > > > > > > That being said, Charles I am definitely looking forward to your > > > report > > > > > of > > > > > > what the upgrade from 0.11 -> 0.12.2-rc1 is like in your cluster! > > > > > > > > > > > > On Wed, Aug 8, 2018 at 12:30 PM Gian Merlino <g...@apache.org> > > > wrote: > > > > > > > > > > > > > My thought is that recently we have started doing small bug-fix > > > > > releases > > > > > > > more often (0.12.1 and 0.12.2 were both small releases) and I > > think > > > > it > > > > > > > makes sense to continue this practice. It makes sense to get > them > > > out > > > > > > > quickly, since shipping bug fixes is good. IMO trying to > validate > > > bug > > > > > fix > > > > > > > releases within the customary Apache style 72 hour voting > period > > > is a > > > > > > good > > > > > > > goal. > > > > > > > > > > > > > > On the other hand we do strive to put out high quality > releases, > > > and > > > > we > > > > > > > don't want bug fix releases to introduce regressions. Testing > > every > > > > > > single > > > > > > > patch in real clusters is an important part of that. All I can > do > > > is > > > > > > > encourage people running real clusters to deploy RCs as fast as > > > they > > > > > can! > > > > > > > Fwiw, we have already incorporated all the 0.12.2 patches into > > our > > > > > Imply > > > > > > > distro of Druid and already have a good number users running > > them. > > > So > > > > > my > > > > > > +1 > > > > > > > earlier incorporated knowledge that the patches have been > > validated > > > > in > > > > > > that > > > > > > > way. > > > > > > > > > > > > > > I agree with Jihoon that we will probably end up doing an > 0.12.3 > > > > soon, > > > > > to > > > > > > > fix the issues he mentioned and a couple of others as well. > > > > > > > > > > > > > > On Wed, Aug 8, 2018 at 12:07 PM Jihoon Son < > jihoon...@apache.org > > > > > > > > wrote: > > > > > > > > > > > > > >> Charles, thank you for doing performance evaluation! > Performance > > > > > numbers > > > > > > >> are always good and helpful. > > > > > > >> > > > > > > >> However, IMO, any kind of performance degradation shouldn't > be a > > > > > blocker > > > > > > >> for this release. 0.12.2 is a minor release and contains only > > bug > > > > > fixes. > > > > > > >> https://github.com/apache/incubator-druid/pull/5878/files is > > the > > > > only > > > > > > one > > > > > > >> tagged with 'Performance', but it can be regarded as a more > > like a > > > > > code > > > > > > >> bug > > > > > > >> rather than architectural performance issue. > > > > > > >> > > > > > > >> Instead, those kinds of performance tests should be performed > > per > > > > > major > > > > > > >> release to catch any kinds of unexpected performance change. > > They > > > > can > > > > > > be a > > > > > > >> blocker if we find any performance regression. > > > > > > >> > > > > > > >> Also, if you find any performance regression for this release, > > we > > > > > > probably > > > > > > >> make another minor release. I think some bug fixes (e.g., > > > > > > >> https://github.com/apache/incubator-druid/issues/6124, > > > > > > >> https://github.com/apache/incubator-druid/issues/6123) are > also > > > > worth > > > > > > to > > > > > > >> be > > > > > > >> included in the minor release. > > > > > > >> > > > > > > >> What do you think? > > > > > > >> > > > > > > >> Best, > > > > > > >> Jihoon > > > > > > >> > > > > > > >> On Wed, Aug 8, 2018 at 10:18 AM Charles Allen > > > > > > >> <charles.al...@snap.com.invalid> wrote: > > > > > > >> > > > > > > >> > I'm hoping to have some numbers for any performance changes > or > > > > other > > > > > > >> > impacts in the next few days (rollouts on big clusters take > a > > > long > > > > > > >> time). I > > > > > > >> > am neutral until the numbers come in. Preliminary indicators > > > show > > > > no > > > > > > >> > significant regression since the 0.11.x series. More data is > > > > > expected > > > > > > >> to be > > > > > > >> > available in a few days as rollout completes. > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > On Wed, Aug 8, 2018 at 9:10 AM Himanshu < > g.himan...@gmail.com > > > > > > > > wrote: > > > > > > >> > > > > > > > >> > > +1 , thanks for coordinating it. > > > > > > >> > > > > > > > > >> > > On Tue, Aug 7, 2018 at 8:05 PM, Gian Merlino < > > g...@apache.org > > > > > > > > > > wrote: > > > > > > >> > > > > > > > > >> > > > +1. Thank you Jihoon for running this release. > > > > > > >> > > > > > > > > > >> > > > On Tue, Aug 7, 2018 at 10:04 AM Jihoon Son < > > > > > jihoon...@apache.org> > > > > > > >> > wrote: > > > > > > >> > > > > > > > > > >> > > > > Sure, > > > > > > >> > > > > > > > > > > >> > > > > the release note is available here: > > > > > > >> > > > > https://github.com/apache/incubator-druid/issues/6116 > . > > > > > > >> > > > > > > > > > > >> > > > > Best, > > > > > > >> > > > > Jihoon > > > > > > >> > > > > > > > > > > >> > > > > On Tue, Aug 7, 2018 at 10:02 AM Charles Allen < > > > > > > cral...@apache.org > > > > > > >> > > > > > > > >> > > > wrote: > > > > > > >> > > > > > > > > > > >> > > > > > ((don't let this ask block the release)) > > > > > > >> > > > > > > > > > > > >> > > > > > Is there a way to get a preview of what the release > > > notice > > > > > > will > > > > > > >> > look > > > > > > >> > > > > like? > > > > > > >> > > > > > > > > > > > >> > > > > > On Mon, Aug 6, 2018 at 3:38 PM Fangjin Yang < > > > > > fang...@imply.io > > > > > > > > > > > > > >> > > wrote: > > > > > > >> > > > > > > > > > > > >> > > > > > > +1 > > > > > > >> > > > > > > > > > > > > >> > > > > > > On Mon, Aug 6, 2018 at 3:03 PM, Jihoon Son < > > > > > > >> jihoon...@apache.org > > > > > > >> > > > > > > > > >> > > > > wrote: > > > > > > >> > > > > > > > > > > > > >> > > > > > > > Hi all, > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > Druid 0.12.2-rc1 ( > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__druid. > > > io_downloads.html&d=DwIBaQ&c=ncDTmphkJTvjIDPh0hpF_w&r= > > > HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m= > > > jpC6qqjxYOq5XtNdLQ8U99CX0aj4Qn7oNQCLnjJ_v-Y&s=wdtYVmGEMAXuXYDvVCvJ_ > > > u6CKSIShdGKA2DpgjX9lfE&e= > > > > > > >> > ) > > > > > > >> > > is available > > > > > > >> > > > now, > > > > > > >> > > > > > and > > > > > > >> > > > > > > I > > > > > > >> > > > > > > > think it's time to vote on the 0.12.2 release. > > > Please > > > > > note > > > > > > >> that > > > > > > >> > > > > 0.12.2 > > > > > > >> > > > > > is > > > > > > >> > > > > > > > not an ASF release. > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > Here is my +1. > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > Best, > > > > > > >> > > > > > > > Jihoon > > > > > > >> > > > > > > > > > > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > >
