Oh probably. Long story short: approximately equivalent, new release has slightly tucked in long tail on the bad side of query time (but also few samples in the long tail overall).
On Thu, Aug 9, 2018 at 10:37 AM Gian Merlino <g...@apache.org> wrote: > Nice!! > > Although I don't see the graphic attached, maybe the mailing list ate it? > > On Wed, Aug 8, 2018 at 4:15 PM Charles Allen <charles.al...@snap.com > .invalid> > wrote: > > > Blue is 0.12.2 with some minor backports not perf related. Red is from > the > > 0.11.x series. This is effectively a bucketed PDF of the query times for > a > > live cluster with Timeseries queries as self-reported by historical > nodes. > > I mentioned elsewhere I'm not convinced query/time is a good proxy for > user > > experience, but it does provide a good baseline for comparisons between > > versions. Low query times are suspected due to some aggressive caching or > > complete node misses (node very little data for that time range for that > > datasource). And high query time outliers are often the result of bad GC. > > > > On our side there is a new java version going out with the 0.12.2 > > deployment so it is unclear how much is attributed to the new java > version > > and how much is attributed to the druid jars or other config changes. > > Overall things seem to consistently display a small % improvement in the > > mean with our internal 0.12.2 release. This is good! > > > > Cheers, > > Charles Allen > > > > [image: Screen Shot 2018-08-08 at 4.01.24 PM.png] > > > > > > On Wed, Aug 8, 2018 at 3:11 PM David Lim <david...@apache.org> wrote: > > > >> +1, thank you! > >> > >> On Wed, Aug 8, 2018 at 3:16 PM Jonathan Wei <jon...@apache.org> wrote: > >> > >> > +1, thanks Jihoon! > >> > > >> > On Wed, Aug 8, 2018 at 1:18 PM, Jihoon Son <jihoon...@apache.org> > >> wrote: > >> > > >> > > Awesome! Thanks Charles! > >> > > > >> > > Jihoon > >> > > > >> > > On Wed, Aug 8, 2018 at 1:16 PM Gian Merlino <g...@apache.org> > wrote: > >> > > > >> > > > Thanks, it will be nice to see! > >> > > > > >> > > > On Wed, Aug 8, 2018 at 1:15 PM Charles Allen < > >> charles.al...@snap.com > >> > > > .invalid> > >> > > > wrote: > >> > > > > >> > > > > I don't think it should be a blocker to release, but I have to > run > >> > perf > >> > > > > tests for rollouts anyways so I figured I'd publish what I find > >> :-P > >> > > > > > >> > > > > Cheers, > >> > > > > Charles Allen > >> > > > > > >> > > > > > >> > > > > On Wed, Aug 8, 2018 at 12:33 PM Gian Merlino <g...@apache.org> > >> > wrote: > >> > > > > > >> > > > > > That being said, Charles I am definitely looking forward to > your > >> > > report > >> > > > > of > >> > > > > > what the upgrade from 0.11 -> 0.12.2-rc1 is like in your > >> cluster! > >> > > > > > > >> > > > > > On Wed, Aug 8, 2018 at 12:30 PM Gian Merlino <g...@apache.org > > > >> > > wrote: > >> > > > > > > >> > > > > > > My thought is that recently we have started doing small > >> bug-fix > >> > > > > releases > >> > > > > > > more often (0.12.1 and 0.12.2 were both small releases) and > I > >> > think > >> > > > it > >> > > > > > > makes sense to continue this practice. It makes sense to get > >> them > >> > > out > >> > > > > > > quickly, since shipping bug fixes is good. IMO trying to > >> validate > >> > > bug > >> > > > > fix > >> > > > > > > releases within the customary Apache style 72 hour voting > >> period > >> > > is a > >> > > > > > good > >> > > > > > > goal. > >> > > > > > > > >> > > > > > > On the other hand we do strive to put out high quality > >> releases, > >> > > and > >> > > > we > >> > > > > > > don't want bug fix releases to introduce regressions. > Testing > >> > every > >> > > > > > single > >> > > > > > > patch in real clusters is an important part of that. All I > >> can do > >> > > is > >> > > > > > > encourage people running real clusters to deploy RCs as fast > >> as > >> > > they > >> > > > > can! > >> > > > > > > Fwiw, we have already incorporated all the 0.12.2 patches > into > >> > our > >> > > > > Imply > >> > > > > > > distro of Druid and already have a good number users running > >> > them. > >> > > So > >> > > > > my > >> > > > > > +1 > >> > > > > > > earlier incorporated knowledge that the patches have been > >> > validated > >> > > > in > >> > > > > > that > >> > > > > > > way. > >> > > > > > > > >> > > > > > > I agree with Jihoon that we will probably end up doing an > >> 0.12.3 > >> > > > soon, > >> > > > > to > >> > > > > > > fix the issues he mentioned and a couple of others as well. > >> > > > > > > > >> > > > > > > On Wed, Aug 8, 2018 at 12:07 PM Jihoon Son < > >> jihoon...@apache.org > >> > > > >> > > > > wrote: > >> > > > > > > > >> > > > > > >> Charles, thank you for doing performance evaluation! > >> Performance > >> > > > > numbers > >> > > > > > >> are always good and helpful. > >> > > > > > >> > >> > > > > > >> However, IMO, any kind of performance degradation shouldn't > >> be a > >> > > > > blocker > >> > > > > > >> for this release. 0.12.2 is a minor release and contains > only > >> > bug > >> > > > > fixes. > >> > > > > > >> https://github.com/apache/incubator-druid/pull/5878/files > is > >> > the > >> > > > only > >> > > > > > one > >> > > > > > >> tagged with 'Performance', but it can be regarded as a more > >> > like a > >> > > > > code > >> > > > > > >> bug > >> > > > > > >> rather than architectural performance issue. > >> > > > > > >> > >> > > > > > >> Instead, those kinds of performance tests should be > performed > >> > per > >> > > > > major > >> > > > > > >> release to catch any kinds of unexpected performance > change. > >> > They > >> > > > can > >> > > > > > be a > >> > > > > > >> blocker if we find any performance regression. > >> > > > > > >> > >> > > > > > >> Also, if you find any performance regression for this > >> release, > >> > we > >> > > > > > probably > >> > > > > > >> make another minor release. I think some bug fixes (e.g., > >> > > > > > >> https://github.com/apache/incubator-druid/issues/6124, > >> > > > > > >> https://github.com/apache/incubator-druid/issues/6123) are > >> also > >> > > > worth > >> > > > > > to > >> > > > > > >> be > >> > > > > > >> included in the minor release. > >> > > > > > >> > >> > > > > > >> What do you think? > >> > > > > > >> > >> > > > > > >> Best, > >> > > > > > >> Jihoon > >> > > > > > >> > >> > > > > > >> On Wed, Aug 8, 2018 at 10:18 AM Charles Allen > >> > > > > > >> <charles.al...@snap.com.invalid> wrote: > >> > > > > > >> > >> > > > > > >> > I'm hoping to have some numbers for any performance > >> changes or > >> > > > other > >> > > > > > >> > impacts in the next few days (rollouts on big clusters > >> take a > >> > > long > >> > > > > > >> time). I > >> > > > > > >> > am neutral until the numbers come in. Preliminary > >> indicators > >> > > show > >> > > > no > >> > > > > > >> > significant regression since the 0.11.x series. More data > >> is > >> > > > > expected > >> > > > > > >> to be > >> > > > > > >> > available in a few days as rollout completes. > >> > > > > > >> > > >> > > > > > >> > > >> > > > > > >> > > >> > > > > > >> > On Wed, Aug 8, 2018 at 9:10 AM Himanshu < > >> g.himan...@gmail.com > >> > > > >> > > > > wrote: > >> > > > > > >> > > >> > > > > > >> > > +1 , thanks for coordinating it. > >> > > > > > >> > > > >> > > > > > >> > > On Tue, Aug 7, 2018 at 8:05 PM, Gian Merlino < > >> > g...@apache.org > >> > > > > >> > > > > > wrote: > >> > > > > > >> > > > >> > > > > > >> > > > +1. Thank you Jihoon for running this release. > >> > > > > > >> > > > > >> > > > > > >> > > > On Tue, Aug 7, 2018 at 10:04 AM Jihoon Son < > >> > > > > jihoon...@apache.org> > >> > > > > > >> > wrote: > >> > > > > > >> > > > > >> > > > > > >> > > > > Sure, > >> > > > > > >> > > > > > >> > > > > > >> > > > > the release note is available here: > >> > > > > > >> > > > > > >> https://github.com/apache/incubator-druid/issues/6116. > >> > > > > > >> > > > > > >> > > > > > >> > > > > Best, > >> > > > > > >> > > > > Jihoon > >> > > > > > >> > > > > > >> > > > > > >> > > > > On Tue, Aug 7, 2018 at 10:02 AM Charles Allen < > >> > > > > > cral...@apache.org > >> > > > > > >> > > >> > > > > > >> > > > wrote: > >> > > > > > >> > > > > > >> > > > > > >> > > > > > ((don't let this ask block the release)) > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > Is there a way to get a preview of what the > release > >> > > notice > >> > > > > > will > >> > > > > > >> > look > >> > > > > > >> > > > > like? > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > On Mon, Aug 6, 2018 at 3:38 PM Fangjin Yang < > >> > > > > fang...@imply.io > >> > > > > > > > >> > > > > > >> > > wrote: > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > > +1 > >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > On Mon, Aug 6, 2018 at 3:03 PM, Jihoon Son < > >> > > > > > >> jihoon...@apache.org > >> > > > > > >> > > > >> > > > > > >> > > > > wrote: > >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > Hi all, > >> > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > Druid 0.12.2-rc1 ( > >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> > >> > > > > > > >> > > > > > >> > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__druid. > >> > > io_downloads.html&d=DwIBaQ&c=ncDTmphkJTvjIDPh0hpF_w&r= > >> > > HrLGT1qWNhseJBMYABL0GFSZESht5gBoLejor3SqMSo&m= > >> > > jpC6qqjxYOq5XtNdLQ8U99CX0aj4Qn7oNQCLnjJ_v-Y&s=wdtYVmGEMAXuXYDvVCvJ_ > >> > > u6CKSIShdGKA2DpgjX9lfE&e= > >> > > > > > >> > ) > >> > > > > > >> > > is available > >> > > > > > >> > > > now, > >> > > > > > >> > > > > > and > >> > > > > > >> > > > > > > I > >> > > > > > >> > > > > > > > think it's time to vote on the 0.12.2 > release. > >> > > Please > >> > > > > note > >> > > > > > >> that > >> > > > > > >> > > > > 0.12.2 > >> > > > > > >> > > > > > is > >> > > > > > >> > > > > > > > not an ASF release. > >> > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > Here is my +1. > >> > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > Best, > >> > > > > > >> > > > > > > > Jihoon > >> > > > > > >> > > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > >> > > > >> > > > > > >> > > >> > > > > > >> > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > > >
