That is very helpful, thank you!

+1 for continuing with 0.12.2-RC1

On Tue, Jul 10, 2018 at 6:51 PM Clint Wylie <clint.wy...@imply.io> wrote:

> Heya, sorry for the delay (and missing the sync, i'll try to get better
> about showing up). I've fixed a handful of coordinator bugs post 0.12.0
> (and
> not backported to 0.12.1), some of these issues go far back, some back to
> when segment assignment priority for different tiers of historicals was
> introduced, some are just some oddities on the behavior of the balancer
> that I am unsure when were introduced. This is the complete list of fixes
> that are currently in 0.12.2 afaik, with a small description (see PRs and
> associated issues for more details)
>
> https://github.com/apache/incubator-druid/pull/5528 fixed an issue that
> movement did not drop the segment from the server the segment was being
> moved from (this one goes waaaay back, to batch segment announcements)
>
> https://github.com/apache/incubator-druid/pull/5529 changed behavior of
> drop to use the balancer to choose where to drop segments from, based on
> behavior observed caused by the issue of 5528
>
> https://github.com/apache/incubator-druid/pull/5532 fixes an issue where
> primary assignment during load rule processing would assign an unavailable
> segment to every server with capacity until at least 1 historical had the
> segment (and drop it from all the others if they all loaded at the same
> time), choking load queues from doing useful things
>
> https://github.com/apache/incubator-druid/pull/5555 fixed a way for http
> based coordinator to get stuck loading or dropping segments and a companion
> PR that fixed a lambda that wasn't friendly to older jvm versions
> https://github.com/apache/incubator-druid/pull/5591
>
> https://github.com/apache/incubator-druid/pull/5888 makes balancing honor
> a
> load rule max load queue depth setting to help prevent movement from
> starving loading
>
> https://github.com/apache/incubator-druid/pull/5928 doesn't really fix
> anything, just does an early return to avoid doing pointless work
>
> Additionally, there are a couple of pairs of PRs that are not currently in
> 0.12.2: https://github.com/druid-io/druid/pull/5927 and
> https://github.com/apache/incubator-druid/pull/5929 and their respective
> fixes which have yet to be merged, but have been performing well on our
> test cluster, https://github.com/apache/incubator-druid/pull/5987 and
> https://github.com/apache/incubator-druid/pull/5988. One of them makes
> balancing behave in a way more consistent with expectations by always
> trying to move maxSegmentsToMove and more correctly tracking what the
> balancer is doing, and one just adds better logging (without much extra log
> volume) due to frustrations I had chasing down all these other issues. Both
> of these were slated for 0.12.2 but were pulled out because of the issues
> (which the open PRs fix afaict). I would be in favor of sliding them in
> there, pending review of the fixes, but understand if they won't make the
> cut since they maybe fall a bit more on the cosmetic side of things. I'm
> pretty happy of the state of things on our test cluster right now, but
> without these 4 patches things should still be operating more correctly
> than they were before, just the differences being with balancing moving
> somewhere between 0 and max, and less useful logging making future issues
> (which I have no doubts still lurk) harder to diagnose.
>
> Cheers,
> Clint
>
> On Tue, Jul 10, 2018 at 10:30 AM, Charles Allen <cral...@apache.org>
> wrote:
>
> > Brought this up in the dev sync:
> >
> > I saw a lot of PRs and fixes for Coordinator segment balancing related to
> > some regressions that happened in 0.12.x . Is anyone able to give a
> rundown
> > of the state of coordinator segment management for the 0.12.2 RC?
> >
> > On Tue, Jul 10, 2018 at 10:26 AM Nishant Bangarwa <
> > nbanga...@hortonworks.com>
> > wrote:
> >
> > > +1
> > >
> > > --
> > > Nishant Bangarwa
> > >
> > > Hortonworks
> > >
> > > ´╗┐On 7/10/18, 3:57 AM, "Jihoon Son" <jihoon...@apache.org> wrote:
> > >
> > >     Related thread:
> > >
> > > https://lists.apache.org/thread.html/76755aecfddb1210fcc3f08b1d4631
> > 784a8a5eede64d22718c271841@%3Cdev.druid.apache.org%3E
> > >     .
> > >
> > >     Jihoon
> > >
> > >     On Mon, Jul 9, 2018 at 3:25 PM Jihoon Son <jihoon...@apache.org>
> > > wrote:
> > >
> > >     > Hi all,
> > >     >
> > >     > We have no open issues and PRs for 0.12.2 (
> > >     > https://github.com/apache/incubator-druid/milestone/27). The
> > 0.12.2
> > >     > branch is already available and all PRs for 0.12.2 have merged
> into
> > > that
> > >     > branch.
> > >     >
> > >     > Let's vote on releasing RC1. Here is my +1.
> > >     >
> > >     > This is a non-ASF release.
> > >     >
> > >     > Best,
> > >     > Jihoon
> > >     >
> > >
> > >
> > >
> >
>

Reply via email to