Hi Enrico,
Thanks for the comments. Please see my replies inline below:
On Wed, Dec 14, 2016 at 4:08 AM, Enrico Olivelli <[email protected]>
wrote:
> Very interesting,
> some other notes for the discussion:
> - as we are now on java8 can you add a "default" no-op implementation
> to the new method in EnsemblePlacementPolicy#updateBookieInfo, this
> way we will not break compatibility with custom policies
>
OK. I wasn't familiar with it. I'll explore it.
> - inside the new BookieInfo can we provide a set of string "labels"
> for the bookie, to be configured statically on bookie configuration
> (ServerConfiguration). maybe this could be another implementation, but
> I would like that the BookieInfo will be extensible enough to add new
> fields in the future without protocol changes and without "protocol
> extensions"
>
My current implementation requires the client to request what information
it needs and the server will only respond to those.
> - I wonder if we can change actual out-of-the-box policies (RackAware,
> Default...) or just create a new policy ResourceAwarePlacementPolicy
> which takes into account this new info
>
>
I think creating a new one will not be easy. I'll need to specialize each
of the existing one: Default, RackAware, RegionAware, etc. Instead I chose
to add weighted property to the existing policies and it is controlled with
a separate config parameter.
> Thanks for the BP
> Enrico
>
> 2016-12-14 4:12 GMT+01:00 Sijie Guo <[email protected]>:
> > Hi Rithin,
> >
> > The proposal looks good to me. I have a few comments:
> >
> > - How will the BookieInfo structure look like?
> > - I like the idea of a general GET_BOOKIE_INFO request. Do you have any
> > proposed wire protocol changes?
> > - I assume that you might need the clients to configure a function to
> > compute the weight for a bookie, right? So we can also compute the weight
> > based on network bandwidth?
> > - How does bookie collect the resource information?
> >
> > - Sijie
> >
> > On Mon, Dec 12, 2016 at 11:09 AM, Rithin Shetty <[email protected]>
> wrote:
> >
> >> Hi Sijie,
> >>
> >> I've updated the proposal page now:
> >> https://cwiki.apache.org/confluence/display/BOOKKEEPER/
> >> BP-2+-+Resource+aware+data+placement.
> >> Sorry about the delay.
> >>
> >> Thanks,
> >> --Rithin
> >>
> >> On Sat, Dec 10, 2016 at 1:45 AM, Sijie Guo <[email protected]> wrote:
> >>
> >> > As a reminder, we will be discussing the resource aware placement at
> the
> >> > meeting next week.
> >> >
> >> > Rithin, do you mind updating the proposal by next Tue? so we have
> time to
> >> > check before the meeting.
> >> >
> >> > - Sijie
> >> >
> >> > On Tue, Nov 1, 2016 at 2:29 AM, Sijie Guo <[email protected]> wrote:
> >> >
> >> > > https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP-2+
> >> > > -+Resource+aware+data+placement
> >> > >
> >> > > I created a wiki page for place holder. Anyone that is interested in
> >> this
> >> > > topic can help fill a basic proposal and drive the discussion?
> >> > >
> >> > > *Problem*
> >> > >
> >> > > There are bunch of tickets around talking about placement policy :
> >> > > - handle different types of storage (tier storage)
> >> > > - handle storage node having different type of resources (resource
> >> aware)
> >> > > - better to canary/bucket-testing new version (manageability)
> >> > >
> >> > > It would be good to consolidate these thoughts to provide a common
> >> > > framework/solution to meet those basic requirements.
> >> > >
> >> > > *Proposal*
> >> > >
> >> > > [TBD]
> >> > >
> >> > > - Sijie
> >> > >
> >> > >
> >> >
> >>
>
