Very interesting, some other notes for the discussion: - as we are now on java8 can you add a "default" no-op implementation to the new method in EnsemblePlacementPolicy#updateBookieInfo, this way we will not break compatibility with custom policies - inside the new BookieInfo can we provide a set of string "labels" for the bookie, to be configured statically on bookie configuration (ServerConfiguration). maybe this could be another implementation, but I would like that the BookieInfo will be extensible enough to add new fields in the future without protocol changes and without "protocol extensions" - I wonder if we can change actual out-of-the-box policies (RackAware, Default...) or just create a new policy ResourceAwarePlacementPolicy which takes into account this new info
Thanks for the BP Enrico 2016-12-14 4:12 GMT+01:00 Sijie Guo <[email protected]>: > Hi Rithin, > > The proposal looks good to me. I have a few comments: > > - How will the BookieInfo structure look like? > - I like the idea of a general GET_BOOKIE_INFO request. Do you have any > proposed wire protocol changes? > - I assume that you might need the clients to configure a function to > compute the weight for a bookie, right? So we can also compute the weight > based on network bandwidth? > - How does bookie collect the resource information? > > - Sijie > > On Mon, Dec 12, 2016 at 11:09 AM, Rithin Shetty <[email protected]> wrote: > >> Hi Sijie, >> >> I've updated the proposal page now: >> https://cwiki.apache.org/confluence/display/BOOKKEEPER/ >> BP-2+-+Resource+aware+data+placement. >> Sorry about the delay. >> >> Thanks, >> --Rithin >> >> On Sat, Dec 10, 2016 at 1:45 AM, Sijie Guo <[email protected]> wrote: >> >> > As a reminder, we will be discussing the resource aware placement at the >> > meeting next week. >> > >> > Rithin, do you mind updating the proposal by next Tue? so we have time to >> > check before the meeting. >> > >> > - Sijie >> > >> > On Tue, Nov 1, 2016 at 2:29 AM, Sijie Guo <[email protected]> wrote: >> > >> > > https://cwiki.apache.org/confluence/display/BOOKKEEPER/BP-2+ >> > > -+Resource+aware+data+placement >> > > >> > > I created a wiki page for place holder. Anyone that is interested in >> this >> > > topic can help fill a basic proposal and drive the discussion? >> > > >> > > *Problem* >> > > >> > > There are bunch of tickets around talking about placement policy : >> > > - handle different types of storage (tier storage) >> > > - handle storage node having different type of resources (resource >> aware) >> > > - better to canary/bucket-testing new version (manageability) >> > > >> > > It would be good to consolidate these thoughts to provide a common >> > > framework/solution to meet those basic requirements. >> > > >> > > *Proposal* >> > > >> > > [TBD] >> > > >> > > - Sijie >> > > >> > > >> > >>
