Hi Gordon, Thank you. Any help from you and the folks in your network would be much appreciated. Let me know if there is any more information you need from me or if you'd like to setup a meeting at some point to discuss.
As far as the balance policy goes, I think for the first go-round we might want to have the system automatically balance when nodes are added and include a BALANCE command to be run manually if/when the system becomes out of balance. - Doug On Mon, Dec 20, 2010 at 10:38 AM, Gordon <[email protected]> wrote: > Hi Doug, > > Thanks for the nice writeup -- at this point we would like to start carving > out what the API would like to allow people to extend with their own > intelligent load balancing algorithms. Load balancing of this type is > recognized as a hard research problem(*) and I would like to see Hypertable > be used as central to that research. > > We'll need a set of abstractions and interfaces that allow a variety of > approaches that people might like to apply -- reinforcement learning, > constraint programming, or other AI style planning approaches. I'd like to > collect some feedback from a few different sources to see if we can solve > for the minimal interface that gives everyone the inputs and the controls > they need to devise a balancer. Then, we can produce an API recommendation. > > Also, we should add a bit more detail to talk about policy for calling the > balancer. Intelligence in the balancing strategy might probably better be > served by allowing the balancer policy itself to be learned or optimized. > For example, the system might be better left in a slightly unbalanced > condition rather than pay the balancing cost if it's learned that the > unbalanced condition is temporary. > > I'll point out the problem and this mailing list to folks doing research in > ML (particularly SysML or ML applied to systems management) but also from > the constraint computing field. > > (*) see for example: > http://www2.cs.uni-paderborn.de/cs/ag-monien/RESEARCH/LOADBAL/ > > > > On Sun, Dec 19, 2010 at 9:15 PM, Doug Judd <[email protected]> wrote: > >> We're now 100% focused on load balancing. This will handle the addition >> of new nodes as well as normal load imbalances. I've come up with a design >> and have described it in the following document: >> >> http://code.google.com/p/hypertable/wiki/LoadBalancing >> >> Please read it and send your feedback. Thanks! >> >> - Doug >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Hypertable Development" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<hypertable-dev%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/hypertable-dev?hl=en. >> > > > > -- > Gordon Rios -- Cork Constraint Computation Centre > http://www.4c.ucc.ie/web/people.jsp?id=144 > http://www.linkedin.com/in/gordonrios > Ireland: +353 86 089 2416 > USA: +1 650 906 3473 > > -- > You received this message because you are subscribed to the Google Groups > "Hypertable Development" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<hypertable-dev%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/hypertable-dev?hl=en. > -- You received this message because you are subscribed to the Google Groups "Hypertable Development" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/hypertable-dev?hl=en.
