Hi Josh, My idea's really pretty simple - make "DB proxy" and "Task workflow" separate services, and allow people to co-locate them if they want to.
Cheers. Phil > -----Original Message----- > From: Joshua Harlow [mailto:[email protected]] > Sent: 17 July 2013 14:57 > To: OpenStack Development Mailing List > Cc: OpenStack Development Mailing List > Subject: Re: [openstack-dev] Moving task flow to conductor - concern about > scale > > Hi Phil, > > I understand and appreciate your concern and I think everyone is trying to > keep > that in mind. It still appears to me to be to early in this refactoring and > task > restructuring effort to tell where it may "end up". I think that's also good > news > since we can get these kinds of ideas (componentized conductors if u will) to > handle your (and mine) scaling concerns. It would be pretty neat if said > conductors could be scaled at different rates depending on there component, > although as u said we need to get much much better with handling said > patterns (as u said just 2 schedulers is a pita right now). I believe we can > do it, > given the right kind of design and scaling "principles" we build in from the > start > (right now). > > Would like to hear more of your ideas so they get incorporated earlier rather > than later. > > Sent from my really tiny device.. > > On Jul 16, 2013, at 9:55 AM, "Dan Smith" <[email protected]> wrote: > > >> In the original context of using Conductor as a database proxy then > >> the number of conductor instances is directly related to the number > >> of compute hosts I need them to serve. > > > > Just a point of note, as far as I know, the plan has always been to > > establish conductor as a thing that sits between the api and compute > > nodes. However, we started with the immediate need, which was the > > offloading of database traffic. > > > >> What I not sure is that I would also want to have the same number of > >> conductor instances for task control flow - historically even running > >> 2 schedulers has been a problem, so the thought of having 10's of > >> them makes me very concerned at the moment. However I can't see any > >> way to specialise a conductor to only handle one type of request. > > > > Yeah, I don't think the way it's currently being done allows for > > specialization. > > > > Since you were reviewing actual task code, can you offer any specifics > > about the thing(s) that concern you? I think that scaling conductor > > (and its tasks) horizontally is an important point we need to achieve, > > so if you see something that needs tweaking, please point it out. > > > > Based on what is there now and proposed soon, I think it's mostly > > fairly safe, straightforward, and really no different than what two > > computes do when working together for something like resize or migrate. > > > >> So I guess my question is, given that it may have to address two > >> independent scale drivers, is putting task work flow and DB proxy > >> functionality into the same service really the right thing to do - or > >> should there be some separation between them. > > > > I think that we're going to need more than one "task" node, and so it > > seems appropriate to locate one scales-with-computes function with > > another. > > > > Thanks! > > > > --Dan > > > > _______________________________________________ > > OpenStack-dev mailing list > > [email protected] > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > _______________________________________________ > OpenStack-dev mailing list > [email protected] > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
