Hi Tim, All,
1) Step 3: The VM-placement engine is also a "datalog engine" . Right?
When policies are delegated:
when policies are inserted? When the VM-placement engine has already registered
itself all policies are given to it?
"In our example, this would mean the domain-specific policy engine executes the
following API call over the DSE"
ð "domain-agnostic" ....
2) Step 4:
Ok
But finally: if Congress will likely "delegate"
3) Step 5: Compilation of subpolicy to LP in VM-placement engine
For the PoC, it is likely that the LP program ( in PuLP or some other ML) is
*not* completely generated by compiler/translator.
ð Right?
You also indicate that some category of constraints ("the LP solver
doesn't know what the relationship between assign[i][j], hMemUse[j], and
vMemUse[i] actually is, so the VM-placement engine must also include
constraints") .
These constraints must be "explicitly" written? (e.g. max_ram_allocation
etc that are constraints used in the solver-scheduler's package).
So what "parts" will be generated:
Cost function :
Constraint from Policy : memory usage < 75%
Then the rest should be "filled" up?
Could we convene on an intermediary "modeling language"?
@Yathi: do you think we could use some thing like AMPL ? Is this
proprietary?
A detail: the example "Y[host1] = hMemUse[host1] > 0.75 * hMemCap[host1]"
ð To be changed to a linear form (mi - Mi > 0 then Yi = 1 else Yi = 0) so
something like (mi - Mi) < 100 yi
4) Step 6: This is completely internal to the VM-placement engine (and we
could say this is "transparent" to Congress)
We should allow configuration of a solver (this itself could be a policy :) )
How to invoke the solver API ?
The domain-specific placement engine could send out to DSE (action_handler:
data)?
3) Step 7 : Perform the migrations (according to the assignments computed in
the step 6)
This part invokes OpenStack API (to perform migrations).
We may suppose that there are services implementing "action handlers"?
It can listen on the DSE and execute the action.
5) Nova tables to use
Policy warning(id) :-
nova:host(id, name, service, zone, memory_capacity),
legacy:special_zone(zone),
ceilometer:statistics(id, "memory", avg, count, duration,
durstart, durend, max, min, period,
perstart, perend, sum, unit),
avg > 0.75 * memory_capacity
I believe that ceilometer gives usage of VMs and not hosts. The host table
(ComputeNode table) should give current used capacity.
6) One of the issues highlighted in OpenStack (scheduler) and also
elsewhere (e.g. Omega scheduler by google) is :
Reading "host utilization" state from the data bases and DB (nova:host table)
updates and overhead of maintaining in-memory state uptodate.
ð This is expensive and current nova-scheduler does face this issue (many
blogs/discussions).
While the first goal is a PoC, this will likely become a concern in terms
of adoption.
7) While in this document you have changed the "example" policy, could we
drill down the set of policies for the PoC (the server under utilization ?)
ð As a reference
Ruby
De : Yathiraj Udupi (yudupi) [mailto:[email protected]]
Envoyé : mardi 24 février 2015 20:01
À : OpenStack Development Mailing List (not for usage questions); Tim Hinrichs
Cc : Debo Dutta (dedutta)
Objet : Re: [openstack-dev] [Congress][Delegation] Initial workflow design
Hi Tim,
Thanks for your updated doc on Delegation from Congress to a domain-specific
policy engine, in this case, you are planning to build a LP-based VM-Placement
engine to be the domain specific policy engine.
I agree your main goal is to first get the delegation interface sorted out. It
will be good so that external services (like Solver-Scheduler) can also easily
integrate to the delegation model.
>From the Solver-Scheduler point of view, we would actually want to start
>working on a PoC effort to start integrating Congress and the Solver-Scheduler.
We believe rather than pushing this effort to a long-term, it would add value
to both the Solver Scheduler effort, as well as the Congress effort to try some
early integration now, as most of the LP solver work for VM placements is ready
available now in Solver scheduler, and we need to spend some time thinking
about translating your domain-agnostic policy to constraints that the Solver
scheduler can use.
I would definitely need your help from the Congress interfaces and I hope you
will share your early interfaces for the delegation, so I can start the effort
from the Solver scheduler side for integration.
I will reach out to you to get some initial help for integration w.r.t.
Congress, and also keep you posted about the progress from our side.
Thanks,
Yathi.
On 2/23/15, 11:28 AM, "Tim Hinrichs"
<[email protected]<mailto:[email protected]>> wrote:
Hi all,
I made a heavy editing pass of the Delegation google doc, incorporating many of
your comments and my latest investigations into VM-placement. I left the old
stuff in place at the end of the doc and put the new stuff at the top.
My goal was to propose an end-to-end workflow for a PoC that we could put
together quickly to help us explore the delegation interface. We should
iterate on this design until we have something that we think is workable. And
by all means pipe up if you think we need a totally different starting point to
begin the iteration.
(BTW I'm thinking of the integration with solver-scheduler as a long-term
solution to VM-placement, once we get the delegation interface sorted out.)
https://docs.google.com/document/d/1ksDilJYXV-5AXWON8PLMedDKr9NpS8VbT0jIy_MIEtI/edit#<https://docs.google.com/document/d/1ksDilJYXV-5AXWON8PLMedDKr9NpS8VbT0jIy_MIEtI/edit>
Tim
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: [email protected]?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
