Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-17 Thread Tim Hinrichs
Hi Ruby,

I was envisioning the VM-placement engine choosing the hard-coded 
implementation or converting to LP.

There are 2 places that logic could go: in the VM-placement engine wrapper that 
runs on the DSE message bus, or within the VM-placement engine itself (assuming 
the two are different).  I think we’re still trying to figure out which one 
(and the initial PoC and the long-term solution may be different).

Right now I’m thinking that the VM-placement engine wrapper running on the DSE 
bus should subscribe to whatever data it needs.

Tim

On Feb 16, 2015, at 9:05 AM, 
ruby.krishnasw...@orange.commailto:ruby.krishnasw...@orange.com 
ruby.krishnasw...@orange.commailto:ruby.krishnasw...@orange.com wrote:

Hi Tim

What I’d like to see is the best of both worlds.  Users write Datalog policies 
describing whatever VM-placement policy they want.  If the policy they’ve 
written is on the solver-scheduler’s list of options, we use the hard-coded 
implementation, but if the policy isn’t on that list we translate directly to 
LP.


ð  How (calling the hard-coded implementation) ?

o   Through the message bus?

ð  Is it Congress that will send out the data or should each implementation (of 
a policy) read it in directly?

Ruby

De : Tim Hinrichs [mailto:thinri...@vmware.com]
Envoyé : jeudi 12 février 2015 19:03
À : OpenStack Development Mailing List (not for usage questions)
Objet : Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

Hi Debo and Yathiraj,

I took a third look at the solver-scheduler docs and code with your comments in 
mind.  A few things jumped out.

1)  Choice of LP solver.

I see solver-scheduler uses Pulp, which was on the Congress short list as well. 
 So we’re highly aligned on the choice of underlying solver.

2) User control over VM-placement.

To choose the criteria for VM-placement, the solver-scheduler user picks from a 
list of predefined options, e.g. ActiveHostConstraint, 
MaxRamAllocationPerHostConstraint.

We’re investigating a slightly different approach, where the user defines the 
criteria for VM-placement by writing any policy they like in Datalog.  Under 
the hood we then convert that Datalog to an LP problem.  From the developer’s 
perspective, with the Congress approach we don’t attempt to anticipate the 
different policies the user might want and write code for each policy; instead, 
we as developers write a translator from Datalog to LP.  From the user’s 
perspective, the difference is that if the option they want isn’t on the 
solver-scheduler's list, they’re out of luck or need to write the code 
themselves.  But with the Congress approach, they can write any VM-placement 
policy they like.

What I’d like to see is the best of both worlds.  Users write Datalog policies 
describing whatever VM-placement policy they want.  If the policy they’ve 
written is on the solver-scheduler’s list of options, we use the hard-coded 
implementation, but if the policy isn’t on that list we translate directly to 
LP.  This approach gives us the ability to write custom code to handle common 
cases while at the same time letting users write whatever policy they like.

3) API and architecture.

Today the solver-scheduler's VM-placement policy is defined at config-time 
(i.e. not run-time).  Am I correct that this limitation is only because there’s 
no API call to set the solver-scheduler’s policy?  Or is there some other 
reason the policy is set at config-time?

Congress policies change at runtime, so we’ll definitely need a VM-placement 
engine whose policy can be changed at run-time as well.

If we focus on just migration (and not provisioning), we can build a 
VM-placement engine that sits outside of Nova that has an API call that allows 
us to set policy at runtime.  We can also set up that engine to get data 
updates that influence the policy.  We were planning on creating this kind of 
VM-placement engine within Congress as a node on the DSE (our message bus).  
This is convenient because all nodes on the DSE run in their own thread, any 
node on the DSE can subscribe to any data from any other node (e.g. 
ceilometer’s data), and the algorithms for translating Datalog to LP look to be 
quite similar to the algorithms we’re using in our domain-agnostic policy 
engine.

Tim


On Feb 11, 2015, at 4:50 PM, Debojyoti Dutta 
ddu...@gmail.commailto:ddu...@gmail.com wrote:


Hi Tim: moving our thread to the mailer. Excited to collaborate!



From: Debo~ Dutta dedu...@cisco.commailto:dedu...@cisco.com
Date: Wednesday, February 11, 2015 at 4:48 PM
To: Tim Hinrichs thinri...@vmware.commailto:thinri...@vmware.com
Cc: Yathiraj Udupi (yudupi) yud...@cisco.commailto:yud...@cisco.com, 
Gokul B Kandiraju go...@us.ibm.commailto:go...@us.ibm.com, Prabhakar Kudva 
ku...@us.ibm.commailto:ku...@us.ibm.com, 
ruby.krishnasw...@orange.commailto:ruby.krishnasw...@orange.com 
ruby.krishnasw...@orange.commailto:ruby.krishnasw...@orange.com, 
dilik...@in.ibm.commailto:dilik

Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-16 Thread ruby.krishnaswamy
Hi Tim

What I'd like to see is the best of both worlds.  Users write Datalog policies 
describing whatever VM-placement policy they want.  If the policy they've 
written is on the solver-scheduler's list of options, we use the hard-coded 
implementation, but if the policy isn't on that list we translate directly to 
LP.


ð  How (calling the hard-coded implementation) ?

o   Through the message bus?

ð  Is it Congress that will send out the data or should each implementation (of 
a policy) read it in directly?

Ruby

De : Tim Hinrichs [mailto:thinri...@vmware.com]
Envoyé : jeudi 12 février 2015 19:03
À : OpenStack Development Mailing List (not for usage questions)
Objet : Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

Hi Debo and Yathiraj,

I took a third look at the solver-scheduler docs and code with your comments in 
mind.  A few things jumped out.

1)  Choice of LP solver.

I see solver-scheduler uses Pulp, which was on the Congress short list as well. 
 So we're highly aligned on the choice of underlying solver.

2) User control over VM-placement.

To choose the criteria for VM-placement, the solver-scheduler user picks from a 
list of predefined options, e.g. ActiveHostConstraint, 
MaxRamAllocationPerHostConstraint.

We're investigating a slightly different approach, where the user defines the 
criteria for VM-placement by writing any policy they like in Datalog.  Under 
the hood we then convert that Datalog to an LP problem.  From the developer's 
perspective, with the Congress approach we don't attempt to anticipate the 
different policies the user might want and write code for each policy; instead, 
we as developers write a translator from Datalog to LP.  From the user's 
perspective, the difference is that if the option they want isn't on the 
solver-scheduler's list, they're out of luck or need to write the code 
themselves.  But with the Congress approach, they can write any VM-placement 
policy they like.

What I'd like to see is the best of both worlds.  Users write Datalog policies 
describing whatever VM-placement policy they want.  If the policy they've 
written is on the solver-scheduler's list of options, we use the hard-coded 
implementation, but if the policy isn't on that list we translate directly to 
LP.  This approach gives us the ability to write custom code to handle common 
cases while at the same time letting users write whatever policy they like.

3) API and architecture.

Today the solver-scheduler's VM-placement policy is defined at config-time 
(i.e. not run-time).  Am I correct that this limitation is only because there's 
no API call to set the solver-scheduler's policy?  Or is there some other 
reason the policy is set at config-time?

Congress policies change at runtime, so we'll definitely need a VM-placement 
engine whose policy can be changed at run-time as well.

If we focus on just migration (and not provisioning), we can build a 
VM-placement engine that sits outside of Nova that has an API call that allows 
us to set policy at runtime.  We can also set up that engine to get data 
updates that influence the policy.  We were planning on creating this kind of 
VM-placement engine within Congress as a node on the DSE (our message bus).  
This is convenient because all nodes on the DSE run in their own thread, any 
node on the DSE can subscribe to any data from any other node (e.g. 
ceilometer's data), and the algorithms for translating Datalog to LP look to be 
quite similar to the algorithms we're using in our domain-agnostic policy 
engine.

Tim


On Feb 11, 2015, at 4:50 PM, Debojyoti Dutta 
ddu...@gmail.commailto:ddu...@gmail.com wrote:


Hi Tim: moving our thread to the mailer. Excited to collaborate!



From: Debo~ Dutta dedu...@cisco.commailto:dedu...@cisco.com
Date: Wednesday, February 11, 2015 at 4:48 PM
To: Tim Hinrichs thinri...@vmware.commailto:thinri...@vmware.com
Cc: Yathiraj Udupi (yudupi) yud...@cisco.commailto:yud...@cisco.com, 
Gokul B Kandiraju go...@us.ibm.commailto:go...@us.ibm.com, Prabhakar Kudva 
ku...@us.ibm.commailto:ku...@us.ibm.com, 
ruby.krishnasw...@orange.commailto:ruby.krishnasw...@orange.com 
ruby.krishnasw...@orange.commailto:ruby.krishnasw...@orange.com, 
dilik...@in.ibm.commailto:dilik...@in.ibm.com 
dilik...@in.ibm.commailto:dilik...@in.ibm.com, Norival Figueira 
nfigu...@brocade.commailto:nfigu...@brocade.com, Ramki Krishnan 
r...@brocade.commailto:r...@brocade.com, Xinyuan Huang (xinyuahu) 
xinyu...@cisco.commailto:xinyu...@cisco.com, Rishabh Jain -X (rishabja - 
AAP3 INC at Cisco) risha...@cisco.commailto:risha...@cisco.com
Subject: Re: Nova solver scheduler and Congress

Hi Tim

To address your particular questions:

  1.  translate some policy language into constraints for the LP/CVP and we had 
left that to congress hoping to integrate when the policy efforts in openstack 
were ready (our initial effort was pre congress)
  2.  For migrations: we are currently doing that - its about incremental

Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-13 Thread Debojyoti Dutta
Tim

Wanted to clarify a bit. As I have mentioned before: Solver scheduler is
work done before this work (Datalog-constraints) but we had kept it
very generic to be integrated with something like congress. In fact Ramki
(who was one of the members of the original thread when you reached out to
us) joined us to in talk in Atlanta where we described some of the same use
cases using PULP  congress was still ramping up then. We were not aware
of the Datalog-constraints work that you guys were doing, else we would
have joined hands before.

The question is this: going forward, how do build this cool stuff together
in the community? I am hoping the scheduler folks will be very excited too!

debo

On Thu, Feb 12, 2015 at 11:27 AM, Yathiraj Udupi (yudupi) yud...@cisco.com
wrote:

  Hi Tim,

  Thanks for your response.  Excited too to extend the collaboration and
 ensure there is no need to duplicate effort in the open source community.
  My responses inline.

   1)  Choice of LP solver.

  I see solver-scheduler uses Pulp, which was on the Congress short list
 as well.  So we’re highly aligned on the choice of underlying solver.


  YATHI - This makes me wonder why can’t we easily adapt the
 solver-scheduler to your needs, rather than duplicating the effort!


  2) User control over VM-placement.


  To choose the criteria for VM-placement, the solver-scheduler user picks
 from a list of predefined options, e.g. ActiveHostConstraint,
 MaxRamAllocationPerHostConstraint.

  We’re investigating a slightly different approach, where the user
 defines the criteria for VM-placement by writing any policy they like in
 Datalog.  Under the hood we then convert that Datalog to an LP problem.
 From the developer’s perspective, with the Congress approach we don’t
 attempt to anticipate the different policies the user might want and write
 code for each policy; instead, we as developers write a translator from
 Datalog to LP.  From the user’s perspective, the difference is that if the
 option they want isn’t on the solver-scheduler's list, they’re out of luck
 or need to write the code themselves.  But with the Congress approach, they
 can write any VM-placement policy they like.

  What I’d like to see is the best of both worlds.  Users write Datalog
 policies describing whatever VM-placement policy they want.  If the policy
 they’ve written is on the solver-scheduler’s list of options, we use the
 hard-coded implementation, but if the policy isn’t on that list we
 translate directly to LP.  This approach gives us the ability to write
 custom code to handle common cases while at the same time letting users
 write whatever policy they like.


  YATHI -  The idea of providing some default constraint classes in Solver
 Scheduler was to enable easy pluggability for various placement policy
 scenarios.  We can easily add a custom constraint class in solver
 scheduler, that enables adding additional constraints at runtime (PulP
 model or any other models we can use and support).  It will just take in
 any external policy (say Datalog in Congress example), and it can easily
 add those set of resulting translated constraints via this custom
 constraint builder class.  This is something we can definitely add value to
 solver scheduler by implementing and adding here.


  3) API and architecture.

  Today the solver-scheduler's VM-placement policy is defined at
 config-time (i.e. not run-time).  Am I correct that this limitation is only
 because there’s no API call to set the solver-scheduler’s policy?  Or is
 there some other reason the policy is set at config-time?

  Congress policies change at runtime, so we’ll definitely need a
 VM-placement engine whose policy can be changed at run-time as well.

   YATHI -  We have working code to set VM placement policies at run-time
 to dynamically select the constraint or cost classes to use.   It is yet to
 upstreamed to solver scheduler stackforge repo, but will be soon.  But yeah
 I agree with you, this is definitely needed for any policy-driven VM
 placement engine, as the policies are dynamic. Short answer, yes solver
 scheduler has abilities to support this.


  If we focus on just migration (and not provisioning), we can build a
 VM-placement engine that sits outside of Nova that has an API call that
 allows us to set policy at runtime.  We can also set up that engine to get
 data updates that influence the policy.  We were planning on creating this
 kind of VM-placement engine within Congress as a node on the DSE (our
 message bus).  This is convenient because all nodes on the DSE run in their
 own thread, any node on the DSE can subscribe to any data from any other
 node (e.g. ceilometer’s data), and the algorithms for translating Datalog
 to LP look to be quite similar to the algorithms we’re using in our
 domain-agnostic policy engine.


  YATHI – The entire scheduling community in Nova is planning on an
 external scheduler (Gantt), and we are pitching solver scheduler also 

Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-13 Thread Debojyoti Dutta
Hi Ruby

Good point. If you assume congress to be present, then scheduling and most
actions are a result of a policy decision and might not impact the
scheduler. The results of the LP/CVP would allow you to spawn resources at
end points.

However if you assume that there is congress + other entities, then it
might be better/cleaner to use a separate solver scheduling layer to decide
if the policies and constraints from congress are not in conflict with
other entities that might be asking for resources.

I guess its about how the community wants to layer the components. We
wanted to first build the simple constraint solver layer and then hope that
policy layers would talk to the advanced scheduler and this scheduler
driver would fit into nova scheduler or Gantt.

debo

On Fri, Feb 13, 2015 at 7:51 AM, ruby.krishnasw...@orange.com wrote:

  Hello Debo/Tim



 My understanding is that with Congress things like filters (e.g.
 anti-affinity or other aggregates) will be replaced to be written as
 policies with Datalog.

 Goals (a Policy), Constraints (policies in Congress) will also get
 translated to (for example) linear programs in some modelling language
 (e.g. PuLP).



 So this is likely to be a major change to the scheduler?



 Ruby



 *De :* Debojyoti Dutta [mailto:ddu...@gmail.com]
 *Envoyé :* vendredi 13 février 2015 14:06
 *À :* OpenStack Development Mailing List (not for usage questions)
 *Objet :* Re: [openstack-dev] [Congress][Delegation] Google doc for
 working notes



 Tim



 Wanted to clarify a bit. As I have mentioned before: Solver scheduler is
 work done before this work (Datalog-constraints) but we had kept it
 very generic to be integrated with something like congress. In fact Ramki
 (who was one of the members of the original thread when you reached out to
 us) joined us to in talk in Atlanta where we described some of the same use
 cases using PULP  congress was still ramping up then. We were not aware
 of the Datalog-constraints work that you guys were doing, else we would
 have joined hands before.



 The question is this: going forward, how do build this cool stuff together
 in the community? I am hoping the scheduler folks will be very excited too!



 debo



 On Thu, Feb 12, 2015 at 11:27 AM, Yathiraj Udupi (yudupi) 
 yud...@cisco.com wrote:

 Hi Tim,



 Thanks for your response.  Excited too to extend the collaboration and
 ensure there is no need to duplicate effort in the open source community.

  My responses inline.



  1)  Choice of LP solver.



 I see solver-scheduler uses Pulp, which was on the Congress short list as
 well.  So we’re highly aligned on the choice of underlying solver.



 YATHI - This makes me wonder why can’t we easily adapt the
 solver-scheduler to your needs, rather than duplicating the effort!





 2) User control over VM-placement.





 To choose the criteria for VM-placement, the solver-scheduler user picks
 from a list of predefined options, e.g. ActiveHostConstraint,
 MaxRamAllocationPerHostConstraint.



 We’re investigating a slightly different approach, where the user defines
 the criteria for VM-placement by writing any policy they like in Datalog.
 Under the hood we then convert that Datalog to an LP problem.  From the
 developer’s perspective, with the Congress approach we don’t attempt to
 anticipate the different policies the user might want and write code for
 each policy; instead, we as developers write a translator from Datalog to
 LP.  From the user’s perspective, the difference is that if the option they
 want isn’t on the solver-scheduler's list, they’re out of luck or need to
 write the code themselves.  But with the Congress approach, they can write
 any VM-placement policy they like.



 What I’d like to see is the best of both worlds.  Users write Datalog
 policies describing whatever VM-placement policy they want.  If the policy
 they’ve written is on the solver-scheduler’s list of options, we use the
 hard-coded implementation, but if the policy isn’t on that list we
 translate directly to LP.  This approach gives us the ability to write
 custom code to handle common cases while at the same time letting users
 write whatever policy they like.





 YATHI -  The idea of providing some default constraint classes in Solver
 Scheduler was to enable easy pluggability for various placement policy
 scenarios.  We can easily add a custom constraint class in solver
 scheduler, that enables adding additional constraints at runtime (PulP
 model or any other models we can use and support).  It will just take in
 any external policy (say Datalog in Congress example), and it can easily
 add those set of resulting translated constraints via this custom
 constraint builder class.  This is something we can definitely add value to
 solver scheduler by implementing and adding here.





 3) API and architecture.



 Today the solver-scheduler's VM-placement policy is defined at config-time
 (i.e. not run-time).  Am I correct

Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-13 Thread ruby.krishnaswamy
Hello Debo/Tim

My understanding is that with Congress things like filters (e.g. anti-affinity 
or other aggregates) will be replaced to be written as policies with Datalog.
Goals (a Policy), Constraints (policies in Congress) will also get translated 
to (for example) linear programs in some modelling language (e.g. PuLP).

So this is likely to be a major change to the scheduler?

Ruby

De : Debojyoti Dutta [mailto:ddu...@gmail.com]
Envoyé : vendredi 13 février 2015 14:06
À : OpenStack Development Mailing List (not for usage questions)
Objet : Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

Tim

Wanted to clarify a bit. As I have mentioned before: Solver scheduler is work 
done before this work (Datalog-constraints) but we had kept it very 
generic to be integrated with something like congress. In fact Ramki (who was 
one of the members of the original thread when you reached out to us) joined us 
to in talk in Atlanta where we described some of the same use cases using PULP 
 congress was still ramping up then. We were not aware of the 
Datalog-constraints work that you guys were doing, else we would have joined 
hands before.

The question is this: going forward, how do build this cool stuff together in 
the community? I am hoping the scheduler folks will be very excited too!

debo

On Thu, Feb 12, 2015 at 11:27 AM, Yathiraj Udupi (yudupi) 
yud...@cisco.commailto:yud...@cisco.com wrote:
Hi Tim,

Thanks for your response.  Excited too to extend the collaboration and ensure 
there is no need to duplicate effort in the open source community.
 My responses inline.

1)  Choice of LP solver.

I see solver-scheduler uses Pulp, which was on the Congress short list as well. 
 So we’re highly aligned on the choice of underlying solver.

YATHI - This makes me wonder why can’t we easily adapt the solver-scheduler to 
your needs, rather than duplicating the effort!


2) User control over VM-placement.


To choose the criteria for VM-placement, the solver-scheduler user picks from a 
list of predefined options, e.g. ActiveHostConstraint, 
MaxRamAllocationPerHostConstraint.

We’re investigating a slightly different approach, where the user defines the 
criteria for VM-placement by writing any policy they like in Datalog.  Under 
the hood we then convert that Datalog to an LP problem.  From the developer’s 
perspective, with the Congress approach we don’t attempt to anticipate the 
different policies the user might want and write code for each policy; instead, 
we as developers write a translator from Datalog to LP.  From the user’s 
perspective, the difference is that if the option they want isn’t on the 
solver-scheduler's list, they’re out of luck or need to write the code 
themselves.  But with the Congress approach, they can write any VM-placement 
policy they like.

What I’d like to see is the best of both worlds.  Users write Datalog policies 
describing whatever VM-placement policy they want.  If the policy they’ve 
written is on the solver-scheduler’s list of options, we use the hard-coded 
implementation, but if the policy isn’t on that list we translate directly to 
LP.  This approach gives us the ability to write custom code to handle common 
cases while at the same time letting users write whatever policy they like.


YATHI -  The idea of providing some default constraint classes in Solver 
Scheduler was to enable easy pluggability for various placement policy 
scenarios.  We can easily add a custom constraint class in solver scheduler, 
that enables adding additional constraints at runtime (PulP model or any other 
models we can use and support).  It will just take in any external policy (say 
Datalog in Congress example), and it can easily add those set of resulting 
translated constraints via this custom constraint builder class.  This is 
something we can definitely add value to solver scheduler by implementing and 
adding here.


3) API and architecture.

Today the solver-scheduler's VM-placement policy is defined at config-time 
(i.e. not run-time).  Am I correct that this limitation is only because there’s 
no API call to set the solver-scheduler’s policy?  Or is there some other 
reason the policy is set at config-time?

Congress policies change at runtime, so we’ll definitely need a VM-placement 
engine whose policy can be changed at run-time as well.

YATHI -  We have working code to set VM placement policies at run-time to 
dynamically select the constraint or cost classes to use.   It is yet to 
upstreamed to solver scheduler stackforge repo, but will be soon.  But yeah I 
agree with you, this is definitely needed for any policy-driven VM placement 
engine, as the policies are dynamic. Short answer, yes solver scheduler has 
abilities to support this.


If we focus on just migration (and not provisioning), we can build a 
VM-placement engine that sits outside of Nova that has an API call that allows 
us to set policy at runtime.  We can also set up that engine

Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-13 Thread Tim Hinrichs
Hi Debo and Yathi,

We’re completely on the same page here.  We’ve known about the solver-scheduler 
for a while now (I even attended your Atlanta talk), and I always expected 
Congress would integrate with it.  As you say, now it’s a matter of getting 
down to the details.

A bit on the context…  The current problem we’re working on in Congress is how 
we might delegate responsibility for policy enforcement to domain-specific 
policy engines, and a number of people were interested in integrating with a 
VM-placement engine.  We started looking at the solver-scheduler (the obvious 
first choice), hence this dialog.  The notes in the google doc are just me 
trying to understand the problem of delegation to a VM-placement engine by 
working through the problem end-to-end.  (I’ve not worked with LP or 
VM-placement much, so my notes are there to help me grapple a bit with the 
domain for the first time.)  How we build a PoC is something we haven’t started 
to discuss.  So you’re joining the discussion at the right time.  The more of 
that PoC we can build by leveraging solver-scheduler, the better.

More detailed comments inline.


On Feb 13, 2015, at 5:05 AM, Debojyoti Dutta 
ddu...@gmail.commailto:ddu...@gmail.com wrote:

Tim

Wanted to clarify a bit. As I have mentioned before: Solver scheduler is work 
done before this work (Datalog-constraints) but we had kept it very 
generic to be integrated with something like congress. In fact Ramki (who was 
one of the members of the original thread when you reached out to us) joined us 
to in talk in Atlanta where we described some of the same use cases using PULP 
 congress was still ramping up then. We were not aware of the 
Datalog-constraints work that you guys were doing, else we would have joined 
hands before.

The question is this: going forward, how do build this cool stuff together in 
the community? I am hoping the scheduler folks will be very excited too!

debo

On Thu, Feb 12, 2015 at 11:27 AM, Yathiraj Udupi (yudupi) 
yud...@cisco.commailto:yud...@cisco.com wrote:
Hi Tim,

Thanks for your response.  Excited too to extend the collaboration and ensure 
there is no need to duplicate effort in the open source community.
 My responses inline.

1)  Choice of LP solver.

I see solver-scheduler uses Pulp, which was on the Congress short list as well. 
 So we’re highly aligned on the choice of underlying solver.

YATHI - This makes me wonder why can’t we easily adapt the solver-scheduler to 
your needs, rather than duplicating the effort!


My primary goal is to build an architecture that makes it easy to integrate 
with domain-specific policy engines (like compute or networking).

What I’m also hearing is that people are interested in building *new* 
domain-specific policy engines within the Congress framework and/or expanding 
the functionality of the Congress policy engine itself to include optimization 
technology.  In both cases, we would need a library for solving optimization 
problems.  Oliver (CC’ed) has proposed adding such a library to Congress.  
Solver-scheduler already has such a library, so it would be great if we could 
all brainstorm about how to make optimization technology easy to use for people 
writing domain-specific policy engines, without reinventing the wheel.

https://blueprints.launchpad.net/congress/+spec/rule-x


2) User control over VM-placement.


To choose the criteria for VM-placement, the solver-scheduler user picks from a 
list of predefined options, e.g. ActiveHostConstraint, 
MaxRamAllocationPerHostConstraint.

We’re investigating a slightly different approach, where the user defines the 
criteria for VM-placement by writing any policy they like in Datalog.  Under 
the hood we then convert that Datalog to an LP problem.  From the developer’s 
perspective, with the Congress approach we don’t attempt to anticipate the 
different policies the user might want and write code for each policy; instead, 
we as developers write a translator from Datalog to LP.  From the user’s 
perspective, the difference is that if the option they want isn’t on the 
solver-scheduler's list, they’re out of luck or need to write the code 
themselves.  But with the Congress approach, they can write any VM-placement 
policy they like.

What I’d like to see is the best of both worlds.  Users write Datalog policies 
describing whatever VM-placement policy they want.  If the policy they’ve 
written is on the solver-scheduler’s list of options, we use the hard-coded 
implementation, but if the policy isn’t on that list we translate directly to 
LP.  This approach gives us the ability to write custom code to handle common 
cases while at the same time letting users write whatever policy they like.


YATHI -  The idea of providing some default constraint classes in Solver 
Scheduler was to enable easy pluggability for various placement policy 
scenarios.  We can easily add a custom constraint class in solver scheduler, 
that enables adding 

Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-13 Thread Yathiraj Udupi (yudupi)
Hi Tim,

Glad to collaborate and work towards nailing down the details.  Yeah in terms 
of policy enforcement from Congress, it makes sense to delegate to  
domain-specific policy engines.  It will be good to go through this PoC and to 
start thinking about the integration points of Congress with Solver scheduler, 
with a good set of APIs supported from both sides.

Some more comments inline to your questions:

On 2/13/15, 9:35 AM, Tim Hinrichs 
thinri...@vmware.commailto:thinri...@vmware.com wrote:

Hi Debo and Yathi,

We’re completely on the same page here.  We’ve known about the solver-scheduler 
for a while now (I even attended your Atlanta talk), and I always expected 
Congress would integrate with it.  As you say, now it’s a matter of getting 
down to the details.

A bit on the context…  The current problem we’re working on in Congress is how 
we might delegate responsibility for policy enforcement to domain-specific 
policy engines, and a number of people were interested in integrating with a 
VM-placement engine.  We started looking at the solver-scheduler (the obvious 
first choice), hence this dialog.  The notes in the google doc are just me 
trying to understand the problem of delegation to a VM-placement engine by 
working through the problem end-to-end.  (I’ve not worked with LP or 
VM-placement much, so my notes are there to help me grapple a bit with the 
domain for the first time.)  How we build a PoC is something we haven’t started 
to discuss.  So you’re joining the discussion at the right time.  The more of 
that PoC we can build by leveraging solver-scheduler, the better.

More detailed comments inline.


On Feb 13, 2015, at 5:05 AM, Debojyoti Dutta 
ddu...@gmail.commailto:ddu...@gmail.com wrote:

Tim

Wanted to clarify a bit. As I have mentioned before: Solver scheduler is work 
done before this work (Datalog-constraints) but we had kept it very 
generic to be integrated with something like congress. In fact Ramki (who was 
one of the members of the original thread when you reached out to us) joined us 
to in talk in Atlanta where we described some of the same use cases using PULP 
 congress was still ramping up then. We were not aware of the 
Datalog-constraints work that you guys were doing, else we would have joined 
hands before.

The question is this: going forward, how do build this cool stuff together in 
the community? I am hoping the scheduler folks will be very excited too!

debo

On Thu, Feb 12, 2015 at 11:27 AM, Yathiraj Udupi (yudupi) 
yud...@cisco.commailto:yud...@cisco.com wrote:
Hi Tim,

Thanks for your response.  Excited too to extend the collaboration and ensure 
there is no need to duplicate effort in the open source community.
 My responses inline.

1)  Choice of LP solver.

I see solver-scheduler uses Pulp, which was on the Congress short list as well. 
 So we’re highly aligned on the choice of underlying solver.

YATHI - This makes me wonder why can’t we easily adapt the solver-scheduler to 
your needs, rather than duplicating the effort!


My primary goal is to build an architecture that makes it easy to integrate 
with domain-specific policy engines (like compute or networking).

What I’m also hearing is that people are interested in building *new* 
domain-specific policy engines within the Congress framework and/or expanding 
the functionality of the Congress policy engine itself to include optimization 
technology.  In both cases, we would need a library for solving optimization 
problems.  Oliver (CC’ed) has proposed adding such a library to Congress.  
Solver-scheduler already has such a library, so it would be great if we could 
all brainstorm about how to make optimization technology easy to use for people 
writing domain-specific policy engines, without reinventing the wheel.

https://blueprints.launchpad.net/congress/+spec/rule-x

YATHI: It is an interesting thought in this direction.  However it is good to 
build an architecture with easy integration of Congress to separate 
domain-specific engines in compute, networking, and storage.I feel some of 
these policy validation/enforcement workflows are sometimes best within the 
domain-specific engines like VM placement/scheduling for example.   But 
definitely optimization technology is a good choice for this kind of problems.



2) User control over VM-placement.


To choose the criteria for VM-placement, the solver-scheduler user picks from a 
list of predefined options, e.g. ActiveHostConstraint, 
MaxRamAllocationPerHostConstraint.

We’re investigating a slightly different approach, where the user defines the 
criteria for VM-placement by writing any policy they like in Datalog.  Under 
the hood we then convert that Datalog to an LP problem.  From the developer’s 
perspective, with the Congress approach we don’t attempt to anticipate the 
different policies the user might want and write code for each policy; instead, 
we as developers write a translator from Datalog to LP.  From the user’s 

Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-12 Thread Yathiraj Udupi (yudupi)
Hi Tim,

Thanks for your response.  Excited too to extend the collaboration and ensure 
there is no need to duplicate effort in the open source community.
 My responses inline.

1)  Choice of LP solver.

I see solver-scheduler uses Pulp, which was on the Congress short list as well. 
 So we’re highly aligned on the choice of underlying solver.

YATHI - This makes me wonder why can’t we easily adapt the solver-scheduler to 
your needs, rather than duplicating the effort!


2) User control over VM-placement.


To choose the criteria for VM-placement, the solver-scheduler user picks from a 
list of predefined options, e.g. ActiveHostConstraint, 
MaxRamAllocationPerHostConstraint.

We’re investigating a slightly different approach, where the user defines the 
criteria for VM-placement by writing any policy they like in Datalog.  Under 
the hood we then convert that Datalog to an LP problem.  From the developer’s 
perspective, with the Congress approach we don’t attempt to anticipate the 
different policies the user might want and write code for each policy; instead, 
we as developers write a translator from Datalog to LP.  From the user’s 
perspective, the difference is that if the option they want isn’t on the 
solver-scheduler's list, they’re out of luck or need to write the code 
themselves.  But with the Congress approach, they can write any VM-placement 
policy they like.

What I’d like to see is the best of both worlds.  Users write Datalog policies 
describing whatever VM-placement policy they want.  If the policy they’ve 
written is on the solver-scheduler’s list of options, we use the hard-coded 
implementation, but if the policy isn’t on that list we translate directly to 
LP.  This approach gives us the ability to write custom code to handle common 
cases while at the same time letting users write whatever policy they like.


YATHI -  The idea of providing some default constraint classes in Solver 
Scheduler was to enable easy pluggability for various placement policy 
scenarios.  We can easily add a custom constraint class in solver scheduler, 
that enables adding additional constraints at runtime (PulP model or any other 
models we can use and support).  It will just take in any external policy (say 
Datalog in Congress example), and it can easily add those set of resulting 
translated constraints via this custom constraint builder class.  This is 
something we can definitely add value to solver scheduler by implementing and 
adding here.


3) API and architecture.

Today the solver-scheduler's VM-placement policy is defined at config-time 
(i.e. not run-time).  Am I correct that this limitation is only because there’s 
no API call to set the solver-scheduler’s policy?  Or is there some other 
reason the policy is set at config-time?

Congress policies change at runtime, so we’ll definitely need a VM-placement 
engine whose policy can be changed at run-time as well.

YATHI -  We have working code to set VM placement policies at run-time to 
dynamically select the constraint or cost classes to use.   It is yet to 
upstreamed to solver scheduler stackforge repo, but will be soon.  But yeah I 
agree with you, this is definitely needed for any policy-driven VM placement 
engine, as the policies are dynamic. Short answer, yes solver scheduler has 
abilities to support this.


If we focus on just migration (and not provisioning), we can build a 
VM-placement engine that sits outside of Nova that has an API call that allows 
us to set policy at runtime.  We can also set up that engine to get data 
updates that influence the policy.  We were planning on creating this kind of 
VM-placement engine within Congress as a node on the DSE (our message bus).  
This is convenient because all nodes on the DSE run in their own thread, any 
node on the DSE can subscribe to any data from any other node (e.g. 
ceilometer’s data), and the algorithms for translating Datalog to LP look to be 
quite similar to the algorithms we’re using in our domain-agnostic policy 
engine.

YATHI – The entire scheduling community in Nova is planning on an external 
scheduler (Gantt), and we are pitching solver scheduler also as a stand-alone 
placement engine a scheduler as a service.  Nova integration is just to ensure 
it fits within the Nova workflow.   I am not quite familiar with the DSE 
architecture yet,  but the simple idea we have is, Congress policies, as part 
of the enforcement workflow, should set the VM placement constraints, and feed 
any additional data to be used for scheduling/placement decisions, which will 
be consumed dynamically by the Solver Scheduler, and after the delegation, the 
Solver scheduler module will calculate the placement decisions, and complete 
the VM initial placement or call the VM migration APIs and enable the required 
migrations.



Thanks,
Yathi.


On 2/12/15, 10:02 AM, Tim Hinrichs 
thinri...@vmware.commailto:thinri...@vmware.com wrote:

Hi Debo and Yathiraj,

I took a third look at the 

Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-12 Thread Tim Hinrichs
Hi Debo and Yathiraj,

I took a third look at the solver-scheduler docs and code with your comments in 
mind.  A few things jumped out.

1)  Choice of LP solver.

I see solver-scheduler uses Pulp, which was on the Congress short list as well. 
 So we’re highly aligned on the choice of underlying solver.

2) User control over VM-placement.

To choose the criteria for VM-placement, the solver-scheduler user picks from a 
list of predefined options, e.g. ActiveHostConstraint, 
MaxRamAllocationPerHostConstraint.

We’re investigating a slightly different approach, where the user defines the 
criteria for VM-placement by writing any policy they like in Datalog.  Under 
the hood we then convert that Datalog to an LP problem.  From the developer’s 
perspective, with the Congress approach we don’t attempt to anticipate the 
different policies the user might want and write code for each policy; instead, 
we as developers write a translator from Datalog to LP.  From the user’s 
perspective, the difference is that if the option they want isn’t on the 
solver-scheduler's list, they’re out of luck or need to write the code 
themselves.  But with the Congress approach, they can write any VM-placement 
policy they like.

What I’d like to see is the best of both worlds.  Users write Datalog policies 
describing whatever VM-placement policy they want.  If the policy they’ve 
written is on the solver-scheduler’s list of options, we use the hard-coded 
implementation, but if the policy isn’t on that list we translate directly to 
LP.  This approach gives us the ability to write custom code to handle common 
cases while at the same time letting users write whatever policy they like.

3) API and architecture.

Today the solver-scheduler's VM-placement policy is defined at config-time 
(i.e. not run-time).  Am I correct that this limitation is only because there’s 
no API call to set the solver-scheduler’s policy?  Or is there some other 
reason the policy is set at config-time?

Congress policies change at runtime, so we’ll definitely need a VM-placement 
engine whose policy can be changed at run-time as well.

If we focus on just migration (and not provisioning), we can build a 
VM-placement engine that sits outside of Nova that has an API call that allows 
us to set policy at runtime.  We can also set up that engine to get data 
updates that influence the policy.  We were planning on creating this kind of 
VM-placement engine within Congress as a node on the DSE (our message bus).  
This is convenient because all nodes on the DSE run in their own thread, any 
node on the DSE can subscribe to any data from any other node (e.g. 
ceilometer’s data), and the algorithms for translating Datalog to LP look to be 
quite similar to the algorithms we’re using in our domain-agnostic policy 
engine.

Tim


On Feb 11, 2015, at 4:50 PM, Debojyoti Dutta 
ddu...@gmail.commailto:ddu...@gmail.com wrote:

Hi Tim: moving our thread to the mailer. Excited to collaborate!



From: Debo~ Dutta dedu...@cisco.commailto:dedu...@cisco.com
Date: Wednesday, February 11, 2015 at 4:48 PM
To: Tim Hinrichs thinri...@vmware.commailto:thinri...@vmware.com
Cc: Yathiraj Udupi (yudupi) yud...@cisco.commailto:yud...@cisco.com, 
Gokul B Kandiraju go...@us.ibm.commailto:go...@us.ibm.com, Prabhakar Kudva 
ku...@us.ibm.commailto:ku...@us.ibm.com, 
ruby.krishnasw...@orange.commailto:ruby.krishnasw...@orange.com 
ruby.krishnasw...@orange.commailto:ruby.krishnasw...@orange.com, 
dilik...@in.ibm.commailto:dilik...@in.ibm.com 
dilik...@in.ibm.commailto:dilik...@in.ibm.com, Norival Figueira 
nfigu...@brocade.commailto:nfigu...@brocade.com, Ramki Krishnan 
r...@brocade.commailto:r...@brocade.com, Xinyuan Huang (xinyuahu) 
xinyu...@cisco.commailto:xinyu...@cisco.com, Rishabh Jain -X (rishabja - 
AAP3 INC at Cisco) risha...@cisco.commailto:risha...@cisco.com
Subject: Re: Nova solver scheduler and Congress

Hi Tim

To address your particular questions:

  1.  translate some policy language into constraints for the LP/CVP and we had 
left that to congress hoping to integrate when the policy efforts in openstack 
were ready (our initial effort was pre congress)
  2.  For migrations: we are currently doing that – its about incremental 
constraints into the same solver. Hence its a small deal ….

Joining forces is a terrific idea. Would love to join the IRC call and see how 
we can build cool stuff in the community together. I hope we don’t have to 
replicate the vm placement engine while the work that was done in the community 
does something very similar (and be adapted)

debo

From: Tim Hinrichs thinri...@vmware.commailto:thinri...@vmware.com
Date: Wednesday, February 11, 2015 at 4:43 PM
To: Debo~ Dutta dedu...@cisco.commailto:dedu...@cisco.com
Cc: Yathiraj Udupi (yudupi) yud...@cisco.commailto:yud...@cisco.com, 
Gokul B Kandiraju go...@us.ibm.commailto:go...@us.ibm.com, Prabhakar Kudva 
ku...@us.ibm.commailto:ku...@us.ibm.com, 

[openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-11 Thread Tim Hinrichs
Hi all,

A (growing) group of folks are interested in working on the problem of 
delegating policy from Congress to domain-specific policy engines.  We started 
looking at an NFV use case: migrating VMs to reduce energy consumption.  In 
particular we’re looking into building a VM-placement policy engine built on 
top of a linear programming solver.  Here’s a doc with some working notes where 
we’re trying to figure out how to do the translation from Congress policy to 
the linear program.

https://docs.google.com/document/d/1ksDilJYXV-5AXWON8PLMedDKr9NpS8VbT0jIy_MIEtI/edit?usp=sharing

Tim
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Congress][Delegation] Google doc for working notes

2015-02-11 Thread Debojyoti Dutta
Hi Tim: moving our thread to the mailer. Excited to collaborate!



From: Debo~ Dutta dedu...@cisco.com
Date: Wednesday, February 11, 2015 at 4:48 PM
To: Tim Hinrichs thinri...@vmware.com
Cc: Yathiraj Udupi (yudupi) yud...@cisco.com, Gokul B Kandiraju 
go...@us.ibm.com, Prabhakar Kudva ku...@us.ibm.com, 
ruby.krishnasw...@orange.com ruby.krishnasw...@orange.com, 
dilik...@in.ibm.com dilik...@in.ibm.com, Norival Figueira 
nfigu...@brocade.com, Ramki Krishnan r...@brocade.com, Xinyuan Huang
(xinyuahu) xinyu...@cisco.com, Rishabh Jain -X (rishabja - AAP3 INC at
Cisco) risha...@cisco.com
Subject: Re: Nova solver scheduler and Congress

Hi Tim

To address your particular questions:

   1. translate some policy language into constraints for the LP/CVP and we
   had left that to congress hoping to integrate when the policy efforts in
   openstack were ready (our initial effort was pre congress)
   2. For migrations: we are currently doing that – its about incremental
   constraints into the same solver. Hence its a small deal ….

Joining forces is a terrific idea. Would love to join the IRC call and see
how we can build cool stuff in the community together. I hope we don’t have
to replicate the vm placement engine while the work that was done in the
community does something very similar (and be adapted)

debo

From: Tim Hinrichs thinri...@vmware.com
Date: Wednesday, February 11, 2015 at 4:43 PM
To: Debo~ Dutta dedu...@cisco.com
Cc: Yathiraj Udupi (yudupi) yud...@cisco.com, Gokul B Kandiraju 
go...@us.ibm.com, Prabhakar Kudva ku...@us.ibm.com, 
ruby.krishnasw...@orange.com ruby.krishnasw...@orange.com, 
dilik...@in.ibm.com dilik...@in.ibm.com, Norival Figueira 
nfigu...@brocade.com, Ramki Krishnan r...@brocade.com, Xinyuan Huang
(xinyuahu) xinyu...@cisco.com, Rishabh Jain -X (rishabja - AAP3 INC at
Cisco) risha...@cisco.com
Subject: Re: Nova solver scheduler and Congress

Hi Debo,

The 2 efforts share great similarities, which was why we investigated the
state of solver-scheduler.  From what I understand, (i) solver-scheduler
doesn’t currently have a policy language and (ii) it doesn’t do migrations.
 (I realize these are both in the works.)  We needed both and wanted to
make progress before those were complete.

In the long run, it may make perfect sense to replace our vm-placement
engine with yours.  So joining forces sounds like a good idea.  At the very
*least* we ought to keep up to date with each other’s progress.

I’m starting to wonder if we ought to schedule a (bi-) weekly IRC for this
topic.

Tim

On Feb 11, 2015, at 4:35 PM, Debo Dutta (dedutta) dedu...@cisco.com wrote:

Hi Tim

This looks awesome. Trying to figure out how this approach is different
from the solver scheduler effort we did? We would be happy to fold our
solver scheduler effort into this (that way you also get code up and
running)

Will also respond on the thread.

thx
debo

From: Tim Hinrichs thinri...@vmware.com
Date: Wednesday, February 11, 2015 at 4:11 PM
To: Yathiraj Udupi (yudupi) yud...@cisco.com
Cc: Gokul B Kandiraju go...@us.ibm.com, Prabhakar Kudva ku...@us.ibm.com,
ruby.krishnasw...@orange.com ruby.krishnasw...@orange.com, 
dilik...@in.ibm.com dilik...@in.ibm.com, Norival Figueira 
nfigu...@brocade.com, Ramki Krishnan r...@brocade.com, Xinyuan Huang
(xinyuahu) xinyu...@cisco.com, Rishabh Jain -X (rishabja - AAP3 INC at
Cisco) risha...@cisco.com, Debo~ Dutta dedu...@cisco.com
Subject: Re: Nova solver scheduler and Congress

Hi Yathiraj,

The group is getting big enough that we’ve decided to move the entire
discussion to the openstack-dev mailing list.  I sent a note today with the
google doc we’re working on.  We’re trying to include
[Congress][Delegation] in the subject line of relevant posts.  Here’s the
gdoc.

https://docs.google.com/document/d/1ksDilJYXV-5AXWON8PLMedDKr9NpS8VbT0jIy_MIEtI/edit?usp=sharing
https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.google.com_document_d_1ksDilJYXV-2D5AXWON8PLMedDKr9NpS8VbT0jIy-5FMIEtI_edit-3Fusp-3Dsharingd=AwMF-gc=Sqcl0Ez6M0X8aeM67LKIiDJAXVeAw-YihVMNtXt-uEsr=B6BWd4kFfgOzAREgThxkmTZKy7dDXE2-eBAmL0PBK7sm=no-emyiErtYa3_zneDNhY78LG0mCHc0bgMpXi1StZ7As=rLY2ACQqD5EQn3MgOnoX8M_zr9254v-FqhF56wfGpice=

Tim

On Feb 10, 2015, at 11:10 AM, Yathiraj Udupi (yudupi) yud...@cisco.com
wrote:

Hi Tim,

Thanks for your response.  I think Congress will have to appreciate
different APIs interacting with multiple components in the OpenStack
ecosystem.  So I will be happy to help figure out the integration plan in
general from the Congress side.

I will  be very interested and glad to participate in the discussions of
designing these interfaces in Congress.   Please share any preliminary
designs you may have.   I had participated in one of the Congress mid-cycle
meet ups, and I am interested in the upcoming work on these kind of
enforcement aspects (reactive, proactive) of Congress.  In terms of Nova
scheduling via Solver scheduler, it will also help us be ready with the
integration points.

Let’s be