subject:"\[openstack\-dev\] Fwd\: FW\: \[Neutron\] Group Based Policy and the way forward"

Can you link to the etherpad you mentioned?

In the mean time, apologies for another analogy in
advance. :-)

If I give you an API to sort a list, I'm free to implement it however I
want as long as I return a sorted list. However, there is no way me to know
based on a call to this API that you might only be looking for the second
largest element, so it won't be the most efficient approach because I will
always have to sort the entire list.
If I give you a higher level API to declare that you want elements of a
list that match a criteria in a certain order, then the API can make the
optimization to not actually sort the whole list if you just need the first
of the largest two elements.

The former is analogous to the security groups API, and the latter to the
GBP API.
On Aug 7, 2014 4:00 PM, Aaron Rosen aaronoro...@gmail.com wrote:




 On Thu, Aug 7, 2014 at 12:08 PM, Kevin Benton blak...@gmail.com wrote:

 I mean't 'side stepping' why GBP allows for the comment you made
 previous, With the latter, a mapping driver could determine that
 communication between these two hosts can be prevented by using an ACL on a
 router or a switch, which doesn't violate the user's intent and buys a
 performance improvement and works with ports that don't support security
 groups..

 Neutron's current API is a logical abstraction and enforcement can be
 done however one chooses to implement it. I'm really trying to understand
 at the network level why GBP allows for these optimizations and performance
 improvements you talked about.

 You absolutely cannot enforce security groups on a firewall/router that
 sits at the boundary between networks. If you try, you are lying to the
 end-user because it's not enforced at the port level. The current neutron
 APIs force you to decide where things like that are implemented.


 The current neutron API's are just logical abstractions. Where and how
 things are actually enforced are 100% an implementation detail of a vendors
 system.  Anyways, moving the discussion to the etherpad...



 The higher level abstractions give you the freedom to move the
 enforcement by allowing the expression of broad connectivity requirements.

 Why are you bringing up logging connections?

 This was brought up as a feature proposal to FWaaS because this is a
 basic firewall feature missing from OpenStack. However, this does not
 preclude a FWaaS vendor from logging.

 Personally, I think one could easily write up a very short document
 probably less than one page with examples showing/exampling how the current
 neutron API works even without a much networking background.

 The difficulty of the API for establishing basic connectivity isn't
 really the problem. It's when you have to compose a bunch of requirements
 and make sure nothing is violating auditing and connectivity constraints
 that it becomes a problem. We are arguing about the levels of abstraction.
 You could also write up a short document explaining to novice programmers
 how to use C to read and write database entries to an sqlite database, but
 that doesn't mean it's the best level of abstraction for what the users are
 trying to accomplish.

 I'll let someone else explain the current GBP API because I'm not working
 on that. I'm just trying to convince you of the value of declarative
 network configuration.


 On Thu, Aug 7, 2014 at 12:02 PM, Aaron Rosen aaronoro...@gmail.com
 wrote:




 On Thu, Aug 7, 2014 at 9:54 AM, Kevin Benton blak...@gmail.com wrote:

 You said you had no idea what group based policy was buying us so I
 tried to illustrate what the difference between declarative and imperative
 network configuration looks like. That's the major selling point of GBP so
 I'm not sure how that's 'side stepping' any points. It removes the need for
 the user to pick between implementation details like security
 groups/FWaaS/ACLs.


 I mean't 'side stepping' why GBP allows for the comment you made
 previous, With the latter, a mapping driver could determine that
 communication between these two hosts can be prevented by using an ACL on a
 router or a switch, which doesn't violate the user's intent and buys a
 performance improvement and works with ports that don't support security
 groups..

 Neutron's current API is a logical abstraction and enforcement can be
 done however one chooses to implement it. I'm really trying to understand
 at the network level why GBP allows for these optimizations and performance
 improvements you talked about.



 So are you saying that GBP allows someone to be able to configure an
 application that at the end of the day is equivalent  to
 networks/router/FWaaS rules without understanding networking concepts?

 It's one thing to understand the ports an application leverages and
 another to understand the differences between configuring VM firewalls,
 security groups, FWaaS, and router ACLs.


 Sure, but how does group based policy solve this. Security Groups and
 FWaaS are just different places of

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Wuhongning

Does it make sense to move all advanced extension out of ML2, like security 
group, qos...? Then we can just talk about advanced service itself, without 
bothering basic neutron object (network/subnet/port)

Traditionally, SG is applied in CN, and FWaas is applied in NN (bound to L3 
agent), however in DVR they are bluing, for each host has a L3 agent, this 
brings opportunity to move all service related features out of L2 agent, and 
coordinate  the two security modules (SG  FW for W-E traffic). However, 
neutron agents (l2/l3/advanced service) needs some rework.

When all these service features is detached from ML2, we can easily keep the 
mainstream OVS continually as L2 backend, and vendor specific hardware as the 
service policy enforcement backend such as acl/qos for high performance.

So maybe a trade-off is better: an optionally new policy service plugin 
together with SG and FWaas, cloud operate can choose what to present to 
enduser, but without concept of EP/EPG/BD/RD(now renamed to L2Pand L3P), Policy 
only focus on service layer, and policy template will be applied to existing 
neutron port object.

EP/EPG/L2P/L3P is really not so friendly to endusers facing. I asked some SMB 
IT staff and personal user of amazon AWS (they all have very limited networking 
knowledge), they can easily understand port/network/subnet, but can't 
understand those GBP objects. So, maybe apply advanced service policy to 
existing basic neutron object is more smoothly for endusers to accept.


From: Kevin Benton [blak...@gmail.com]
Sent: Friday, August 08, 2014 2:42 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way 
forward


Can you link to the etherpad you mentioned?

In the mean time, apologies for another analogy in
advance. :-)

If I give you an API to sort a list, I'm free to implement it however I want as 
long as I return a sorted list. However, there is no way me to know based on a 
call to this API that you might only be looking for the second largest element, 
so it won't be the most efficient approach because I will always have to sort 
the entire list.
If I give you a higher level API to declare that you want elements of a list 
that match a criteria in a certain order, then the API can make the 
optimization to not actually sort the whole list if you just need the first of 
the largest two elements.

The former is analogous to the security groups API, and the latter to the GBP 
API.

On Aug 7, 2014 4:00 PM, Aaron Rosen 
aaronoro...@gmail.commailto:aaronoro...@gmail.com wrote:



On Thu, Aug 7, 2014 at 12:08 PM, Kevin Benton 
blak...@gmail.commailto:blak...@gmail.com wrote:
I mean't 'side stepping' why GBP allows for the comment you made previous, 
With the latter, a mapping driver could determine that communication between 
these two hosts can be prevented by using an ACL on a router or a switch, 
which doesn't violate the user's intent and buys a performance improvement and 
works with ports that don't support security groups..

Neutron's current API is a logical abstraction and enforcement can be done 
however one chooses to implement it. I'm really trying to understand at the 
network level why GBP allows for these optimizations and performance 
improvements you talked about.

You absolutely cannot enforce security groups on a firewall/router that sits at 
the boundary between networks. If you try, you are lying to the end-user 
because it's not enforced at the port level. The current neutron APIs force you 
to decide where things like that are implemented.

The current neutron API's are just logical abstractions. Where and how things 
are actually enforced are 100% an implementation detail of a vendors system.  
Anyways, moving the discussion to the etherpad...

The higher level abstractions give you the freedom to move the enforcement by 
allowing the expression of broad connectivity requirements.
Why are you bringing up logging connections?

This was brought up as a feature proposal to FWaaS because this is a basic 
firewall feature missing from OpenStack. However, this does not preclude a 
FWaaS vendor from logging.

Personally, I think one could easily write up a very short document probably 
less than one page with examples showing/exampling how the current neutron API 
works even without a much networking background.

The difficulty of the API for establishing basic connectivity isn't really the 
problem. It's when you have to compose a bunch of requirements and make sure 
nothing is violating auditing and connectivity constraints that it becomes a 
problem. We are arguing about the levels of abstraction. You could also write 
up a short document explaining to novice programmers how to use C to read and 
write database entries to an sqlite database, but that doesn't mean it's the 
best level of abstraction for what the users are trying to accomplish.

I'll let

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread CARVER, PAUL

Wuhongning [mailto:wuhongn...@huawei.com] wrote:

Does it make sense to move all advanced extension out of ML2, like security
group, qos...? Then we can just talk about advanced service itself, without
bothering basic neutron object (network/subnet/port)

A modular layer 3 (ML3) analogous to ML2 sounds like a good idea. I still
think it's too late in the game to be shooting down all the work that the
GBP team has put in unless there's a really clean and effective way of
running AND iterating on GBP in conjunction with Neutron without being
part of the Juno release. As far as I can tell they've worked really
hard to follow the process and accommodate input. They shouldn't have
to wait multiple more releases on a hypothetical refactoring of how L3+ vs
L2 is structured.

But, just so I'm not making a horrible mistake, can someone reassure me
that GBP isn't removing the constructs of network/subnet/port from Neutron?

I'm under the impression that GBP is adding a higher level abstraction
but that it's not ripping basic constructs like network/subnet/port out
of the existing API. If I'm wrong about that I'll have to change my
opinion. We need those fundamental networking constructs to be present
and accessible to users that want/need to deal with them. I'm viewing
GBP as just a higher level abstraction over the top.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

The existing constructs will not change.
On Aug 8, 2014 9:49 AM, CARVER, PAUL pc2...@att.com wrote:

 Wuhongning [mailto:wuhongn...@huawei.com] wrote:

 Does it make sense to move all advanced extension out of ML2, like
 security
 group, qos...? Then we can just talk about advanced service itself,
 without
 bothering basic neutron object (network/subnet/port)

 A modular layer 3 (ML3) analogous to ML2 sounds like a good idea. I still
 think it's too late in the game to be shooting down all the work that the
 GBP team has put in unless there's a really clean and effective way of
 running AND iterating on GBP in conjunction with Neutron without being
 part of the Juno release. As far as I can tell they've worked really
 hard to follow the process and accommodate input. They shouldn't have
 to wait multiple more releases on a hypothetical refactoring of how L3+ vs
 L2 is structured.

 But, just so I'm not making a horrible mistake, can someone reassure me
 that GBP isn't removing the constructs of network/subnet/port from Neutron?

 I'm under the impression that GBP is adding a higher level abstraction
 but that it's not ripping basic constructs like network/subnet/port out
 of the existing API. If I'm wrong about that I'll have to change my
 opinion. We need those fundamental networking constructs to be present
 and accessible to users that want/need to deal with them. I'm viewing
 GBP as just a higher level abstraction over the top.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Ivar Lazzaro

Hi Paul,

Don't need to worry, you are perfectly right, GBP API is not replacing
anything :).

Also thanks for sharing your opinion on this matter.

Thanks,
Ivar.


On Fri, Aug 8, 2014 at 5:46 PM, CARVER, PAUL pc2...@att.com wrote:

 Wuhongning [mailto:wuhongn...@huawei.com] wrote:

 Does it make sense to move all advanced extension out of ML2, like
 security
 group, qos...? Then we can just talk about advanced service itself,
 without
 bothering basic neutron object (network/subnet/port)

 A modular layer 3 (ML3) analogous to ML2 sounds like a good idea. I still
 think it's too late in the game to be shooting down all the work that the
 GBP team has put in unless there's a really clean and effective way of
 running AND iterating on GBP in conjunction with Neutron without being
 part of the Juno release. As far as I can tell they've worked really
 hard to follow the process and accommodate input. They shouldn't have
 to wait multiple more releases on a hypothetical refactoring of how L3+ vs
 L2 is structured.

 But, just so I'm not making a horrible mistake, can someone reassure me
 that GBP isn't removing the constructs of network/subnet/port from Neutron?

 I'm under the impression that GBP is adding a higher level abstraction
 but that it's not ripping basic constructs like network/subnet/port out
 of the existing API. If I'm wrong about that I'll have to change my
 opinion. We need those fundamental networking constructs to be present
 and accessible to users that want/need to deal with them. I'm viewing
 GBP as just a higher level abstraction over the top.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Akash Gangil

Quick Question:
From what I understand, GBP is a high level declarative way of configuring
the network which ultimately gets mapped to basic Neutron API's via some
business logic. Why can't it be in a module of it own? In that way users
who want to use it can just install that and use it as an interface to
interact with Neutron while the rest continue with their lives as usual.




On Fri, Aug 8, 2014 at 9:25 PM, Kevin Benton blak...@gmail.com wrote:

 The existing constructs will not change.
 On Aug 8, 2014 9:49 AM, CARVER, PAUL pc2...@att.com wrote:

 Wuhongning [mailto:wuhongn...@huawei.com] wrote:

 Does it make sense to move all advanced extension out of ML2, like
 security
 group, qos...? Then we can just talk about advanced service itself,
 without
 bothering basic neutron object (network/subnet/port)

 A modular layer 3 (ML3) analogous to ML2 sounds like a good idea. I still
 think it's too late in the game to be shooting down all the work that the
 GBP team has put in unless there's a really clean and effective way of
 running AND iterating on GBP in conjunction with Neutron without being
 part of the Juno release. As far as I can tell they've worked really
 hard to follow the process and accommodate input. They shouldn't have
 to wait multiple more releases on a hypothetical refactoring of how L3+ vs
 L2 is structured.

 But, just so I'm not making a horrible mistake, can someone reassure me
 that GBP isn't removing the constructs of network/subnet/port from
 Neutron?

 I'm under the impression that GBP is adding a higher level abstraction
 but that it's not ripping basic constructs like network/subnet/port out
 of the existing API. If I'm wrong about that I'll have to change my
 opinion. We need those fundamental networking constructs to be present
 and accessible to users that want/need to deal with them. I'm viewing
 GBP as just a higher level abstraction over the top.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Akash
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Jay Pipes

On 08/08/2014 08:55 AM, Kevin Benton wrote:

The existing constructs will not change.

A followup question on the above...

If GPB API is merged into Neutron, the next logical steps (from what I
can tell) will be to add drivers that handle policy-based payloads/requests.

Some of these drivers, AFAICT, will *not* be deconstructing these policy
requests into the low-level port, network, and subnet
creation/attachment/detachment commands, but instead will be calling out
as-is to hardware that speaks the higher-level abstraction API [1], not
the lower-level port/subnet/network APIs. The low-level APIs would
essentially be consumed entirely within the policy-based driver, which
would effectively mean that the only way a system would be able to
orchestrate networking in systems using these drivers would be via the
high-level policy API.

Is that correct? Very sorry if I haven't explained clearly my
question... this is a tough question to frame eloquently :(

Thanks,
-jay

[1]
http://www.cisco.com/c/en/us/solutions/data-center-virtualization/application-centric-infrastructure/index.html

On Aug 8, 2014 9:49 AM, CARVER, PAUL pc2...@att.com
mailto:pc2...@att.com wrote:

Wuhongning [mailto:wuhongn...@huawei.com
mailto:wuhongn...@huawei.com] wrote:

Does it make sense to move all advanced extension out of ML2, like
security
group, qos...? Then we can just talk about advanced service
itself, without
bothering basic neutron object (network/subnet/port)

A modular layer 3 (ML3) analogous to ML2 sounds like a good idea. I
still
think it's too late in the game to be shooting down all the work
that the
GBP team has put in unless there's a really clean and effective way of
running AND iterating on GBP in conjunction with Neutron without being
part of the Juno release. As far as I can tell they've worked really
hard to follow the process and accommodate input. They shouldn't have
to wait multiple more releases on a hypothetical refactoring of how
L3+ vs
L2 is structured.

But, just so I'm not making a horrible mistake, can someone reassure me
that GBP isn't removing the constructs of network/subnet/port from
Neutron?

I'm under the impression that GBP is adding a higher level abstraction
but that it's not ripping basic constructs like network/subnet/port out
of the existing API. If I'm wrong about that I'll have to change my
opinion. We need those fundamental networking constructs to be present
and accessible to users that want/need to deal with them. I'm viewing
GBP as just a higher level abstraction over the top.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Ivar Lazzaro

Hi Jay,

You can choose. The whole purpose of this is about flexibility, if you want
to use GBP API 'only' with a specific driver you just can.
Additionally, given the 'ML2 like' architecture, the reference mapping
driver can ideally run alongside by filling the core Neutron constructs
without ever 'disturbing' your own driver (I'm not entirely sure about this
but it seems feasible).

I hope this answers your question,
Ivar.


On Fri, Aug 8, 2014 at 6:28 PM, Jay Pipes jaypi...@gmail.com wrote:

 On 08/08/2014 08:55 AM, Kevin Benton wrote:

 The existing constructs will not change.


 A followup question on the above...

 If GPB API is merged into Neutron, the next logical steps (from what I can
 tell) will be to add drivers that handle policy-based payloads/requests.

 Some of these drivers, AFAICT, will *not* be deconstructing these policy
 requests into the low-level port, network, and subnet
 creation/attachment/detachment commands, but instead will be calling out
 as-is to hardware that speaks the higher-level abstraction API [1], not the
 lower-level port/subnet/network APIs. The low-level APIs would essentially
 be consumed entirely within the policy-based driver, which would
 effectively mean that the only way a system would be able to orchestrate
 networking in systems using these drivers would be via the high-level
 policy API.

 Is that correct? Very sorry if I haven't explained clearly my question...
 this is a tough question to frame eloquently :(

 Thanks,
 -jay

 [1] http://www.cisco.com/c/en/us/solutions/data-center-
 virtualization/application-centric-infrastructure/index.html

  On Aug 8, 2014 9:49 AM, CARVER, PAUL pc2...@att.com
 mailto:pc2...@att.com wrote:

 Wuhongning [mailto:wuhongn...@huawei.com
 mailto:wuhongn...@huawei.com] wrote:

  Does it make sense to move all advanced extension out of ML2, like
 security
  group, qos...? Then we can just talk about advanced service
 itself, without
  bothering basic neutron object (network/subnet/port)

 A modular layer 3 (ML3) analogous to ML2 sounds like a good idea. I
 still
 think it's too late in the game to be shooting down all the work
 that the
 GBP team has put in unless there's a really clean and effective way of
 running AND iterating on GBP in conjunction with Neutron without being
 part of the Juno release. As far as I can tell they've worked really
 hard to follow the process and accommodate input. They shouldn't have
 to wait multiple more releases on a hypothetical refactoring of how
 L3+ vs
 L2 is structured.

 But, just so I'm not making a horrible mistake, can someone reassure
 me
 that GBP isn't removing the constructs of network/subnet/port from
 Neutron?

 I'm under the impression that GBP is adding a higher level abstraction
 but that it's not ripping basic constructs like network/subnet/port
 out
 of the existing API. If I'm wrong about that I'll have to change my
 opinion. We need those fundamental networking constructs to be present
 and accessible to users that want/need to deal with them. I'm viewing
 GBP as just a higher level abstraction over the top.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 mailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Salvatore Orlando

It might be because of the wording used, but it seems to me that you're
making it sound like the group policy effort could have been completely
orthogonal to neutron as we know it now.

What I understood is that the declarative abstraction offered by group
policy could do without any existing neutron entity leveraging native
drivers, but can actually be used also with existing neutron plugins
through the mapping driver - which will provide a sort of backward
compatibility. And still in that case I'm not sure one would be able to use
traditional neutron API (or legacy as it has been called), since I
don't know if the mapping driver is bidirectional.

I know this probably stems from my ignorance on the subject - I had
unfortunately very little time to catch-up with this effort in the past
months.

Salvatore

On 8 August 2014 18:49, Ivar Lazzaro ivarlazz...@gmail.com wrote:

 Hi Jay,

 You can choose. The whole purpose of this is about flexibility, if you
 want to use GBP API 'only' with a specific driver you just can.
 Additionally, given the 'ML2 like' architecture, the reference mapping
 driver can ideally run alongside by filling the core Neutron constructs
 without ever 'disturbing' your own driver (I'm not entirely sure about this
 but it seems feasible).

 I hope this answers your question,
 Ivar.


 On Fri, Aug 8, 2014 at 6:28 PM, Jay Pipes jaypi...@gmail.com wrote:

 On 08/08/2014 08:55 AM, Kevin Benton wrote:

 The existing constructs will not change.


 A followup question on the above...

 If GPB API is merged into Neutron, the next logical steps (from what I
 can tell) will be to add drivers that handle policy-based payloads/requests.

 Some of these drivers, AFAICT, will *not* be deconstructing these policy
 requests into the low-level port, network, and subnet
 creation/attachment/detachment commands, but instead will be calling out
 as-is to hardware that speaks the higher-level abstraction API [1], not the
 lower-level port/subnet/network APIs. The low-level APIs would essentially
 be consumed entirely within the policy-based driver, which would
 effectively mean that the only way a system would be able to orchestrate
 networking in systems using these drivers would be via the high-level
 policy API.

 Is that correct? Very sorry if I haven't explained clearly my question...
 this is a tough question to frame eloquently :(

 Thanks,
 -jay

 [1] http://www.cisco.com/c/en/us/solutions/data-center-
 virtualization/application-centric-infrastructure/index.html

  On Aug 8, 2014 9:49 AM, CARVER, PAUL pc2...@att.com
 mailto:pc2...@att.com wrote:

 Wuhongning [mailto:wuhongn...@huawei.com
 mailto:wuhongn...@huawei.com] wrote:

  Does it make sense to move all advanced extension out of ML2, like
 security
  group, qos...? Then we can just talk about advanced service
 itself, without
  bothering basic neutron object (network/subnet/port)

 A modular layer 3 (ML3) analogous to ML2 sounds like a good idea. I
 still
 think it's too late in the game to be shooting down all the work
 that the
 GBP team has put in unless there's a really clean and effective way
 of
 running AND iterating on GBP in conjunction with Neutron without
 being
 part of the Juno release. As far as I can tell they've worked really
 hard to follow the process and accommodate input. They shouldn't have
 to wait multiple more releases on a hypothetical refactoring of how
 L3+ vs
 L2 is structured.

 But, just so I'm not making a horrible mistake, can someone reassure
 me
 that GBP isn't removing the constructs of network/subnet/port from
 Neutron?

 I'm under the impression that GBP is adding a higher level
 abstraction
 but that it's not ripping basic constructs like network/subnet/port
 out
 of the existing API. If I'm wrong about that I'll have to change my
 opinion. We need those fundamental networking constructs to be
 present
 and accessible to users that want/need to deal with them. I'm viewing
 GBP as just a higher level abstraction over the top.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 mailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

There is an enforcement component to the group policy that allows you to
use the current APIs and it's the reason that group policy is integrated
into the neutron project. If someone uses the current APIs, the group
policy plugin will make sure they don't violate any policy constraints
before passing the request into the regular core/service plugins.


On Fri, Aug 8, 2014 at 11:02 AM, Salvatore Orlando sorla...@nicira.com
wrote:

 It might be because of the wording used, but it seems to me that you're
 making it sound like the group policy effort could have been completely
 orthogonal to neutron as we know it now.

 What I understood is that the declarative abstraction offered by group
 policy could do without any existing neutron entity leveraging native
 drivers, but can actually be used also with existing neutron plugins
 through the mapping driver - which will provide a sort of backward
 compatibility. And still in that case I'm not sure one would be able to use
 traditional neutron API (or legacy as it has been called), since I
 don't know if the mapping driver is bidirectional.

 I know this probably stems from my ignorance on the subject - I had
 unfortunately very little time to catch-up with this effort in the past
 months.

 Salvatore


 On 8 August 2014 18:49, Ivar Lazzaro ivarlazz...@gmail.com wrote:

 Hi Jay,

 You can choose. The whole purpose of this is about flexibility, if you
 want to use GBP API 'only' with a specific driver you just can.
 Additionally, given the 'ML2 like' architecture, the reference mapping
 driver can ideally run alongside by filling the core Neutron constructs
 without ever 'disturbing' your own driver (I'm not entirely sure about this
 but it seems feasible).

 I hope this answers your question,
 Ivar.


 On Fri, Aug 8, 2014 at 6:28 PM, Jay Pipes jaypi...@gmail.com wrote:

 On 08/08/2014 08:55 AM, Kevin Benton wrote:

 The existing constructs will not change.


 A followup question on the above...

 If GPB API is merged into Neutron, the next logical steps (from what I
 can tell) will be to add drivers that handle policy-based payloads/requests.

 Some of these drivers, AFAICT, will *not* be deconstructing these policy
 requests into the low-level port, network, and subnet
 creation/attachment/detachment commands, but instead will be calling out
 as-is to hardware that speaks the higher-level abstraction API [1], not the
 lower-level port/subnet/network APIs. The low-level APIs would essentially
 be consumed entirely within the policy-based driver, which would
 effectively mean that the only way a system would be able to orchestrate
 networking in systems using these drivers would be via the high-level
 policy API.

 Is that correct? Very sorry if I haven't explained clearly my
 question... this is a tough question to frame eloquently :(

 Thanks,
 -jay

 [1] http://www.cisco.com/c/en/us/solutions/data-center-
 virtualization/application-centric-infrastructure/index.html

  On Aug 8, 2014 9:49 AM, CARVER, PAUL pc2...@att.com
 mailto:pc2...@att.com wrote:

 Wuhongning [mailto:wuhongn...@huawei.com
 mailto:wuhongn...@huawei.com] wrote:

  Does it make sense to move all advanced extension out of ML2, like
 security
  group, qos...? Then we can just talk about advanced service
 itself, without
  bothering basic neutron object (network/subnet/port)

 A modular layer 3 (ML3) analogous to ML2 sounds like a good idea. I
 still
 think it's too late in the game to be shooting down all the work
 that the
 GBP team has put in unless there's a really clean and effective way
 of
 running AND iterating on GBP in conjunction with Neutron without
 being
 part of the Juno release. As far as I can tell they've worked really
 hard to follow the process and accommodate input. They shouldn't
 have
 to wait multiple more releases on a hypothetical refactoring of how
 L3+ vs
 L2 is structured.

 But, just so I'm not making a horrible mistake, can someone
 reassure me
 that GBP isn't removing the constructs of network/subnet/port from
 Neutron?

 I'm under the impression that GBP is adding a higher level
 abstraction
 but that it's not ripping basic constructs like network/subnet/port
 out
 of the existing API. If I'm wrong about that I'll have to change my
 opinion. We need those fundamental networking constructs to be
 present
 and accessible to users that want/need to deal with them. I'm
 viewing
 GBP as just a higher level abstraction over the top.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 mailto:OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Sumit Naiksatam

Hi Jay, To extend Ivar's response here, the core resources and core
plugin configuration does not change with the addition of these
extensions. The mechanism to implement the GBP extensions is via a
service plugin. So even in a deployment where a GBP service plugin is
deployed with a driver which interfaces with a backend that perhaps
directly understands some of the GBP constructs, that system would
still need to have a core plugin configured that honors Neutron's core
resources. Hence my earlier comment that GBP extensions are
complementary to the existing core resources (in much the same way as
the existing extensions in Neutron).

Thanks,
~Sumit.

On Fri, Aug 8, 2014 at 9:49 AM, Ivar Lazzaro ivarlazz...@gmail.com wrote:
Hi Jay,

You can choose. The whole purpose of this is about flexibility, if you want
to use GBP API 'only' with a specific driver you just can.
Additionally, given the 'ML2 like' architecture, the reference mapping
driver can ideally run alongside by filling the core Neutron constructs
without ever 'disturbing' your own driver (I'm not entirely sure about this
but it seems feasible).

I hope this answers your question,
Ivar.

On Fri, Aug 8, 2014 at 6:28 PM, Jay Pipes jaypi...@gmail.com wrote:

On 08/08/2014 08:55 AM, Kevin Benton wrote:

The existing constructs will not change.

A followup question on the above...

If GPB API is merged into Neutron, the next logical steps (from what I can
tell) will be to add drivers that handle policy-based payloads/requests.

Some of these drivers, AFAICT, will *not* be deconstructing these policy
requests into the low-level port, network, and subnet
creation/attachment/detachment commands, but instead will be calling out
as-is to hardware that speaks the higher-level abstraction API [1], not the
lower-level port/subnet/network APIs. The low-level APIs would essentially
be consumed entirely within the policy-based driver, which would effectively
mean that the only way a system would be able to orchestrate networking in
systems using these drivers would be via the high-level policy API.

Is that correct? Very sorry if I haven't explained clearly my question...
this is a tough question to frame eloquently :(

Thanks,
-jay

[1]
http://www.cisco.com/c/en/us/solutions/data-center-virtualization/application-centric-infrastructure/index.html

On Aug 8, 2014 9:49 AM, CARVER, PAUL pc2...@att.com
mailto:pc2...@att.com wrote:

Wuhongning [mailto:wuhongn...@huawei.com
mailto:wuhongn...@huawei.com] wrote:

A modular layer 3 (ML3) analogous to ML2 sounds like a good idea. I
still
think it's too late in the game to be shooting down all the work
that the
GBP team has put in unless there's a really clean and effective way
of
running AND iterating on GBP in conjunction with Neutron without
being
part of the Juno release. As far as I can tell they've worked really
hard to follow the process and accommodate input. They shouldn't have
to wait multiple more releases on a hypothetical refactoring of how
L3+ vs
L2 is structured.

But, just so I'm not making a horrible mistake, can someone reassure
me
that GBP isn't removing the constructs of network/subnet/port from
Neutron?

I'm under the impression that GBP is adding a higher level
abstraction
but that it's not ripping basic constructs like network/subnet/port
out
of the existing API. If I'm wrong about that I'll have to change my
opinion. We need those fundamental networking constructs to be
present
and accessible to users that want/need to deal with them. I'm viewing
GBP as just a higher level abstraction over the top.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
mailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Jay Pipes


On 08/08/2014 12:29 PM, Sumit Naiksatam wrote:

Hi Jay, To extend Ivar's response here, the core resources and core
plugin configuration does not change with the addition of these
extensions. The mechanism to implement the GBP extensions is via a
service plugin. So even in a deployment where a GBP service plugin is
deployed with a driver which interfaces with a backend that perhaps
directly understands some of the GBP constructs, that system would
still need to have a core plugin configured that honors Neutron's core
resources. Hence my earlier comment that GBP extensions are
complementary to the existing core resources (in much the same way as
the existing extensions in Neutron).


OK, thanks Sumit. That clearly explains things for me.

Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On 8 August 2014 10:56, Kevin Benton blak...@gmail.com wrote:

 There is an enforcement component to the group policy that allows you to
 use the current APIs and it's the reason that group policy is integrated
 into the neutron project. If someone uses the current APIs, the group
 policy plugin will make sure they don't violate any policy constraints
 before passing the request into the regular core/service plugins.


This is the statement that makes me trip over, and I don't understand why
GBP and Neutron Core need to be 'integrated' together as they have. Policy
decision points can be decentralized from the system under scrutiny, we
don't need to have one giant monolithic system that does everything; it's
an architectural decision that would make difficult to achieve
composability and all the other good -ilities of software systems.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Sumit Naiksatam

On Fri, Aug 8, 2014 at 12:45 PM, Armando M. arma...@gmail.com wrote:
 On 8 August 2014 10:56, Kevin Benton blak...@gmail.com wrote:

 There is an enforcement component to the group policy that allows you to
 use the current APIs and it's the reason that group policy is integrated
 into the neutron project. If someone uses the current APIs, the group policy
 plugin will make sure they don't violate any policy constraints before
 passing the request into the regular core/service plugins.


 This is the statement that makes me trip over, and I don't understand why
 GBP and Neutron Core need to be 'integrated' together as they have. Policy
 decision points can be decentralized from the system under scrutiny, we
 don't need to have one giant monolithic system that does everything; it's an
 architectural decision that would make difficult to achieve composability
 and all the other good -ilities of software systems.


Adding the GBP extension to Neutron does not change the nature of the
software architecture of Neutron making it more or less monolithic. It
fulfills a gap that is currently present in the Neutron API, namely -
to complement the current imperative abstractions with a app
-developer/deployer friendly declarative abstraction [1]. To
reiterate, it has been proposed as an “extension”, and not a
replacement of the core abstractions or the way those are consumed. If
this is understood and interpreted correctly, I doubt that there
should be reason for concern.

[1] https://review.openstack.org/#/c/89469

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Prasad Vellanki

GBP is about networking policy and hence limited to networking constructs.
It enhances the networking constructs. Since it follows more or less the
plugin model, it is not in one monolithic module but fans out to the policy
module and is done via  extension.


On Fri, Aug 8, 2014 at 12:45 PM, Armando M. arma...@gmail.com wrote:

 On 8 August 2014 10:56, Kevin Benton blak...@gmail.com wrote:

 There is an enforcement component to the group policy that allows you to
 use the current APIs and it's the reason that group policy is integrated
 into the neutron project. If someone uses the current APIs, the group
 policy plugin will make sure they don't violate any policy constraints
 before passing the request into the regular core/service plugins.


 This is the statement that makes me trip over, and I don't understand why
 GBP and Neutron Core need to be 'integrated' together as they have. Policy
 decision points can be decentralized from the system under scrutiny, we
 don't need to have one giant monolithic system that does everything; it's
 an architectural decision that would make difficult to achieve
 composability and all the other good -ilities of software systems.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward


 Adding the GBP extension to Neutron does not change the nature of the
 software architecture of Neutron making it more or less monolithic.


I agree with this statement...partially: the way GBP was developed is in
accordance to the same principles and architectural choices made for the
service plugins and frameworks we have right now, and yes it does not make
Neutron more monolithic but certainly not less. These same very principles
have unveiled limitations we have realized need to be addressed, according
to Neutron's busy agenda. That said, if I were to be given the opportunity
to revise some architectural decisions during the new groundbreaking work
(regardless of the nature), I would.

For instance, I hate that the service plugins live in the same address
space of Neutron Server, I hate that I have one Neutron Server that does
L2, L3, IPAM, ...; we could break it down and make sure every entity can
have its own lifecycle: we can compose and integrate more easily if we did.
Isn't that what years of middleware and distributed systems taught us?

I suggested in the past that GBP would best integrate to Neutron via a
stable and RESTful interface, just like any other OpenStack project does. I
have been unable to be convinced otherwise, and I would love to be able to
change my opinion.


 It
 fulfills a gap that is currently present in the Neutron API, namely -
 to complement the current imperative abstractions with a app
 -developer/deployer friendly declarative abstraction [1]. To
 reiterate, it has been proposed as an “extension”, and not a
 replacement of the core abstractions or the way those are consumed.

If
 this is understood and interpreted correctly, I doubt that there
 should be reason for concern.


I never said that GBP did (mean to replace the core abstractions): I am
talking purely architecture and system integration. Not sure if this
statement is directed to my comment.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

This is the statement that makes me trip over,

I don't know what that means. Does it mean that you are so incredibly
shocked by the stupidity of that statement that you fall down? Or does it
mean something else?

Policy decision points can be decentralized from the system under scrutiny,


Unfortunately they can't in this case where some policy needs to be
enforced between plugins. If we could refactor the communication between
service and core plugins to use the API as well, then we probably could
build this as a middleware.


On Fri, Aug 8, 2014 at 1:45 PM, Armando M. arma...@gmail.com wrote:

 On 8 August 2014 10:56, Kevin Benton blak...@gmail.com wrote:

 There is an enforcement component to the group policy that allows you to
 use the current APIs and it's the reason that group policy is integrated
 into the neutron project. If someone uses the current APIs, the group
 policy plugin will make sure they don't violate any policy constraints
 before passing the request into the regular core/service plugins.


 This is the statement that makes me trip over, and I don't understand why
 GBP and Neutron Core need to be 'integrated' together as they have. Policy
 decision points can be decentralized from the system under scrutiny, we
 don't need to have one giant monolithic system that does everything; it's
 an architectural decision that would make difficult to achieve
 composability and all the other good -ilities of software systems.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On 8 August 2014 14:55, Kevin Benton blak...@gmail.com wrote:

 This is the statement that makes me trip over,

 I don't know what that means. Does it mean that you are so incredibly
 shocked by the stupidity of that statement that you fall down? Or does it
 mean something else?


Why would you think that? I trip over the obstacle that prevents me from
understanding! If at all, I would blame my stupidity, not the one of the
statement :)



 Policy decision points can be decentralized from the system under
 scrutiny,

 Unfortunately they can't in this case where some policy needs to be
 enforced between plugins. If we could refactor the communication between
 service and core plugins to use the API as well, then we probably could
 build this as a middleware.


Assumed I agreed they couldn't, which I find hard to believe, instead of
going after the better approach, we stick with the less optimal one?



 On Fri, Aug 8, 2014 at 1:45 PM, Armando M. arma...@gmail.com wrote:

 On 8 August 2014 10:56, Kevin Benton blak...@gmail.com wrote:

 There is an enforcement component to the group policy that allows you to
 use the current APIs and it's the reason that group policy is integrated
 into the neutron project. If someone uses the current APIs, the group
 policy plugin will make sure they don't violate any policy constraints
 before passing the request into the regular core/service plugins.


 This is the statement that makes me trip over, and I don't understand why
 GBP and Neutron Core need to be 'integrated' together as they have. Policy
 decision points can be decentralized from the system under scrutiny, we
 don't need to have one giant monolithic system that does everything; it's
 an architectural decision that would make difficult to achieve
 composability and all the other good -ilities of software systems.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 --
 Kevin Benton

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Prasad Vellanki

On Fri, Aug 8, 2014 at 2:21 PM, Armando M. arma...@gmail.com wrote:

 Adding the GBP extension to Neutron does not change the nature of the
 software architecture of Neutron making it more or less monolithic.


 I agree with this statement...partially: the way GBP was developed is in
 accordance to the same principles and architectural choices made for the
 service plugins and frameworks we have right now, and yes it does not make
 Neutron more monolithic but certainly not less. These same very principles
 have unveiled limitations we have realized need to be addressed, according
 to Neutron's busy agenda. That said, if I were to be given the opportunity
 to revise some architectural decisions during the new groundbreaking work
 (regardless of the nature), I would.

 For instance, I hate that the service plugins live in the same address
 space of Neutron Server, I hate that I have one Neutron Server that does
 L2, L3, IPAM, ...; we could break it down and make sure every entity can
 have its own lifecycle: we can compose and integrate more easily if we did.
 Isn't that what years of middleware and distributed systems taught us?

 I suggested in the past that GBP would best integrate to Neutron via a
 stable and RESTful interface, just like any other OpenStack project does. I
 have been unable to be convinced otherwise, and I would love to be able to
 change my opinion.





 One advantage of the service plugin is that one can leverage the neutron
common framework such as Keystone authentication where common scoping is
done. It would be important in the policy type of framework to have such
scoping

While the service plugin has scalability issues as pointed above that it
resides in neutron server, it is however stable and user configurable and a
lot of common code is executed for networking services. So while we make
the next generation services framework more distributed and scalable, it is
ok to do it under the current framework especially since it has provision
for the user to opt in when needed.




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

  One advantage of the service plugin is that one can leverage the neutron
 common framework such as Keystone authentication where common scoping is
 done. It would be important in the policy type of framework to have such
 scoping


The framework you're referring to is common and already reusable, it's not
a prerogative of Neutron.



 While the service plugin has scalability issues as pointed above that it
 resides in neutron server, it is however stable and user configurable and a
 lot of common code is executed for networking services.


This is what static or dynamic libraries are for and reused for; I can have
a building block and reuse it many times the way I see fit keeping my
components' lifecycles separate.


 So while we make the next generation services framework more distributed
 and scalable, it is ok to do it under the current framework especially
 since it has provision for the user to opt in when needed.


A next generation services framework is not a prerequisite to integrating
two OpenStack projects via REST APIs. I don't see how we would associate
the two concepts together.






 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-08 Thread Hemanth Ravi

On Fri, Aug 8, 2014 at 5:38 PM, Armando M. arma...@gmail.com wrote:



  One advantage of the service plugin is that one can leverage the neutron
 common framework such as Keystone authentication where common scoping is
 done. It would be important in the policy type of framework to have such
 scoping


 The framework you're referring to is common and already reusable, it's not
 a prerogative of Neutron.


Are you suggesting that Service Plugins, L3, IPAM etc become individual
endpoints, resulting in redundant authentication round-trips for each of
the components.

Wouldn't this result in degraded performance and potential consistency
issues?





 While the service plugin has scalability issues as pointed above that it
 resides in neutron server, it is however stable and user configurable and a
 lot of common code is executed for networking services.


 This is what static or dynamic libraries are for and reused for; I can
 have a building block and reuse it many times the way I see fit keeping my
 components' lifecycles separate.


 So while we make the next generation services framework more distributed
 and scalable, it is ok to do it under the current framework especially
 since it has provision for the user to opt in when needed.


 A next generation services framework is not a prerequisite to integrating
 two OpenStack projects via REST APIs. I don't see how we would associate
 the two concepts together.






 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward



 On Fri, Aug 8, 2014 at 5:38 PM, Armando M. arma...@gmail.com wrote:



   One advantage of the service plugin is that one can leverage the
 neutron common framework such as Keystone authentication where common
 scoping is done. It would be important in the policy type of framework to
 have such scoping


 The framework you're referring to is common and already reusable, it's
 not a prerogative of Neutron.


  Are you suggesting that Service Plugins, L3, IPAM etc become individual
 endpoints, resulting in redundant authentication round-trips for each of
 the components.

 Wouldn't this result in degraded performance and potential consistency
 issues?


The endpoint - in the OpenStack lingo - that exposes the API abstractions
(concepts and operations) can be, logically and physically, different from
the worker that implements these abstractions; authentication is orthogonal
to this and I am not suggesting what you mention.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-07 Thread Kevin Benton

I mean't 'side stepping' why GBP allows for the comment you made previous,
With the latter, a mapping driver could determine that communication
between these two hosts can be prevented by using an ACL on a router or a
switch, which doesn't violate the user's intent and buys a performance
improvement and works with ports that don't support security groups..

Neutron's current API is a logical abstraction and enforcement can be done
however one chooses to implement it. I'm really trying to understand at the
network level why GBP allows for these optimizations and performance
improvements you talked about.

You absolutely cannot enforce security groups on a firewall/router that
sits at the boundary between networks. If you try, you are lying to the
end-user because it's not enforced at the port level. The current neutron
APIs force you to decide where things like that are implemented. The higher
level abstractions give you the freedom to move the enforcement by allowing
the expression of broad connectivity requirements.

Why are you bringing up logging connections?

This was brought up as a feature proposal to FWaaS because this is a basic
firewall feature missing from OpenStack. However, this does not preclude a
FWaaS vendor from logging.

Personally, I think one could easily write up a very short document
probably less than one page with examples showing/exampling how the current
neutron API works even without a much networking background.

The difficulty of the API for establishing basic connectivity isn't really
the problem. It's when you have to compose a bunch of requirements and make
sure nothing is violating auditing and connectivity constraints that it
becomes a problem. We are arguing about the levels of abstraction. You
could also write up a short document explaining to novice programmers how
to use C to read and write database entries to an sqlite database, but that
doesn't mean it's the best level of abstraction for what the users are
trying to accomplish.

I'll let someone else explain the current GBP API because I'm not working
on that. I'm just trying to convince you of the value of declarative
network configuration.


On Thu, Aug 7, 2014 at 12:02 PM, Aaron Rosen aaronoro...@gmail.com wrote:




 On Thu, Aug 7, 2014 at 9:54 AM, Kevin Benton blak...@gmail.com wrote:

 You said you had no idea what group based policy was buying us so I tried
 to illustrate what the difference between declarative and imperative
 network configuration looks like. That's the major selling point of GBP so
 I'm not sure how that's 'side stepping' any points. It removes the need for
 the user to pick between implementation details like security
 groups/FWaaS/ACLs.


 I mean't 'side stepping' why GBP allows for the comment you made previous,
 With the latter, a mapping driver could determine that communication
 between these two hosts can be prevented by using an ACL on a router or a
 switch, which doesn't violate the user's intent and buys a performance
 improvement and works with ports that don't support security groups..

 Neutron's current API is a logical abstraction and enforcement can be done
 however one chooses to implement it. I'm really trying to understand at the
 network level why GBP allows for these optimizations and performance
 improvements you talked about.



 So are you saying that GBP allows someone to be able to configure an
 application that at the end of the day is equivalent  to
 networks/router/FWaaS rules without understanding networking concepts?

 It's one thing to understand the ports an application leverages and
 another to understand the differences between configuring VM firewalls,
 security groups, FWaaS, and router ACLs.


 Sure, but how does group based policy solve this. Security Groups and
 FWaaS are just different places of enforcement. Say I want different
 security enforcement on my router than on my instances. One still needs to
 know enough to tell group based policy this right?  They need to know
 enough that there are different enforcement points? How is doing this with
 Group based policy make it easier?



  I'm also curious how this GBP is really less error prone than the
 model we have today as it seems the user will basically have to tell
 neutron the same information about how he wants his networking to function.

 With GBP, the user just gives the desired end result (e.g. allow
 connectivity between endpoint groups via TCP port 22 with all connections
 logged). Without it, the user has to do the following:


 Why are you bringing up logging connections? Neutron has no concept of
 this at all today in it's code base. Is logging something related to GBP?


1. create a network/subnet for each endpoint group
2. allow all traffic on the security groups since the logging would
need to be accomplished with FWaaS
3. create an FWaaS instance
4. attach the FWaaS to both networks

 Today FWaaS api is still incomplete as there is no real point of
 enforcement

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-07 Thread Aaron Rosen

On Thu, Aug 7, 2014 at 12:08 PM, Kevin Benton blak...@gmail.com wrote:

 I mean't 'side stepping' why GBP allows for the comment you made
 previous, With the latter, a mapping driver could determine that
 communication between these two hosts can be prevented by using an ACL on a
 router or a switch, which doesn't violate the user's intent and buys a
 performance improvement and works with ports that don't support security
 groups..

 Neutron's current API is a logical abstraction and enforcement can be
 done however one chooses to implement it. I'm really trying to understand
 at the network level why GBP allows for these optimizations and performance
 improvements you talked about.

 You absolutely cannot enforce security groups on a firewall/router that
 sits at the boundary between networks. If you try, you are lying to the
 end-user because it's not enforced at the port level. The current neutron
 APIs force you to decide where things like that are implemented.


The current neutron API's are just logical abstractions. Where and how
things are actually enforced are 100% an implementation detail of a vendors
system.  Anyways, moving the discussion to the etherpad...



 The higher level abstractions give you the freedom to move the enforcement
 by allowing the expression of broad connectivity requirements.

Why are you bringing up logging connections?

 This was brought up as a feature proposal to FWaaS because this is a basic
 firewall feature missing from OpenStack. However, this does not preclude a
 FWaaS vendor from logging.

 Personally, I think one could easily write up a very short document
 probably less than one page with examples showing/exampling how the current
 neutron API works even without a much networking background.

 The difficulty of the API for establishing basic connectivity isn't really
 the problem. It's when you have to compose a bunch of requirements and make
 sure nothing is violating auditing and connectivity constraints that it
 becomes a problem. We are arguing about the levels of abstraction. You
 could also write up a short document explaining to novice programmers how
 to use C to read and write database entries to an sqlite database, but that
 doesn't mean it's the best level of abstraction for what the users are
 trying to accomplish.

 I'll let someone else explain the current GBP API because I'm not working
 on that. I'm just trying to convince you of the value of declarative
 network configuration.


 On Thu, Aug 7, 2014 at 12:02 PM, Aaron Rosen aaronoro...@gmail.com
 wrote:




 On Thu, Aug 7, 2014 at 9:54 AM, Kevin Benton blak...@gmail.com wrote:

 You said you had no idea what group based policy was buying us so I
 tried to illustrate what the difference between declarative and imperative
 network configuration looks like. That's the major selling point of GBP so
 I'm not sure how that's 'side stepping' any points. It removes the need for
 the user to pick between implementation details like security
 groups/FWaaS/ACLs.


 I mean't 'side stepping' why GBP allows for the comment you made
 previous, With the latter, a mapping driver could determine that
 communication between these two hosts can be prevented by using an ACL on a
 router or a switch, which doesn't violate the user's intent and buys a
 performance improvement and works with ports that don't support security
 groups..

 Neutron's current API is a logical abstraction and enforcement can be
 done however one chooses to implement it. I'm really trying to understand
 at the network level why GBP allows for these optimizations and performance
 improvements you talked about.



 So are you saying that GBP allows someone to be able to configure an
 application that at the end of the day is equivalent  to
 networks/router/FWaaS rules without understanding networking concepts?

 It's one thing to understand the ports an application leverages and
 another to understand the differences between configuring VM firewalls,
 security groups, FWaaS, and router ACLs.


 Sure, but how does group based policy solve this. Security Groups and
 FWaaS are just different places of enforcement. Say I want different
 security enforcement on my router than on my instances. One still needs to
 know enough to tell group based policy this right?  They need to know
 enough that there are different enforcement points? How is doing this with
 Group based policy make it easier?



  I'm also curious how this GBP is really less error prone than the
 model we have today as it seems the user will basically have to tell
 neutron the same information about how he wants his networking to function.

 With GBP, the user just gives the desired end result (e.g. allow
 connectivity between endpoint groups via TCP port 22 with all connections
 logged). Without it, the user has to do the following:


 Why are you bringing up logging connections? Neutron has no concept of
 this at all today in it's code base. Is logging something related to GBP?

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-07 Thread Mohammad Banikazemi

Thierry Carrez thie...@openstack.org wrote on 08/07/2014 06:23:56 AM:

 From: Thierry Carrez thie...@openstack.org
 To: openstack-dev@lists.openstack.org
 Date: 08/07/2014 06:25 AM
 Subject: Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy
 and the way forward

 Armando M. wrote:
  This thread is moving so fast I can't keep up!

  The fact that troubles me is that I am unable to grasp how we move
  forward, which was the point of this thread to start with. It seems we
  have 2 options:

  - We make GBP to merge as is, in the Neutron tree, with some minor
  revision (e.g. naming?);
  - We make GBP a stackforge project, that integrates with Neutron in
some
  shape or form;

  Another option, might be something in between, where GBP is in tree,
but
  in some sort of experimental staging area (even though I am not sure
how
  well baked this idea is).

  Now, as a community we all need make a decision; arguing about the fact
  that the blueprint was approved is pointless.
 I agree with you: it is possible to change your mind on a topic and
 revisit past decisions.
 In past OpenStack history we did revert merged
 commits and remove existing functionality because we felt it wasn't that
 much of a great idea after all. Here we are talking about making the
 right decision *before* the final merging and shipping into a release,
 which is kind of an improvement. The spec system was supposed to help
 limit such cases, but it's not bullet-proof.

 In the end, if there is no consensus on that question within the Neutron
 project (and I hear both sides have good arguments), our governance
 gives the elected Neutron PTL the power to make the final call. If the
 disagreement is between projects (like if Nova disagreed with the
 Neutron decision), then the issue could be escalated to the TC.

It is good to know that the OpenStack governance provides a way to resolve
these issues but I really hope that we can reach a consensus.

Best,

Mohammad

 Regards,

 --
 Thierry Carrez (ttx)

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-07 Thread Subrahmanyam Ongole

I am one of the developers on the project, so I have a strong preference
for option (1).

I think a 3rd option is also possible, which offers a scale down version of
GBP APIs. Contracts could be added in kilo. Provide EndPoints,
EndPointGroups, Rules and Policies. This is the simpler approach suggested
in GBP document, where you have a policy with a single rule (classifier 
action) applied between 2 EPGs. This approach minimizes complexity and
therefore saves precious reviewer's time. This requires some code reorg,
which may not be preferable to other developers on the project.

Alternatively, contracts could be added as optional vendor extensions in
Juno.



On Wed, Aug 6, 2014 at 8:50 PM, Alan Kavanagh alan.kavan...@ericsson.com
wrote:

 +1
 I believe Pedro has a very valid point here, and that is the the
 community to approve the spec and that decision should be respected. It
 makes sense to again clearly denote the process and governance and have
 this noted on the thread Stefano started earlier today.

 /Alan

 -Original Message-
 From: Pedro Marques [mailto:pedro.r.marq...@gmail.com]
 Sent: August-06-14 4:52 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the
 way forward


 On Aug 6, 2014, at 1:27 PM, Jay Pipes jaypi...@gmail.com wrote:
 
  However, it seems to me that the end-goal of the GBP effort is
 *actually* to provide a higher-layer API to Neutron that would essentially
 enable proprietary vendors to entirely swap out all of Neutron core for a
 new set of drivers that spoke proprietary device APIs.
 
  If this is the end-goal, it should be stated more clearly, IMO.

 I believe that people should be considered innocent until proven
 otherwise. Is there a reason to believe there is some hidden reason behind
 this proposal ? It seems to me that this is uncalled for.

 Neutron allows vendors to speak to proprietary device APIs, it was
 designed to do so, AFAIK. It is also possibly to entirely swap out all of
 the Neutron core... the proponents of the group based policy didn't have
 to go through so much trouble if that was their intent. As far as i know
 most plugins talk to a proprietary API.

 I happen to disagree technically with a couple of choices made by this
 proposal; but the blueprint was approved. Which means that i lost the
 argument, or didn't raise it on time, or didn't argue convincingly...
 regardless of the reason, the time to argue about the goal has passed. The
 decision of the community was to approve the spec and that decision should
 be respected.

   Pedro.
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 

Thanks
OSM
(Subrahmanyam Ongole)
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-07 Thread Thierry Carrez

Armando M. wrote:
 This thread is moving so fast I can't keep up!
 
 The fact that troubles me is that I am unable to grasp how we move
 forward, which was the point of this thread to start with. It seems we
 have 2 options:
 
 - We make GBP to merge as is, in the Neutron tree, with some minor
 revision (e.g. naming?);
 - We make GBP a stackforge project, that integrates with Neutron in some
 shape or form;
 
 Another option, might be something in between, where GBP is in tree, but
 in some sort of experimental staging area (even though I am not sure how
 well baked this idea is).
 
 Now, as a community we all need make a decision; arguing about the fact
 that the blueprint was approved is pointless.

I agree with you: it is possible to change your mind on a topic and
revisit past decisions. In past OpenStack history we did revert merged
commits and remove existing functionality because we felt it wasn't that
much of a great idea after all. Here we are talking about making the
right decision *before* the final merging and shipping into a release,
which is kind of an improvement. The spec system was supposed to help
limit such cases, but it's not bullet-proof.

In the end, if there is no consensus on that question within the Neutron
project (and I hear both sides have good arguments), our governance
gives the elected Neutron PTL the power to make the final call. If the
disagreement is between projects (like if Nova disagreed with the
Neutron decision), then the issue could be escalated to the TC.

Regards,

-- 
Thierry Carrez (ttx)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-07 Thread Nachi Ueno

Hi folks


I think this thread is still mixing topics. I feel we can archive 1000 mails :P
so let me name it and let me write my thought on this.


[Topic1] Nova parity priority

I do understand concern and this is highest priority.
However, group based policy effort won't slower this effort.


Because most skilled developers such as  Mark, Oleg and Carl are
working hard for this making happen.
I'm also trying to POC non-DVR approach such as
migrating nova-network directly to the neutron. (Sorry, this is off
topic, but if you are interested in this,
https://docs.google.com/presentation/d/12w28HhNpLltSpA6pJvKWBiqeEusaI-NM-OZo8HNyH2w/edit#slide=id.p
is my thought )

Even if codes for GBP and Nova parity is really independent, so it
won't conflict.
Note that if you have any task item related with nova parity work, plz ping me.

[Topic2] Neutron community decision making process

I'm super lazy guy, so I'll be very disappointed the code i write rejected..
( http://openstackreactions.enovance.com/2013/09/when-my-patch-get-2/ )

We are discussing this spec for couple of releases…
IMO, we should have voting process such as IETF or let PTL make final
decision like TTX saying.

[Topic3] Group based policy specs

I'm not jumped in this one. I have already shared my thought on this.
This is still extension proposal. so we should let user choose use
this or not.
If it is proven for wide use, then we can discuss making it for core API.

I'm also working on another extension spec. It is deferred for next
release, 
(http://openstackreactions.enovance.com/2014/04/getting-told-that-this-feature-will-not-be-accepted-before-next-release/
)

but if you have any interest on this plz ping me

http://docs-draft.openstack.org/12/93112/16/check/gate-neutron-specs-docs/afb346d/doc/build/html/specs/juno/security_group_action.html
http://docs-draft.openstack.org/12/93112/16/check/gate-neutron-specs-docs/afb346d/doc/build/html/specs/juno/security_group_for_network.html

[Topic4] Where we develop group based policy stuffs. And generally,
service related stuffs.

I do remember the LBaaS new project meeting at the Summit. I remember
someone says I don't trust neutron makes this feature make it working
in Juno timeframe..  I guess he was right..

We should have a new project for service related stuffs.
( 
http://openstackreactions.enovance.com/2013/11/when-a-new-openstack-project-announce-itself/
)


Best
Nachi

2014-08-07 19:23 GMT+09:00 Thierry Carrez thie...@openstack.org:
 Armando M. wrote:
 This thread is moving so fast I can't keep up!

 The fact that troubles me is that I am unable to grasp how we move
 forward, which was the point of this thread to start with. It seems we
 have 2 options:

 - We make GBP to merge as is, in the Neutron tree, with some minor
 revision (e.g. naming?);
 - We make GBP a stackforge project, that integrates with Neutron in some
 shape or form;

 Another option, might be something in between, where GBP is in tree, but
 in some sort of experimental staging area (even though I am not sure how
 well baked this idea is).

 Now, as a community we all need make a decision; arguing about the fact
 that the blueprint was approved is pointless.

 I agree with you: it is possible to change your mind on a topic and
 revisit past decisions. In past OpenStack history we did revert merged
 commits and remove existing functionality because we felt it wasn't that
 much of a great idea after all. Here we are talking about making the
 right decision *before* the final merging and shipping into a release,
 which is kind of an improvement. The spec system was supposed to help
 limit such cases, but it's not bullet-proof.

 In the end, if there is no consensus on that question within the Neutron
 project (and I hear both sides have good arguments), our governance
 gives the elected Neutron PTL the power to make the final call. If the
 disagreement is between projects (like if Nova disagreed with the
 Neutron decision), then the issue could be escalated to the TC.

 Regards,

 --
 Thierry Carrez (ttx)

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-07 Thread Aaron Rosen

Hi Kevin,

I feel as your latest response is completely side stepping the points we
have been trying to get to in the last series of emails. At the end of the
day I don't believe we are changing the laws of networking (or perhaps we
are?).  Thus I think it's important to actually get down to the networking
level to actually figure out why optimizations such as this one are enabled
via GBP and not the model we have:

With the latter, a mapping driver could determine that communication
between these two hosts can be prevented by using an ACL on a router or a
switch, which doesn't violate the user's intent and buys a performance
improvement and works with ports that don't support security groups.



On Wed, Aug 6, 2014 at 7:48 PM, Kevin Benton blak...@gmail.com wrote:

 Do you not see a difference between explicitly configuring networks, a
 router and FWaaS rules with logging and just stating that two groups of
 servers can only communicate via one TCP port with all connections logged?
 The first is very prone to errors for someone deploying an application
 without a strong networking background, and the latter is basically just
 stating the requirements and letting Neutron figure out how to implement
 it.

 So are you saying that GBP allows someone to be able to configure an
application that at the end of the day is equivalent  to
networks/router/FWaaS rules without understanding networking concepts? I'm
also curious how this GBP is really less error prone than the model we have
today as it seems the user will basically have to tell neutron the same
information about how he wants his networking to function.



 Just stating requirements becomes even more important when something like
 the logging requirement comes from someone other than the app deployer
 (e.g. a security team). In the above example, someone could set everything
 up using security groups; however, when the logging requirement came in
 from the security team, they would have to undo all of that work and
 replace it with something like FWaaS that can centrally log all of the
 connections.

 It's the difference between using puppet and bash scripts. Sure you can
 write a script that uses awk/sed to ensure that an ini file has a
 particular setting and then restart a service if the setting is changed,
 but it's much easier and less error prone to just write a puppet manifest
 that uses the INI module with a pointer to the file, the section name, the
 key, and the value with a notification to restart the service.



 On Wed, Aug 6, 2014 at 7:40 PM, Aaron Rosen aaronoro...@gmail.com wrote:


 On Wed, Aug 6, 2014 at 5:27 PM, Kevin Benton blak...@gmail.com wrote:

 Web tier can communicate with anything except for the DB.
 App tier can only communicate with Web and DB.
 DB can communicate with App.

 These high-level constraints can then be implemented as security groups
 like you showed, or network hardware ACLs like I had shown.
 But if you start with the security groups API, you are forcing it to be
 implemented there.


 I still have no idea what group based policy is buying us then. It seems
 to me that the key point we've identified going backing and forth here is
 the difference between the current model and the GBP model is that GBP
 constricts topology which allows us to write these types of enforcement
 rules. The reason we want this is because it yields performance
 optimizations (for some reason, which I don't think we've gotten into).
 Would you agree this is accurate?

 Honestly, I know a lot of work has been put into this. I haven't said I'm
 for or against it either. I'm really just trying to understand what is the
 motivation for this and why does it make neutron better.

 Best,

 Aaron




 On Wed, Aug 6, 2014 at 6:06 PM, Aaron Rosen aaronoro...@gmail.com
 wrote:




 On Wed, Aug 6, 2014 at 4:46 PM, Kevin Benton blak...@gmail.com wrote:

 That's the point. By using security groups, you are forcing a certain
 kind of enforcement that must be honored and might not be necessary if the
 original intent was just to isolate between groups. In the example you
 gave, it cannot be implemented on the router without violating the
 constraints of the security group.


 Hi Kevin,

 Mind proposing an alternative example then. The only way I can see this
 claim to be made is because Group Based policy is actually limiting what a
 tenants desired topology can be?

 Thanks,

 Aaron



  On Wed, Aug 6, 2014 at 5:39 PM, Aaron Rosen aaronoro...@gmail.com
 wrote:




 On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton blak...@gmail.com
 wrote:

 Given this information I don't see any reason why the backend
 system couldn't do enforcement at the logical router and if it did so
 neither parties would know.

 With security groups you are specifying that nothing can contact
 these devices on those ports unless they come from the allowed IP
 addresses. If you tried to enforce this at the router you would be
 violating that specification because devices in the

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-07 Thread Kevin Benton

You said you had no idea what group based policy was buying us so I tried
to illustrate what the difference between declarative and imperative
network configuration looks like. That's the major selling point of GBP so
I'm not sure how that's 'side stepping' any points. It removes the need for
the user to pick between implementation details like security
groups/FWaaS/ACLs.

So are you saying that GBP allows someone to be able to configure an
application that at the end of the day is equivalent  to
networks/router/FWaaS rules without understanding networking concepts?

It's one thing to understand the ports an application leverages and another
to understand the differences between configuring VM firewalls, security
groups, FWaaS, and router ACLs.

 I'm also curious how this GBP is really less error prone than the model
we have today as it seems the user will basically have to tell neutron the
same information about how he wants his networking to function.

With GBP, the user just gives the desired end result (e.g. allow
connectivity between endpoint groups via TCP port 22 with all connections
logged). Without it, the user has to do the following:

   1. create a network/subnet for each endpoint group
   2. allow all traffic on the security groups since the logging would need
   to be accomplished with FWaaS
   3. create an FWaaS instance
   4. attach the FWaaS to both networks
   5. add an FWaaS policy and the FWaaS rules to allow the correct traffic

The declarative approach is less error prone because the user can give
neutron the desired state of connectivity rather than a mentally compiled
set of instructions describing how to configure a bunch of individual
network components. How well do you think someone will handle the latter
approach that got all of their networking knowledge from one college course
5 years ago?

IIRC one of the early nova parity requests was an API to automagically
setup a neutron network and router so no extra work would be required to
get instances connected to the Internet. That wasn't requested because
people thought neutron networks were too easy to setup already. :-)


On Thu, Aug 7, 2014 at 9:10 AM, Aaron Rosen aaronoro...@gmail.com wrote:

 Hi Kevin,

 I feel as your latest response is completely side stepping the points we
 have been trying to get to in the last series of emails. At the end of the
 day I don't believe we are changing the laws of networking (or perhaps we
 are?).  Thus I think it's important to actually get down to the networking
 level to actually figure out why optimizations such as this one are enabled
 via GBP and not the model we have:


 With the latter, a mapping driver could determine that communication
 between these two hosts can be prevented by using an ACL on a router or a
 switch, which doesn't violate the user's intent and buys a performance
 improvement and works with ports that don't support security groups.



 On Wed, Aug 6, 2014 at 7:48 PM, Kevin Benton blak...@gmail.com wrote:

 Do you not see a difference between explicitly configuring networks, a
 router and FWaaS rules with logging and just stating that two groups of
 servers can only communicate via one TCP port with all connections logged?
 The first is very prone to errors for someone deploying an application
 without a strong networking background, and the latter is basically just
 stating the requirements and letting Neutron figure out how to implement
 it.

 So are you saying that GBP allows someone to be able to configure an
 application that at the end of the day is equivalent  to
 networks/router/FWaaS rules without understanding networking concepts? I'm
 also curious how this GBP is really less error prone than the model we have
 today as it seems the user will basically have to tell neutron the same
 information about how he wants his networking to function.



 Just stating requirements becomes even more important when something like
 the logging requirement comes from someone other than the app deployer
 (e.g. a security team). In the above example, someone could set everything
 up using security groups; however, when the logging requirement came in
 from the security team, they would have to undo all of that work and
 replace it with something like FWaaS that can centrally log all of the
 connections.

 It's the difference between using puppet and bash scripts. Sure you can
 write a script that uses awk/sed to ensure that an ini file has a
 particular setting and then restart a service if the setting is changed,
 but it's much easier and less error prone to just write a puppet manifest
 that uses the INI module with a pointer to the file, the section name, the
 key, and the value with a notification to restart the service.



 On Wed, Aug 6, 2014 at 7:40 PM, Aaron Rosen aaronoro...@gmail.com
 wrote:


 On Wed, Aug 6, 2014 at 5:27 PM, Kevin Benton blak...@gmail.com wrote:

 Web tier can communicate with anything except for the DB.
 App tier can only communicate with Web

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-07 Thread Aaron Rosen

On Thu, Aug 7, 2014 at 9:54 AM, Kevin Benton blak...@gmail.com wrote:

 You said you had no idea what group based policy was buying us so I tried
 to illustrate what the difference between declarative and imperative
 network configuration looks like. That's the major selling point of GBP so
 I'm not sure how that's 'side stepping' any points. It removes the need for
 the user to pick between implementation details like security
 groups/FWaaS/ACLs.


I mean't 'side stepping' why GBP allows for the comment you made previous,
With the latter, a mapping driver could determine that communication
between these two hosts can be prevented by using an ACL on a router or a
switch, which doesn't violate the user's intent and buys a performance
improvement and works with ports that don't support security groups..

Neutron's current API is a logical abstraction and enforcement can be done
however one chooses to implement it. I'm really trying to understand at the
network level why GBP allows for these optimizations and performance
improvements you talked about.



 So are you saying that GBP allows someone to be able to configure an
 application that at the end of the day is equivalent  to
 networks/router/FWaaS rules without understanding networking concepts?

 It's one thing to understand the ports an application leverages and
 another to understand the differences between configuring VM firewalls,
 security groups, FWaaS, and router ACLs.


Sure, but how does group based policy solve this. Security Groups and FWaaS
are just different places of enforcement. Say I want different security
enforcement on my router than on my instances. One still needs to know
enough to tell group based policy this right?  They need to know enough
that there are different enforcement points? How is doing this with Group
based policy make it easier?



  I'm also curious how this GBP is really less error prone than the model
 we have today as it seems the user will basically have to tell neutron the
 same information about how he wants his networking to function.

 With GBP, the user just gives the desired end result (e.g. allow
 connectivity between endpoint groups via TCP port 22 with all connections
 logged). Without it, the user has to do the following:


Why are you bringing up logging connections? Neutron has no concept of this
at all today in it's code base. Is logging something related to GBP?


1. create a network/subnet for each endpoint group
2. allow all traffic on the security groups since the logging would
need to be accomplished with FWaaS
3. create an FWaaS instance
4. attach the FWaaS to both networks

 Today FWaaS api is still incomplete as there is no real point of
enforcement in it's api (though it really seems that it should just be
router ports) it's just global on the router right  now.



1. add an FWaaS policy and the FWaaS rules to allow the correct traffic

 I'd more or less agree with these steps. Would you mind also giving the
steps involved in group based policy so we can compare?



 The declarative approach is less error prone because the user can give
 neutron the desired state of connectivity rather than a mentally compiled
 set of instructions describing how to configure a bunch of individual
 network components. How well do you think someone will handle the latter
 approach that got all of their networking knowledge from one college course
 5 years ago?


Personally, I think one could easily write up a very short document
probably less than one page with examples showing/exampling how the current
neutron API works even without a much networking background.  The funny
thing is I've been trying to completely understand the proposed group
policy api for a little while now and I'm still having trouble. Seems like
we're taking abstractions that are quite well known/understood and changing
them to a different model that requires one to know about this new
terminology:


Endpoint (EP): An L2/L3 addressable entity.
Endpoint Group (EPG): A collection of endpoints.
Contract: It defines how the application services provided by an EPG can be
accessed. In effect it specifies how an EPG communicates with other EPGs. A
Contract consists of Policy Rules.
Policy Rule: These are individual rules used to define the communication
criteria between EPGs. Each rule contains a Filter, Classifier, and Action.
Classifier: Characterizes the traffic that a particular Policy Rule acts
on. Corresponding action is taken on traffic that satisfies this
classification criteria.
Action: The action that is taken for a matching Policy Rule defined in a
Contract.
Filter: Provides a way to tag a Policy Rule with Capability and Role labels.
Capability: It is a Policy Label that defines what part of a Contract a
particular EPG provides.
Role: It is a Policy Label that defines what part of a Contract an EPG
wants to consume.
Contract Scope: An EPG conveys its intent to provide or consume a Contract
(or its part) by

[openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Stefano Santini

Hi,

In my company (Vodafone), we (DC network architecture) are following very
closely the work happening on Group Based Policy since we see a great value
on the new paradigm to drive network configurations with an advanced logic.

We're working on a new production project for an internal private cloud
deployment targeting Juno release where we plan to introduce the
capabilities based on using Group Policy and we don't want to see it
delayed.
We strongly request/vote to see this complete as proposed without such
changes to allow to move forward with the evolution of the network
capabilities

Thanks and kind regards
Stefano
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward


On 08/06/2014 04:30 AM, Stefano Santini wrote:

Hi,

In my company (Vodafone), we (DC network architecture) are following
very closely the work happening on Group Based Policy since we see a
great value on the new paradigm to drive network configurations with an
advanced logic.

We're working on a new production project for an internal private cloud
deployment targeting Juno release where we plan to introduce the
capabilities based on using Group Policy and we don't want to see it
delayed.
We strongly request/vote to see this complete as proposed without such
changes to allow to move forward with the evolution of the network
capabilities


Hi Stefano,

AFAICT, there is nothing that can be done with the GBP API that cannot 
be done with the low-level regular Neutron API.


Further, if the Nova integration of the GBP API does not occur in the 
Juno timeframe, what benefit will GBP in Neutron give you? Specifics on 
the individual API calls that you would change would be most appreciated.


Thanks in advance for your input!
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

Hi,

I've made my way through the group based policy code and blueprints and I'd
like ask several questions about it.  My first question really is what is
the advantage that the new proposed group based policy model buys us?


Bobs says, The group-based policy BP approved for Juno addresses the
 critical need for a more usable, declarative, intent-based interface for
 cloud application developers and deployers, that can co-exist with
 Neutron's current networking-hardware-oriented API and work nicely with all
 existing core plugins. Additionally, we believe that this declarative
 approach is what is needed to properly integrate advanced services into
 Neutron, and will go a long way towards resolving the difficulties so far
 trying to integrate LBaaS, FWaaS, and VPNaaS APIs into the current Neutron
 model.

My problem with the current blueprint and that comment above is it does not
provide any evidence or data of where the current neutron abstractions
(ports/networks/subnets/routers) provide difficulties and what benefit this
new model will provide.

In the current proposed implementation of group policy, it's implementation
maps onto the existing neutron primitives and the neutron back end(s)
remains unchanged. Because of this one can map the new abstractions onto
the previous ones so I'm curious why we want to move this complexity into
neutron and not have it done externally similarly to how heat works or a
client that abstracts this complexity on it's own end.

From the group-based policy blueprint that was submitted [1]:


The current Neutron model of networks, ports, subnets, routers, and security
 groups provides the necessary building blocks to build a logical network
 topology for connectivity. However, it does not provide the right level
 of abstraction for an application administrator who understands the
 application's details (like application port numbers), but not the
 infrastructure details likes networks and routes.

It looks to me that application administrators still need to understand
network primitives as the concept of networks/ports/routers are still
present though just carrying a different name. For example, in
ENDPOINT_GROUPS there is an attribute l2_policy_id which maps to something
that you use to describe a l2_network and contains an attribute
l3_policy_id which is used to describe an L3 network. This looks similar to
the abstraction we have today where a l2_policy (network) then can have
multiple l3_policies (subnets) mapping to it.  Because of this I'm curious
how the GBP abstraction really provides a different level of abstraction
for application administrators.


 Not only that, the current
 abstraction puts the burden of maintaining the consistency of the network
 topology on the user. The lack of application developer/administrator
 focussed
 abstractions supported by a declarative model make it hard for those users
 to consume Neutron as a connectivity layer.

What is the problem in the current abstraction that puts a burden of
maintaining the consistency of networking topology on users? It seems to me
that the complexity of having to know about topology should be abstracted
at the client layer if desired (and neutron should expose the basic
building blocks for networking). For example, Horizon/Heat or the CLI could
hide the requirement of topology by automatically creating a GROUP  (which
is a network+subnet on a router uplinked to an external network)
simplifying this need for the tenant to understand topology. In addition,
topology still seems to be present in the group policy model proposed just
in a different way as I see it.

From the proposed change section the following is stated:


This proposal suggests a model that allows application administrators to
 express their networking requirements using group and policy abstractions,
 with
 the specifics of policy enforcement and implementation left to the
 underlying
 policy driver. The main advantage of the extensions described in this
 blueprint
 is that they allow for an application-centric interface to Neutron that
 complements the existing network-centric interface.


How is the Application-centric interface complementary to the
network-centric interface?  Is the intention that one would use both
interfaces at one once?

More specifically the new abstractions will achieve the following:
 * Show clear separation of concerns between application and infrastructure
 administrator.


I'm not quite sure I understand this point, how is this different than what
we have today?


 - The application administrator can then deal with a higher level
 abstraction
 that does not concern itself with networking specifics like
 networks/routers/etc.


It seems like the proposed abstraction still requires one to concern
themselves with networking specifics (l2_policies, l3_policies).  I'd
really like to see more evidence backing this. Now they have to deal with
specifies like: Endpoint, Endpoint Group, Contract, Policy Rule,

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Ryan Moats




Jay Pipes jaypi...@gmail.com wrote on 08/06/2014 01:04:41 PM:

[snip]

 AFAICT, there is nothing that can be done with the GBP API that cannot
 be done with the low-level regular Neutron API.

I'll take you up on that, Jay :)

How exactly do I specify behavior between two collections of ports residing
in the same IP subnet (an example of this is a bump-in-the-wire network
appliance).

I've looked around regular Neutron and all I've come up with so far is:
(1) use security groups on the ports
(2) set allow_overlapping_ips to true, set up two networks with
identical CIDR block subnets and disjoint allocation pools and put a
vRouter between them.

Now #1 only works for basic allow/deny access and adds the complexity of
needing to specify per-IP address security rules, which means you need the
ports to have IP addresses already and then manually add them into the
security groups, which doesn't seem particularly very orchestration
friendly.

Now #2 handles both allow/deny access as well as provides a potential
attachment point for other behaviors, *but* you have to know to set up the
disjoint allocation pools, and your depending on your drivers to handle the
case of a router that isn't really a router (i.e. it's got two interfaces
in the same subnet, possibly with the same address (unless you thought of
that when you set things up)).

You can say that both of these are *possible*, but they both look more
complex to me than just having two groups of ports and specifying a policy
between them.

Ryan Moats___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

Hi Aaron,

These are good questions, but can we move this to a different thread
labeled what is the point of group policy?

I don't want to derail this one again and we should stick to Salvatore's
options about the way to move forward with these code changes.
On Aug 6, 2014 12:42 PM, Aaron Rosen aaronoro...@gmail.com wrote:

 Hi,

 I've made my way through the group based policy code and blueprints and
 I'd like ask several questions about it.  My first question really is what
 is the advantage that the new proposed group based policy model buys us?


 Bobs says, The group-based policy BP approved for Juno addresses the
 critical need for a more usable, declarative, intent-based interface for
 cloud application developers and deployers, that can co-exist with
 Neutron's current networking-hardware-oriented API and work nicely with all
 existing core plugins. Additionally, we believe that this declarative
 approach is what is needed to properly integrate advanced services into
 Neutron, and will go a long way towards resolving the difficulties so far
 trying to integrate LBaaS, FWaaS, and VPNaaS APIs into the current Neutron
 model.

 My problem with the current blueprint and that comment above is it does
 not provide any evidence or data of where the current neutron abstractions
 (ports/networks/subnets/routers) provide difficulties and what benefit this
 new model will provide.

 In the current proposed implementation of group policy, it's
 implementation maps onto the existing neutron primitives and the neutron
 back end(s) remains unchanged. Because of this one can map the new
 abstractions onto the previous ones so I'm curious why we want to move this
 complexity into neutron and not have it done externally similarly to how
 heat works or a client that abstracts this complexity on it's own end.

 From the group-based policy blueprint that was submitted [1]:


 The current Neutron model of networks, ports, subnets, routers, and
 security
 groups provides the necessary building blocks to build a logical network
 topology for connectivity. However, it does not provide the right level
 of abstraction for an application administrator who understands the
 application's details (like application port numbers), but not the
 infrastructure details likes networks and routes.

 It looks to me that application administrators still need to understand
 network primitives as the concept of networks/ports/routers are still
 present though just carrying a different name. For example, in
 ENDPOINT_GROUPS there is an attribute l2_policy_id which maps to something
 that you use to describe a l2_network and contains an attribute
 l3_policy_id which is used to describe an L3 network. This looks similar to
 the abstraction we have today where a l2_policy (network) then can have
 multiple l3_policies (subnets) mapping to it.  Because of this I'm curious
 how the GBP abstraction really provides a different level of abstraction
 for application administrators.


  Not only that, the current
 abstraction puts the burden of maintaining the consistency of the network
 topology on the user. The lack of application developer/administrator
 focussed
 abstractions supported by a declarative model make it hard for those users
 to consume Neutron as a connectivity layer.

 What is the problem in the current abstraction that puts a burden of
 maintaining the consistency of networking topology on users? It seems to me
 that the complexity of having to know about topology should be abstracted
 at the client layer if desired (and neutron should expose the basic
 building blocks for networking). For example, Horizon/Heat or the CLI could
 hide the requirement of topology by automatically creating a GROUP  (which
 is a network+subnet on a router uplinked to an external network)
 simplifying this need for the tenant to understand topology. In addition,
 topology still seems to be present in the group policy model proposed just
 in a different way as I see it.

 From the proposed change section the following is stated:


 This proposal suggests a model that allows application administrators to
 express their networking requirements using group and policy
 abstractions, with
 the specifics of policy enforcement and implementation left to the
 underlying
 policy driver. The main advantage of the extensions described in this
 blueprint
 is that they allow for an application-centric interface to Neutron that
 complements the existing network-centric interface.


 How is the Application-centric interface complementary to the
 network-centric interface?  Is the intention that one would use both
 interfaces at one once?

  More specifically the new abstractions will achieve the following:
 * Show clear separation of concerns between application and infrastructure
 administrator.


 I'm not quite sure I understand this point, how is this different than
 what we have today?


 - The application administrator can then deal with a higher level

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

Hi Ryan,


On Wed, Aug 6, 2014 at 11:55 AM, Ryan Moats rmo...@us.ibm.com wrote:

 Jay Pipes jaypi...@gmail.com wrote on 08/06/2014 01:04:41 PM:

 [snip]


  AFAICT, there is nothing that can be done with the GBP API that cannot
  be done with the low-level regular Neutron API.

 I'll take you up on that, Jay :)

 How exactly do I specify behavior between two collections of ports
 residing in the same IP subnet (an example of this is a bump-in-the-wire
 network appliance).

 Would you mind explaining what behavior you want between the two
collection of ports?


 I've looked around regular Neutron and all I've come up with so far is:
  (1) use security groups on the ports
  (2) set allow_overlapping_ips to true, set up two networks with identical
 CIDR block subnets and disjoint allocation pools and put a vRouter between
 them.

 Now #1 only works for basic allow/deny access and adds the complexity of
 needing to specify per-IP address security rules, which means you need the
 ports to have IP addresses already and then manually add them into the
 security groups, which doesn't seem particularly very orchestration
 friendly.


I believe the referential security group rules solve this problem (unless
I'm not understanding):

neutron security-group-create group1
neutron security-group-create group2

# allow members of group1 to ssh into group2 (but not the other way around):
neutron security-group-rule-create --direction ingress --port-range-min 22
--port-range-max 22 --protocol TCP --remote-group-id group1 group2

# allow members of group2 to be able to access TCP 80 from members of
group1 (but not the other way around):
neutron security-group-rule-create --direction ingress --port-range-min 80
--port-range-max 80 --protocol TCP --remote-group-id group2 group1

# Now when you create ports just place these in the desired security groups
and neutron will automatically handle this orchestration for you (and you
don't have to deal with ip_addresses and updates).

neutron port-create --security-groups group1 network1
neutron port-create --security-groups group2 network1



 Now #2 handles both allow/deny access as well as provides a potential
 attachment point for other behaviors, *but* you have to know to set up the
 disjoint allocation pools, and your depending on your drivers to handle the
 case of a router that isn't really a router (i.e. it's got two interfaces
 in the same subnet, possibly with the same address (unless you thought of
 that when you set things up)).


Are you talking about the firewall as a service stuff here?


 You can say that both of these are *possible*, but they both look more
 complex to me than just having two groups of ports and specifying a policy
 between them.


Would you mind proposing how this is done in the Group policy api? From
what I can tell in the new proposed api you'd need to map both of these
groups to different endpoints i.e networks.



 Ryan Moats


 Best,

Aaron

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

Hi Kevin,

I think we should keep these threads together as we need to understand the
benefit collectively before we move forward with what to do.

Aaron


On Wed, Aug 6, 2014 at 12:03 PM, Kevin Benton blak...@gmail.com wrote:

 Hi Aaron,

 These are good questions, but can we move this to a different thread
 labeled what is the point of group policy?

 I don't want to derail this one again and we should stick to Salvatore's
 options about the way to move forward with these code changes.
 On Aug 6, 2014 12:42 PM, Aaron Rosen aaronoro...@gmail.com wrote:

 Hi,

 I've made my way through the group based policy code and blueprints and
 I'd like ask several questions about it.  My first question really is what
 is the advantage that the new proposed group based policy model buys us?


 Bobs says, The group-based policy BP approved for Juno addresses the
 critical need for a more usable, declarative, intent-based interface for
 cloud application developers and deployers, that can co-exist with
 Neutron's current networking-hardware-oriented API and work nicely with all
 existing core plugins. Additionally, we believe that this declarative
 approach is what is needed to properly integrate advanced services into
 Neutron, and will go a long way towards resolving the difficulties so far
 trying to integrate LBaaS, FWaaS, and VPNaaS APIs into the current Neutron
 model.

 My problem with the current blueprint and that comment above is it does
 not provide any evidence or data of where the current neutron abstractions
 (ports/networks/subnets/routers) provide difficulties and what benefit this
 new model will provide.

 In the current proposed implementation of group policy, it's
 implementation maps onto the existing neutron primitives and the neutron
 back end(s) remains unchanged. Because of this one can map the new
 abstractions onto the previous ones so I'm curious why we want to move this
 complexity into neutron and not have it done externally similarly to how
 heat works or a client that abstracts this complexity on it's own end.

 From the group-based policy blueprint that was submitted [1]:


 The current Neutron model of networks, ports, subnets, routers, and
 security
 groups provides the necessary building blocks to build a logical network
 topology for connectivity. However, it does not provide the right level
 of abstraction for an application administrator who understands the
 application's details (like application port numbers), but not the
 infrastructure details likes networks and routes.

 It looks to me that application administrators still need to understand
 network primitives as the concept of networks/ports/routers are still
 present though just carrying a different name. For example, in
 ENDPOINT_GROUPS there is an attribute l2_policy_id which maps to something
 that you use to describe a l2_network and contains an attribute
 l3_policy_id which is used to describe an L3 network. This looks similar to
 the abstraction we have today where a l2_policy (network) then can have
 multiple l3_policies (subnets) mapping to it.  Because of this I'm curious
 how the GBP abstraction really provides a different level of abstraction
 for application administrators.


  Not only that, the current
 abstraction puts the burden of maintaining the consistency of the network
 topology on the user. The lack of application developer/administrator
 focussed
 abstractions supported by a declarative model make it hard for those
 users
 to consume Neutron as a connectivity layer.

 What is the problem in the current abstraction that puts a burden of
 maintaining the consistency of networking topology on users? It seems to me
 that the complexity of having to know about topology should be abstracted
 at the client layer if desired (and neutron should expose the basic
 building blocks for networking). For example, Horizon/Heat or the CLI could
 hide the requirement of topology by automatically creating a GROUP  (which
 is a network+subnet on a router uplinked to an external network)
 simplifying this need for the tenant to understand topology. In addition,
 topology still seems to be present in the group policy model proposed just
 in a different way as I see it.

 From the proposed change section the following is stated:


 This proposal suggests a model that allows application administrators to
 express their networking requirements using group and policy
 abstractions, with
 the specifics of policy enforcement and implementation left to the
 underlying
 policy driver. The main advantage of the extensions described in this
 blueprint
 is that they allow for an application-centric interface to Neutron that
 complements the existing network-centric interface.


 How is the Application-centric interface complementary to the
 network-centric interface?  Is the intention that one would use both
 interfaces at one once?

  More specifically the new abstractions will achieve the following:
 * Show clear separation of

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Salvatore Orlando

As long as the discussion stays focused on how group policies are
beneficial for the user community and how the Neutron developer community
should move forward, I reckon it's fine to keep the discussion in this
thread.

Salvatore
Il 06/ago/2014 21:18 Aaron Rosen aaronoro...@gmail.com ha scritto:

 Hi Kevin,

 I think we should keep these threads together as we need to understand the
 benefit collectively before we move forward with what to do.

 Aaron


 On Wed, Aug 6, 2014 at 12:03 PM, Kevin Benton blak...@gmail.com wrote:

 Hi Aaron,

 These are good questions, but can we move this to a different thread
 labeled what is the point of group policy?

 I don't want to derail this one again and we should stick to Salvatore's
 options about the way to move forward with these code changes.
 On Aug 6, 2014 12:42 PM, Aaron Rosen aaronoro...@gmail.com wrote:

 Hi,

 I've made my way through the group based policy code and blueprints and
 I'd like ask several questions about it.  My first question really is what
 is the advantage that the new proposed group based policy model buys us?


 Bobs says, The group-based policy BP approved for Juno addresses the
 critical need for a more usable, declarative, intent-based interface for
 cloud application developers and deployers, that can co-exist with
 Neutron's current networking-hardware-oriented API and work nicely with all
 existing core plugins. Additionally, we believe that this declarative
 approach is what is needed to properly integrate advanced services into
 Neutron, and will go a long way towards resolving the difficulties so far
 trying to integrate LBaaS, FWaaS, and VPNaaS APIs into the current Neutron
 model.

 My problem with the current blueprint and that comment above is it does
 not provide any evidence or data of where the current neutron abstractions
 (ports/networks/subnets/routers) provide difficulties and what benefit this
 new model will provide.

 In the current proposed implementation of group policy, it's
 implementation maps onto the existing neutron primitives and the neutron
 back end(s) remains unchanged. Because of this one can map the new
 abstractions onto the previous ones so I'm curious why we want to move this
 complexity into neutron and not have it done externally similarly to how
 heat works or a client that abstracts this complexity on it's own end.

 From the group-based policy blueprint that was submitted [1]:


 The current Neutron model of networks, ports, subnets, routers, and
 security
 groups provides the necessary building blocks to build a logical network
 topology for connectivity. However, it does not provide the right level
 of abstraction for an application administrator who understands the
 application's details (like application port numbers), but not the
 infrastructure details likes networks and routes.

 It looks to me that application administrators still need to understand
 network primitives as the concept of networks/ports/routers are still
 present though just carrying a different name. For example, in
 ENDPOINT_GROUPS there is an attribute l2_policy_id which maps to something
 that you use to describe a l2_network and contains an attribute
 l3_policy_id which is used to describe an L3 network. This looks similar to
 the abstraction we have today where a l2_policy (network) then can have
 multiple l3_policies (subnets) mapping to it.  Because of this I'm curious
 how the GBP abstraction really provides a different level of abstraction
 for application administrators.


  Not only that, the current
 abstraction puts the burden of maintaining the consistency of the
 network
 topology on the user. The lack of application developer/administrator
 focussed
 abstractions supported by a declarative model make it hard for those
 users
 to consume Neutron as a connectivity layer.

 What is the problem in the current abstraction that puts a burden of
 maintaining the consistency of networking topology on users? It seems to me
 that the complexity of having to know about topology should be abstracted
 at the client layer if desired (and neutron should expose the basic
 building blocks for networking). For example, Horizon/Heat or the CLI could
 hide the requirement of topology by automatically creating a GROUP  (which
 is a network+subnet on a router uplinked to an external network)
 simplifying this need for the tenant to understand topology. In addition,
 topology still seems to be present in the group policy model proposed just
 in a different way as I see it.

 From the proposed change section the following is stated:


 This proposal suggests a model that allows application administrators to
 express their networking requirements using group and policy
 abstractions, with
 the specifics of policy enforcement and implementation left to the
 underlying
 policy driver. The main advantage of the extensions described in this
 blueprint
 is that they allow for an application-centric interface to Neutron that

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Ivar Lazzaro

Hi Aaron,

Please note that the user using the current reference implementation
doesn't need to create Networks, Ports, or anything else. As a matter of
fact, the mapping is done implicitly.

Also, I agree with Kevin when he says that this is a whole different
discussion.

Thanks,
Ivar.


On Wed, Aug 6, 2014 at 9:12 PM, Aaron Rosen aaronoro...@gmail.com wrote:

 Hi Ryan,


 On Wed, Aug 6, 2014 at 11:55 AM, Ryan Moats rmo...@us.ibm.com wrote:

 Jay Pipes jaypi...@gmail.com wrote on 08/06/2014 01:04:41 PM:

 [snip]


  AFAICT, there is nothing that can be done with the GBP API that cannot
  be done with the low-level regular Neutron API.

 I'll take you up on that, Jay :)

 How exactly do I specify behavior between two collections of ports
 residing in the same IP subnet (an example of this is a bump-in-the-wire
 network appliance).

 Would you mind explaining what behavior you want between the two
 collection of ports?


  I've looked around regular Neutron and all I've come up with so far is:
  (1) use security groups on the ports
  (2) set allow_overlapping_ips to true, set up two networks with
 identical CIDR block subnets and disjoint allocation pools and put a
 vRouter between them.

 Now #1 only works for basic allow/deny access and adds the complexity of
 needing to specify per-IP address security rules, which means you need the
 ports to have IP addresses already and then manually add them into the
 security groups, which doesn't seem particularly very orchestration
 friendly.


 I believe the referential security group rules solve this problem (unless
 I'm not understanding):

 neutron security-group-create group1
 neutron security-group-create group2

 # allow members of group1 to ssh into group2 (but not the other way
 around):
 neutron security-group-rule-create --direction ingress --port-range-min 22
 --port-range-max 22 --protocol TCP --remote-group-id group1 group2

 # allow members of group2 to be able to access TCP 80 from members of
 group1 (but not the other way around):
 neutron security-group-rule-create --direction ingress --port-range-min 80
 --port-range-max 80 --protocol TCP --remote-group-id group2 group1

 # Now when you create ports just place these in the desired security
 groups and neutron will automatically handle this orchestration for you
 (and you don't have to deal with ip_addresses and updates).

 neutron port-create --security-groups group1 network1
 neutron port-create --security-groups group2 network1



 Now #2 handles both allow/deny access as well as provides a potential
 attachment point for other behaviors, *but* you have to know to set up the
 disjoint allocation pools, and your depending on your drivers to handle the
 case of a router that isn't really a router (i.e. it's got two interfaces
 in the same subnet, possibly with the same address (unless you thought of
 that when you set things up)).


 Are you talking about the firewall as a service stuff here?


  You can say that both of these are *possible*, but they both look more
 complex to me than just having two groups of ports and specifying a policy
 between them.


 Would you mind proposing how this is done in the Group policy api? From
 what I can tell in the new proposed api you'd need to map both of these
 groups to different endpoints i.e networks.



 Ryan Moats


 Best,

 Aaron

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On Wed, Aug 6, 2014 at 12:25 PM, Ivar Lazzaro ivarlazz...@gmail.com wrote:

 Hi Aaron,

 Please note that the user using the current reference implementation
 doesn't need to create Networks, Ports, or anything else. As a matter of
 fact, the mapping is done implicitly.



The user still needs to create an endpointgroup. What is being done
implicitly here? I fail to see the difference.



 Also, I agree with Kevin when he says that this is a whole different
 discussion.

 Thanks,
 Ivar.


 On Wed, Aug 6, 2014 at 9:12 PM, Aaron Rosen aaronoro...@gmail.com wrote:

 Hi Ryan,


 On Wed, Aug 6, 2014 at 11:55 AM, Ryan Moats rmo...@us.ibm.com wrote:

 Jay Pipes jaypi...@gmail.com wrote on 08/06/2014 01:04:41 PM:

 [snip]


  AFAICT, there is nothing that can be done with the GBP API that cannot
  be done with the low-level regular Neutron API.

 I'll take you up on that, Jay :)

 How exactly do I specify behavior between two collections of ports
 residing in the same IP subnet (an example of this is a bump-in-the-wire
 network appliance).

 Would you mind explaining what behavior you want between the two
 collection of ports?


  I've looked around regular Neutron and all I've come up with so far is:
  (1) use security groups on the ports
  (2) set allow_overlapping_ips to true, set up two networks with
 identical CIDR block subnets and disjoint allocation pools and put a
 vRouter between them.

 Now #1 only works for basic allow/deny access and adds the complexity of
 needing to specify per-IP address security rules, which means you need the
 ports to have IP addresses already and then manually add them into the
 security groups, which doesn't seem particularly very orchestration
 friendly.


 I believe the referential security group rules solve this problem (unless
 I'm not understanding):

 neutron security-group-create group1
 neutron security-group-create group2

 # allow members of group1 to ssh into group2 (but not the other way
 around):
 neutron security-group-rule-create --direction ingress --port-range-min
 22 --port-range-max 22 --protocol TCP --remote-group-id group1 group2

 # allow members of group2 to be able to access TCP 80 from members of
 group1 (but not the other way around):
 neutron security-group-rule-create --direction ingress --port-range-min
 80 --port-range-max 80 --protocol TCP --remote-group-id group2 group1

 # Now when you create ports just place these in the desired security
 groups and neutron will automatically handle this orchestration for you
 (and you don't have to deal with ip_addresses and updates).

 neutron port-create --security-groups group1 network1
 neutron port-create --security-groups group2 network1



 Now #2 handles both allow/deny access as well as provides a potential
 attachment point for other behaviors, *but* you have to know to set up the
 disjoint allocation pools, and your depending on your drivers to handle the
 case of a router that isn't really a router (i.e. it's got two interfaces
 in the same subnet, possibly with the same address (unless you thought of
 that when you set things up)).


 Are you talking about the firewall as a service stuff here?


  You can say that both of these are *possible*, but they both look more
 complex to me than just having two groups of ports and specifying a policy
 between them.


  Would you mind proposing how this is done in the Group policy api? From
 what I can tell in the new proposed api you'd need to map both of these
 groups to different endpoints i.e networks.



 Ryan Moats


 Best,

 Aaron

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

I believe the referential security group rules solve this problem (unless
I'm not understanding):

I think the disconnect is that you are comparing the way to current mapping
driver implements things for the reference implementation with the existing
APIs. Under this light, it's not going to look like there is a point to
this code being in Neutron since, as you said, the abstraction could happen
at a client. However, this changes once new mapping drivers can be added
that implement things differently.

Let's take the security groups example. Using the security groups API
directly is imperative (put a firewall rule on this port that blocks this
IP) compared to a higher level declarative abstraction (make sure these
two endpoints cannot communicate). With the former, the ports must support
security groups and there is nowhere except for the firewall rules on that
port to implement it without violating the user's expectation. With the
latter, a mapping driver could determine that communication between these
two hosts can be prevented by using an ACL on a router or a switch, which
doesn't violate the user's intent and buys a performance improvement and
works with ports that don't support security groups.

Group based policy is trying to move the requests into the declarative
abstraction so optimizations like the one above can be made.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Ryan Moats

Aaron Rosen aaronoro...@gmail.com wrote on 08/06/2014 02:12:05 PM:

 From: Aaron Rosen aaronoro...@gmail.com
 To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Date: 08/06/2014 02:12 PM
 Subject: Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy
 and the way forward

 Hi Ryan,

 On Wed, Aug 6, 2014 at 11:55 AM, Ryan Moats rmo...@us.ibm.com wrote:
 Jay Pipes jaypi...@gmail.com wrote on 08/06/2014 01:04:41 PM:

 [snip]

  AFAICT, there is nothing that can be done with the GBP API that cannot
  be done with the low-level regular Neutron API.

 I'll take you up on that, Jay :)

 How exactly do I specify behavior between two collections of ports
 residing in the same IP subnet (an example of this is a bump-in-the-
 wire network appliance).

 Would you mind explaining what behavior you want between the two
 collection of ports?

That's the point - I want a framework that will work regardless of the
specifics of the behavior later on.

 I've looked around regular Neutron and all I've come up with so far is:
 (1) use security groups on the ports
 (2) set allow_overlapping_ips to true, set up two networks with
 identical CIDR block subnets and disjoint allocation pools and put a
 vRouter between them.

 Now #1 only works for basic allow/deny access and adds the
 complexity of needing to specify per-IP address security rules,
 which means you need the ports to have IP addresses already and then
 manually add them into the security groups, which doesn't seem
 particularly very orchestration friendly.

 I believe the referential security group rules solve this problem
 (unless I'm not understanding):

 neutron security-group-create group1
 neutron security-group-create group2

 # allow members of group1 to ssh into group2 (but not the other way
around):
 neutron security-group-rule-create --direction ingress --port-range-
 min 22 --port-range-max 22 --protocol TCP --remote-group-id group1 group2

 # allow members of group2 to be able to access TCP 80 from members
 of group1 (but not the other way around):
 neutron security-group-rule-create --direction ingress --port-range-
 min 80 --port-range-max 80 --protocol TCP --remote-group-id group2 group1

 # Now when you create ports just place these in the desired security
 groups and neutron will automatically handle this orchestration for
 you (and you don't have to deal with ip_addresses and updates).

 neutron port-create --security-groups group1 network1
 neutron port-create --security-groups group2 network1

While those rule instances aren't particularly interesting, I concede that
security groups can handle allow/deny. However, allow/deny is the least
interesting part of the problem :)

 Now #2 handles both allow/deny access as well as provides a
 potential attachment point for other behaviors, *but* you have to
 know to set up the disjoint allocation pools, and your depending on
 your drivers to handle the case of a router that isn't really a
 router (i.e. it's got two interfaces in the same subnet, possibly
 with the same address (unless you thought of that when you set things
up)).

 Are you talking about the firewall as a service stuff here?

No, I'm being more general than any of the curret *aaS items, I'm thinking
of the general service insertion case.

 You can say that both of these are *possible*, but they both look
 more complex to me than just having two groups of ports and
 specifying a policy between them.

 Would you mind proposing how this is done in the Group policy api?
 From what I can tell in the new proposed api you'd need to map both
 of these groups to different endpoints i.e networks.

Note: I'm not going to use the pure group policy API as I have been a fan
of the
2-group approach from day 1, I'm going to use the commands as stated in the
patch set, and I'm going to assume (dangerous I know) that I can specify
endpoint-group on port create

neutron grouppolicy-endpoint-group-create group1
neutron grouppolicy-endpoint-group-create group2
neutron port-create --endpoint-group group1 network1
neutron port-create --endpoint-group group2 network1
neutron grouppolicy-policy-action-create action
neutron grouppolicy-policy-classifier-create classifier
neutron grouppolicy-policy-rule-create --action action --classifier
classifer rule
neutron policy-apply rule group1 group2

One major difference between GP and current neutron (other than security
groups) is in your last statement about tying things to networks.  Like
security groups, groups in GP can have ports that span networks.  Unlike
security groups, GP allows more rich policy than just allow.

That of course begs the question of extending security groups (think this
blueprint https://review.openstack.org/#/c/93112/ ) but that approach may
have its own issues.

Ryan

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

The translation to creating a network is what is being done implicitly.
Another plugin could choose to implement this entirely using something like
security groups on a single giant network.


On Wed, Aug 6, 2014 at 1:38 PM, Aaron Rosen aaronoro...@gmail.com wrote:




 On Wed, Aug 6, 2014 at 12:25 PM, Ivar Lazzaro ivarlazz...@gmail.com
 wrote:

 Hi Aaron,

 Please note that the user using the current reference implementation
 doesn't need to create Networks, Ports, or anything else. As a matter of
 fact, the mapping is done implicitly.



 The user still needs to create an endpointgroup. What is being done
 implicitly here? I fail to see the difference.



 Also, I agree with Kevin when he says that this is a whole different
 discussion.

 Thanks,
 Ivar.


 On Wed, Aug 6, 2014 at 9:12 PM, Aaron Rosen aaronoro...@gmail.com
 wrote:

 Hi Ryan,


 On Wed, Aug 6, 2014 at 11:55 AM, Ryan Moats rmo...@us.ibm.com wrote:

 Jay Pipes jaypi...@gmail.com wrote on 08/06/2014 01:04:41 PM:

 [snip]


  AFAICT, there is nothing that can be done with the GBP API that
 cannot
  be done with the low-level regular Neutron API.

 I'll take you up on that, Jay :)

 How exactly do I specify behavior between two collections of ports
 residing in the same IP subnet (an example of this is a bump-in-the-wire
 network appliance).

 Would you mind explaining what behavior you want between the two
 collection of ports?


  I've looked around regular Neutron and all I've come up with so far
 is:
  (1) use security groups on the ports
  (2) set allow_overlapping_ips to true, set up two networks with
 identical CIDR block subnets and disjoint allocation pools and put a
 vRouter between them.

 Now #1 only works for basic allow/deny access and adds the complexity
 of needing to specify per-IP address security rules, which means you need
 the ports to have IP addresses already and then manually add them into the
 security groups, which doesn't seem particularly very orchestration
 friendly.


 I believe the referential security group rules solve this problem
 (unless I'm not understanding):

 neutron security-group-create group1
 neutron security-group-create group2

 # allow members of group1 to ssh into group2 (but not the other way
 around):
 neutron security-group-rule-create --direction ingress --port-range-min
 22 --port-range-max 22 --protocol TCP --remote-group-id group1 group2

 # allow members of group2 to be able to access TCP 80 from members of
 group1 (but not the other way around):
 neutron security-group-rule-create --direction ingress --port-range-min
 80 --port-range-max 80 --protocol TCP --remote-group-id group2 group1

 # Now when you create ports just place these in the desired security
 groups and neutron will automatically handle this orchestration for you
 (and you don't have to deal with ip_addresses and updates).

 neutron port-create --security-groups group1 network1
 neutron port-create --security-groups group2 network1



 Now #2 handles both allow/deny access as well as provides a potential
 attachment point for other behaviors, *but* you have to know to set up the
 disjoint allocation pools, and your depending on your drivers to handle the
 case of a router that isn't really a router (i.e. it's got two interfaces
 in the same subnet, possibly with the same address (unless you thought of
 that when you set things up)).


 Are you talking about the firewall as a service stuff here?


  You can say that both of these are *possible*, but they both look
 more complex to me than just having two groups of ports and specifying a
 policy between them.


  Would you mind proposing how this is done in the Group policy api? From
 what I can tell in the new proposed api you'd need to map both of these
 groups to different endpoints i.e networks.



 Ryan Moats


 Best,

 Aaron

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward


On 08/06/2014 03:46 PM, Kevin Benton wrote:

 I believe the referential security group rules solve this problem
(unless I'm not understanding):

I think the disconnect is that you are comparing the way to current
mapping driver implements things for the reference implementation with
the existing APIs. Under this light, it's not going to look like there
is a point to this code being in Neutron since, as you said, the
abstraction could happen at a client. However, this changes once new
mapping drivers can be added that implement things differently.

Let's take the security groups example. Using the security groups API
directly is imperative (put a firewall rule on this port that blocks
this IP) compared to a higher level declarative abstraction (make sure
these two endpoints cannot communicate). With the former, the ports
must support security groups and there is nowhere except for the
firewall rules on that port to implement it without violating the user's
expectation. With the latter, a mapping driver could determine that
communication between these two hosts can be prevented by using an ACL
on a router or a switch, which doesn't violate the user's intent and
buys a performance improvement and works with ports that don't support
security groups.

Group based policy is trying to move the requests into the declarative
abstraction so optimizations like the one above can be made.


So, in short, GBP is an effort to provide an API that substitutes 
generic names for specific names in order to allow custom proprietary 
hardware vendors to wire in their hardware using an API that better 
matches their own internal modelling.


Would that be a good statement of the end-goal of GBP?

-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Sumit Naiksatam

On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton blak...@gmail.com wrote:
I believe the referential security group rules solve this problem (unless
 I'm not understanding):

 I think the disconnect is that you are comparing the way to current mapping
 driver implements things for the reference implementation with the existing
 APIs. Under this light, it's not going to look like there is a point to this
 code being in Neutron since, as you said, the abstraction could happen at a
 client. However, this changes once new mapping drivers can be added that
 implement things differently.

 Let's take the security groups example. Using the security groups API
 directly is imperative (put a firewall rule on this port that blocks this
 IP) compared to a higher level declarative abstraction (make sure these
 two endpoints cannot communicate). With the former, the ports must support
 security groups and there is nowhere except for the firewall rules on that
 port to implement it without violating the user's expectation. With the
 latter, a mapping driver could determine that communication between these
 two hosts can be prevented by using an ACL on a router or a switch, which
 doesn't violate the user's intent and buys a performance improvement and
 works with ports that don't support security groups.

 Group based policy is trying to move the requests into the declarative
 abstraction so optimizations like the one above can be made.


Kevin, you have captured the GBP value prop and difference very
succinctly. The major difference is in the declarative (GBP) versus
imperative (current) style of programming.

This has been stated very clearly and explicitly in the blueprint
spec. If one does not appreciate the difference or advantage of one
over the other, then this discussion is pointless.

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On Wed, Aug 6, 2014 at 12:45 PM, Ryan Moats rmo...@us.ibm.com wrote:

 Aaron Rosen aaronoro...@gmail.com wrote on 08/06/2014 02:12:05 PM:

  From: Aaron Rosen aaronoro...@gmail.com

  To: OpenStack Development Mailing List (not for usage questions)
  openstack-dev@lists.openstack.org
  Date: 08/06/2014 02:12 PM
  Subject: Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy
  and the way forward

  Hi Ryan,

  On Wed, Aug 6, 2014 at 11:55 AM, Ryan Moats rmo...@us.ibm.com wrote:
  Jay Pipes jaypi...@gmail.com wrote on 08/06/2014 01:04:41 PM:

  [snip]

   AFAICT, there is nothing that can be done with the GBP API that cannot
   be done with the low-level regular Neutron API.

  I'll take you up on that, Jay :)

  How exactly do I specify behavior between two collections of ports
  residing in the same IP subnet (an example of this is a bump-in-the-
  wire network appliance).

  Would you mind explaining what behavior you want between the two
  collection of ports?

 That's the point - I want a framework that will work regardless of the
 specifics of the behavior later on.

  I've looked around regular Neutron and all I've come up with so far is:
  (1) use security groups on the ports
  (2) set allow_overlapping_ips to true, set up two networks with
  identical CIDR block subnets and disjoint allocation pools and put a
  vRouter between them.

  Now #1 only works for basic allow/deny access and adds the
  complexity of needing to specify per-IP address security rules,
  which means you need the ports to have IP addresses already and then
  manually add them into the security groups, which doesn't seem
  particularly very orchestration friendly.

  I believe the referential security group rules solve this problem
  (unless I'm not understanding):

  neutron security-group-create group1
  neutron security-group-create group2

  # allow members of group1 to ssh into group2 (but not the other way
 around):
  neutron security-group-rule-create --direction ingress --port-range-
  min 22 --port-range-max 22 --protocol TCP --remote-group-id group1 group2

  # allow members of group2 to be able to access TCP 80 from members
  of group1 (but not the other way around):
  neutron security-group-rule-create --direction ingress --port-range-
  min 80 --port-range-max 80 --protocol TCP --remote-group-id group2 group1

  # Now when you create ports just place these in the desired security
  groups and neutron will automatically handle this orchestration for
  you (and you don't have to deal with ip_addresses and updates).

  neutron port-create --security-groups group1 network1
  neutron port-create --security-groups group2 network1

 While those rule instances aren't particularly interesting, I concede that
 security groups can handle allow/deny. However, allow/deny is the least
 interesting part of the problem :)

  Now #2 handles both allow/deny access as well as provides a
  potential attachment point for other behaviors, *but* you have to
  know to set up the disjoint allocation pools, and your depending on
  your drivers to handle the case of a router that isn't really a
  router (i.e. it's got two interfaces in the same subnet, possibly
  with the same address (unless you thought of that when you set things
 up)).

  Are you talking about the firewall as a service stuff here?

 No, I'm being more general than any of the curret *aaS items, I'm thinking
 of the general service insertion case.

  You can say that both of these are *possible*, but they both look
  more complex to me than just having two groups of ports and
  specifying a policy between them.

  Would you mind proposing how this is done in the Group policy api?
  From what I can tell in the new proposed api you'd need to map both
  of these groups to different endpoints i.e networks.

 Note: I'm not going to use the pure group policy API as I have been a fan
 of the
 2-group approach from day 1, I'm going to use the commands as stated in
 the patch set, and I'm going to assume (dangerous I know) that I can
 specify endpoint-group on port create

 neutron grouppolicy-endpoint-group-create group1
 neutron grouppolicy-endpoint-group-create group2

 neutron port-create --endpoint-group group1 network1
 neutron port-create --endpoint-group group2 network1

Sorry, I don't follow what this port-create command does? Here ---
https://docs.google.com/presentation/d/1Nn1HjghAvk2RTPwvltSrnCUJkidWKWY2ckU7OYAVNpo/edit#slide=id.g12c5a79d7_4078
 --- shows that endpoint-groups map to networks so i'm confused why
network1 is coming into play here?

 neutron grouppolicy-policy-action-create action
 neutron grouppolicy-policy-classifier-create classifier
 neutron grouppolicy-policy-rule-create --action action --classifier
 classifer rule
 neutron policy-apply rule group1 group2

Mind mapping this to my example so we can see how to achieve the same thing
in group policy (with protocols and all)?  Looks like you would also need
to specify

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward


On 08/06/2014 04:13 PM, Sumit Naiksatam wrote:

On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton blak...@gmail.com wrote:

I believe the referential security group rules solve this problem (unless
I'm not understanding):


I think the disconnect is that you are comparing the way to current mapping
driver implements things for the reference implementation with the existing
APIs. Under this light, it's not going to look like there is a point to this
code being in Neutron since, as you said, the abstraction could happen at a
client. However, this changes once new mapping drivers can be added that
implement things differently.

Let's take the security groups example. Using the security groups API
directly is imperative (put a firewall rule on this port that blocks this
IP) compared to a higher level declarative abstraction (make sure these
two endpoints cannot communicate). With the former, the ports must support
security groups and there is nowhere except for the firewall rules on that
port to implement it without violating the user's expectation. With the
latter, a mapping driver could determine that communication between these
two hosts can be prevented by using an ACL on a router or a switch, which
doesn't violate the user's intent and buys a performance improvement and
works with ports that don't support security groups.

Group based policy is trying to move the requests into the declarative
abstraction so optimizations like the one above can be made.



Kevin, you have captured the GBP value prop and difference very
succinctly. The major difference is in the declarative (GBP) versus
imperative (current) style of programming.

This has been stated very clearly and explicitly in the blueprint
spec. If one does not appreciate the difference or advantage of one
over the other, then this discussion is pointless.


One does appreciate the value of having porcelain APIs overlay a 
plumbing API. This discussion was about the proper way and place to 
introduce such functionality.


However, it seems to me that the end-goal of the GBP effort is 
*actually* to provide a higher-layer API to Neutron that would 
essentially enable proprietary vendors to entirely swap out all of 
Neutron core for a new set of drivers that spoke proprietary device APIs.


If this is the end-goal, it should be stated more clearly, IMO.

The classic plumbing versus porcelain API conversation [1] is a good 
one, and one worth having in the context of Neutron.


It's almost like some Neutron contributor folks are saying let's add a 
porcelain API so we can ditch all the existing plumbing APIs and replace 
with our own stuff. And that's not what the point of plumbing vs. 
porcelain is.


-jay

[1] http://git-scm.com/book/en/Git-Internals-Plumbing-and-Porcelain

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Mohammad Banikazemi

Jay Pipes jaypi...@gmail.com wrote on 08/06/2014 04:09:20 PM:

 From: Jay Pipes jaypi...@gmail.com
 To: openstack-dev@lists.openstack.org
 Date: 08/06/2014 04:12 PM
 Subject: Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy
 and the way forward

 On 08/06/2014 03:46 PM, Kevin Benton wrote:
   I believe the referential security group rules solve this problem
  (unless I'm not understanding):

  I think the disconnect is that you are comparing the way to current
  mapping driver implements things for the reference implementation with
  the existing APIs. Under this light, it's not going to look like there
  is a point to this code being in Neutron since, as you said, the
  abstraction could happen at a client. However, this changes once new
  mapping drivers can be added that implement things differently.

  Let's take the security groups example. Using the security groups API
  directly is imperative (put a firewall rule on this port that blocks
  this IP) compared to a higher level declarative abstraction (make
sure
  these two endpoints cannot communicate). With the former, the ports
  must support security groups and there is nowhere except for the
  firewall rules on that port to implement it without violating the
user's
  expectation. With the latter, a mapping driver could determine that
  communication between these two hosts can be prevented by using an ACL
  on a router or a switch, which doesn't violate the user's intent and
  buys a performance improvement and works with ports that don't support
  security groups.

  Group based policy is trying to move the requests into the declarative
  abstraction so optimizations like the one above can be made.

 So, in short, GBP is an effort to provide an API that substitutes
 generic names for specific names in order to allow custom proprietary
 hardware vendors to wire in their hardware using an API that better
 matches their own internal modelling.

 Would that be a good statement of the end-goal of GBP?

No. That would be incorrect.
If you have the time, please see the following article. Even though it is
more than a year old, it may help in clarifying the intent of the policy
work.

M. Banikazemi, D, Olshefski, A. Shaikh, J. Tracey, G. Wang, Meridian: an
SDN platform for cloud network services, IEEE Communications Magazine,
vol. 51, no. 2, February 2013

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=truearnumber=6461196

Best,

-Mohammad

 -jay

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

Vendors swapping out components is only one possibility. Once people use
this declarative model, optimizations can be made on the reference side as
well. ACLs can be placed on L3 agents to reduce the rule count on
individual compute nodes, etc. Everyone benefits from the abstraction, not
just hardware vendors.

We don't use SQL because Oracle or Microsoft have good optimizations. We
use it because it lets people say what they want without worrying about how
it happens.


On Wed, Aug 6, 2014 at 2:27 PM, Jay Pipes jaypi...@gmail.com wrote:

 On 08/06/2014 04:13 PM, Sumit Naiksatam wrote:

 On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton blak...@gmail.com wrote:

 I believe the referential security group rules solve this problem (unless
 I'm not understanding):


 I think the disconnect is that you are comparing the way to current
 mapping
 driver implements things for the reference implementation with the
 existing
 APIs. Under this light, it's not going to look like there is a point to
 this
 code being in Neutron since, as you said, the abstraction could happen
 at a
 client. However, this changes once new mapping drivers can be added that
 implement things differently.

 Let's take the security groups example. Using the security groups API
 directly is imperative (put a firewall rule on this port that blocks
 this
 IP) compared to a higher level declarative abstraction (make sure these
 two endpoints cannot communicate). With the former, the ports must
 support
 security groups and there is nowhere except for the firewall rules on
 that
 port to implement it without violating the user's expectation. With the
 latter, a mapping driver could determine that communication between these
 two hosts can be prevented by using an ACL on a router or a switch, which
 doesn't violate the user's intent and buys a performance improvement and
 works with ports that don't support security groups.

 Group based policy is trying to move the requests into the declarative
 abstraction so optimizations like the one above can be made.


 Kevin, you have captured the GBP value prop and difference very
 succinctly. The major difference is in the declarative (GBP) versus
 imperative (current) style of programming.

 This has been stated very clearly and explicitly in the blueprint
 spec. If one does not appreciate the difference or advantage of one
 over the other, then this discussion is pointless.


 One does appreciate the value of having porcelain APIs overlay a
 plumbing API. This discussion was about the proper way and place to
 introduce such functionality.

 However, it seems to me that the end-goal of the GBP effort is *actually*
 to provide a higher-layer API to Neutron that would essentially enable
 proprietary vendors to entirely swap out all of Neutron core for a new set
 of drivers that spoke proprietary device APIs.

 If this is the end-goal, it should be stated more clearly, IMO.

 The classic plumbing versus porcelain API conversation [1] is a good one,
 and one worth having in the context of Neutron.

 It's almost like some Neutron contributor folks are saying let's add a
 porcelain API so we can ditch all the existing plumbing APIs and replace
 with our own stuff. And that's not what the point of plumbing vs.
 porcelain is.

 -jay

 [1] http://git-scm.com/book/en/Git-Internals-Plumbing-and-Porcelain


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Sumit Naiksatam

On Wed, Aug 6, 2014 at 1:27 PM, Jay Pipes jaypi...@gmail.com wrote:
On 08/06/2014 04:13 PM, Sumit Naiksatam wrote:

On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton blak...@gmail.com wrote:

I believe the referential security group rules solve this problem
(unless
I'm not understanding):

I think the disconnect is that you are comparing the way to current
mapping
driver implements things for the reference implementation with the
existing
APIs. Under this light, it's not going to look like there is a point to
this
code being in Neutron since, as you said, the abstraction could happen at
a
client. However, this changes once new mapping drivers can be added that
implement things differently.

Let's take the security groups example. Using the security groups API
directly is imperative (put a firewall rule on this port that blocks
this
IP) compared to a higher level declarative abstraction (make sure these
two endpoints cannot communicate). With the former, the ports must
support
security groups and there is nowhere except for the firewall rules on
that
port to implement it without violating the user's expectation. With the
latter, a mapping driver could determine that communication between these
two hosts can be prevented by using an ACL on a router or a switch, which
doesn't violate the user's intent and buys a performance improvement and
works with ports that don't support security groups.

Group based policy is trying to move the requests into the declarative
abstraction so optimizations like the one above can be made.

Kevin, you have captured the GBP value prop and difference very
succinctly. The major difference is in the declarative (GBP) versus
imperative (current) style of programming.

This has been stated very clearly and explicitly in the blueprint
spec. If one does not appreciate the difference or advantage of one
over the other, then this discussion is pointless.

One does appreciate the value of having porcelain APIs overlay a plumbing
API. This discussion was about the proper way and place to introduce such
functionality.

However, it seems to me that the end-goal of the GBP effort is *actually* to
provide a higher-layer API to Neutron that would essentially enable
proprietary vendors to entirely swap out all of Neutron core for a new set
of drivers that spoke proprietary device APIs.

If this is the end-goal, it should be stated more clearly, IMO.

The goal and design intent is unambiguously stated in the spec [1];

L36-L38
The policy framework described in this blueprint complements the current
Neutron model with the notion of policies that can be applied between
groups of endpoints.

Note the choice of words - complements. The implementation also has
been faithful to this intent.

I am not sure why you would draw the conclusion that you did.

[1]
https://review.openstack.org/#/c/89469/10/specs/juno/group-based-policy-abstraction.rst,cm

The classic plumbing versus porcelain API conversation [1] is a good one,
and one worth having in the context of Neutron.

It's almost like some Neutron contributor folks are saying let's add a
porcelain API so we can ditch all the existing plumbing APIs and replace
with our own stuff. And that's not what the point of plumbing vs. porcelain
is.

-jay

[1] http://git-scm.com/book/en/Git-Internals-Plumbing-and-Porcelain

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Pedro Marques


On Aug 6, 2014, at 1:27 PM, Jay Pipes jaypi...@gmail.com wrote:
 
 However, it seems to me that the end-goal of the GBP effort is *actually* to 
 provide a higher-layer API to Neutron that would essentially enable 
 proprietary vendors to entirely swap out all of Neutron core for a new set of 
 drivers that spoke proprietary device APIs.
 
 If this is the end-goal, it should be stated more clearly, IMO.

I believe that people should be considered innocent until proven otherwise. Is 
there a reason to believe there is some hidden reason behind this proposal ? It 
seems to me that this is uncalled for.

Neutron allows vendors to speak to proprietary device APIs, it was designed to 
do so, AFAIK. It is also possibly to entirely swap out all of the Neutron 
core... the proponents of the group based policy didn't have to go through so 
much trouble if that was their intent. As far as i know most plugins talk to a 
proprietary API.

I happen to disagree technically with a couple of choices made by this 
proposal; but the blueprint was approved. Which means that i lost the argument, 
or didn't raise it on time, or didn't argue convincingly... regardless of the 
reason, the time to argue about the goal has passed. The decision of the 
community was to approve the spec and that decision should be respected.

  Pedro.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Prasad Vellanki

Jay
Doesnt the current plugin model in neutron work the way you are saying. We
have a a core set of APIs that there is a reference model for and
individual vendors have substituted plugins that enhance and sometimes
replace networking component. GBP in that respect does not change. There is
a reference implementation in neutron for declarative model in neutron and
vendors can substitute their implementation to enhance what is in
reference.

But what you need to understand is the declarative model that it provides
which Ryan has elaborated which current neutron does not provide.

prasadv


On Wed, Aug 6, 2014 at 1:27 PM, Jay Pipes jaypi...@gmail.com wrote:

 On 08/06/2014 04:13 PM, Sumit Naiksatam wrote:

 On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton blak...@gmail.com wrote:

 I believe the referential security group rules solve this problem (unless
 I'm not understanding):


 I think the disconnect is that you are comparing the way to current
 mapping
 driver implements things for the reference implementation with the
 existing
 APIs. Under this light, it's not going to look like there is a point to
 this
 code being in Neutron since, as you said, the abstraction could happen
 at a
 client. However, this changes once new mapping drivers can be added that
 implement things differently.

 Let's take the security groups example. Using the security groups API
 directly is imperative (put a firewall rule on this port that blocks
 this
 IP) compared to a higher level declarative abstraction (make sure these
 two endpoints cannot communicate). With the former, the ports must
 support
 security groups and there is nowhere except for the firewall rules on
 that
 port to implement it without violating the user's expectation. With the
 latter, a mapping driver could determine that communication between these
 two hosts can be prevented by using an ACL on a router or a switch, which
 doesn't violate the user's intent and buys a performance improvement and
 works with ports that don't support security groups.

 Group based policy is trying to move the requests into the declarative
 abstraction so optimizations like the one above can be made.


 Kevin, you have captured the GBP value prop and difference very
 succinctly. The major difference is in the declarative (GBP) versus
 imperative (current) style of programming.

 This has been stated very clearly and explicitly in the blueprint
 spec. If one does not appreciate the difference or advantage of one
 over the other, then this discussion is pointless.


 One does appreciate the value of having porcelain APIs overlay a
 plumbing API. This discussion was about the proper way and place to
 introduce such functionality.

 However, it seems to me that the end-goal of the GBP effort is *actually*
 to provide a higher-layer API to Neutron that would essentially enable
 proprietary vendors to entirely swap out all of Neutron core for a new set
 of drivers that spoke proprietary device APIs.

 If this is the end-goal, it should be stated more clearly, IMO.

 The classic plumbing versus porcelain API conversation [1] is a good one,
 and one worth having in the context of Neutron.

 It's almost like some Neutron contributor folks are saying let's add a
 porcelain API so we can ditch all the existing plumbing APIs and replace
 with our own stuff. And that's not what the point of plumbing vs.
 porcelain is.

 -jay

 [1] http://git-scm.com/book/en/Git-Internals-Plumbing-and-Porcelain


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward