date:20140806

Do you not see a difference between explicitly configuring networks, a
router and FWaaS rules with logging and just stating that two groups of
servers can only communicate via one TCP port with all connections logged?
The first is very prone to errors for someone deploying an application
without a strong networking background, and the latter is basically just
stating the requirements and letting Neutron figure out how to implement
it.

Just stating requirements becomes even more important when something like
the logging requirement comes from someone other than the app deployer
(e.g. a security team). In the above example, someone could set everything
up using security groups; however, when the logging requirement came in
from the security team, they would have to undo all of that work and
replace it with something like FWaaS that can centrally log all of the
connections.

It's the difference between using puppet and bash scripts. Sure you can
write a script that uses awk/sed to ensure that an ini file has a
particular setting and then restart a service if the setting is changed,
but it's much easier and less error prone to just write a puppet manifest
that uses the INI module with a pointer to the file, the section name, the
key, and the value with a notification to restart the service.



On Wed, Aug 6, 2014 at 7:40 PM, Aaron Rosen  wrote:

>
> On Wed, Aug 6, 2014 at 5:27 PM, Kevin Benton  wrote:
>
>> Web tier can communicate with anything except for the DB.
>> App tier can only communicate with Web and DB.
>> DB can communicate with App.
>>
>> These high-level constraints can then be implemented as security groups
>> like you showed, or network hardware ACLs like I had shown.
>> But if you start with the security groups API, you are forcing it to be
>> implemented there.
>>
>>
> I still have no idea what group based policy is buying us then. It seems
> to me that the key point we've identified going backing and forth here is
> the difference between the current model and the GBP model is that GBP
> constricts topology which allows us to write these types of enforcement
> rules. The reason we want this is because it yields performance
> optimizations (for some reason, which I don't think we've gotten into).
> Would you agree this is accurate?
>
> Honestly, I know a lot of work has been put into this. I haven't said I'm
> for or against it either. I'm really just trying to understand what is the
> motivation for this and why does it make neutron better.
>
> Best,
>
> Aaron
>
>
>
>>
>> On Wed, Aug 6, 2014 at 6:06 PM, Aaron Rosen 
>> wrote:
>>
>>>
>>>
>>>
>>> On Wed, Aug 6, 2014 at 4:46 PM, Kevin Benton  wrote:
>>>
 That's the point. By using security groups, you are forcing a certain
 kind of enforcement that must be honored and might not be necessary if the
 original intent was just to isolate between groups. In the example you
 gave, it cannot be implemented on the router without violating the
 constraints of the security group.


 Hi Kevin,
>>>
>>> Mind proposing an alternative example then. The only way I can see this
>>> claim to be made is because Group Based policy is actually limiting what a
>>> tenants desired topology can be?
>>>
>>> Thanks,
>>>
>>> Aaron
>>>
>>>
>>>
  On Wed, Aug 6, 2014 at 5:39 PM, Aaron Rosen 
 wrote:

>
>
>
> On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton 
> wrote:
>
>> >Given this information I don't see any reason why the backend
>> system couldn't do enforcement at the logical router and if it did so
>> neither parties would know.
>>
>> With security groups you are specifying that nothing can contact
>> these devices on those ports unless they come from the allowed IP
>> addresses. If you tried to enforce this at the router you would be
>> violating that specification because devices in the same subnet would be
>> able to communicate on those blocked ports.
>>
>
> Sure, though this is a problem of where you are doing your
> enforcement. If the backend system chooses to implement this optimization
> in this fashion (which was the example you gave previously [1]). Then, if
> the topology changes, i.e adding a port to the same network with
> conflicting security group rules, the backend system can no longer 
> optimize
> in this same fashion at the router level and a more fine grain filtering
> will need to be done. How would this be any different with group based
> policy?
>
>
> [1] - With the latter, a mapping driver could determine that
> communication between these two hosts can be prevented by using an ACL on 
> a
> router or a switch, which doesn't violate the user's intent and buys a
> performance improvement and works with ports that don't support security
> groups.
>
> states
>
>
>
>>
>>
>> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen 
>> wrote:
>>
>

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Prasad Vellanki

I don't think people are choosing because of the shortest route but rather
these may be two different items that are not completely exclusive but
different enough.
Nova parity addresses the scope to have existing well understood and stable
api currently supported in nova to be supported in neutron and a dash to
make that code stable. The goal is to move quickly to enable replacement of
nova networking.
While group policy is offering additional abstractions which are richer
that enables easier usage of networking.

Both teams have been working diligently towards that goal. As pedro points
out there are several people from different organizations working on GBP to
ensure stability and closely reviewed code is checked in.

I think both nova parity and GBP can go in parallel, hence my choice of
option 1

On Wed, Aug 6, 2014 at 6:13 PM, Armando M.  wrote:

> On 6 August 2014 17:34, Prasad Vellanki <
> prasad.vella...@oneconvergence.com> wrote:
>
>> It seems like Option  1 would be preferable. User can use this right
>> away.
>>
>>
> People choosing Option 1 may think that the shortest route may be the
> best, that said the drawback I identified is not to be dismissed either
> (and I am sure there many more pros/cons): an immature product is of good
> use to no-one, and we still have the nova parity that haunts us.
>
> I think this could be another reason why people associated GBP and
> nova-network parity in this thread: the fact that new abstractions are
> introduced without solidifying the foundations of the project is a risk to
> GBP as well as Neutron itself.
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Well-tested guides for OpenStack Icehouse installation and Instance creation with Neutron

2014-08-06 Thread Baohua Yang

Nice work.
Suggest add some watch points/problem solution/diagnosis in the
installations.
Btw, recently we find both the RDO and devstack are very unstable for
installation, wish there will be more test and improvements soon.


On Wed, Aug 6, 2014 at 3:27 AM, chayma ghribi  wrote:

>
> Hi all,
>
>
> I want to share with you our well tested OpenStack Icehouse Installation
> Guide for Ubuntu 14.04.
>
>
> https://github.com/ChaimaGhribi/OpenStack-Icehouse-Installation/blob/master/OpenStack-Icehouse-Installation.rst
>
> If you want to create your first instance with Neutron, follow the
> instructions in our VM creation guide available here:
>
>
> https://github.com/ChaimaGhribi/OpenStack-Icehouse-Installation/blob/master/Create-your-first-instance-with-Neutron.rst
>
> Hope this will be helpful !
> Your questions and suggestions are welcome :)
>
>
> Regards,
>
> Chaima Ghribi
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best wishes!
Baohua
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][third-party] Arista CI hits 10, 000 runs this morning

Nice work Sukhdev, worth commending! Thanks for sharing!!

On Wed, Aug 6, 2014 at 7:06 PM, Baohua Yang  wrote:
> Woo~
> Really nice work!
>
>
> On Thu, Aug 7, 2014 at 7:09 AM, Sukhdev Kapur 
> wrote:
>>
>> Folks,
>>
>> Just wanted to share with you that Arista CI has been up and running 24x7
>> since the beginning of this year with no down time.
>>
>> This morning it posted a vote on 10,000th Neutron patch
>>
>> cheers..
>> -Sukhdev
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Best wishes!
> Baohua
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Rudra Rugge

On Wed, Aug 6, 2014 at 6:40 PM, Aaron Rosen  wrote:

>
> On Wed, Aug 6, 2014 at 5:27 PM, Kevin Benton  wrote:
>
>> Web tier can communicate with anything except for the DB.
>> App tier can only communicate with Web and DB.
>> DB can communicate with App.
>>
>> These high-level constraints can then be implemented as security groups
>> like you showed, or network hardware ACLs like I had shown.
>> But if you start with the security groups API, you are forcing it to be
>> implemented there.
>>
>>
> I still have no idea what group based policy is buying us then. It seems
> to me that the key point we've identified going backing and forth here is
> the difference between the current model and the GBP model is that GBP
> constricts topology which allows us to write these types of enforcement
> rules. The reason we want this is because it yields performance
> optimizations (for some reason, which I don't think we've gotten into).
> Would you agree this is accurate?
>
>

Policy framework is a progression towards providing a richer configuration.
Network ACLs help with aggregation (network or other aggregations) and
attaching to broader bind points as opposed to just ports. Giving the
operator a choice of all options is necessary at this point with such
features being standard in public clouds (eg: VPC feature set in AWS)

Rudra



> Aaron
>
>
>
>>
>> On Wed, Aug 6, 2014 at 6:06 PM, Aaron Rosen 
>> wrote:
>>
>>>
>>>
>>>
>>> On Wed, Aug 6, 2014 at 4:46 PM, Kevin Benton  wrote:
>>>
 That's the point. By using security groups, you are forcing a certain
 kind of enforcement that must be honored and might not be necessary if the
 original intent was just to isolate between groups. In the example you
 gave, it cannot be implemented on the router without violating the
 constraints of the security group.


 Hi Kevin,
>>>
>>> Mind proposing an alternative example then. The only way I can see this
>>> claim to be made is because Group Based policy is actually limiting what a
>>> tenants desired topology can be?
>>>
>>> Thanks,
>>>
>>> Aaron
>>>
>>>
>>>
  On Wed, Aug 6, 2014 at 5:39 PM, Aaron Rosen 
 wrote:

>
>
>
> On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton 
> wrote:
>
>> >Given this information I don't see any reason why the backend
>> system couldn't do enforcement at the logical router and if it did so
>> neither parties would know.
>>
>> With security groups you are specifying that nothing can contact
>> these devices on those ports unless they come from the allowed IP
>> addresses. If you tried to enforce this at the router you would be
>> violating that specification because devices in the same subnet would be
>> able to communicate on those blocked ports.
>>
>
> Sure, though this is a problem of where you are doing your
> enforcement. If the backend system chooses to implement this optimization
> in this fashion (which was the example you gave previously [1]). Then, if
> the topology changes, i.e adding a port to the same network with
> conflicting security group rules, the backend system can no longer 
> optimize
> in this same fashion at the router level and a more fine grain filtering
> will need to be done. How would this be any different with group based
> policy?
>
>
> [1] - With the latter, a mapping driver could determine that
> communication between these two hosts can be prevented by using an ACL on 
> a
> router or a switch, which doesn't violate the user's intent and buys a
> performance improvement and works with ports that don't support security
> groups.
>
> states
>
>
>
>>
>>
>> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen 
>> wrote:
>>
>>>
>>> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton 
>>> wrote:
>>>
 By working at the port level you have already eliminated your
 ability to implement the filtering at different components of the 
 network.
 They now need to be implemented in stateful rules at the port level 
 and the
 device has to support security groups.

>>>
>>>
>>> Lets take this example where we setup a 2 tier app with web-servers
>>> and db-servers that are connected on two different networks attached to 
>>> a
>>> router. We add a security group rules such that web-servers can talk to
>>> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers 
>>> from
>>> anywhere.
>>>
>>> neutron net-create web_net
>>> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>>>
>>> neutron net-create db_net
>>> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>>>
>>> neutron router-create myrouter
>>> neutron router-interface-add myrouter web_subnet
>>> neutron router-interface-add myrouter db_subnet
>>>
>>>

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

2014-08-06 Thread Baohua Yang

+1.
And the review process should be more efficient!



On Thu, Aug 7, 2014 at 3:07 AM, Stefano Maffulli 
wrote:

> On 08/06/2014 11:19 AM, Edgar Magana wrote:
> > That is the beauty of the open source projects, there is always a
> smartest
> > reviewer catching out the facts that you don¹t.
>
> And yet, the specification clearly talks about 'endpoints' and nobody
> caught it where it supposed to be caught so I fear that something failed
> badly here:
>
> https://review.openstack.org/#/c/89469/10
>
> What failed and how we make sure this doesn't happen again? This to me
> is the most important question to answer.  If I remember correctly we
> introduced the concept of Specs exactly to discuss on the ideas *before*
> the implementation starts. We wanted things like architecture, naming
> conventions and other important decisions to be socialized and agreed
> upon *before* code was proposed. We wanted to avoid developers to spend
> time implementing features in ways that are incompatible and likely to
> be rejected at code review time. And yet, here we are.
>
> Something failed and I would ask for all core reviewers to sit down and
> do an exercise to identify the root cause. If you want we can start from
> this specific case, do some simple root cause analysis together and take
> GBP as an example. Thoughts?
>
> /stef
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Best wishes!
Baohua
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][third-party] Arista CI hits 10, 000 runs this morning

2014-08-06 Thread Baohua Yang

Woo~
Really nice work!


On Thu, Aug 7, 2014 at 7:09 AM, Sukhdev Kapur 
wrote:

> Folks,
>
> Just wanted to share with you that Arista CI has been up and running 24x7
> since the beginning of this year with no down time.
>
> This morning it posted a vote on 10,000th Neutron patch
>
> cheers..
> -Sukhdev
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best wishes!
Baohua
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-Infra] [Neutron][third-party] Arista CI hits 10, 000 runs this morning

2014-08-06 Thread Sukhdev Kapur

Thanks Salvatore.

I originally started with all tests and then started to back off to come up
with a set of tests which gave good stable results. Otherwise I was getting
65% - 80% pass rates. And, when a test fails, I will go back and triage it
only to realize that running that test did not really test the Arista ML2
driver, so, I filtered those tests. This is how I came up with the present
set of tests.

I am looking into the scenario tests. Having some issues with them. Will
soon be adding a few to the test suite.

Thanks for the feedback.

regards..
-Sukhdev

On Wed, Aug 6, 2014 at 4:19 PM, Salvatore Orlando 
wrote:

> Congratulations!
> And to celebrate this milestone, I would consider running something more
> than ~280 api tests... perhaps also a few scenario tests?
>
> Salvatore
>
>
> On 7 August 2014 01:09, Sukhdev Kapur  wrote:
>
>> Folks,
>>
>> Just wanted to share with you that Arista CI has been up and running 24x7
>> since the beginning of this year with no down time.
>>
>> This morning it posted a vote on 10,000th Neutron patch
>>
>> cheers..
>> -Sukhdev
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-Infra mailing list
> openstack-in...@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On Wed, Aug 6, 2014 at 5:27 PM, Kevin Benton  wrote:

> Web tier can communicate with anything except for the DB.
> App tier can only communicate with Web and DB.
> DB can communicate with App.
>
> These high-level constraints can then be implemented as security groups
> like you showed, or network hardware ACLs like I had shown.
> But if you start with the security groups API, you are forcing it to be
> implemented there.
>
>
I still have no idea what group based policy is buying us then. It seems to
me that the key point we've identified going backing and forth here is the
difference between the current model and the GBP model is that GBP
constricts topology which allows us to write these types of enforcement
rules. The reason we want this is because it yields performance
optimizations (for some reason, which I don't think we've gotten into).
Would you agree this is accurate?

Honestly, I know a lot of work has been put into this. I haven't said I'm
for or against it either. I'm really just trying to understand what is the
motivation for this and why does it make neutron better.

Best,

Aaron



>
> On Wed, Aug 6, 2014 at 6:06 PM, Aaron Rosen  wrote:
>
>>
>>
>>
>> On Wed, Aug 6, 2014 at 4:46 PM, Kevin Benton  wrote:
>>
>>> That's the point. By using security groups, you are forcing a certain
>>> kind of enforcement that must be honored and might not be necessary if the
>>> original intent was just to isolate between groups. In the example you
>>> gave, it cannot be implemented on the router without violating the
>>> constraints of the security group.
>>>
>>>
>>> Hi Kevin,
>>
>> Mind proposing an alternative example then. The only way I can see this
>> claim to be made is because Group Based policy is actually limiting what a
>> tenants desired topology can be?
>>
>> Thanks,
>>
>> Aaron
>>
>>
>>
>>>  On Wed, Aug 6, 2014 at 5:39 PM, Aaron Rosen 
>>> wrote:
>>>



 On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton  wrote:

> >Given this information I don't see any reason why the backend system
> couldn't do enforcement at the logical router and if it did so neither
> parties would know.
>
> With security groups you are specifying that nothing can contact these
> devices on those ports unless they come from the allowed IP addresses. If
> you tried to enforce this at the router you would be violating that
> specification because devices in the same subnet would be able to
> communicate on those blocked ports.
>

 Sure, though this is a problem of where you are doing your enforcement.
 If the backend system chooses to implement this optimization in this
 fashion (which was the example you gave previously [1]). Then, if the
 topology changes, i.e adding a port to the same network with conflicting
 security group rules, the backend system can no longer optimize in this
 same fashion at the router level and a more fine grain filtering will need
 to be done. How would this be any different with group based policy?


 [1] - With the latter, a mapping driver could determine that
 communication between these two hosts can be prevented by using an ACL on a
 router or a switch, which doesn't violate the user's intent and buys a
 performance improvement and works with ports that don't support security
 groups.

 states



>
>
> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen 
> wrote:
>
>>
>> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton 
>> wrote:
>>
>>> By working at the port level you have already eliminated your
>>> ability to implement the filtering at different components of the 
>>> network.
>>> They now need to be implemented in stateful rules at the port level and 
>>> the
>>> device has to support security groups.
>>>
>>
>>
>> Lets take this example where we setup a 2 tier app with web-servers
>> and db-servers that are connected on two different networks attached to a
>> router. We add a security group rules such that web-servers can talk to
>> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers 
>> from
>> anywhere.
>>
>> neutron net-create web_net
>> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>>
>> neutron net-create db_net
>> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>>
>> neutron router-create myrouter
>> neutron router-interface-add myrouter web_subnet
>> neutron router-interface-add myrouter db_subnet
>>
>> neutron security-group-create  web-servers;
>> neutron security-group-create db-servers;
>>
>> # add rule to allow web members to talk to the db-servers on TCP 3306
>> for their db connection;
>> neutron security-group-rule-create --protocol TCP --port-range-min
>> 3306 --port-range-max 3306 --remote-group-id web-servers db-servers
>>
>> # add rule

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

>I think this could be another reason why people associated GBP and
nova-network parity in this thread: the fact that new abstractions are
introduced without solidifying the foundations of the project is a risk to
GBP as well as Neutron itself.

How does a separate project fix this? It seems to me like it just incurs
all of the extra overhead of an extra project and doesn't do anything to
improve the parity efforts.


On Wed, Aug 6, 2014 at 7:13 PM, Armando M.  wrote:

> On 6 August 2014 17:34, Prasad Vellanki <
> prasad.vella...@oneconvergence.com> wrote:
>
>> It seems like Option  1 would be preferable. User can use this right
>> away.
>>
>>
> People choosing Option 1 may think that the shortest route may be the
> best, that said the drawback I identified is not to be dismissed either
> (and I am sure there many more pros/cons): an immature product is of good
> use to no-one, and we still have the nova parity that haunts us.
>
> I think this could be another reason why people associated GBP and
> nova-network parity in this thread: the fact that new abstractions are
> introduced without solidifying the foundations of the project is a risk to
> GBP as well as Neutron itself.
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On 6 August 2014 17:34, Prasad Vellanki 
wrote:

> It seems like Option  1 would be preferable. User can use this right away.
>
>
People choosing Option 1 may think that the shortest route may be the best,
that said the drawback I identified is not to be dismissed either (and I am
sure there many more pros/cons): an immature product is of good use to
no-one, and we still have the nova parity that haunts us.

I think this could be another reason why people associated GBP and
nova-network parity in this thread: the fact that new abstractions are
introduced without solidifying the foundations of the project is a risk to
GBP as well as Neutron itself.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Which program for Rally

2014-08-06 Thread Yingjun Li

From a user’s aspect i do think Rally is more suitable for a product-ready 
cloud, and seems like it is where it focused on.  It’s very easy to evaluate 
that if the performance of the cloud is better after we adjust some configs or 
some other tuning. It also provides SLA which maybe not
so powerful currently but it’s a good start. So I think Rally is good enough to 
be in separated program.

I totally agree that tempest shouldn’t try to cover everything, simple makes a 
thing better.

On Aug 7, 2014, at 5:48, John Griffith  wrote:

> I have to agree with Duncan here.  I also don't know if I fully understand 
> the limit in options.  Stress test seems like it could/should be different 
> (again overlap isn't a horrible thing) and I don't see it as siphoning off 
> resources so not sure of the issue.  We've become quite wrapped up in 
> projects, programs and the like lately and it seems to hinder forward 
> progress more than anything else.
> 
> I'm also not convinced that Tempest is where all things belong, in fact I've 
> been thinking more and more that a good bit of what Tempest does today should 
> fall more on the responsibility of the projects themselves.  For example 
> functional testing of features etc, ideally I'd love to have more of that 
> fall on the projects and their respective teams.  That might even be 
> something as simple to start as saying "if you contribute a new feature, you 
> have to also provide a link to a contribution to the Tempest test-suite that 
> checks it".  Sort of like we do for unit tests, cross-project tracking is 
> difficult of course, but it's a start.  The other idea is maybe functional 
> test harnesses live in their respective projects.
> 
> Honestly I think who better to write tests for a project than the folks 
> building and contributing to the project.  At some point IMO the QA team 
> isn't going to scale.  I wonder if maybe we should be thinking about 
> proposals for delineating responsibility and goals in terms of functional 
> testing?
> 
> 
> 
> 
> On Wed, Aug 6, 2014 at 12:25 PM, Duncan Thomas  
> wrote:
> I'm not following here - you complain about rally being monolithic,
> then suggest that parts of it should be baked into tempest - a tool
> that is already huge and difficult to get into. I'd rather see tools
> that do one thing well and some overlap than one tool to rule them
> all.
> 
> On 6 August 2014 14:44, Sean Dague  wrote:
> > On 08/06/2014 09:11 AM, Russell Bryant wrote:
> >> On 08/06/2014 06:30 AM, Thierry Carrez wrote:
> >>> Hi everyone,
> >>>
> >>> At the TC meeting yesterday we discussed Rally program request and
> >>> incubation request. We quickly dismissed the incubation request, as
> >>> Rally appears to be able to live happily on top of OpenStack and would
> >>> benefit from having a release cycle decoupled from the OpenStack
> >>> "integrated release".
> >>>
> >>> That leaves the question of the program. OpenStack programs are created
> >>> by the Technical Committee, to bless existing efforts and teams that are
> >>> considered *essential* to the production of the "OpenStack" integrated
> >>> release and the completion of the OpenStack project mission. There are 3
> >>> ways to look at Rally and official programs at this point:
> >>>
> >>> 1. Rally as an essential QA tool
> >>> Performance testing (and especially performance regression testing) is
> >>> an essential QA function, and a feature that Rally provides. If the QA
> >>> team is happy to use Rally to fill that function, then Rally can
> >>> obviously be adopted by the (already-existing) QA program. That said,
> >>> that would put Rally under the authority of the QA PTL, and that raises
> >>> a few questions due to the current architecture of Rally, which is more
> >>> product-oriented. There needs to be further discussion between the QA
> >>> core team and the Rally team to see how that could work and if that
> >>> option would be acceptable for both sides.
> >>>
> >>> 2. Rally as an essential operator tool
> >>> Regular benchmarking of OpenStack deployments is a best practice for
> >>> cloud operators, and a feature that Rally provides. With a bit of a
> >>> stretch, we could consider that benchmarking is essential to the
> >>> completion of the OpenStack project mission. That program could one day
> >>> evolve to include more such "operations best practices" tools. In
> >>> addition to the slight stretch already mentioned, one concern here is
> >>> that we still want to have performance testing in QA (which is clearly
> >>> essential to the production of "OpenStack"). Letting Rally primarily be
> >>> an operational tool might make that outcome more difficult.
> >>>
> >>> 3. Let Rally be a product on top of OpenStack
> >>> The last option is to not have Rally in any program, and not consider it
> >>> *essential* to the production of the "OpenStack" integrated release or
> >>> the completion of the OpenStack project mission. Rally can happily exist
> >>> as an operator

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Stephen Wong

Hi,

Thanks to Armando for steering the discussion back to the original
intent.

On Wed, Aug 6, 2014 at 3:56 PM, Armando M.  wrote:

>
> On 6 August 2014 15:47, Kevin Benton  wrote:
>
>> I think we should merge it and just prefix the API for now with
>> '/your_application_will_break_after_juno_if_you_use_this/'
>>
>
> And you make your call based and what pros and cons exactly, If I am ask?
>
> Let me start:
>
> Option 1:
>   - pros
> - immediate delivery vehicle for consumption by operators
>

Buried inside these 100+ posts are posts from two OpenStack users
pleading for their desire to use GBP APIs for their Juno deployments. While
that is a small sample size, it does prove that there is legitimate
interests from our user base to get their hands on this feature.

User feedback is the best way to evolve the APIs moving forward - as
long as these APIs/implementation do not jeopardize the stability of
Neutron. And as many folks in the thread had pointed out, the GBP
implementation currently has really gone the extra mile to ensure it does
NOT do that.

>   - cons
> - code is burder from a number of standpoints (review, test, etc)
>

This is a legitimate concern - that said, if you take a look at the
first patch:

https://review.openstack.org/#/c/95900/

there are 30 human reviewers (non-CI) signed up to review the patch at
this time, and among them 9 Neutron core members (8 if you don't count
Sumit, who is the author), as well as a Nova core. From the reception, I
believe the community does not generally treat reviewing GBP related
patches as a burden, but likely as an item of interest. Additionally, with
such board and strong community base willing to involve in reviewing the
code, I think with these "many eyes" it will hopefully help lessen the
burden on Neutron cores to review and merge these set of patches.

>
> Option 2:
>   - pros
> - enable a small set of Illuminati to iterate faster
>

As a subgroup, we have already iterated the model and APIs for about a
year, with around 40 IRC meetings for community discussions, a PoC demo
that was presented to about 300 audiences back at J-Summit, and actual
implementations in gerrit for months now. Indeed with about 35+ people
responding to this thread, I have yet to see anyone making a claim that
"GBP model and APIs as they are now are crap, we have to scrap it and
rethink". I would like to think that we are at a point where we will do
phase by phase enhancements - as should practically any other APIs in
OpenStack - rather than rapid iterations within a cycle. While we already
have some user feedbacks, we would love to get more and more user and
developer community feedbacks to evolve GBP to better fit their needs, and
stackforge unfortunately does not serve that purpose well.

>   - cons
> - integration burden with other OpenStack projects (keystone, nova,
> neutron, etc)
>

That is indeed a major con - evidenced by the fact that the number of
Nova folks joining the discussion on this thread to express integration
concern. We would like to get GBP integration with other OpenStack projects
done as soon as possible. As you pointed out, stackforge is a terrible
option as it pushes the burden of this integration to much later, which
inevitably will incur much higher integration costs moving forward.

Thanks,
- Stephen

>
> Cheers,
> Armando
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

I definitely agree that such cross-pollination across projects is ideal.

However, I think (and not to deviate from the general discussion on
making blueprint specs review more effective), Kevin's question was
specifically in the context of the GBP blueprint. It is not clear in
that case that a Nova reviewer would have caught the terminology
overlap. Or in other words, anyone else could have caught that, and it
did not have to be a Nova reviewer (perhaps a Keystone reviewer might
have been more perceptive).

Also the model being proposed in the GBP extensions is more
user-centrci (user being the app deployer). Nova's interaction with
Neutron is more in the consumption of the current network/port level
imperative APIs.

On Wed, Aug 6, 2014 at 5:40 PM, Richard Woo  wrote:
> I agreed with Jay. Nova is one of the consumer of Neutron project, someone
> from Nova project should participate reviewing related blueprint in neutron
> project.
>
> Richard
>
>
> On Wed, Aug 6, 2014 at 8:28 PM, Jay Pipes  wrote:
>>
>> On 08/06/2014 07:54 PM, Kevin Benton wrote:
>>>
>>> I'm curious, how would having Nova reviewers look at this have helped?
>>
>>
>> As I mentioned on a previous email, Nova is the pre-eminent consumer of
>> Neutron's API.
>>
>> Best,
>> -jay
>>
>>> On Wed, Aug 6, 2014 at 5:41 PM, Jay Pipes >> > wrote:
>>>
>>> On 08/06/2014 07:08 PM, CARVER, PAUL wrote:
>>>
>>> On Aug 6, 2014, at 2:01 PM, Mohammad Banikazemi >> >>
>>> >>
>>>wrote:
>>>
>>> Yes, indeed.
>>> I do not want to be over dramatic but the discussion on the
>>> original "Group
>>> Based Policy and the way forward" thread is nothing short of
>>> heartbreaking.
>>> After months and months of discussions, three presentations
>>> at the past three
>>> summits, a design session at the last summit, and (most
>>> relevant to this
>>> thread) the approval of the spec, why are we talking about
>>> the merits of the
>>> work now?
>>>
>>> I understand if people think this is not a good idea or this
>>> is not a good
>>> time. What I do not understand is why these concerns were
>>> not raised clearly
>>> and openly earlier.
>>>
>>>
>>> I have to agree here. I'm not sure whether my organization needs
>>> GBP or not.
>>> It's certainly not our top priority for Neutron given a variety
>>> of other more
>>> important functional gaps. However, I saw their demo at the
>>> summit and it was
>>> clear that a lot of work had gone into it even before Icehouse.
>>>  From the demo
>>> it was clearly a useful enhancement to Neutron even if it wasn't
>>> at the top
>>> of my priority list.
>>>
>>> For people to be asking to justify the "why" this far into the
>>> Juno cycle
>>> when the spec was approved and the code was demoed at the summit
>>> really
>>> brings the OpenStack process into question. It's one thing to
>>> discuss
>>> technical merits of contributions but it's totally different to
>>> pull the rug
>>> out from under a group of contributors at the last minute after
>>> such a long
>>> period of development, discussion, and demo.
>>>
>>> Seeing this sort of last minute rejection of a contribution
>>> after so much
>>> time has been invested in it could very easily have a chilling
>>> effect on
>>> contributors.
>>>
>>>
>>> I don't disagree with you, Paul.
>>>
>>> I blame myself for not paying the attention I should have to this
>>> earlier in the process.
>>>
>>> FWIW, I had a good conversation with Sumit and Kevin on
>>> #openstack-neutron this afternoon about this particular topic. We
>>> agree on some things; disagree on others.
>>>
>>> Bottom line, I go back to what I said in a previous email: the Nova
>>> and Neutron development teams need to do a much better job in being
>>> directly involved in each other's spec discussions and design
>>> conversations.
>>>
>>> Best,
>>> -jay
>>>
>>>
>>> _
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.__org
>>> 
>>> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
>>> 
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Kevin Benton
>>>
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

2014-08-06 Thread Richard Woo

I agreed with Jay. Nova is one of the consumer of Neutron project, someone
from Nova project should participate reviewing related blueprint in neutron
project.

Richard


On Wed, Aug 6, 2014 at 8:28 PM, Jay Pipes  wrote:

> On 08/06/2014 07:54 PM, Kevin Benton wrote:
>
>> I'm curious, how would having Nova reviewers look at this have helped?
>>
>
> As I mentioned on a previous email, Nova is the pre-eminent consumer of
> Neutron's API.
>
> Best,
> -jay
>
>  On Wed, Aug 6, 2014 at 5:41 PM, Jay Pipes > > wrote:
>>
>> On 08/06/2014 07:08 PM, CARVER, PAUL wrote:
>>
>> On Aug 6, 2014, at 2:01 PM, Mohammad Banikazemi > >
>> >>
>>wrote:
>>
>> Yes, indeed.
>> I do not want to be over dramatic but the discussion on the
>> original "Group
>> Based Policy and the way forward" thread is nothing short of
>> heartbreaking.
>> After months and months of discussions, three presentations
>> at the past three
>> summits, a design session at the last summit, and (most
>> relevant to this
>> thread) the approval of the spec, why are we talking about
>> the merits of the
>> work now?
>>
>> I understand if people think this is not a good idea or this
>> is not a good
>> time. What I do not understand is why these concerns were
>> not raised clearly
>> and openly earlier.
>>
>>
>> I have to agree here. I'm not sure whether my organization needs
>> GBP or not.
>> It's certainly not our top priority for Neutron given a variety
>> of other more
>> important functional gaps. However, I saw their demo at the
>> summit and it was
>> clear that a lot of work had gone into it even before Icehouse.
>>  From the demo
>> it was clearly a useful enhancement to Neutron even if it wasn't
>> at the top
>> of my priority list.
>>
>> For people to be asking to justify the "why" this far into the
>> Juno cycle
>> when the spec was approved and the code was demoed at the summit
>> really
>> brings the OpenStack process into question. It's one thing to
>> discuss
>> technical merits of contributions but it's totally different to
>> pull the rug
>> out from under a group of contributors at the last minute after
>> such a long
>> period of development, discussion, and demo.
>>
>> Seeing this sort of last minute rejection of a contribution
>> after so much
>> time has been invested in it could very easily have a chilling
>> effect on
>> contributors.
>>
>>
>> I don't disagree with you, Paul.
>>
>> I blame myself for not paying the attention I should have to this
>> earlier in the process.
>>
>> FWIW, I had a good conversation with Sumit and Kevin on
>> #openstack-neutron this afternoon about this particular topic. We
>> agree on some things; disagree on others.
>>
>> Bottom line, I go back to what I said in a previous email: the Nova
>> and Neutron development teams need to do a much better job in being
>> directly involved in each other's spec discussions and design
>> conversations.
>>
>> Best,
>> -jay
>>
>>
>> _
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.__org
>> 
>> http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev
>> 
>>
>>
>>
>>
>>
>> --
>> Kevin Benton
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] The future of the integrated release

2014-08-06 Thread Michael Still

On Thu, Aug 7, 2014 at 10:05 AM, Stefano Maffulli  wrote:
> On Wed 06 Aug 2014 02:10:23 PM PDT, Michael Still wrote:
>>  - we rate limit the total number of blueprints under code review at
>> any one time to a fixed number of "slots". I secretly prefer the term
>> "runway", so I am going to use that for the rest of this email. A
>> suggested initial number of runways was proposed at ten.
>
> oh, I like the 'slots/runaway model'. Sounds to me like kan ban (in
> the Toyota sense not the hipster developer sense).
>
> A light in my head just went on.
>
> Let me translate what you're thinking about in other terms: the
> slot/runaway model would switch what is now a push model into a "pull"
> model. Currently we have patches coming in, pushed up for review. We
> have then on gerrit reviewers and core reviewers shuffling through
> these changesets, doing work and approve/comment. The reviewers have
> little to no way to notice when they're overloaded and managers have no
> way either. There is no way to identify when the process is suffering,
> slowing down or not satisfying demand, if not when the backlog blows
> up. As recent discussions demonstrate, this model is failing under our
> growth.

I agree, although I hadn't thought of the length of the backlog as
something that we could use to report on our overload until you
mentioned it. I think there is a risk here, because it would be naive
to assume that the solution to a long review backlog would be to add
more core reviewers -- we need to be careful about quality as well as
quantity.

> By switching to a model where we have a set of slots/runaway (buckets,
> in Toyota's terminology) reviewers would have a clear way to *pull* new
> reviews into their workstations to be processed. It's as simple as a
> supermaket aisle: when there is no more pasta on the shelf, a clerk
> goes in the backend and gets more pasta to refurbish the shelf. There
> is no sophisticated algorithm to predict demand: it's the demand of
> pasta that drives new pull requests (of pasta or changes to review).
>
> This pull mechanism would help make it very visible where the
> bottlenecks are. At Toyota, for example, the amount of kanbans is the
> visible way to understand the capacity of the plant. The amount of
> slots/runaways would probably give us similar overview of the capacity
> of each project and give us tools to solve bottlenecks before they
> become emergencies.
>
> I think you guys are on to something, I'd be very happy to formalize a
> proposal. Who should I talk to?

You should ping John Garbutt and Dan Smith, as they're the ones I
recall tricking.

Michael

-- 
Rackspace Australia

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Prasad Vellanki

It seems like Option  1 would be preferable. User can use this right away.

The code and to a large extent BP has gone through quite a bit of review
already so it seems to me that there actually going lot less than usual. I
dont see a whole of lot Con on this. Though there are still some issues
with naming which can be cleared fairly quickly


On Wed, Aug 6, 2014 at 4:28 PM, Pedro Marques 
wrote:

>
> On Aug 6, 2014, at 3:56 PM, Armando M.  wrote:
>
>
> On 6 August 2014 15:47, Kevin Benton  wrote:
>
>> I think we should merge it and just prefix the API for now with
>> '/your_application_will_break_after_juno_if_you_use_this/'
>>
>
> And you make your call based and what pros and cons exactly, If I am ask?
>
> Let me start:
>
> Option 1:
>   - pros
> - immediate delivery vehicle for consumption by operators
>
>
> Since our collective goal is to maximize the benefits for OpenStack
> users/operators, that seems to be the winner.
>
>   - cons
> - code is burder from a number of standpoints (review, test, etc)
>
>
> Any piece of code is a burden.
>
>
> Option 2:
>   - pros
> - enable a small set of Illuminati to iterate faster
>
>
> This is probably not intentional from your part ,but your choice of words
> make it seem that you are deriding the efforts of the team behind this
> effort. While i may disagree technically here and there with their current
> design, it seems to me that the effort in question is rather broad based in
> terms of support (from multiple different organizations) and that the team
> has put a non trivial effort in making the effort public. I don't think we
> can characterize the team either as a "secret group" or a "small set".
>
>   Pedro.
>
>
>   - cons
> - integration burden with other OpenStack projects (keystone, nova,
> neutron, etc)
>
> Cheers,
> Armando
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)


On 08/06/2014 07:54 PM, Kevin Benton wrote:

I'm curious, how would having Nova reviewers look at this have helped?


As I mentioned on a previous email, Nova is the pre-eminent consumer of 
Neutron's API.


Best,
-jay


On Wed, Aug 6, 2014 at 5:41 PM, Jay Pipes mailto:jaypi...@gmail.com>> wrote:

On 08/06/2014 07:08 PM, CARVER, PAUL wrote:

On Aug 6, 2014, at 2:01 PM, Mohammad Banikazemi mailto:m...@us.ibm.com>>>
   wrote:

Yes, indeed.
I do not want to be over dramatic but the discussion on the
original "Group
Based Policy and the way forward" thread is nothing short of
heartbreaking.
After months and months of discussions, three presentations
at the past three
summits, a design session at the last summit, and (most
relevant to this
thread) the approval of the spec, why are we talking about
the merits of the
work now?

I understand if people think this is not a good idea or this
is not a good
time. What I do not understand is why these concerns were
not raised clearly
and openly earlier.


I have to agree here. I'm not sure whether my organization needs
GBP or not.
It's certainly not our top priority for Neutron given a variety
of other more
important functional gaps. However, I saw their demo at the
summit and it was
clear that a lot of work had gone into it even before Icehouse.
 From the demo
it was clearly a useful enhancement to Neutron even if it wasn't
at the top
of my priority list.

For people to be asking to justify the "why" this far into the
Juno cycle
when the spec was approved and the code was demoed at the summit
really
brings the OpenStack process into question. It's one thing to
discuss
technical merits of contributions but it's totally different to
pull the rug
out from under a group of contributors at the last minute after
such a long
period of development, discussion, and demo.

Seeing this sort of last minute rejection of a contribution
after so much
time has been invested in it could very easily have a chilling
effect on
contributors.


I don't disagree with you, Paul.

I blame myself for not paying the attention I should have to this
earlier in the process.

FWIW, I had a good conversation with Sumit and Kevin on
#openstack-neutron this afternoon about this particular topic. We
agree on some things; disagree on others.

Bottom line, I go back to what I said in a previous email: the Nova
and Neutron development teams need to do a much better job in being
directly involved in each other's spec discussions and design
conversations.

Best,
-jay


_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org

http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 





--
Kevin Benton


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

Web tier can communicate with anything except for the DB.
App tier can only communicate with Web and DB.
DB can communicate with App.

These high-level constraints can then be implemented as security groups
like you showed, or network hardware ACLs like I had shown.
But if you start with the security groups API, you are forcing it to be
implemented there.


On Wed, Aug 6, 2014 at 6:06 PM, Aaron Rosen  wrote:

>
>
>
> On Wed, Aug 6, 2014 at 4:46 PM, Kevin Benton  wrote:
>
>> That's the point. By using security groups, you are forcing a certain
>> kind of enforcement that must be honored and might not be necessary if the
>> original intent was just to isolate between groups. In the example you
>> gave, it cannot be implemented on the router without violating the
>> constraints of the security group.
>>
>>
>> Hi Kevin,
>
> Mind proposing an alternative example then. The only way I can see this
> claim to be made is because Group Based policy is actually limiting what a
> tenants desired topology can be?
>
> Thanks,
>
> Aaron
>
>
>
>> On Wed, Aug 6, 2014 at 5:39 PM, Aaron Rosen 
>> wrote:
>>
>>>
>>>
>>>
>>> On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton  wrote:
>>>
 >Given this information I don't see any reason why the backend system
 couldn't do enforcement at the logical router and if it did so neither
 parties would know.

 With security groups you are specifying that nothing can contact these
 devices on those ports unless they come from the allowed IP addresses. If
 you tried to enforce this at the router you would be violating that
 specification because devices in the same subnet would be able to
 communicate on those blocked ports.

>>>
>>> Sure, though this is a problem of where you are doing your enforcement.
>>> If the backend system chooses to implement this optimization in this
>>> fashion (which was the example you gave previously [1]). Then, if the
>>> topology changes, i.e adding a port to the same network with conflicting
>>> security group rules, the backend system can no longer optimize in this
>>> same fashion at the router level and a more fine grain filtering will need
>>> to be done. How would this be any different with group based policy?
>>>
>>>
>>> [1] - With the latter, a mapping driver could determine that
>>> communication between these two hosts can be prevented by using an ACL on a
>>> router or a switch, which doesn't violate the user's intent and buys a
>>> performance improvement and works with ports that don't support security
>>> groups.
>>>
>>> states
>>>
>>>
>>>


 On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen 
 wrote:

>
> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton 
> wrote:
>
>> By working at the port level you have already eliminated your ability
>> to implement the filtering at different components of the network. They 
>> now
>> need to be implemented in stateful rules at the port level and the device
>> has to support security groups.
>>
>
>
> Lets take this example where we setup a 2 tier app with web-servers
> and db-servers that are connected on two different networks attached to a
> router. We add a security group rules such that web-servers can talk to
> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers 
> from
> anywhere.
>
> neutron net-create web_net
> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>
> neutron net-create db_net
> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>
> neutron router-create myrouter
> neutron router-interface-add myrouter web_subnet
> neutron router-interface-add myrouter db_subnet
>
> neutron security-group-create  web-servers;
> neutron security-group-create db-servers;
>
> # add rule to allow web members to talk to the db-servers on TCP 3306
> for their db connection;
> neutron security-group-rule-create --protocol TCP --port-range-min
> 3306 --port-range-max 3306 --remote-group-id web-servers db-servers
>
> # add rule to allow TCP 80 into the web-server sg
> neutron security-group-rule-create --protocol TCP --port-range-min 80
> --port-range-max 80 web-servers db-servers
>
> # create some ports with desired security profiles.
> neutron port-create  --security-group web-servers web_net
> neutron port-create  --security-group web-servers web_net
>
> neutron port-create  --security-group db-servers db_net
> neutron port-create  --security-group db-servers db_net
>
>
> Now to your point:
>
> By working at the port level you have already eliminated your ability
>> to implement the filtering at different components of the network. They 
>> now
>> need to be implemented in stateful rules at the port level and the device
>> has to support security groups.
>>
>>
> Given this information I don't see any

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Hemanth Ravi

Hi,

I agree that Option 1 would be best way to make the GBP API available to
operators and to make forward progress with the API.

Regards,
-hemanth


On Wed, Aug 6, 2014 at 5:04 PM, Ivar Lazzaro  wrote:

> Hi Pedro,
>
> I agree that having as much users/operators as possible using the API is
> the winner option.
> I think you should move this comment also in the main thread since it
> looks that the Subject has been accidentally modified.
>
> Cheers,
> Ivar.
>
>
> On Thu, Aug 7, 2014 at 1:28 AM, Pedro Marques 
> wrote:
>
>>
>> On Aug 6, 2014, at 3:56 PM, Armando M.  wrote:
>>
>>
>> On 6 August 2014 15:47, Kevin Benton  wrote:
>>
>>> I think we should merge it and just prefix the API for now with
>>> '/your_application_will_break_after_juno_if_you_use_this/'
>>>
>>
>> And you make your call based and what pros and cons exactly, If I am ask?
>>
>> Let me start:
>>
>> Option 1:
>>   - pros
>> - immediate delivery vehicle for consumption by operators
>>
>>
>> Since our collective goal is to maximize the benefits for OpenStack
>> users/operators, that seems to be the winner.
>>
>>   - cons
>> - code is burder from a number of standpoints (review, test, etc)
>>
>>
>> Any piece of code is a burden.
>>
>>
>> Option 2:
>>   - pros
>> - enable a small set of Illuminati to iterate faster
>>
>>
>> This is probably not intentional from your part ,but your choice of words
>> make it seem that you are deriding the efforts of the team behind this
>> effort. While i may disagree technically here and there with their current
>> design, it seems to me that the effort in question is rather broad based in
>> terms of support (from multiple different organizations) and that the team
>> has put a non trivial effort in making the effort public. I don't think we
>> can characterize the team either as a "secret group" or a "small set".
>>
>>   Pedro.
>>
>>
>>   - cons
>> - integration burden with other OpenStack projects (keystone, nova,
>> neutron, etc)
>>
>> Cheers,
>> Armando
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][LBaaS] Weekly IRC Agenda

2014-08-06 Thread Doug Wiegley

You should be able to, if logged into launchpad.  I edited it for you.

Doug


On 8/6/14, 6:07 PM, "Vijay Venkatachalam" 
wrote:

>I couldn't edit the wiki. Want to add 2 items
>1. Separating deployment and operational status.
>2. Can  driver interface be called for every API request?
>  Ex. It is not called for create pool.
>
>Sent using 
>CloudMagic 
>
>On Thu, Aug 07, 2014 at 3:54 AM, Jorge Miramontes
> wrote:
>
>
>Hey LBaaS folks,
>
>
>This is you friendly reminder to provide any agenda items for tomorrow's
>weekly IRC meeting. Please add them to the agenda wiki ==>
>https://wiki.openstack.org/wiki/Network/LBaaS#Agenda. The agenda
>currently has these items:
>
>* Review Updates
>
>Cheers,
>--Jorge
>
>
>
>
>
>P.S. Also, please don't forget to update the weekly standup ==>
>https://etherpad.openstack.org/p/neutron-lbaas-weekly-standup
>
>
>
>
>
>
>


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

I prefer merged because moving it to a separate project blocks it from
enforcing policy on the current API (including calls between service
plugins).


On Wed, Aug 6, 2014 at 4:56 PM, Armando M.  wrote:

>
> On 6 August 2014 15:47, Kevin Benton  wrote:
>
>> I think we should merge it and just prefix the API for now with
>> '/your_application_will_break_after_juno_if_you_use_this/'
>>
>
> And you make your call based and what pros and cons exactly, If I am ask?
>
> Let me start:
>
> Option 1:
>- pros
> - immediate delivery vehicle for consumption by operators
>   - cons
> - code is burder from a number of standpoints (review, test, etc)
>
> Option 2:
>   - pros
> - enable a small set of Illuminati to iterate faster
>   - cons
> - integration burden with other OpenStack projects (keystone, nova,
> neutron, etc)
>
> Cheers,
> Armando
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to improve the specs review process

On 08/06/2014 04:54 PM, Kevin Benton wrote:
> I'm curious, how would having Nova reviewers look at this have helped?

In general, we should try to involve users in all spec reviews. With
Nova being a (heavy) user of Neutron, I think, in general, having Nova
people involved would help. In general.

In this case... it's late and we should probably talk about finding
solutions for this specific case following Salvatore's and Armando's
proposals (the other thread, with a different subject).

/stef

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On 08/06/2014 04:57 PM, Aaron Rosen wrote:
> Gah, clearly hitting some kind of gmail bug as i replied to the right
> thread all 3 times i believe. 

gmail is the new Outlook

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][LBaaS] Weekly IRC Agenda

2014-08-06 Thread Vijay Venkatachalam

I couldn't edit the wiki. Want to add 2 items

1. Separating deployment and operational status.
2. Can  driver interface be called for every API request?
  Ex. It is not called for create pool.

Sent using 
CloudMagic

On Thu, Aug 07, 2014 at 3:54 AM, Jorge Miramontes 
mailto:jorge.miramon...@rackspace.com>> wrote:

Hey LBaaS folks,

This is you friendly reminder to provide any agenda items for tomorrow's weekly 
IRC meeting. Please add them to the agenda wiki ==> 
https://wiki.openstack.org/wiki/Network/LBaaS#Agenda. The agenda currently has 
these items:

  *   Review Updates

Cheers,
--Jorge

P.S. Also, please don't forget to update the weekly standup ==> 
https://etherpad.openstack.org/p/neutron-lbaas-weekly-standup
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On Wed, Aug 6, 2014 at 4:46 PM, Kevin Benton  wrote:

> That's the point. By using security groups, you are forcing a certain kind
> of enforcement that must be honored and might not be necessary if the
> original intent was just to isolate between groups. In the example you
> gave, it cannot be implemented on the router without violating the
> constraints of the security group.
>
>
> Hi Kevin,

Mind proposing an alternative example then. The only way I can see this
claim to be made is because Group Based policy is actually limiting what a
tenants desired topology can be?

Thanks,

Aaron



> On Wed, Aug 6, 2014 at 5:39 PM, Aaron Rosen  wrote:
>
>>
>>
>>
>> On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton  wrote:
>>
>>> >Given this information I don't see any reason why the backend system
>>> couldn't do enforcement at the logical router and if it did so neither
>>> parties would know.
>>>
>>> With security groups you are specifying that nothing can contact these
>>> devices on those ports unless they come from the allowed IP addresses. If
>>> you tried to enforce this at the router you would be violating that
>>> specification because devices in the same subnet would be able to
>>> communicate on those blocked ports.
>>>
>>
>> Sure, though this is a problem of where you are doing your enforcement.
>> If the backend system chooses to implement this optimization in this
>> fashion (which was the example you gave previously [1]). Then, if the
>> topology changes, i.e adding a port to the same network with conflicting
>> security group rules, the backend system can no longer optimize in this
>> same fashion at the router level and a more fine grain filtering will need
>> to be done. How would this be any different with group based policy?
>>
>>
>> [1] - With the latter, a mapping driver could determine that
>> communication between these two hosts can be prevented by using an ACL on a
>> router or a switch, which doesn't violate the user's intent and buys a
>> performance improvement and works with ports that don't support security
>> groups.
>>
>> states
>>
>>
>>
>>>
>>>
>>> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen 
>>> wrote:
>>>

 On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton  wrote:

> By working at the port level you have already eliminated your ability
> to implement the filtering at different components of the network. They 
> now
> need to be implemented in stateful rules at the port level and the device
> has to support security groups.
>


 Lets take this example where we setup a 2 tier app with web-servers and
 db-servers that are connected on two different networks attached to a
 router. We add a security group rules such that web-servers can talk to
 db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers from
 anywhere.

 neutron net-create web_net
 neutron subnet-create --name web_subnet web_net 10.0.0.0/24

 neutron net-create db_net
 neutron subnet-create --name db_subnet db_net 10.2.0.0/24

 neutron router-create myrouter
 neutron router-interface-add myrouter web_subnet
 neutron router-interface-add myrouter db_subnet

 neutron security-group-create  web-servers;
 neutron security-group-create db-servers;

 # add rule to allow web members to talk to the db-servers on TCP 3306
 for their db connection;
 neutron security-group-rule-create --protocol TCP --port-range-min 3306
 --port-range-max 3306 --remote-group-id web-servers db-servers

 # add rule to allow TCP 80 into the web-server sg
 neutron security-group-rule-create --protocol TCP --port-range-min 80
 --port-range-max 80 web-servers db-servers

 # create some ports with desired security profiles.
 neutron port-create  --security-group web-servers web_net
 neutron port-create  --security-group web-servers web_net

 neutron port-create  --security-group db-servers db_net
 neutron port-create  --security-group db-servers db_net


 Now to your point:

 By working at the port level you have already eliminated your ability
> to implement the filtering at different components of the network. They 
> now
> need to be implemented in stateful rules at the port level and the device
> has to support security groups.
>
>
 Given this information I don't see any reason why the backend system
 couldn't do enforcement at the logical router and if it did so neither
 parties would know. The backend system should have the full graph of
 everything and be able to do enforcement optimizations where ever it likes.

 btw: I say the enforcement could be done on the logical router though
 the backend system could also do this on the physical fabic as well if it
 wanted to as it should also know that graph. No?


>
> On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen 
> wrote:
>
>

Re: [openstack-dev] [all] The future of the integrated release

On Wed 06 Aug 2014 02:10:23 PM PDT, Michael Still wrote:
>  - we rate limit the total number of blueprints under code review at
> any one time to a fixed number of "slots". I secretly prefer the term
> "runway", so I am going to use that for the rest of this email. A
> suggested initial number of runways was proposed at ten.

oh, I like the 'slots/runaway model'. Sounds to me like kan ban (in 
the Toyota sense not the hipster developer sense).

A light in my head just went on.

Let me translate what you're thinking about in other terms: the 
slot/runaway model would switch what is now a push model into a "pull" 
model. Currently we have patches coming in, pushed up for review. We 
have then on gerrit reviewers and core reviewers shuffling through 
these changesets, doing work and approve/comment. The reviewers have 
little to no way to notice when they're overloaded and managers have no 
way either. There is no way to identify when the process is suffering, 
slowing down or not satisfying demand, if not when the backlog blows 
up. As recent discussions demonstrate, this model is failing under our 
growth.

By switching to a model where we have a set of slots/runaway (buckets, 
in Toyota's terminology) reviewers would have a clear way to *pull* new 
reviews into their workstations to be processed. It's as simple as a 
supermaket aisle: when there is no more pasta on the shelf, a clerk 
goes in the backend and gets more pasta to refurbish the shelf. There 
is no sophisticated algorithm to predict demand: it's the demand of 
pasta that drives new pull requests (of pasta or changes to review).

This pull mechanism would help make it very visible where the 
bottlenecks are. At Toyota, for example, the amount of kanbans is the 
visible way to understand the capacity of the plant. The amount of 
slots/runaways would probably give us similar overview of the capacity 
of each project and give us tools to solve bottlenecks before they 
become emergencies.

I think you guys are on to something, I'd be very happy to formalize a 
proposal. Who should I talk to?

/stef

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

Hi Pedro,

I agree that having as much users/operators as possible using the API is
the winner option.
I think you should move this comment also in the main thread since it looks
that the Subject has been accidentally modified.

Cheers,
Ivar.


On Thu, Aug 7, 2014 at 1:28 AM, Pedro Marques 
wrote:

>
> On Aug 6, 2014, at 3:56 PM, Armando M.  wrote:
>
>
> On 6 August 2014 15:47, Kevin Benton  wrote:
>
>> I think we should merge it and just prefix the API for now with
>> '/your_application_will_break_after_juno_if_you_use_this/'
>>
>
> And you make your call based and what pros and cons exactly, If I am ask?
>
> Let me start:
>
> Option 1:
>   - pros
> - immediate delivery vehicle for consumption by operators
>
>
> Since our collective goal is to maximize the benefits for OpenStack
> users/operators, that seems to be the winner.
>
>   - cons
> - code is burder from a number of standpoints (review, test, etc)
>
>
> Any piece of code is a burden.
>
>
> Option 2:
>   - pros
> - enable a small set of Illuminati to iterate faster
>
>
> This is probably not intentional from your part ,but your choice of words
> make it seem that you are deriding the efforts of the team behind this
> effort. While i may disagree technically here and there with their current
> design, it seems to me that the effort in question is rather broad based in
> terms of support (from multiple different organizations) and that the team
> has put a non trivial effort in making the effort public. I don't think we
> can characterize the team either as a "secret group" or a "small set".
>
>   Pedro.
>
>
>   - cons
> - integration burden with other OpenStack projects (keystone, nova,
> neutron, etc)
>
> Cheers,
> Armando
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Richard Woo

+1
On Aug 6, 2014 7:07 PM, "Salvatore Orlando"  wrote:

> I was asked beforehand what I mean with
>
> * consider GBP an 'experimental' V3 tenant API (this was mentioned
> somewhere in this thread) and treat it accordingly
>
> The group based policies, although implemented as a service plugin, are
> quite different from the ones we have now. Things like firewall, load
> balancer, etc. add something that will be run alongside the "core" API.
> This service plugin instead define a different kind of abstractive
> (declarative vs imperative as it has been mentioned several times) and it
> won't be wrong to say it can replace the core API. To this aim I think it's
> fair to consider it an experimentation around a new tenant API.
>
> In this hypothesis, how should this be treated? The decision ultimately
> does not lie with me.
> I would merely point out that:
> 1) The neutron core team should not stop innovation. If a consistent part
> of the community feels that a declarative base model is that way forward
> and we should experiment with it, than we should do everything we can to
> help this part of the community.
> 2) On the other hand, innovation and experimentation should not deprive
> the core team of significant resources which should be directed first and
> foremost to address the areas identified by the TC as in need of improvement
> In the light of the above, and reflecting on the ultimate question which
> is whether this code should be merged or not. Allow me to say that my final
> thought is that I don't care whether is merged in the main neutron repo or
> somewhere else, as long as it does not constitute additional burden for the
> core team in terms of reviews, maintainability, gate failures (it won't be
> the first time I or other core team members had to chase failures for some
> service plugins digging into logs), and future design decisions (ie: having
> to ask the question - "what about group policy" for every future API
> decision)
>
> Salvatore
>
> PS: did I make post #100?
>
>
>
> On 7 August 2014 00:47, Kevin Benton  wrote:
>
>> I think we should merge it and just prefix the API for now with
>> '/your_application_will_break_after_juno_if_you_use_this/'
>>  On Aug 6, 2014 4:41 PM, "Armando M."  wrote:
>>
>>> This thread is moving so fast I can't keep up!
>>>
>>> The fact that troubles me is that I am unable to grasp how we move
>>> forward, which was the point of this thread to start with. It seems we have
>>> 2 options:
>>>
>>> - We make GBP to merge as is, in the Neutron tree, with some minor
>>> revision (e.g. naming?);
>>> - We make GBP a stackforge project, that integrates with Neutron in some
>>> shape or form;
>>>
>>> Another option, might be something in between, where GBP is in tree, but
>>> in some sort of experimental staging area (even though I am not sure how
>>> well baked this idea is).
>>>
>>> Now, as a community we all need make a decision; arguing about the fact
>>> that the blueprint was approved is pointless. As a matter of fact, I think
>>> that blueprint should be approved, if and only if the code has landed
>>> completely, but I digress!
>>>
>>> Let's together come up with pros and cons of each approach and come up
>>> with an informed decision.
>>>
>>> Just reading free form text, how are we expected to do that? At least I
>>> can't!
>>>
>>> My 2c.
>>> Armando
>>>
>>>
>>> On 6 August 2014 15:03, Aaron Rosen  wrote:
>>>



 On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton 
 wrote:

> >I believe the referential security group rules solve this problem
> (unless I'm not understanding):
>
> I think the disconnect is that you are comparing the way to current
> mapping driver implements things for the reference implementation with the
> existing APIs. Under this light, it's not going to look like there is a
> point to this code being in Neutron since, as you said, the abstraction
> could happen at a client. However, this changes once new mapping drivers
> can be added that implement things differently.
>
> Let's take the security groups example. Using the security groups API
> directly is imperative ("put a firewall rule on this port that blocks this
> IP") compared to a higher level declarative abstraction ("make sure these
> two endpoints cannot communicate"). With the former, the ports must 
> support
> security groups and there is nowhere except for the firewall rules on that
> port to implement it without violating the user's expectation. With the
> latter, a mapping driver could determine that communication between these
> two hosts can be prevented by using an ACL on a router or a switch, which
> doesn't violate the user's intent and buys a performance improvement and
> works with ports that don't support security groups.
>
> Group based policy is trying to move the requests into the declarative
> abstraction so optimizations like the one above can be m

[openstack-dev] [Neutron][Nova] API design and usability

2014-08-06 Thread Robert Collins

On 6 August 2014 22:23, Christopher Yeoh  wrote:
>
> On Wed, Aug 6, 2014 at 5:41 PM, Aaron Rosen  wrote:
>>
>>
>>
>>
>> On Wed, Aug 6, 2014 at 12:59 AM, Gary Kotton  wrote:
>>>
>>>
>>>
>>> From: Aaron Rosen 
>>> Reply-To: OpenStack List 
>>> Date: Wednesday, August 6, 2014 at 10:09 AM
>>>
>>> To: OpenStack List 
>>> Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way
>>> forward
>>>
>>>
>>> On Tue, Aug 5, 2014 at 11:18 PM, Gary Kotton  wrote:



 On 8/5/14, 8:53 PM, "Russell Bryant"  wrote:

 >On 08/05/2014 01:23 PM, Gary Kotton wrote:
 >> Ok, thanks for the clarification. This means that it will not be done
 >> automagically as it is today  the tenant will need to create a
 >> Neutron
 >> port and then pass that through.
 >
 >FWIW, that's the direction we've wanted to move in Nova anyway.  We'd
 >like to get rid of automatic port creation, but can't do that in the
 >current stable API.

 Can you elaborate on what you mean here? What are the issues with port
 creation?

>>>
>>> Having nova-compute create ports for neutron is problematic if timeouts
>>> occur between nova and neutron as you have to garbage collect neutron ports
>>> in nova to cleanup (which was the cause of several bug in the cache handing
>>> allowing ports to leak into the info_cache in nova).  Pushing this out to
>>> the tenant is less orchestration nova has to do.
>>>
>>> [gary] my take on this is that we should allocate this via the n-api and
>>> not via the nova compute (which is far too late in the process. But that is
>>> another discussion :)
>>
>>
>> I agree, I had actually proposed this here:
>> https://blueprints.launchpad.net/nova/+spec/nova-api-quantum-create-port
>> :),   though there are some issues we need to solve in neutron first --
>> allowing the mac_address on the port to be updated in neutron. This is
>> required for bare metal support as when the port is created we don't know
>> which physical mac will need to be mapped to the port.
>>>
>>>
>
>
> I think that in the long term (when we can do an API rev) we should just be
> getting rid of the automatic port creation completely with the updated Nova
> API. I don't see why the Nova API needs to do proxying work to neutron to
> create the port when the client can do it directly with neutron (perhaps via
> some convenience client code if desired). It removes the complexity of the
> garbage collection on failure issues in Nova that we currently have.

I'm astounded by this proposal - it doesn't remove the garbage
collection complexity at all - it transfers it from our code - Nova -
onto end users. So rather than one tested and consolidated
implementation, we'll have one implementation in saltstack, one
implementation in heat, one implementation in Juju, one implementation
in foreman etc.

In what possible way is that an improvement ?

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

Gah, clearly hitting some kind of gmail bug as i replied to the right
thread all 3 times i believe.


On Wed, Aug 6, 2014 at 4:56 PM, Aaron Rosen  wrote:

> [Moving my reply to the correct thread as someone changed the subject
> line. Attempt 3 :'( ]
>
>
>
> On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton  wrote:
>
>> >Given this information I don't see any reason why the backend system
>> couldn't do enforcement at the logical router and if it did so neither
>> parties would know.
>>
>> With security groups you are specifying that nothing can contact these
>> devices on those ports unless they come from the allowed IP addresses. If
>> you tried to enforce this at the router you would be violating that
>> specification because devices in the same subnet would be able to
>> communicate on those blocked ports.
>>
>>
>>
> Sure, though this is a problem of where you are doing your enforcement. If
> the backend system chooses to implement this optimization in this fashion
> (which was the example you gave previously [1]). Then, if the topology
> changes, i.e adding a port to the same network with conflicting security
> group rules, the backend system can no longer optimize in this same fashion
> at the router level and a more fine grain filtering will need to be done.
> How would this be any different with group based policy?
>
> [1] - With the latter, a mapping driver could determine that communication
> between these two hosts can be prevented by using an ACL on a router or a
> switch, which doesn't violate the user's intent and buys a performance
> improvement and works with ports that don't support security groups.
>
>
>
>
>> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen 
>> wrote:
>>
>>>
>>> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton  wrote:
>>>
 By working at the port level you have already eliminated your ability
 to implement the filtering at different components of the network. They now
 need to be implemented in stateful rules at the port level and the device
 has to support security groups.

>>>
>>>
>>> Lets take this example where we setup a 2 tier app with web-servers and
>>> db-servers that are connected on two different networks attached to a
>>> router. We add a security group rules such that web-servers can talk to
>>> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers from
>>> anywhere.
>>>
>>> neutron net-create web_net
>>> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>>>
>>> neutron net-create db_net
>>> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>>>
>>> neutron router-create myrouter
>>> neutron router-interface-add myrouter web_subnet
>>> neutron router-interface-add myrouter db_subnet
>>>
>>> neutron security-group-create  web-servers;
>>> neutron security-group-create db-servers;
>>>
>>> # add rule to allow web members to talk to the db-servers on TCP 3306
>>> for their db connection;
>>> neutron security-group-rule-create --protocol TCP --port-range-min 3306
>>> --port-range-max 3306 --remote-group-id web-servers db-servers
>>>
>>> # add rule to allow TCP 80 into the web-server sg
>>> neutron security-group-rule-create --protocol TCP --port-range-min 80
>>> --port-range-max 80 web-servers db-servers
>>>
>>> # create some ports with desired security profiles.
>>> neutron port-create  --security-group web-servers web_net
>>> neutron port-create  --security-group web-servers web_net
>>>
>>> neutron port-create  --security-group db-servers db_net
>>> neutron port-create  --security-group db-servers db_net
>>>
>>>
>>> Now to your point:
>>>
>>> By working at the port level you have already eliminated your ability to
 implement the filtering at different components of the network. They now
 need to be implemented in stateful rules at the port level and the device
 has to support security groups.


>>> Given this information I don't see any reason why the backend system
>>> couldn't do enforcement at the logical router and if it did so neither
>>> parties would know. The backend system should have the full graph of
>>> everything and be able to do enforcement optimizations where ever it likes.
>>>
>>> btw: I say the enforcement could be done on the logical router though
>>> the backend system could also do this on the physical fabic as well if it
>>> wanted to as it should also know that graph. No?
>>>
>>>

 On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen 
 wrote:

>
>
>
> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton 
> wrote:
>
>> >I believe the referential security group rules solve this problem
>> (unless I'm not understanding):
>>
>> I think the disconnect is that you are comparing the way to current
>> mapping driver implements things for the reference implementation with 
>> the
>> existing APIs. Under this light, it's not going to look like there is a
>> point to this code being in Neutron since, as you said, the abstraction
>> c

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

[Moving my reply to the correct thread as someone changed the subject line.
Attempt 3 :'( ]



On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton  wrote:

> >Given this information I don't see any reason why the backend system
> couldn't do enforcement at the logical router and if it did so neither
> parties would know.
>
> With security groups you are specifying that nothing can contact these
> devices on those ports unless they come from the allowed IP addresses. If
> you tried to enforce this at the router you would be violating that
> specification because devices in the same subnet would be able to
> communicate on those blocked ports.
>
>
>
Sure, though this is a problem of where you are doing your enforcement. If
the backend system chooses to implement this optimization in this fashion
(which was the example you gave previously [1]). Then, if the topology
changes, i.e adding a port to the same network with conflicting security
group rules, the backend system can no longer optimize in this same fashion
at the router level and a more fine grain filtering will need to be done.
How would this be any different with group based policy?

[1] - With the latter, a mapping driver could determine that communication
between these two hosts can be prevented by using an ACL on a router or a
switch, which doesn't violate the user's intent and buys a performance
improvement and works with ports that don't support security groups.




> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen  wrote:
>
>>
>> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton  wrote:
>>
>>> By working at the port level you have already eliminated your ability to
>>> implement the filtering at different components of the network. They now
>>> need to be implemented in stateful rules at the port level and the device
>>> has to support security groups.
>>>
>>
>>
>> Lets take this example where we setup a 2 tier app with web-servers and
>> db-servers that are connected on two different networks attached to a
>> router. We add a security group rules such that web-servers can talk to
>> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers from
>> anywhere.
>>
>> neutron net-create web_net
>> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>>
>> neutron net-create db_net
>> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>>
>> neutron router-create myrouter
>> neutron router-interface-add myrouter web_subnet
>> neutron router-interface-add myrouter db_subnet
>>
>> neutron security-group-create  web-servers;
>> neutron security-group-create db-servers;
>>
>> # add rule to allow web members to talk to the db-servers on TCP 3306 for
>> their db connection;
>> neutron security-group-rule-create --protocol TCP --port-range-min 3306
>> --port-range-max 3306 --remote-group-id web-servers db-servers
>>
>> # add rule to allow TCP 80 into the web-server sg
>> neutron security-group-rule-create --protocol TCP --port-range-min 80
>> --port-range-max 80 web-servers db-servers
>>
>> # create some ports with desired security profiles.
>> neutron port-create  --security-group web-servers web_net
>> neutron port-create  --security-group web-servers web_net
>>
>> neutron port-create  --security-group db-servers db_net
>> neutron port-create  --security-group db-servers db_net
>>
>>
>> Now to your point:
>>
>> By working at the port level you have already eliminated your ability to
>>> implement the filtering at different components of the network. They now
>>> need to be implemented in stateful rules at the port level and the device
>>> has to support security groups.
>>>
>>>
>> Given this information I don't see any reason why the backend system
>> couldn't do enforcement at the logical router and if it did so neither
>> parties would know. The backend system should have the full graph of
>> everything and be able to do enforcement optimizations where ever it likes.
>>
>> btw: I say the enforcement could be done on the logical router though the
>> backend system could also do this on the physical fabic as well if it
>> wanted to as it should also know that graph. No?
>>
>>
>>>
>>> On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen 
>>> wrote:
>>>



 On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton 
 wrote:

> >I believe the referential security group rules solve this problem
> (unless I'm not understanding):
>
> I think the disconnect is that you are comparing the way to current
> mapping driver implements things for the reference implementation with the
> existing APIs. Under this light, it's not going to look like there is a
> point to this code being in Neutron since, as you said, the abstraction
> could happen at a client. However, this changes once new mapping drivers
> can be added that implement things differently.
>
> Let's take the security groups example. Using the security groups API
> directly is imperative ("put a firewall rule on this port that blocks this
> IP")

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

I'm curious, how would having Nova reviewers look at this have helped?


On Wed, Aug 6, 2014 at 5:41 PM, Jay Pipes  wrote:

> On 08/06/2014 07:08 PM, CARVER, PAUL wrote:
>
>> On Aug 6, 2014, at 2:01 PM, Mohammad Banikazemi > m...@us.ibm.com>>
>>   wrote:
>>
>>  Yes, indeed.
>>> I do not want to be over dramatic but the discussion on the original
>>> "Group
>>> Based Policy and the way forward" thread is nothing short of
>>> heartbreaking.
>>> After months and months of discussions, three presentations at the past
>>> three
>>> summits, a design session at the last summit, and (most relevant to this
>>> thread) the approval of the spec, why are we talking about the merits of
>>> the
>>> work now?
>>>
>>> I understand if people think this is not a good idea or this is not a
>>> good
>>> time. What I do not understand is why these concerns were not raised
>>> clearly
>>> and openly earlier.
>>>
>>
>> I have to agree here. I'm not sure whether my organization needs GBP or
>> not.
>> It's certainly not our top priority for Neutron given a variety of other
>> more
>> important functional gaps. However, I saw their demo at the summit and it
>> was
>> clear that a lot of work had gone into it even before Icehouse. From the
>> demo
>> it was clearly a useful enhancement to Neutron even if it wasn't at the
>> top
>> of my priority list.
>>
>> For people to be asking to justify the "why" this far into the Juno cycle
>> when the spec was approved and the code was demoed at the summit really
>> brings the OpenStack process into question. It's one thing to discuss
>> technical merits of contributions but it's totally different to pull the
>> rug
>> out from under a group of contributors at the last minute after such a
>> long
>> period of development, discussion, and demo.
>>
>> Seeing this sort of last minute rejection of a contribution after so much
>> time has been invested in it could very easily have a chilling effect on
>> contributors.
>>
>
> I don't disagree with you, Paul.
>
> I blame myself for not paying the attention I should have to this earlier
> in the process.
>
> FWIW, I had a good conversation with Sumit and Kevin on #openstack-neutron
> this afternoon about this particular topic. We agree on some things;
> disagree on others.
>
> Bottom line, I go back to what I said in a previous email: the Nova and
> Neutron development teams need to do a much better job in being directly
> involved in each other's spec discussions and design conversations.
>
> Best,
> -jay
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

[Moving my reply to the correct thread as someone changed the subject line.
Attempt 2 :) ]


On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton  wrote:

> >Given this information I don't see any reason why the backend system
> couldn't do enforcement at the logical router and if it did so neither
> parties would know.
>
> With security groups you are specifying that nothing can contact these
> devices on those ports unless they come from the allowed IP addresses. If
> you tried to enforce this at the router you would be violating that
> specification because devices in the same subnet would be able to
> communicate on those blocked ports.
>
>
Sure, though this is a problem of where you are doing your enforcement. If
the backend system chooses to implement this optimization in this fashion
(which was the example you gave previously [1]). Then, if the topology
changes, i.e adding a port to the same network with conflicting security
group rules, the backend system can no longer optimize in this same fashion
at the router level and a more fine grain filtering will need to be done.
How would this be any different with group based policy?

[1] - With the latter, a mapping driver could determine that communication
between these two hosts can be prevented by using an ACL on a router or a
switch, which doesn't violate the user's intent and buys a performance
improvement and works with ports that don't support security groups.




>
> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen  wrote:
>
>>
>> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton  wrote:
>>
>>> By working at the port level you have already eliminated your ability to
>>> implement the filtering at different components of the network. They now
>>> need to be implemented in stateful rules at the port level and the device
>>> has to support security groups.
>>>
>>
>>
>> Lets take this example where we setup a 2 tier app with web-servers and
>> db-servers that are connected on two different networks attached to a
>> router. We add a security group rules such that web-servers can talk to
>> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers from
>> anywhere.
>>
>> neutron net-create web_net
>> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>>
>> neutron net-create db_net
>> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>>
>> neutron router-create myrouter
>> neutron router-interface-add myrouter web_subnet
>> neutron router-interface-add myrouter db_subnet
>>
>> neutron security-group-create  web-servers;
>> neutron security-group-create db-servers;
>>
>> # add rule to allow web members to talk to the db-servers on TCP 3306 for
>> their db connection;
>> neutron security-group-rule-create --protocol TCP --port-range-min 3306
>> --port-range-max 3306 --remote-group-id web-servers db-servers
>>
>> # add rule to allow TCP 80 into the web-server sg
>> neutron security-group-rule-create --protocol TCP --port-range-min 80
>> --port-range-max 80 web-servers db-servers
>>
>> # create some ports with desired security profiles.
>> neutron port-create  --security-group web-servers web_net
>> neutron port-create  --security-group web-servers web_net
>>
>> neutron port-create  --security-group db-servers db_net
>> neutron port-create  --security-group db-servers db_net
>>
>>
>> Now to your point:
>>
>> By working at the port level you have already eliminated your ability to
>>> implement the filtering at different components of the network. They now
>>> need to be implemented in stateful rules at the port level and the device
>>> has to support security groups.
>>>
>>>
>> Given this information I don't see any reason why the backend system
>> couldn't do enforcement at the logical router and if it did so neither
>> parties would know. The backend system should have the full graph of
>> everything and be able to do enforcement optimizations where ever it likes.
>>
>> btw: I say the enforcement could be done on the logical router though the
>> backend system could also do this on the physical fabic as well if it
>> wanted to as it should also know that graph. No?
>>
>>
>>>
>>> On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen 
>>> wrote:
>>>



 On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton 
 wrote:

> >I believe the referential security group rules solve this problem
> (unless I'm not understanding):
>
> I think the disconnect is that you are comparing the way to current
> mapping driver implements things for the reference implementation with the
> existing APIs. Under this light, it's not going to look like there is a
> point to this code being in Neutron since, as you said, the abstraction
> could happen at a client. However, this changes once new mapping drivers
> can be added that implement things differently.
>
> Let's take the security groups example. Using the security groups API
> directly is imperative ("put a firewall rule on this port that blocks this
> IP") c

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

[Moving my reply to the correct thread as someone changed the subject line.]


On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton  wrote:

> >Given this information I don't see any reason why the backend system
> couldn't do enforcement at the logical router and if it did so neither
> parties would know.
>
> With security groups you are specifying that nothing can contact these
> devices on those ports unless they come from the allowed IP addresses. If
> you tried to enforce this at the router you would be violating that
> specification because devices in the same subnet would be able to
> communicate on those blocked ports.
>
>
Sure, though this is a problem of where you are doing your enforcement. If
the backend system chooses to implement this optimization in this fashion
(which was the example you gave previously [1]). Then, if the topology
changes, i.e adding a port to the same network with conflicting security
group rules, the backend system can no longer optimize in this same fashion
at the router level and a more fine grain filtering will need to be done.
How would this be any different with group based policy?

[1] - With the latter, a mapping driver could determine that communication
between these two hosts can be prevented by using an ACL on a router or a
switch, which doesn't violate the user's intent and buys a performance
improvement and works with ports that don't support security groups.


>
> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen  wrote:
>
>>
>> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton  wrote:
>>
>>> By working at the port level you have already eliminated your ability to
>>> implement the filtering at different components of the network. They now
>>> need to be implemented in stateful rules at the port level and the device
>>> has to support security groups.
>>>
>>
>>
>> Lets take this example where we setup a 2 tier app with web-servers and
>> db-servers that are connected on two different networks attached to a
>> router. We add a security group rules such that web-servers can talk to
>> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers from
>> anywhere.
>>
>> neutron net-create web_net
>> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>>
>> neutron net-create db_net
>> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>>
>> neutron router-create myrouter
>> neutron router-interface-add myrouter web_subnet
>> neutron router-interface-add myrouter db_subnet
>>
>> neutron security-group-create  web-servers;
>> neutron security-group-create db-servers;
>>
>> # add rule to allow web members to talk to the db-servers on TCP 3306 for
>> their db connection;
>> neutron security-group-rule-create --protocol TCP --port-range-min 3306
>> --port-range-max 3306 --remote-group-id web-servers db-servers
>>
>> # add rule to allow TCP 80 into the web-server sg
>> neutron security-group-rule-create --protocol TCP --port-range-min 80
>> --port-range-max 80 web-servers db-servers
>>
>> # create some ports with desired security profiles.
>> neutron port-create  --security-group web-servers web_net
>> neutron port-create  --security-group web-servers web_net
>>
>> neutron port-create  --security-group db-servers db_net
>> neutron port-create  --security-group db-servers db_net
>>
>>
>> Now to your point:
>>
>> By working at the port level you have already eliminated your ability to
>>> implement the filtering at different components of the network. They now
>>> need to be implemented in stateful rules at the port level and the device
>>> has to support security groups.
>>>
>>>
>> Given this information I don't see any reason why the backend system
>> couldn't do enforcement at the logical router and if it did so neither
>> parties would know. The backend system should have the full graph of
>> everything and be able to do enforcement optimizations where ever it likes.
>>
>> btw: I say the enforcement could be done on the logical router though the
>> backend system could also do this on the physical fabic as well if it
>> wanted to as it should also know that graph. No?
>>
>>
>>>
>>> On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen 
>>> wrote:
>>>



 On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton 
 wrote:

> >I believe the referential security group rules solve this problem
> (unless I'm not understanding):
>
> I think the disconnect is that you are comparing the way to current
> mapping driver implements things for the reference implementation with the
> existing APIs. Under this light, it's not going to look like there is a
> point to this code being in Neutron since, as you said, the abstraction
> could happen at a client. However, this changes once new mapping drivers
> can be added that implement things differently.
>
> Let's take the security groups example. Using the security groups API
> directly is imperative ("put a firewall rule on this port that blocks this
> IP") compared to a hig

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

That's the point. By using security groups, you are forcing a certain kind
of enforcement that must be honored and might not be necessary if the
original intent was just to isolate between groups. In the example you
gave, it cannot be implemented on the router without violating the
constraints of the security group.


On Wed, Aug 6, 2014 at 5:39 PM, Aaron Rosen  wrote:

>
>
>
> On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton  wrote:
>
>> >Given this information I don't see any reason why the backend system
>> couldn't do enforcement at the logical router and if it did so neither
>> parties would know.
>>
>> With security groups you are specifying that nothing can contact these
>> devices on those ports unless they come from the allowed IP addresses. If
>> you tried to enforce this at the router you would be violating that
>> specification because devices in the same subnet would be able to
>> communicate on those blocked ports.
>>
>
> Sure, though this is a problem of where you are doing your enforcement. If
> the backend system chooses to implement this optimization in this fashion
> (which was the example you gave previously [1]). Then, if the topology
> changes, i.e adding a port to the same network with conflicting security
> group rules, the backend system can no longer optimize in this same fashion
> at the router level and a more fine grain filtering will need to be done.
> How would this be any different with group based policy?
>
>
> [1] - With the latter, a mapping driver could determine that communication
> between these two hosts can be prevented by using an ACL on a router or a
> switch, which doesn't violate the user's intent and buys a performance
> improvement and works with ports that don't support security groups.
>
> states
>
>
>
>>
>>
>> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen 
>> wrote:
>>
>>>
>>> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton  wrote:
>>>
 By working at the port level you have already eliminated your ability
 to implement the filtering at different components of the network. They now
 need to be implemented in stateful rules at the port level and the device
 has to support security groups.

>>>
>>>
>>> Lets take this example where we setup a 2 tier app with web-servers and
>>> db-servers that are connected on two different networks attached to a
>>> router. We add a security group rules such that web-servers can talk to
>>> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers from
>>> anywhere.
>>>
>>> neutron net-create web_net
>>> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>>>
>>> neutron net-create db_net
>>> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>>>
>>> neutron router-create myrouter
>>> neutron router-interface-add myrouter web_subnet
>>> neutron router-interface-add myrouter db_subnet
>>>
>>> neutron security-group-create  web-servers;
>>> neutron security-group-create db-servers;
>>>
>>> # add rule to allow web members to talk to the db-servers on TCP 3306
>>> for their db connection;
>>> neutron security-group-rule-create --protocol TCP --port-range-min 3306
>>> --port-range-max 3306 --remote-group-id web-servers db-servers
>>>
>>> # add rule to allow TCP 80 into the web-server sg
>>> neutron security-group-rule-create --protocol TCP --port-range-min 80
>>> --port-range-max 80 web-servers db-servers
>>>
>>> # create some ports with desired security profiles.
>>> neutron port-create  --security-group web-servers web_net
>>> neutron port-create  --security-group web-servers web_net
>>>
>>> neutron port-create  --security-group db-servers db_net
>>> neutron port-create  --security-group db-servers db_net
>>>
>>>
>>> Now to your point:
>>>
>>> By working at the port level you have already eliminated your ability to
 implement the filtering at different components of the network. They now
 need to be implemented in stateful rules at the port level and the device
 has to support security groups.


>>> Given this information I don't see any reason why the backend system
>>> couldn't do enforcement at the logical router and if it did so neither
>>> parties would know. The backend system should have the full graph of
>>> everything and be able to do enforcement optimizations where ever it likes.
>>>
>>> btw: I say the enforcement could be done on the logical router though
>>> the backend system could also do this on the physical fabic as well if it
>>> wanted to as it should also know that graph. No?
>>>
>>>

 On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen 
 wrote:

>
>
>
> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton 
> wrote:
>
>> >I believe the referential security group rules solve this problem
>> (unless I'm not understanding):
>>
>> I think the disconnect is that you are comparing the way to current
>> mapping driver implements things for the reference implementation with 
>> the
>> existing APIs. Under thi

Re: [openstack-dev] [git-review] Supporting development in local branches

2014-08-06 Thread Zane Bitter


On 06/08/14 18:12, Yuriy Taraday wrote:

2. since hacking takes tremendous amount of time (you're doing a Cool
>>Feature (tm), nothing less) you need to update some code from master, so
>>you're just merging master in to your branch (i.e. using Git as you'd
>>use it normally);
>>

>
>This is not how I'd use Git normally.


Well, as per Git author, that's how you should do with not-CVS. You have
cheap merges - use them instead of erasing parts of history.


This is just not true.

http://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg39091.html

Choice quotes from the author of Git:

* 'People can (and probably should) rebase their _private_ trees'
* 'you can go wild on the "git rebase" thing'
* 'we use "git rebase" etc while we work on our problems.'
* '"git rebase" is not wrong.'

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)


On 08/06/2014 07:08 PM, CARVER, PAUL wrote:

On Aug 6, 2014, at 2:01 PM, Mohammad Banikazemi 
mailto:m...@us.ibm.com>>
  wrote:


Yes, indeed.
I do not want to be over dramatic but the discussion on the original "Group
Based Policy and the way forward" thread is nothing short of heartbreaking.
After months and months of discussions, three presentations at the past three
summits, a design session at the last summit, and (most relevant to this
thread) the approval of the spec, why are we talking about the merits of the
work now?

I understand if people think this is not a good idea or this is not a good
time. What I do not understand is why these concerns were not raised clearly
and openly earlier.


I have to agree here. I'm not sure whether my organization needs GBP or not.
It's certainly not our top priority for Neutron given a variety of other more
important functional gaps. However, I saw their demo at the summit and it was
clear that a lot of work had gone into it even before Icehouse. From the demo
it was clearly a useful enhancement to Neutron even if it wasn't at the top
of my priority list.

For people to be asking to justify the "why" this far into the Juno cycle
when the spec was approved and the code was demoed at the summit really
brings the OpenStack process into question. It's one thing to discuss
technical merits of contributions but it's totally different to pull the rug
out from under a group of contributors at the last minute after such a long
period of development, discussion, and demo.

Seeing this sort of last minute rejection of a contribution after so much
time has been invested in it could very easily have a chilling effect on
contributors.


I don't disagree with you, Paul.

I blame myself for not paying the attention I should have to this 
earlier in the process.


FWIW, I had a good conversation with Sumit and Kevin on 
#openstack-neutron this afternoon about this particular topic. We agree 
on some things; disagree on others.


Bottom line, I go back to what I said in a previous email: the Nova and 
Neutron development teams need to do a much better job in being directly 
involved in each other's spec discussions and design conversations.


Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

>
> This is probably not intentional from your part ,but your choice of words
> make it seem that you are deriding the efforts of the team behind this
> effort. While i may disagree technically here and there with their current
> design, it seems to me that the effort in question is rather broad based in
> terms of support (from multiple different organizations) and that the team
> has put a non trivial effort in making the effort public. I don't think we
> can characterize the team either as a "secret group" or a "small set".
>

You misread me completely, please refrain from making these comments: I
deride no-one.

I chose the word in reference to the Enlightenment movement, with emphasis
to breaking the traditional way of thinking (declarative vs imperative),
and I found that the analogy would stick, but apparently not.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On Wed, Aug 6, 2014 at 4:18 PM, Kevin Benton  wrote:

> >Given this information I don't see any reason why the backend system
> couldn't do enforcement at the logical router and if it did so neither
> parties would know.
>
> With security groups you are specifying that nothing can contact these
> devices on those ports unless they come from the allowed IP addresses. If
> you tried to enforce this at the router you would be violating that
> specification because devices in the same subnet would be able to
> communicate on those blocked ports.
>

Sure, though this is a problem of where you are doing your enforcement. If
the backend system chooses to implement this optimization in this fashion
(which was the example you gave previously [1]). Then, if the topology
changes, i.e adding a port to the same network with conflicting security
group rules, the backend system can no longer optimize in this same fashion
at the router level and a more fine grain filtering will need to be done.
How would this be any different with group based policy?


[1] - With the latter, a mapping driver could determine that communication
between these two hosts can be prevented by using an ACL on a router or a
switch, which doesn't violate the user's intent and buys a performance
improvement and works with ports that don't support security groups.

states



>
>
> On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen  wrote:
>
>>
>> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton  wrote:
>>
>>> By working at the port level you have already eliminated your ability to
>>> implement the filtering at different components of the network. They now
>>> need to be implemented in stateful rules at the port level and the device
>>> has to support security groups.
>>>
>>
>>
>> Lets take this example where we setup a 2 tier app with web-servers and
>> db-servers that are connected on two different networks attached to a
>> router. We add a security group rules such that web-servers can talk to
>> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers from
>> anywhere.
>>
>> neutron net-create web_net
>> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>>
>> neutron net-create db_net
>> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>>
>> neutron router-create myrouter
>> neutron router-interface-add myrouter web_subnet
>> neutron router-interface-add myrouter db_subnet
>>
>> neutron security-group-create  web-servers;
>> neutron security-group-create db-servers;
>>
>> # add rule to allow web members to talk to the db-servers on TCP 3306 for
>> their db connection;
>> neutron security-group-rule-create --protocol TCP --port-range-min 3306
>> --port-range-max 3306 --remote-group-id web-servers db-servers
>>
>> # add rule to allow TCP 80 into the web-server sg
>> neutron security-group-rule-create --protocol TCP --port-range-min 80
>> --port-range-max 80 web-servers db-servers
>>
>> # create some ports with desired security profiles.
>> neutron port-create  --security-group web-servers web_net
>> neutron port-create  --security-group web-servers web_net
>>
>> neutron port-create  --security-group db-servers db_net
>> neutron port-create  --security-group db-servers db_net
>>
>>
>> Now to your point:
>>
>> By working at the port level you have already eliminated your ability to
>>> implement the filtering at different components of the network. They now
>>> need to be implemented in stateful rules at the port level and the device
>>> has to support security groups.
>>>
>>>
>> Given this information I don't see any reason why the backend system
>> couldn't do enforcement at the logical router and if it did so neither
>> parties would know. The backend system should have the full graph of
>> everything and be able to do enforcement optimizations where ever it likes.
>>
>> btw: I say the enforcement could be done on the logical router though the
>> backend system could also do this on the physical fabic as well if it
>> wanted to as it should also know that graph. No?
>>
>>
>>>
>>> On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen 
>>> wrote:
>>>



 On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton 
 wrote:

> >I believe the referential security group rules solve this problem
> (unless I'm not understanding):
>
> I think the disconnect is that you are comparing the way to current
> mapping driver implements things for the reference implementation with the
> existing APIs. Under this light, it's not going to look like there is a
> point to this code being in Neutron since, as you said, the abstraction
> could happen at a client. However, this changes once new mapping drivers
> can be added that implement things differently.
>
> Let's take the security groups example. Using the security groups API
> directly is imperative ("put a firewall rule on this port that blocks this
> IP") compared to a higher level declarative abstraction ("make sure these
> two endpoi

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Pedro Marques

On Aug 6, 2014, at 3:56 PM, Armando M.  wrote:

> 
> On 6 August 2014 15:47, Kevin Benton  wrote:
> I think we should merge it and just prefix the API for now with 
> '/your_application_will_break_after_juno_if_you_use_this/' 
> 
> And you make your call based and what pros and cons exactly, If I am ask?
> 
> Let me start:
> 
> Option 1:
>   - pros
> - immediate delivery vehicle for consumption by operators

Since our collective goal is to maximize the benefits for OpenStack 
users/operators, that seems to be the winner.

>   - cons
> - code is burder from a number of standpoints (review, test, etc)

Any piece of code is a burden.

> 
> Option 2:
>   - pros
> - enable a small set of Illuminati to iterate faster

This is probably not intentional from your part ,but your choice of words make 
it seem that you are deriding the efforts of the team behind this effort. While 
i may disagree technically here and there with their current design, it seems 
to me that the effort in question is rather broad based in terms of support 
(from multiple different organizations) and that the team has put a non trivial 
effort in making the effort public. I don't think we can characterize the team 
either as a "secret group" or a "small set".

  Pedro.

>  
>   - cons
> - integration burden with other OpenStack projects (keystone, nova, 
> neutron, etc)
> 
> Cheers,
> Armando
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][third-party] Arista CI hits 10, 000 runs this morning

2014-08-06 Thread Salvatore Orlando

Congratulations!
And to celebrate this milestone, I would consider running something more
than ~280 api tests... perhaps also a few scenario tests?

Salvatore


On 7 August 2014 01:09, Sukhdev Kapur  wrote:

> Folks,
>
> Just wanted to share with you that Arista CI has been up and running 24x7
> since the beginning of this year with no down time.
>
> This morning it posted a vote on 10,000th Neutron patch
>
> cheers..
> -Sukhdev
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

>Given this information I don't see any reason why the backend system
couldn't do enforcement at the logical router and if it did so neither
parties would know.

With security groups you are specifying that nothing can contact these
devices on those ports unless they come from the allowed IP addresses. If
you tried to enforce this at the router you would be violating that
specification because devices in the same subnet would be able to
communicate on those blocked ports.

On Wed, Aug 6, 2014 at 5:00 PM, Aaron Rosen  wrote:

>
> On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton  wrote:
>
>> By working at the port level you have already eliminated your ability to
>> implement the filtering at different components of the network. They now
>> need to be implemented in stateful rules at the port level and the device
>> has to support security groups.
>>
>
>
> Lets take this example where we setup a 2 tier app with web-servers and
> db-servers that are connected on two different networks attached to a
> router. We add a security group rules such that web-servers can talk to
> db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers from
> anywhere.
>
> neutron net-create web_net
> neutron subnet-create --name web_subnet web_net 10.0.0.0/24
>
> neutron net-create db_net
> neutron subnet-create --name db_subnet db_net 10.2.0.0/24
>
> neutron router-create myrouter
> neutron router-interface-add myrouter web_subnet
> neutron router-interface-add myrouter db_subnet
>
> neutron security-group-create  web-servers;
> neutron security-group-create db-servers;
>
> # add rule to allow web members to talk to the db-servers on TCP 3306 for
> their db connection;
> neutron security-group-rule-create --protocol TCP --port-range-min 3306
> --port-range-max 3306 --remote-group-id web-servers db-servers
>
> # add rule to allow TCP 80 into the web-server sg
> neutron security-group-rule-create --protocol TCP --port-range-min 80
> --port-range-max 80 web-servers db-servers
>
> # create some ports with desired security profiles.
> neutron port-create  --security-group web-servers web_net
> neutron port-create  --security-group web-servers web_net
>
> neutron port-create  --security-group db-servers db_net
> neutron port-create  --security-group db-servers db_net
>
>
> Now to your point:
>
> By working at the port level you have already eliminated your ability to
>> implement the filtering at different components of the network. They now
>> need to be implemented in stateful rules at the port level and the device
>> has to support security groups.
>>
>>
> Given this information I don't see any reason why the backend system
> couldn't do enforcement at the logical router and if it did so neither
> parties would know. The backend system should have the full graph of
> everything and be able to do enforcement optimizations where ever it likes.
>
> btw: I say the enforcement could be done on the logical router though the
> backend system could also do this on the physical fabic as well if it
> wanted to as it should also know that graph. No?
>
>
>>
>> On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen 
>> wrote:
>>
>>>
>>>
>>>
>>> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:
>>>
 >I believe the referential security group rules solve this problem
 (unless I'm not understanding):

 I think the disconnect is that you are comparing the way to current
 mapping driver implements things for the reference implementation with the
 existing APIs. Under this light, it's not going to look like there is a
 point to this code being in Neutron since, as you said, the abstraction
 could happen at a client. However, this changes once new mapping drivers
 can be added that implement things differently.

 Let's take the security groups example. Using the security groups API
 directly is imperative ("put a firewall rule on this port that blocks this
 IP") compared to a higher level declarative abstraction ("make sure these
 two endpoints cannot communicate"). With the former, the ports must support
 security groups and there is nowhere except for the firewall rules on that
 port to implement it without violating the user's expectation. With the
 latter, a mapping driver could determine that communication between these
 two hosts can be prevented by using an ACL on a router or a switch, which
 doesn't violate the user's intent and buys a performance improvement and
 works with ports that don't support security groups.

 Group based policy is trying to move the requests into the declarative
 abstraction so optimizations like the one above can be made.

>>>
>>> Hi Kevin,
>>>
>>> Interesting points. Though, let me ask this. Why do we need to move to a
>>> declarative API abstraction in neutron in order to perform this
>>> optimization on the backend? For example, In the current neutron model say
>>> we want to create a port with a security group attache

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

2014-08-06 Thread Ed Hall

On Aug 5, 2014, at 4:52 PM, Joshua Harlow  wrote:

> I'm pretty sure yahoo is another case, with a large set of clusters on 
> nova-network still ;)
> 
> I believe we have been active in these discussions, although I'm unsure what 
> was discussed at the meetup (being that I had planned vacation, right now 
> actually). 
> 
> Anyways I think yahoo is fine with being a use case, but I can check when I 
> get back.
> 

tl;dr: we’re willing to be a use case, but our internal timeline is such that 
in all likelihood
this will be as a post-mortem.

We (Yahoo) have thousands of pets that need migrated as well as an unspecified
number of cattle. A “live" strategy is strongly preferred (I’m not saying 
“live" migration
since in our case it needs to be an in-place operation, not shuffling instances 
around).
But several seconds of network outage? No problem. Disabling VM 
creation/deletion,
or even the entire Nova API for a few hours? Well take the grumbling from our 
internal
teams. A suspend/snapshot/cold-migrate would be an absolute last resort, and 
frankly
could push back our aggressive migration timeline significantly.

We’re interested in Oleg Bondarev’s solution, and I’ve even made some 
suggestions
in review comments as to how it can be made more “live," but it’s clear there 
are a
number of objections the greater Nova community has for it. Chief among these 
are the
addition of code and an API to Nova for what is essentially a one-shot 
operation, inability
to deal with more complicated configurations, and reliance on features only 
available in a
fresh release of libvirt. (As it turns out, only the latter affects us, but 
we’re a bit of an outlier
in the community.) It’s still under consideration for us, even if the community 
rejects the
approach.

As an alternative, we’re looking at DB-to-DB translation, with a one-shot 
script run on
the compute nodes to move network taps. We’d actually worked this out back in 
the
Quantum/Folsom era but backed off due to OVS/device driver issues (don’t ask -- 
I still
get nightmares). This, of course, would require an API outage, and is a "big 
bang"
approach (one of the attractions of Oleg’s approach is that we can migrate a 
few low-
value instances and then examine results carefully before proceeding). But once 
again,
our solution is likely to be of limited interest -- flat network without DHCP, 
no routers or
floating IPs, unconventional (for OpenStack) use of VLANs -- though we’d be 
happy
to share once the dust settles.

-Ed Hall
edh...@yahoo-inc.com

> 
> On Aug 5, 2014, at 7:11 PM, "Joe Gordon"  wrote:
> 
>> 
>> On Aug 5, 2014 12:57 PM, "Jay Pipes"  wrote:
>> >
>> > On 08/05/2014 03:23 PM, Collins, Sean wrote:
>> >>
>> >> On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:
>> >>>
>> >>> However, I think the cost to providing that path far outweighs
>> >>> the benefit in the face of other things on our plate.
>> >>
>> >>
>> >> Perhaps those large operators that are hoping for a
>> >> Nova-Network->Neutron zero-downtime live migration, could dedicate
>> >> resources to this requirement? It is my direct experience that features
>> >> that are important to a large organization will require resources
>> >> from that very organization to be completed.
>> >
>> >
>> > Indeed, that's partly why I called out Metacloud in the original post, as 
>> > they were brought up as a deployer with this potential need. Please, if 
>> > there are any other shops that:
>> Perhaps I am not remembering all the details discussed at the nova 
>> mid-cycle, but Metacloud was brought up as an example company uses nova 
>> network and not neutron, not as a company that needs live migration. And 
>> that getting them to move to neutron would be a good litmus test for 
>> nova-network performance parity, something that is very hard to do in the 
>> gate.   But that was all said without any folks from Metacloud in the room, 
>> so we may both be wrong.
>> 
>> >
>> > * Currently deploy nova-network
>> > * Need to move to Neutron
>> > * Their tenants cannot tolerate any downtime due to a cold migration
>> >
>> > Please do comment on this thread and speak up.
>> >
>> > Best,
>> > -jay
>> >
>> >
>> > ___
>> > OpenStack-dev mailing list
>> > OpenStack-dev@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron][third-party] Arista CI hits 10, 000 runs this morning

2014-08-06 Thread Sukhdev Kapur

Folks,

Just wanted to share with you that Arista CI has been up and running 24x7
since the beginning of this year with no down time.

This morning it posted a vote on 10,000th Neutron patch

cheers..
-Sukhdev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

2014-08-06 Thread CARVER, PAUL

On Aug 6, 2014, at 2:01 PM, Mohammad Banikazemi 
mailto:m...@us.ibm.com>>
 wrote:

>Yes, indeed.
>I do not want to be over dramatic but the discussion on the original "Group
>Based Policy and the way forward" thread is nothing short of heartbreaking.
>After months and months of discussions, three presentations at the past three
>summits, a design session at the last summit, and (most relevant to this
>thread) the approval of the spec, why are we talking about the merits of the
>work now?
>
>I understand if people think this is not a good idea or this is not a good
>time. What I do not understand is why these concerns were not raised clearly
>and openly earlier.

I have to agree here. I'm not sure whether my organization needs GBP or not.
It's certainly not our top priority for Neutron given a variety of other more
important functional gaps. However, I saw their demo at the summit and it was
clear that a lot of work had gone into it even before Icehouse. From the demo
it was clearly a useful enhancement to Neutron even if it wasn't at the top
of my priority list.

For people to be asking to justify the "why" this far into the Juno cycle
when the spec was approved and the code was demoed at the summit really
brings the OpenStack process into question. It's one thing to discuss
technical merits of contributions but it's totally different to pull the rug
out from under a group of contributors at the last minute after such a long
period of development, discussion, and demo.

Seeing this sort of last minute rejection of a contribution after so much
time has been invested in it could very easily have a chilling effect on
contributors.

~

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Salvatore Orlando

I was asked beforehand what I mean with

* consider GBP an 'experimental' V3 tenant API (this was mentioned
somewhere in this thread) and treat it accordingly

The group based policies, although implemented as a service plugin, are
quite different from the ones we have now. Things like firewall, load
balancer, etc. add something that will be run alongside the "core" API.
This service plugin instead define a different kind of abstractive
(declarative vs imperative as it has been mentioned several times) and it
won't be wrong to say it can replace the core API. To this aim I think it's
fair to consider it an experimentation around a new tenant API.

In this hypothesis, how should this be treated? The decision ultimately
does not lie with me.
I would merely point out that:
1) The neutron core team should not stop innovation. If a consistent part
of the community feels that a declarative base model is that way forward
and we should experiment with it, than we should do everything we can to
help this part of the community.
2) On the other hand, innovation and experimentation should not deprive the
core team of significant resources which should be directed first and
foremost to address the areas identified by the TC as in need of improvement
In the light of the above, and reflecting on the ultimate question which is
whether this code should be merged or not. Allow me to say that my final
thought is that I don't care whether is merged in the main neutron repo or
somewhere else, as long as it does not constitute additional burden for the
core team in terms of reviews, maintainability, gate failures (it won't be
the first time I or other core team members had to chase failures for some
service plugins digging into logs), and future design decisions (ie: having
to ask the question - "what about group policy" for every future API
decision)

Salvatore

PS: did I make post #100?

On 7 August 2014 00:47, Kevin Benton  wrote:

> I think we should merge it and just prefix the API for now with
> '/your_application_will_break_after_juno_if_you_use_this/'
>  On Aug 6, 2014 4:41 PM, "Armando M."  wrote:
>
>> This thread is moving so fast I can't keep up!
>>
>> The fact that troubles me is that I am unable to grasp how we move
>> forward, which was the point of this thread to start with. It seems we have
>> 2 options:
>>
>> - We make GBP to merge as is, in the Neutron tree, with some minor
>> revision (e.g. naming?);
>> - We make GBP a stackforge project, that integrates with Neutron in some
>> shape or form;
>>
>> Another option, might be something in between, where GBP is in tree, but
>> in some sort of experimental staging area (even though I am not sure how
>> well baked this idea is).
>>
>> Now, as a community we all need make a decision; arguing about the fact
>> that the blueprint was approved is pointless. As a matter of fact, I think
>> that blueprint should be approved, if and only if the code has landed
>> completely, but I digress!
>>
>> Let's together come up with pros and cons of each approach and come up
>> with an informed decision.
>>
>> Just reading free form text, how are we expected to do that? At least I
>> can't!
>>
>> My 2c.
>> Armando
>>
>>
>> On 6 August 2014 15:03, Aaron Rosen  wrote:
>>
>>>
>>>
>>>
>>> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:
>>>
 >I believe the referential security group rules solve this problem
 (unless I'm not understanding):

 I think the disconnect is that you are comparing the way to current
 mapping driver implements things for the reference implementation with the
 existing APIs. Under this light, it's not going to look like there is a
 point to this code being in Neutron since, as you said, the abstraction
 could happen at a client. However, this changes once new mapping drivers
 can be added that implement things differently.

 Let's take the security groups example. Using the security groups API
 directly is imperative ("put a firewall rule on this port that blocks this
 IP") compared to a higher level declarative abstraction ("make sure these
 two endpoints cannot communicate"). With the former, the ports must support
 security groups and there is nowhere except for the firewall rules on that
 port to implement it without violating the user's expectation. With the
 latter, a mapping driver could determine that communication between these
 two hosts can be prevented by using an ACL on a router or a switch, which
 doesn't violate the user's intent and buys a performance improvement and
 works with ports that don't support security groups.

 Group based policy is trying to move the requests into the declarative
 abstraction so optimizations like the one above can be made.

>>>
>>> Hi Kevin,
>>>
>>> Interesting points. Though, let me ask this. Why do we need to move to a
>>> declarative API abstraction in neutron in order to perform this
>>> optimization on the back

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On Wed, Aug 6, 2014 at 3:35 PM, Kevin Benton  wrote:

> By working at the port level you have already eliminated your ability to
> implement the filtering at different components of the network. They now
> need to be implemented in stateful rules at the port level and the device
> has to support security groups.
>


Lets take this example where we setup a 2 tier app with web-servers and
db-servers that are connected on two different networks attached to a
router. We add a security group rules such that web-servers can talk to
db-servers on tcp:3306 and a rule to allow tcp:80 into the web-servers from
anywhere.

neutron net-create web_net
neutron subnet-create --name web_subnet web_net 10.0.0.0/24

neutron net-create db_net
neutron subnet-create --name db_subnet db_net 10.2.0.0/24

neutron router-create myrouter
neutron router-interface-add myrouter web_subnet
neutron router-interface-add myrouter db_subnet

neutron security-group-create  web-servers;
neutron security-group-create db-servers;

# add rule to allow web members to talk to the db-servers on TCP 3306 for
their db connection;
neutron security-group-rule-create --protocol TCP --port-range-min 3306
--port-range-max 3306 --remote-group-id web-servers db-servers

# add rule to allow TCP 80 into the web-server sg
neutron security-group-rule-create --protocol TCP --port-range-min 80
--port-range-max 80 web-servers db-servers

# create some ports with desired security profiles.
neutron port-create  --security-group web-servers web_net
neutron port-create  --security-group web-servers web_net

neutron port-create  --security-group db-servers db_net
neutron port-create  --security-group db-servers db_net


Now to your point:

By working at the port level you have already eliminated your ability to
> implement the filtering at different components of the network. They now
> need to be implemented in stateful rules at the port level and the device
> has to support security groups.
>
>
Given this information I don't see any reason why the backend system
couldn't do enforcement at the logical router and if it did so neither
parties would know. The backend system should have the full graph of
everything and be able to do enforcement optimizations where ever it likes.

btw: I say the enforcement could be done on the logical router though the
backend system could also do this on the physical fabic as well if it
wanted to as it should also know that graph. No?


>
> On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen  wrote:
>
>>
>>
>>
>> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:
>>
>>> >I believe the referential security group rules solve this problem
>>> (unless I'm not understanding):
>>>
>>> I think the disconnect is that you are comparing the way to current
>>> mapping driver implements things for the reference implementation with the
>>> existing APIs. Under this light, it's not going to look like there is a
>>> point to this code being in Neutron since, as you said, the abstraction
>>> could happen at a client. However, this changes once new mapping drivers
>>> can be added that implement things differently.
>>>
>>> Let's take the security groups example. Using the security groups API
>>> directly is imperative ("put a firewall rule on this port that blocks this
>>> IP") compared to a higher level declarative abstraction ("make sure these
>>> two endpoints cannot communicate"). With the former, the ports must support
>>> security groups and there is nowhere except for the firewall rules on that
>>> port to implement it without violating the user's expectation. With the
>>> latter, a mapping driver could determine that communication between these
>>> two hosts can be prevented by using an ACL on a router or a switch, which
>>> doesn't violate the user's intent and buys a performance improvement and
>>> works with ports that don't support security groups.
>>>
>>> Group based policy is trying to move the requests into the declarative
>>> abstraction so optimizations like the one above can be made.
>>>
>>
>> Hi Kevin,
>>
>> Interesting points. Though, let me ask this. Why do we need to move to a
>> declarative API abstraction in neutron in order to perform this
>> optimization on the backend? For example, In the current neutron model say
>> we want to create a port with a security group attached to it called web
>> that allows TCP:80 in and members who are in a security group called
>> database. From this mapping I fail to see how it's really any different
>> from the declarative model? The ports in neutron are logical abstractions
>> and the backend system could be implemented in order to determine that the
>> communication between these two hosts could be prevented by using an ACL on
>> a router or switch as well.
>>
>> Best,
>>
>> Aaron
>>
>>
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> _

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On 6 August 2014 15:47, Kevin Benton  wrote:

> I think we should merge it and just prefix the API for now with
> '/your_application_will_break_after_juno_if_you_use_this/'
>

And you make your call based and what pros and cons exactly, If I am ask?

Let me start:

Option 1:
  - pros
- immediate delivery vehicle for consumption by operators
  - cons
- code is burder from a number of standpoints (review, test, etc)

Option 2:
  - pros
- enable a small set of Illuminati to iterate faster
  - cons
- integration burden with other OpenStack projects (keystone, nova,
neutron, etc)

Cheers,
Armando
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Edgar Magana

Ivar,

You are totally right. I just want to clarify that I haven’t express any 
disagreement on the plugin, I actually (as Sumit mentioned) +2 this patch 
before. I just pointed our that I have expressed previously that GBP should use 
the standard Policy Terminology. I did not push hard enough at the right time 
because I consider it a minor thing but Jay has a valid point and I think it 
has to be reconsidered IMHO.

Let’s be careful with our public statements!

Edgar

From: Ivar Lazzaro mailto:ivarlazz...@gmail.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Date: Wednesday, August 6, 2014 at 2:52 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way forward

Edgar,

Actually, As Pedro said, I think that the time for discussing these kind of 
concerns was the BP approval. The fact that it has been approved after many 
proposals and reviews means that a community effort wanted the GBP to be 
implemented in this release cycle the way it was presented at that time.

With this, I absolutely don't want to say that you should not express your 
disagreement! I'm just saying that it should be expressed differently (a BP to 
propose your model in K?). Otherwise, the whole BP process becomes just 
pointless.

Meanwhile, imho, blocking the patch feels really unfair.

Ivar.

On Aug 6, 2014 11:00 PM, "Edgar Magana" 
mailto:edgar.mag...@workday.com>> wrote:
Ivar,

Of course and this is why we are having this conversation, in order to merge 
our different opinions.

Edgar

From: Ivar Lazzaro mailto:ivarlazz...@gmail.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Date: Wednesday, August 6, 2014 at 1:41 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way forward

Hi Edgar,

Actually, I think that other reviewers saw that name clash, and still thought 
it was ok to use the same terminology in such a different context.
BP reviews are a community effort right? So of course someones' idea may be 
different from yours.

Regards,
Ivar.

On Wed, Aug 6, 2014 at 10:29 PM, Edgar Magana 
mailto:edgar.mag...@workday.com>> wrote:
Basically, I am admitting that I did not catch in my review the part of
the endpoint term that Jay was pointing out.

Edgar

On 8/6/14, 11:32 AM, "Sumit Naiksatam" 
mailto:sumitnaiksa...@gmail.com>> wrote:

>Not sure what you are talking about? You claim now that you had
>suggestion which was not considered, yet you +2'ed a patch, by stating
>that "All looks good to me!".
>
>On Wed, Aug 6, 2014 at 11:19 AM, Edgar Magana 
>mailto:edgar.mag...@workday.com>>
>wrote:
>> That is the beauty of the open source projects, there is always a
>>smartest
>> reviewer catching out the facts that you don¹t.
>>
>> Edgar
>>
>> On 8/6/14, 10:55 AM, "Sumit Naiksatam" 
>> mailto:sumitnaiksa...@gmail.com>> wrote:
>>
>>>Edgar, you seemed to have +2'ed this patch on July 2nd [1]:
>>>
>>>"
>>>Edgar Magana
>>>Jul 2 8:42 AM
>>>
>>>Patch Set 13: Code-Review+2
>>>
>>>All looks good to me! I am not approving yet because Nachi was also
>>>reviewing this code and I would like to see his opinion as well.
>>>"
>>>
>>>That would suggest that you were happy with what was in it. I don't
>>>see anything in the review comments that suggests otherwise.
>>>
>>>[1]  https://review.openstack.org/#/c/95900/
>>>
>>>On Wed, Aug 6, 2014 at 10:39 AM, Edgar Magana 
>>>mailto:edgar.mag...@workday.com>>
>>>wrote:
 This is the consequence of a proposal that is not following the
standardized
 terminology (IETF - RFC) for any Policy-based System:
 http://tools.ietf.org/html/rfc3198

 Well, I did bring  this point during the Hong Kong Summit but as you
can see
 my comments were totally ignored:

https://docs.google.com/document/d/1ZbOFxAoibZbJmDWx1oOrOsDcov6Cuom5aaB
Ir
upCD9E/edit

 I clearly saw this kind of issues coming. Let me quote myself what I
 suggested: "For instance: "endpoints" should be "enforcement point"

 I do not understand why GBP did not include this suggestionŠ

 Edgar

 From: Kevin Benton mailto:blak...@gmail.com>>
 Reply-To: "OpenStack Development Mailing List (not for usage
questions)"
 mailto:openstack-dev@lists.openstack.org>>
 Date: Wednesday, August 6, 2014 at 10:22 AM
 To: "OpenStack Development Mailing List (not for usage questions)"
 mailto:openstack-dev@lists.openstack.org>>

 Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way
 forward

 What I was referring to was also not Keystone's definition of an
endpoint.
 It's almost as if the term has many uses and was not invente

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

I would reword that to:

'/your_application_may_break_after_juno_if_you_use_this/'

in the event of the possibility that it doesn't break. ;-)

On Wed, Aug 6, 2014 at 3:47 PM, Kevin Benton  wrote:
> I think we should merge it and just prefix the API for now with
> '/your_application_will_break_after_juno_if_you_use_this/'
>
> On Aug 6, 2014 4:41 PM, "Armando M."  wrote:
>>
>> This thread is moving so fast I can't keep up!
>>
>> The fact that troubles me is that I am unable to grasp how we move
>> forward, which was the point of this thread to start with. It seems we have
>> 2 options:
>>
>> - We make GBP to merge as is, in the Neutron tree, with some minor
>> revision (e.g. naming?);
>> - We make GBP a stackforge project, that integrates with Neutron in some
>> shape or form;
>>
>> Another option, might be something in between, where GBP is in tree, but
>> in some sort of experimental staging area (even though I am not sure how
>> well baked this idea is).
>>
>> Now, as a community we all need make a decision; arguing about the fact
>> that the blueprint was approved is pointless. As a matter of fact, I think
>> that blueprint should be approved, if and only if the code has landed
>> completely, but I digress!
>>
>> Let's together come up with pros and cons of each approach and come up
>> with an informed decision.
>>
>> Just reading free form text, how are we expected to do that? At least I
>> can't!
>>
>> My 2c.
>> Armando
>>
>>
>> On 6 August 2014 15:03, Aaron Rosen  wrote:
>>>
>>>
>>>
>>>
>>> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:

 >I believe the referential security group rules solve this problem
 > (unless I'm not understanding):

 I think the disconnect is that you are comparing the way to current
 mapping driver implements things for the reference implementation with the
 existing APIs. Under this light, it's not going to look like there is a
 point to this code being in Neutron since, as you said, the abstraction
 could happen at a client. However, this changes once new mapping drivers 
 can
 be added that implement things differently.

 Let's take the security groups example. Using the security groups API
 directly is imperative ("put a firewall rule on this port that blocks this
 IP") compared to a higher level declarative abstraction ("make sure these
 two endpoints cannot communicate"). With the former, the ports must support
 security groups and there is nowhere except for the firewall rules on that
 port to implement it without violating the user's expectation. With the
 latter, a mapping driver could determine that communication between these
 two hosts can be prevented by using an ACL on a router or a switch, which
 doesn't violate the user's intent and buys a performance improvement and
 works with ports that don't support security groups.

 Group based policy is trying to move the requests into the declarative
 abstraction so optimizations like the one above can be made.
>>>
>>>
>>> Hi Kevin,
>>>
>>> Interesting points. Though, let me ask this. Why do we need to move to a
>>> declarative API abstraction in neutron in order to perform this optimization
>>> on the backend? For example, In the current neutron model say we want to
>>> create a port with a security group attached to it called web that allows
>>> TCP:80 in and members who are in a security group called database. From this
>>> mapping I fail to see how it's really any different from the declarative
>>> model? The ports in neutron are logical abstractions and the backend system
>>> could be implemented in order to determine that the communication between
>>> these two hosts could be prevented by using an ACL on a router or switch as
>>> well.
>>>
>>> Best,
>>>
>>> Aaron
>>>


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

>>>
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

I think we should merge it and just prefix the API for now with
'/your_application_will_break_after_juno_if_you_use_this/'
 On Aug 6, 2014 4:41 PM, "Armando M."  wrote:

> This thread is moving so fast I can't keep up!
>
> The fact that troubles me is that I am unable to grasp how we move
> forward, which was the point of this thread to start with. It seems we have
> 2 options:
>
> - We make GBP to merge as is, in the Neutron tree, with some minor
> revision (e.g. naming?);
> - We make GBP a stackforge project, that integrates with Neutron in some
> shape or form;
>
> Another option, might be something in between, where GBP is in tree, but
> in some sort of experimental staging area (even though I am not sure how
> well baked this idea is).
>
> Now, as a community we all need make a decision; arguing about the fact
> that the blueprint was approved is pointless. As a matter of fact, I think
> that blueprint should be approved, if and only if the code has landed
> completely, but I digress!
>
> Let's together come up with pros and cons of each approach and come up
> with an informed decision.
>
> Just reading free form text, how are we expected to do that? At least I
> can't!
>
> My 2c.
> Armando
>
>
> On 6 August 2014 15:03, Aaron Rosen  wrote:
>
>>
>>
>>
>> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:
>>
>>> >I believe the referential security group rules solve this problem
>>> (unless I'm not understanding):
>>>
>>> I think the disconnect is that you are comparing the way to current
>>> mapping driver implements things for the reference implementation with the
>>> existing APIs. Under this light, it's not going to look like there is a
>>> point to this code being in Neutron since, as you said, the abstraction
>>> could happen at a client. However, this changes once new mapping drivers
>>> can be added that implement things differently.
>>>
>>> Let's take the security groups example. Using the security groups API
>>> directly is imperative ("put a firewall rule on this port that blocks this
>>> IP") compared to a higher level declarative abstraction ("make sure these
>>> two endpoints cannot communicate"). With the former, the ports must support
>>> security groups and there is nowhere except for the firewall rules on that
>>> port to implement it without violating the user's expectation. With the
>>> latter, a mapping driver could determine that communication between these
>>> two hosts can be prevented by using an ACL on a router or a switch, which
>>> doesn't violate the user's intent and buys a performance improvement and
>>> works with ports that don't support security groups.
>>>
>>> Group based policy is trying to move the requests into the declarative
>>> abstraction so optimizations like the one above can be made.
>>>
>>
>> Hi Kevin,
>>
>> Interesting points. Though, let me ask this. Why do we need to move to a
>> declarative API abstraction in neutron in order to perform this
>> optimization on the backend? For example, In the current neutron model say
>> we want to create a port with a security group attached to it called web
>> that allows TCP:80 in and members who are in a security group called
>> database. From this mapping I fail to see how it's really any different
>> from the declarative model? The ports in neutron are logical abstractions
>> and the backend system could be implemented in order to determine that the
>> communication between these two hosts could be prevented by using an ACL on
>> a router or switch as well.
>>
>> Best,
>>
>> Aaron
>>
>>
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

This thread is moving so fast I can't keep up!

The fact that troubles me is that I am unable to grasp how we move forward,
which was the point of this thread to start with. It seems we have 2
options:

- We make GBP to merge as is, in the Neutron tree, with some minor revision
(e.g. naming?);
- We make GBP a stackforge project, that integrates with Neutron in some
shape or form;

Another option, might be something in between, where GBP is in tree, but in
some sort of experimental staging area (even though I am not sure how well
baked this idea is).

Now, as a community we all need make a decision; arguing about the fact
that the blueprint was approved is pointless. As a matter of fact, I think
that blueprint should be approved, if and only if the code has landed
completely, but I digress!

Let's together come up with pros and cons of each approach and come up with
an informed decision.

Just reading free form text, how are we expected to do that? At least I
can't!

My 2c.
Armando


On 6 August 2014 15:03, Aaron Rosen  wrote:

>
>
>
> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:
>
>> >I believe the referential security group rules solve this problem
>> (unless I'm not understanding):
>>
>> I think the disconnect is that you are comparing the way to current
>> mapping driver implements things for the reference implementation with the
>> existing APIs. Under this light, it's not going to look like there is a
>> point to this code being in Neutron since, as you said, the abstraction
>> could happen at a client. However, this changes once new mapping drivers
>> can be added that implement things differently.
>>
>> Let's take the security groups example. Using the security groups API
>> directly is imperative ("put a firewall rule on this port that blocks this
>> IP") compared to a higher level declarative abstraction ("make sure these
>> two endpoints cannot communicate"). With the former, the ports must support
>> security groups and there is nowhere except for the firewall rules on that
>> port to implement it without violating the user's expectation. With the
>> latter, a mapping driver could determine that communication between these
>> two hosts can be prevented by using an ACL on a router or a switch, which
>> doesn't violate the user's intent and buys a performance improvement and
>> works with ports that don't support security groups.
>>
>> Group based policy is trying to move the requests into the declarative
>> abstraction so optimizations like the one above can be made.
>>
>
> Hi Kevin,
>
> Interesting points. Though, let me ask this. Why do we need to move to a
> declarative API abstraction in neutron in order to perform this
> optimization on the backend? For example, In the current neutron model say
> we want to create a port with a security group attached to it called web
> that allows TCP:80 in and members who are in a security group called
> database. From this mapping I fail to see how it's really any different
> from the declarative model? The ports in neutron are logical abstractions
> and the backend system could be implemented in order to determine that the
> communication between these two hosts could be prevented by using an ACL on
> a router or switch as well.
>
> Best,
>
> Aaron
>
>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

By working at the port level you have already eliminated your ability to
implement the filtering at different components of the network. They now
need to be implemented in stateful rules at the port level and the device
has to support security groups.


On Wed, Aug 6, 2014 at 4:03 PM, Aaron Rosen  wrote:

>
>
>
> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:
>
>> >I believe the referential security group rules solve this problem
>> (unless I'm not understanding):
>>
>> I think the disconnect is that you are comparing the way to current
>> mapping driver implements things for the reference implementation with the
>> existing APIs. Under this light, it's not going to look like there is a
>> point to this code being in Neutron since, as you said, the abstraction
>> could happen at a client. However, this changes once new mapping drivers
>> can be added that implement things differently.
>>
>> Let's take the security groups example. Using the security groups API
>> directly is imperative ("put a firewall rule on this port that blocks this
>> IP") compared to a higher level declarative abstraction ("make sure these
>> two endpoints cannot communicate"). With the former, the ports must support
>> security groups and there is nowhere except for the firewall rules on that
>> port to implement it without violating the user's expectation. With the
>> latter, a mapping driver could determine that communication between these
>> two hosts can be prevented by using an ACL on a router or a switch, which
>> doesn't violate the user's intent and buys a performance improvement and
>> works with ports that don't support security groups.
>>
>> Group based policy is trying to move the requests into the declarative
>> abstraction so optimizations like the one above can be made.
>>
>
> Hi Kevin,
>
> Interesting points. Though, let me ask this. Why do we need to move to a
> declarative API abstraction in neutron in order to perform this
> optimization on the backend? For example, In the current neutron model say
> we want to create a port with a security group attached to it called web
> that allows TCP:80 in and members who are in a security group called
> database. From this mapping I fail to see how it's really any different
> from the declarative model? The ports in neutron are logical abstractions
> and the backend system could be implemented in order to determine that the
> communication between these two hosts could be prevented by using an ACL on
> a router or switch as well.
>
> Best,
>
> Aaron
>
>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [git-review] Supporting development in local branches

2014-08-06 Thread Yuriy Taraday

Oh, looks like we got a bit of a race condition in messages. I hope you
don't mind.


On Wed, Aug 6, 2014 at 11:00 PM, Ben Nemec  wrote:

> On 08/06/2014 01:42 PM, Yuriy Taraday wrote:
> > On Wed, Aug 6, 2014 at 6:20 PM, Ben Nemec 
> wrote:
> >
> >> On 08/06/2014 03:35 AM, Yuriy Taraday wrote:
> >>> I'd like to stress this to everyone: I DO NOT propose squashing
> together
> >>> commits that should belong to separate change requests. I DO NOT
> propose
> >> to
> >>> upload all your changes at once. I DO propose letting developers to
> keep
> >>> local history of all iterations they have with a change request. The
> >>> history that absolutely doesn't matter to anyone but this developer.
> >>
> >> Right, I understand that may not be the intent, but it's almost
> >> certainly going to be the end result.  You can't control how people are
> >> going to use this feature, and history suggests if it can be abused, it
> >> will be.
> >>
> >
> > Can you please outline the abuse scenario that isn't present nowadays?
> > People upload huge changes and are encouraged to split them during
> review.
> > The same will happen within proposed workflow. More experienced
> developers
> > split their change into a set of change requests. The very same will
> happen
> > within proposed workflow.
>
> There will be a documented option in git-review that automatically
> squashes all commits.  People _will_ use that incorrectly because from a
> submitter perspective it's easier to deal with one review than multiple,
> but from a reviewer perspective it's exactly the opposite.
>

It won't be documented as such. It will include "use with care" and "years
of Git experience: 3+" stickers. Autosquashing will never be mentioned
there. Only a detailed explanation of how to work with it and (probably)
how it works. No rogue dev will get through it without getting the true
understanding.

By the way, currently git-review suggests to squash your outstanding
commits but there is no overwhelming flow of overly huge change requests,
is there?

>>> On Wed, Aug 6, 2014 at 12:03 PM, Martin Geisler 
> >> wrote:
> >>>
>  Ben Nemec  writes:
> 
> > On 08/05/2014 03:14 PM, Yuriy Taraday wrote:
> >>
> >> When you're developing some big change you'll end up with trying
> >> dozens of different approaches and make thousands of mistakes. For
> >> reviewers this is just unnecessary noise (commit title "Scratch my
> >> last CR, that was bullshit") while for you it's a precious history
> >> that can provide basis for future research or bug-hunting.
> >
> > So basically keeping a record of how not to do it?  I get that, but I
> > think I'm more onboard with the suggestion of sticking those dead end
> > changes into a separate branch.  There's no particular reason to keep
> > them on your working branch anyway since they'll never merge to
> master.
> >  They're basically unnecessary conflicts waiting to happen.
> 
>  Yeah, I would never keep broken or unfinished commits around like
> this.
>  In my opinion (as a core Mercurial developer), the best workflow is to
>  work on a feature and make small and large commits as you go along.
> When
>  the feature works, you begin squashing/splitting the commits to make
>  them into logical pieces, if they aren't already in good shape. You
> then
>  submit the branch for review and iterate on it until it is accepted.
> 
> >>>
> >>> Absolutely true. And it's mostly the same workflow that happens in
> >>> OpenStack: you do your cool feature, you carve meaningful small
> >>> self-contained pieces out of it, you submit series of change requests.
> >>> And nothing in my proposal conflicts with it. It just provides a way to
> >>> make developer's side of this simpler (which is the intent of
> git-review,
> >>> isn't it?) while not changing external artifacts of one's work: the
> same
> >>> change requests, with the same granularity.
> >>>
> >>>
>  As a reviewer, it cannot be stressed enough how much small, atomic,
>  commits help. Squashing things together into large commits make
> reviews
>  very tricky and removes the possibility of me accepting a later commit
>  while still discussing or rejecting earlier commits (cherry-picking).
> 
> >>>
> >>> That's true, too. But please don't think I'm proposing to squash
> >> everything
> >>> together and push 10k-loc patches. I hate that, too. I'm proposing to
> let
> >>> developer use one's tools (Git) in a simpler way.
> >>> And the simpler way (for some of us) would be to have one local branch
> >> for
> >>> every change request, not one branch for the whole series. Switching
> >>> between branches is very well supported by Git and doesn't require
> extra
> >>> thinking. Jumping around in detached HEAD state and editing commits
> >> during
> >>> rebase requires remembering all those small details.
> >>>
>  FWIW, I have had long-lived patch series, and I don't really s

Re: [openstack-dev] [Octavia] Weekly meetings resuming + agenda

2014-08-06 Thread Stephen Balukoff

Action items from today's Octavia meeting:

1. We're going to hold off for a couple days on merging the constitution
and preliminary road map to give people (and in particular Ebay) a chance
to review and comment.

2. Stephen is going to try to get Octavia v0.5 design docs into gerrit
review by the end of the week, or early next week at the latest.

3. If those with specific networking concerns could codify this and/or
figure out a way to write these down and share with the list, that would be
great. This is going to be important to ensure that our "operator-grade
load balancer" solution can actually meet the needs of the operators
developing it.

Thanks,

Stephen

On Tue, Aug 5, 2014 at 2:34 PM, Stephen Balukoff 
wrote:

> Hello!
>
> We plan on resuming weekly meetings to discuss things related to the
> Octavia project starting tomorrow: August 6th at 13:00PDT (20:00UTC). In
> order to facilitate high-bandwidth discussion as we bootstrap the project,
> we have decided to hold these meetings via webex, with the plan to
> eventually transition to IRC. Please contact me directly if you would like
> to get in on the webex.
>
> Tomorrow's meeting agenda is currently as follows:
>
> * Discuss Octavia constitution and project direction documents currently
> under gerrit review:
> https://review.openstack.org/#/c/110563/
>
> * Discuss reviews of design proposals currently under gerrit review:
> https://review.openstack.org/#/c/111440/
> https://review.openstack.org/#/c/111445/
>
> * Discuss operator network topology requirements based on data currently
> being collected by HP, Rackspace and Blue Box. (Other operators are
> certainly welcome to collect and share their data as well! I'm looking at
> you, Ebay. ;) )
>
> Please feel free to respond with additional agenda items!
>
> Stephen
>
> --
> Stephen Balukoff
> Blue Box Group, LLC
> (800)613-4305 x807
>

-- 
Stephen Balukoff
Blue Box Group, LLC
(800)613-4305 x807
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Neutron][LBaaS] Weekly IRC Agenda

2014-08-06 Thread Jorge Miramontes

Hey LBaaS folks,

This is you friendly reminder to provide any agenda items for tomorrow's weekly 
IRC meeting. Please add them to the agenda wiki ==> 
https://wiki.openstack.org/wiki/Network/LBaaS#Agenda. The agenda currently has 
these items:

  *   Review Updates

Cheers,
--Jorge

P.S. Also, please don't forget to update the weekly standup ==> 
https://etherpad.openstack.org/p/neutron-lbaas-weekly-standup
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [git-review] Supporting development in local branches

2014-08-06 Thread Yuriy Taraday

I'll start using pictures now, so let's assume M is the latest commit on
the master.

On Wed, Aug 6, 2014 at 9:31 PM, Zane Bitter  wrote:

> On 04/08/14 19:18, Yuriy Taraday wrote:
>
>> Hello, git-review users!
>>
>> I'd like to gather feedback on a feature I want to implement that might
>> turn out useful for you.
>>
>> I like using Git for development. It allows me to keep track of current
>> development process, it remembers everything I ever did with the code
>> (and more).
>>
>
> _CVS_ allowed you to remember everything you ever did; Git is _much_ more
> useful.
>
>
>  I also really like using Gerrit for code review. It provides clean
>> interfaces, forces clean histories (who needs to know that I changed one
>> line of code in 3am on Monday?) and allows productive collaboration.
>>
>
> +1
>
>
>  What I really hate is having to throw away my (local, precious for me)
>> history for all change requests because I need to upload a change to
>> Gerrit.
>>
>
> IMO Ben is 100% correct and, to be blunt, the problem here is your
> workflow.
>

Well... That's the workflow that was born with Git. Keeping track of all
changes, do extremely cheap merges, and all that.

Don't get me wrong, I sympathise - really, I do. Nobody likes to change
> their workflow. I *hate* it when I have to change mine. However what you're
> proposing is to modify the tools to make it easy for other people - perhaps
> new developers - to use a bad workflow instead of to learn a good one from
> the beginning, and that would be a colossal mistake. All of the things you
> want to be made easy are currently hard because doing them makes the world
> a worse place.
>

And when OpenStack switched to Gerrit I was really glad. Instead of ugly

master: ...-M-.-o-o-...
 \   /
  a1-b1-a2-a3-b2-c1-b3-c2

where a[1-3], b[1-3] and c[1-2] are iterations over the same piece of the
feature, we can have pretty

master: ...-M-.o-.-o-...
 \/   /
  A^-B^-C^

where A^, B^ and C^ are the perfect self-contained, independently
reviewable and mergeable pieces of the feature.

And this looked splendid and my workflow seemed clear. Suppose I have smth
like:

master: ...-M
 \
  A3-B2-C1

and I need to update B to B3 and C to C2. So I go:
$ git rebase -i M  # and add edit action to B commit
$ vim # do some changes, test them, etc
$ git rebase --continue
now I have

master: ...-M
 \
  A3-B2-C1
\
 B3-C1'

Then I fix C commit, amend it and get:

master: ...-M
 \
  A3-B2-C1
\
 B3-C1'
   \
С2

Now everything's cool, isn't it? But world isn't fair. And C2 fails a test
that I didn't expect to fail. Or the test suite failed to fail earlier. I'd
like to see if I broke it just now or were it broken after rebase. How do I
do it? With your workflow - I don't. I play smart and guess where the
problem was or dig into reflog to find C1' (or C1), etc. Let's see what
else I can't find. After full iteration over this feature (as in the first
picture) I end up with total history like this:

master: ...-M
|\
| A1-B1
|\
| A2-B1'
 \
  A3-B1''
   |\
   | B2-C1
\
 B3-C1'
   \
С2

With only A3, B3 and C2 available, the rest are practically unreachable.
Now you find out that something that you were sure was working in B1 is
broken (you'll tell me "hey, you're supposed to have tests with
everything!" - I'll answer: "what if you've found a problem in the test
suite that gave false success?"). You can do absolutely nothing to localize
the issue now. Just go and dig into your B code (which might've been
written months ago).
Or you slap your head understanding that the function you thought is not
needed in B2 is actually needed. Well, you can hope you did upload B2 to
Gerrit and you'll find it there. Or you didn't because you decided to make
that change the minute after you committed C1, created B3 and B2 never
existed now...

Now imagine you could somehow link together all As, Bs and Cs. Let's draw
vertical edges between them. And let's transpose the picture, shall we?

master: ...-M
 \
  A1-A2--A3
\  \   \\  \
 B1-B1'-B1''-B2-B3
   \  \   \
C1-C1'-C2

Note that all commits here are absolutely the same as in previous picture.
They just have additional parents (and consequently differen IDs). No
changes to any code in them happen. No harm done.

So now it looks way better. I can just do:
$ git checkout B3
$ git diff HEAD~
and find my lost function!

Now let's be honest and admit that As, Bs and Cs are essentially branches -
labels your commits h

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:

> >I believe the referential security group rules solve this problem
> (unless I'm not understanding):
>
> I think the disconnect is that you are comparing the way to current
> mapping driver implements things for the reference implementation with the
> existing APIs. Under this light, it's not going to look like there is a
> point to this code being in Neutron since, as you said, the abstraction
> could happen at a client. However, this changes once new mapping drivers
> can be added that implement things differently.
>
> Let's take the security groups example. Using the security groups API
> directly is imperative ("put a firewall rule on this port that blocks this
> IP") compared to a higher level declarative abstraction ("make sure these
> two endpoints cannot communicate"). With the former, the ports must support
> security groups and there is nowhere except for the firewall rules on that
> port to implement it without violating the user's expectation. With the
> latter, a mapping driver could determine that communication between these
> two hosts can be prevented by using an ACL on a router or a switch, which
> doesn't violate the user's intent and buys a performance improvement and
> works with ports that don't support security groups.
>
> Group based policy is trying to move the requests into the declarative
> abstraction so optimizations like the one above can be made.
>

Hi Kevin,

Interesting points. Though, let me ask this. Why do we need to move to a
declarative API abstraction in neutron in order to perform this
optimization on the backend? For example, In the current neutron model say
we want to create a port with a security group attached to it called web
that allows TCP:80 in and members who are in a security group called
database. From this mapping I fail to see how it's really any different
from the declarative model? The ports in neutron are logical abstractions
and the backend system could be implemented in order to determine that the
communication between these two hosts could be prevented by using an ACL on
a router or switch as well.

Best,

Aaron

>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Win The Enterprise Work Group Update

2014-08-06 Thread Barrett, Carol L

I want to provide the community an update on the Win The Enterprise work group 
that came together in a BoF session in Atlanta.

The work group led a discussion with the OpenStack Board at their 7/22 meeting 
on the findings of our analysis of Enterprise IT requirements gaps. A summary 
of the presentation and next steps can be found here: 
https://drive.google.com/file/d/0BxtM4AiszlEySmJwMHpDTGFDZHc/edit?usp=sharing

Based upon the analysis and discussion, the actions for the work group are:
1)  Form a Deployment team to take on the Deployment oriented requirements 
that came up from the different teams. This team will have both Technical and 
Marketing members.
a.  Please let me know if you're interested in joining
2)  Form a Monitoring team to take on the Monitoring oriented requirements 
that came up from the different teams. This team will have both Technical and 
Marketing members.
a.  Please let me know if you're interested in joining
  For Technical gaps, we need to assess final accepted Juno blueprints 
versus requirements and develop additional blueprints through community 
participation and implementation support to bring into the Kilo Design Summit
3)  For Documentation gaps, we need to work with either existing 
documentation teams or the Marketing team to create.
4)  For Marketing Perceptions, we need to create a content and collateral 
plan with owners and execute.

Our goals are:
1)  Prepare and intercept the Kilo Design Summit pre-plannning and sessions 
in Paris with new BPs that implement the requirements
2)  Intercept Paris Summit Analyst and Press outreach plans with content 
addressing top perception issues
3)  Complete the needed documentation/collateral ahead of the Paris summit
4)  Target the Enterprise IT Strategy track in Paris on the key Enterprise 
IT requirements to address documentation gaps, and provide how-to info for 
deployments.

Call to Action: Please let me know if you want to be involved in any of the 
work group activities. Lots of opportunities for you to help advance OpenStack 
adoption in this segment!

If you have any questions or want more info, pls get in touch.
Carol Barrett
Intel Corp
+1 503 712 7623



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

+1 Kevin. I really fail to see how a patch which has been ready for a long
time now is the worst enemy of Nova Parity. This is starting to feel kind
of ad-hoc...
On Aug 6, 2014 11:42 PM, "Kevin Benton"  wrote:

> >In all seriousness, folks, I'm bringing up points about the proposed API
> because I see the current mess of Neutron integration with Nova, I see that
> nova-network has been "undeprecated" due to continuing lack of parity and
> HA concerns in Neutron, and I want things to be better, not worse.
>
> Again, I haven't seen any evidence that ongoing development of this new
> API is preventing any of the parity work from happening. Nobody is
> advocating that the parity work be delayed because of this. We are all
> aware of the threats the TC has put forth with regard to demoting Neutron
> to incubation unless the parity demands are met. If you want ongoing
> development to stop, this should be a clear requirement and it should be
> evenly applied to all work (LBaaS separation, ML2 enhancements, any third
> party drivers, etc).
>
>
>
> On Wed, Aug 6, 2014 at 3:10 PM, Jay Pipes  wrote:
>
>> On 08/06/2014 04:51 PM, Pedro Marques wrote:
>>
>>> Neutron allows vendors to speak to proprietary device APIs, it was
>>> designed to do so, AFAIK. It is also possibly to "entirely swap out
>>> all of the Neutron core"... the proponents of the group based policy
>>> didn't have to go through so much trouble if that was their intent.
>>> As far as i know most plugins talk to a proprietary API.
>>>
>>> I happen to disagree technically with a couple of choices made by
>>> this proposal; but the blueprint was approved. Which means that i
>>> lost the argument, or didn't raise it on time, or didn't argue
>>> convincingly... regardless of the reason, the time to argue about the
>>> goal has passed. The decision of the community was to approve the
>>> spec and that decision should be respected.
>>>
>>
>> Sure, no problem. I'll just go back to Nova and wait around to help clean
>> up the mess.
>>
>> In all seriousness, folks, I'm bringing up points about the proposed API
>> because I see the current mess of Neutron integration with Nova, I see that
>> nova-network has been "undeprecated" due to continuing lack of parity and
>> HA concerns in Neutron, and I want things to be better, not worse.
>>
>> Neutron contributors need to recognize that Nova is the pre-eminent
>> consumer of Neutron interfaces, and until those interfaces are stable,
>> consistent regardless of underlying hardware or driver choices, and
>> generally preferable for Nova to recommend as its default network driver,
>> then the Neutron project is sitting as an island unto itself.
>>
>> The fact that not enough Nova developers (including yours truly) are
>> paying attention to what is going on in Neutron spec-land and roadmap is a
>> serious problem we should deal with directly (cross-project spec meetings,
>> better documentation and communication, shared bug triaging and
>> verification meetings, etc). Otherwise, these kinds of conversations are
>> likely to continue.
>>
>> Best,
>> -jay
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Kevin Benton
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Group Based Policy and the way forward

Edgar,

Actually, As Pedro said, I think that the time for discussing these kind of
concerns was the BP approval. The fact that it has been approved after many
proposals and reviews means that a community effort wanted the GBP to be
implemented in this release cycle the way it was presented at that time.

With this, I absolutely don't want to say that you should not express your
disagreement! I'm just saying that it should be expressed differently (a BP
to propose your model in K?). Otherwise, the whole BP process becomes just
pointless.

Meanwhile, imho, blocking the patch feels really unfair.

Ivar.
 On Aug 6, 2014 11:00 PM, "Edgar Magana"  wrote:

>  Ivar,
>
>  Of course and this is why we are having this conversation, in order to
> merge our different opinions.
>
>  Edgar
>
>   From: Ivar Lazzaro 
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: Wednesday, August 6, 2014 at 1:41 PM
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way
> forward
>
>   Hi Edgar,
>
>  Actually, I think that other reviewers saw that name clash, and still
> thought it was ok to use the same terminology in such a different context.
> BP reviews are a community effort right? So of course someones' idea may
> be different from yours.
>
>  Regards,
> Ivar.
>
>
> On Wed, Aug 6, 2014 at 10:29 PM, Edgar Magana 
> wrote:
>
>> Basically, I am admitting that I did not catch in my review the part of
>> the endpoint term that Jay was pointing out.
>>
>> Edgar
>>
>> On 8/6/14, 11:32 AM, "Sumit Naiksatam"  wrote:
>>
>> >Not sure what you are talking about? You claim now that you had
>> >suggestion which was not considered, yet you +2'ed a patch, by stating
>> >that "All looks good to me!".
>> >
>> >On Wed, Aug 6, 2014 at 11:19 AM, Edgar Magana 
>> >wrote:
>> >> That is the beauty of the open source projects, there is always a
>> >>smartest
>> >> reviewer catching out the facts that you don¹t.
>> >>
>> >> Edgar
>> >>
>> >> On 8/6/14, 10:55 AM, "Sumit Naiksatam" 
>> wrote:
>> >>
>> >>>Edgar, you seemed to have +2'ed this patch on July 2nd [1]:
>> >>>
>> >>>"
>> >>>Edgar Magana
>> >>>Jul 2 8:42 AM
>> >>>
>> >>>Patch Set 13: Code-Review+2
>> >>>
>> >>>All looks good to me! I am not approving yet because Nachi was also
>> >>>reviewing this code and I would like to see his opinion as well.
>> >>>"
>> >>>
>> >>>That would suggest that you were happy with what was in it. I don't
>> >>>see anything in the review comments that suggests otherwise.
>> >>>
>> >>>[1]  https://review.openstack.org/#/c/95900/
>> >>>
>> >>>On Wed, Aug 6, 2014 at 10:39 AM, Edgar Magana <
>> edgar.mag...@workday.com>
>> >>>wrote:
>>  This is the consequence of a proposal that is not following the
>> standardized
>>  terminology (IETF - RFC) for any Policy-based System:
>>  http://tools.ietf.org/html/rfc3198
>> 
>>  Well, I did bring  this point during the Hong Kong Summit but as you
>> can see
>>  my comments were totally ignored:
>> 
>> 
>> https://docs.google.com/document/d/1ZbOFxAoibZbJmDWx1oOrOsDcov6Cuom5aaB
>> Ir
>> upCD9E/edit
>> 
>>  I clearly saw this kind of issues coming. Let me quote myself what I
>>  suggested: "For instance: "endpoints" should be "enforcement point"
>> 
>>  I do not understand why GBP did not include this suggestionŠ
>> 
>>  Edgar
>> 
>>  From: Kevin Benton 
>>  Reply-To: "OpenStack Development Mailing List (not for usage
>> questions)"
>>  
>>  Date: Wednesday, August 6, 2014 at 10:22 AM
>>  To: "OpenStack Development Mailing List (not for usage questions)"
>>  
>> 
>>  Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way
>>  forward
>> 
>>  What I was referring to was also not Keystone's definition of an
>> endpoint.
>>  It's almost as if the term has many uses and was not invented for
>> Keystone.
>>  :-)
>> 
>>  http://www.wireshark.org/docs/wsug_html_chunked/ChStatEndpoints.html
>> 
>>  Did a similar discussion occur when Heat wanted to use the word
>> 'template'
>>  since this was clearly already in use by Horizon?
>> 
>>  On Aug 6, 2014 9:24 AM, "Jay Pipes"  wrote:
>> >
>> > On 08/06/2014 02:12 AM, Kevin Benton wrote:
>> >>
>> >> Given that, pointing to the Nova parity work seems a bit like a red
>> >> herring. This new API is being developed orthogonally to the
>> >>existing
>> >> API endpoints
>> >
>> >
>> > You see how you used the term endpoints there? :P
>> >
>> > -jay
>> >
>> > ___
>> > OpenStack-dev mailing list
>> > OpenStack-dev@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>>

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

2014-08-06 Thread Richard Woo

+1,

I think in future we should invite the other project(i.e. Nova) core member
to review the blueprint if it is related to that specific project.





On Wed, Aug 6, 2014 at 5:01 PM, Mohammad Banikazemi  wrote:

> Yes, indeed.
> I do not want to be over dramatic but the discussion on the original "Group
> Based Policy and the way forward" thread is nothing short of heartbreaking.
> After months and months of discussions, three presentations at the past
> three summits, a design session at the last summit, and (most relevant to
> this thread) the approval of the spec, why are we talking about the merits
> of the work now?
>
> I understand if people think this is not a good idea or this is not a good
> time. What I do not understand is why these concerns were not raised
> clearly and openly earlier.
>
> Best,
>
> Mohammad
>
>
> [image: Inactive hide details for Stefano Maffulli ---08/06/2014 04:47:21
> PM---On Wed 06 Aug 2014 01:21:26 PM PDT, Eugene Nikanorov wro]Stefano
> Maffulli ---08/06/2014 04:47:21 PM---On Wed 06 Aug 2014 01:21:26 PM PDT,
> Eugene Nikanorov wrote: > So I don't think it's fair to blame re
>
> From: Stefano Maffulli 
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: 08/06/2014 04:47 PM
> Subject: Re: [openstack-dev] How to improve the specs review process (was
> Re: [Neutron] Group Based Policy and the way forward)
> --
>
>
>
> On Wed 06 Aug 2014 01:21:26 PM PDT, Eugene Nikanorov wrote:
> > So I don't think it's fair to blame reviewers here.
>
> Just want to be super clear: there is no 'blaming' here. This is a
> request to regroup and look at why we are having this conversation about
> GBP now, this late in cycle, and about such fundamental topics (the
> choice of 'endpoint' as name is only one of them), after in-person
> conversations over more than one release cycle and summits.
>
> I am available for the meeting on Monday, Kyle.
>
> In order to prepare for the meeting we should agree on the scope of the
> root cause analysis. I think the problem should be framed around the
> message Mark McClain sent, especially the "Why this email" which I quote
> below:
>
> > Our community has been discussing and working on Group Based Policy
> > (GBP) for many months.  I think the discussion has reached a point
> > where we need to openly discuss a few issues before moving forward.
> [...]
>
> I think the fact that this very fair question has been raised so late is
> the problem we need to find the cause for. Would you agree?
>
> We'll use time during the meeting on Monday to use a simple technique to
> investigate this deeply, no need to spend time now and via email.
>
> /stef
>
> --
> Ask and answer questions on https://ask.openstack.org
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Which program for Rally

2014-08-06 Thread John Griffith

I have to agree with Duncan here.  I also don't know if I fully understand
the limit in options.  Stress test seems like it could/should be different
(again overlap isn't a horrible thing) and I don't see it as siphoning off
resources so not sure of the issue.  We've become quite wrapped up in
projects, programs and the like lately and it seems to hinder forward
progress more than anything else.

I'm also not convinced that Tempest is where all things belong, in fact
I've been thinking more and more that a good bit of what Tempest does today
should fall more on the responsibility of the projects themselves.  For
example functional testing of features etc, ideally I'd love to have more
of that fall on the projects and their respective teams.  That might even
be something as simple to start as saying "if you contribute a new feature,
you have to also provide a link to a contribution to the Tempest test-suite
that checks it".  Sort of like we do for unit tests, cross-project tracking
is difficult of course, but it's a start.  The other idea is maybe
functional test harnesses live in their respective projects.

Honestly I think who better to write tests for a project than the folks
building and contributing to the project.  At some point IMO the QA team
isn't going to scale.  I wonder if maybe we should be thinking about
proposals for delineating responsibility and goals in terms of functional
testing?




On Wed, Aug 6, 2014 at 12:25 PM, Duncan Thomas 
wrote:

> I'm not following here - you complain about rally being monolithic,
> then suggest that parts of it should be baked into tempest - a tool
> that is already huge and difficult to get into. I'd rather see tools
> that do one thing well and some overlap than one tool to rule them
> all.
>
> On 6 August 2014 14:44, Sean Dague  wrote:
> > On 08/06/2014 09:11 AM, Russell Bryant wrote:
> >> On 08/06/2014 06:30 AM, Thierry Carrez wrote:
> >>> Hi everyone,
> >>>
> >>> At the TC meeting yesterday we discussed Rally program request and
> >>> incubation request. We quickly dismissed the incubation request, as
> >>> Rally appears to be able to live happily on top of OpenStack and would
> >>> benefit from having a release cycle decoupled from the OpenStack
> >>> "integrated release".
> >>>
> >>> That leaves the question of the program. OpenStack programs are created
> >>> by the Technical Committee, to bless existing efforts and teams that
> are
> >>> considered *essential* to the production of the "OpenStack" integrated
> >>> release and the completion of the OpenStack project mission. There are
> 3
> >>> ways to look at Rally and official programs at this point:
> >>>
> >>> 1. Rally as an essential QA tool
> >>> Performance testing (and especially performance regression testing) is
> >>> an essential QA function, and a feature that Rally provides. If the QA
> >>> team is happy to use Rally to fill that function, then Rally can
> >>> obviously be adopted by the (already-existing) QA program. That said,
> >>> that would put Rally under the authority of the QA PTL, and that raises
> >>> a few questions due to the current architecture of Rally, which is more
> >>> product-oriented. There needs to be further discussion between the QA
> >>> core team and the Rally team to see how that could work and if that
> >>> option would be acceptable for both sides.
> >>>
> >>> 2. Rally as an essential operator tool
> >>> Regular benchmarking of OpenStack deployments is a best practice for
> >>> cloud operators, and a feature that Rally provides. With a bit of a
> >>> stretch, we could consider that benchmarking is essential to the
> >>> completion of the OpenStack project mission. That program could one day
> >>> evolve to include more such "operations best practices" tools. In
> >>> addition to the slight stretch already mentioned, one concern here is
> >>> that we still want to have performance testing in QA (which is clearly
> >>> essential to the production of "OpenStack"). Letting Rally primarily be
> >>> an operational tool might make that outcome more difficult.
> >>>
> >>> 3. Let Rally be a product on top of OpenStack
> >>> The last option is to not have Rally in any program, and not consider
> it
> >>> *essential* to the production of the "OpenStack" integrated release or
> >>> the completion of the OpenStack project mission. Rally can happily
> exist
> >>> as an operator tool on top of OpenStack. It is built as a monolithic
> >>> product: that approach works very well for external complementary
> >>> solutions... Also be more integrated in OpenStack or part of the
> >>> OpenStack programs might come at a cost (slicing some functionality out
> >>> of rally to make it more a framework and less a product) that might not
> >>> be what its authors want.
> >>>
> >>> Let's explore each option to see which ones are viable, and the pros
> and
> >>> cons of each.
> >>
> >> My feeling right now is that Rally is trying to accomplish too much at
> >> the start (both #1 and #2).  I

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

+1 Mohammad.
On Aug 6, 2014 11:08 PM, "Mohammad Banikazemi"  wrote:

> Yes, indeed.
> I do not want to be over dramatic but the discussion on the original "Group
> Based Policy and the way forward" thread is nothing short of heartbreaking.
> After months and months of discussions, three presentations at the past
> three summits, a design session at the last summit, and (most relevant to
> this thread) the approval of the spec, why are we talking about the merits
> of the work now?
>
> I understand if people think this is not a good idea or this is not a good
> time. What I do not understand is why these concerns were not raised
> clearly and openly earlier.
>
> Best,
>
> Mohammad
>
>
> [image: Inactive hide details for Stefano Maffulli ---08/06/2014 04:47:21
> PM---On Wed 06 Aug 2014 01:21:26 PM PDT, Eugene Nikanorov wro]Stefano
> Maffulli ---08/06/2014 04:47:21 PM---On Wed 06 Aug 2014 01:21:26 PM PDT,
> Eugene Nikanorov wrote: > So I don't think it's fair to blame re
>
> From: Stefano Maffulli 
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: 08/06/2014 04:47 PM
> Subject: Re: [openstack-dev] How to improve the specs review process (was
> Re: [Neutron] Group Based Policy and the way forward)
> --
>
>
>
> On Wed 06 Aug 2014 01:21:26 PM PDT, Eugene Nikanorov wrote:
> > So I don't think it's fair to blame reviewers here.
>
> Just want to be super clear: there is no 'blaming' here. This is a
> request to regroup and look at why we are having this conversation about
> GBP now, this late in cycle, and about such fundamental topics (the
> choice of 'endpoint' as name is only one of them), after in-person
> conversations over more than one release cycle and summits.
>
> I am available for the meeting on Monday, Kyle.
>
> In order to prepare for the meeting we should agree on the scope of the
> root cause analysis. I think the problem should be framed around the
> message Mark McClain sent, especially the "Why this email" which I quote
> below:
>
> > Our community has been discussing and working on Group Based Policy
> > (GBP) for many months.  I think the discussion has reached a point
> > where we need to openly discuss a few issues before moving forward.
> [...]
>
> I think the fact that this very fair question has been raised so late is
> the problem we need to find the cause for. Would you agree?
>
> We'll use time during the meeting on Monday to use a simple technique to
> investigate this deeply, no need to spend time now and via email.
>
> /stef
>
> --
> Ask and answer questions on https://ask.openstack.org
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Carlino, Chuck (OpenStack TripleO, Neutron)

>In all seriousness, folks, I'm bringing up points about the proposed API
because I see the current mess of Neutron integration with Nova, I see that
nova-network has been "undeprecated" due to continuing lack of parity and
HA concerns in Neutron, and I want things to be better, not worse.

Again, I haven't seen any evidence that ongoing development of this new API
is preventing any of the parity work from happening. Nobody is advocating
that the parity work be delayed because of this. We are all aware of the
threats the TC has put forth with regard to demoting Neutron to incubation
unless the parity demands are met. If you want ongoing development to stop,
this should be a clear requirement and it should be evenly applied to all
work (LBaaS separation, ML2 enhancements, any third party drivers, etc).



On Wed, Aug 6, 2014 at 3:10 PM, Jay Pipes  wrote:

> On 08/06/2014 04:51 PM, Pedro Marques wrote:
>
>> Neutron allows vendors to speak to proprietary device APIs, it was
>> designed to do so, AFAIK. It is also possibly to "entirely swap out
>> all of the Neutron core"... the proponents of the group based policy
>> didn't have to go through so much trouble if that was their intent.
>> As far as i know most plugins talk to a proprietary API.
>>
>> I happen to disagree technically with a couple of choices made by
>> this proposal; but the blueprint was approved. Which means that i
>> lost the argument, or didn't raise it on time, or didn't argue
>> convincingly... regardless of the reason, the time to argue about the
>> goal has passed. The decision of the community was to approve the
>> spec and that decision should be respected.
>>
>
> Sure, no problem. I'll just go back to Nova and wait around to help clean
> up the mess.
>
> In all seriousness, folks, I'm bringing up points about the proposed API
> because I see the current mess of Neutron integration with Nova, I see that
> nova-network has been "undeprecated" due to continuing lack of parity and
> HA concerns in Neutron, and I want things to be better, not worse.
>
> Neutron contributors need to recognize that Nova is the pre-eminent
> consumer of Neutron interfaces, and until those interfaces are stable,
> consistent regardless of underlying hardware or driver choices, and
> generally preferable for Nova to recommend as its default network driver,
> then the Neutron project is sitting as an island unto itself.
>
> The fact that not enough Nova developers (including yours truly) are
> paying attention to what is going on in Neutron spec-land and roadmap is a
> serious problem we should deal with directly (cross-project spec meetings,
> better documentation and communication, shared bug triaging and
> verification meetings, etc). Otherwise, these kinds of conversations are
> likely to continue.
>
> Best,
> -jay
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Kevin Benton
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

Given how this discussion has gone, I understand Mohammad's despair.  But it 
seems like people are treating the Stackforge proposal as really nothing more 
than a black hole.  I'm a relative newcomer to this community, so that's 
probably why I took Mark at his word when he presented it as a way to quickly 
improve API design.  Well, that, and I'm a complete believer that iterating on 
running code is 10x better than any form of doc review.

Chuck

On Aug 6, 2014, at 2:01 PM, Mohammad Banikazemi 
mailto:m...@us.ibm.com>>
 wrote:

Yes, indeed.
I do not want to be over dramatic but the discussion on the original "Group 
Based Policy and the way forward" thread is nothing short of heartbreaking. 
After months and months of discussions, three presentations at the past three 
summits, a design session at the last summit, and (most relevant to this 
thread) the approval of the spec, why are we talking about the merits of the 
work now?

I understand if people think this is not a good idea or this is not a good 
time. What I do not understand is why these concerns were not raised clearly 
and openly earlier.

Best,

Mohammad

Stefano Maffulli ---08/06/2014 04:47:21 PM---On Wed 06 Aug 2014 
01:21:26 PM PDT, Eugene Nikanorov wrote: > So I don't think it's fair to blame 
re

From: Stefano Maffulli mailto:stef...@openstack.org>>
To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Date: 08/06/2014 04:47 PM
Subject: Re: [openstack-dev] How to improve the specs review process (was Re: 
[Neutron] Group Based Policy and the way forward)

On Wed 06 Aug 2014 01:21:26 PM PDT, Eugene Nikanorov wrote:
> So I don't think it's fair to blame reviewers here.

Just want to be super clear: there is no 'blaming' here. This is a
request to regroup and look at why we are having this conversation about
GBP now, this late in cycle, and about such fundamental topics (the
choice of 'endpoint' as name is only one of them), after in-person
conversations over more than one release cycle and summits.

I am available for the meeting on Monday, Kyle.

In order to prepare for the meeting we should agree on the scope of the
root cause analysis. I think the problem should be framed around the
message Mark McClain sent, especially the "Why this email" which I quote
below:

> Our community has been discussing and working on Group Based Policy
> (GBP) for many months.  I think the discussion has reached a point
> where we need to openly discuss a few issues before moving forward.
[...]

I think the fact that this very fair question has been raised so late is
the problem we need to find the cause for. Would you agree?

We'll use time during the meeting on Monday to use a simple technique to
investigate this deeply, no need to spend time now and via email.

/stef

--
Ask and answer questions on 
https://ask.openstack.org

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward


On 08/06/2014 04:51 PM, Pedro Marques wrote:

Neutron allows vendors to speak to proprietary device APIs, it was
designed to do so, AFAIK. It is also possibly to "entirely swap out
all of the Neutron core"... the proponents of the group based policy
didn't have to go through so much trouble if that was their intent.
As far as i know most plugins talk to a proprietary API.

I happen to disagree technically with a couple of choices made by
this proposal; but the blueprint was approved. Which means that i
lost the argument, or didn't raise it on time, or didn't argue
convincingly... regardless of the reason, the time to argue about the
goal has passed. The decision of the community was to approve the
spec and that decision should be respected.


Sure, no problem. I'll just go back to Nova and wait around to help 
clean up the mess.


In all seriousness, folks, I'm bringing up points about the proposed API 
because I see the current mess of Neutron integration with Nova, I see 
that nova-network has been "undeprecated" due to continuing lack of 
parity and HA concerns in Neutron, and I want things to be better, not 
worse.


Neutron contributors need to recognize that Nova is the pre-eminent 
consumer of Neutron interfaces, and until those interfaces are stable, 
consistent regardless of underlying hardware or driver choices, and 
generally preferable for Nova to recommend as its default network 
driver, then the Neutron project is sitting as an island unto itself.


The fact that not enough Nova developers (including yours truly) are 
paying attention to what is going on in Neutron spec-land and roadmap is 
a serious problem we should deal with directly (cross-project spec 
meetings, better documentation and communication, shared bug triaging 
and verification meetings, etc). Otherwise, these kinds of conversations 
are likely to continue.


Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] The future of the integrated release

2014-08-06 Thread Michael Still

On Wed, Aug 6, 2014 at 2:03 AM, Thierry Carrez  wrote:

> We seem to be unable to address some key issues in the software we
> produce, and part of it is due to strategic contributors (and core
> reviewers) being overwhelmed just trying to stay afloat of what's
> happening. For such projects, is it time for a pause ? Is it time to
> define key cycle goals and defer everything else ?

The nova team has been thinking about these issues recently too --
especially at our mid cycle meetup last week. We are drawing similar
conclusions to be honest.

Two nova cores were going to go away and write up a proposal for how
nova could handle a more focussed attempt to land code in Kilo, but
they haven't had a chance to do that yet. To keep this conversation
rolling, here's a quick summary of what they proposed:

 - we rate limit the total number of blueprints under code review at
any one time to a fixed number of "slots". I secretly prefer the term
"runway", so I am going to use that for the rest of this email. A
suggested initial number of runways was proposed at ten.

 - the development process would be much like juno for a blueprint --
you propose a spec, get it approved, write some code, and then you
request a runway to land the code in. Depending on your relative
priority compared to other code attempting to land, you queue until
traffic control assigns you a runway.

 - code occupying a runway gets nova core review attention, with the
expectation of fast iteration. If we find a blueprint has stalled in a
runway, it is removed and put back onto the queue based on its
priority (you don't get punished for being bumped).

This proposal is limiting the number of simultaneous proposals a core
needs to track, not the total number landed. The expectation is that
the time taken on a runway is short, and then someone else will occupy
it. Its mostly about focus -- instead of doing 100 core reviews on 100
patches so they never land, trying to do those reviews on the 10
patches so they all land.

We also talked about tweaking the ratio of "tech debt" runways vs
'feature" runways. So, perhaps every second release is focussed on
burning down tech debt and stability, whilst the others are focussed
on adding features. I would suggest if we do such a thing, Kilo should
be a "stability' release.

Michael

-- 
Rackspace Australia

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ironic] Proposal for slight change in our spec process

2014-08-06 Thread Chris K

I also am in favor of this. +1

Chris Krelle


On Wed, Aug 6, 2014 at 9:04 AM, Jay Faulkner  wrote:

> Similarly, I appreciated this idea when we discussed it at the mid-cycle
> and doing so here.
>
> +1
>
> -Jay Faulkner
>
> 
> From: Lucas Alvares Gomes 
> Sent: Wednesday, August 06, 2014 2:34 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [Ironic] Proposal for slight change in our
> specprocess
>
> Already agreed with the idea at the midcycle, but just making it public: +1
>
> On Tue, Aug 5, 2014 at 8:54 PM, Roman Prykhodchenko
>  wrote:
> > Hi!
> >
> > I think this is a nice idea indeed. Do you plan to use this process
> starting
> > from Juno or as soon as possible?
>
> It will start in Kilo
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Ryan Moats


[snipping to save BW]

Aaron Rosen  wrote on 08/06/2014 03:26:24 PM:

> Note: I'm not going to use the pure group policy API as I have been
> a fan of the
> 2-group approach from day 1, I'm going to use the commands as stated
> in the patch set, and I'm going to assume (dangerous I know) that I
> can specify endpoint-group on port create
>
> neutron grouppolicy-endpoint-group-create group1
> neutron grouppolicy-endpoint-group-create group2
>
> neutron port-create --endpoint-group group1 network1
> neutron port-create --endpoint-group group2 network1
>
> Sorry, I don't follow what this port-create command does? Here ---
> https://docs.google.com/presentation/d/
>
1Nn1HjghAvk2RTPwvltSrnCUJkidWKWY2ckU7OYAVNpo/edit#slide=id.g12c5a79d7_4078
>  --- shows that endpoint-groups map to networks so i'm confused why
> network1 is coming into play here?

Let's be careful that we aren't mixing concepts. First, this is where I
said "I'm going to assume I can do this" above comes in and so the network
applies to the port-create, not to the group.

Second, the mapping you site is optional and a subtle point is that while
a group maps to a multiplicity of subnets, those subnets can themselves be
in different networks, so a group does not automatically equate to a
network.

> neutron grouppolicy-policy-action-create action
> neutron grouppolicy-policy-classifier-create classifier
> neutron grouppolicy-policy-rule-create --action action --classifier
> classifer rule
> neutron policy-apply rule group1 group2
>
> Mind mapping this to my example so we can see how to achieve the
> same thing in group policy (with protocols and all)?  Looks like you
> would also need to specify (--provided-contracts, --consumed-
> contracts) when you create the endpoint-groups. Also, I don't see a
> cli command policy-apply so i'm not sure really what that does.

This is where I invoke my "I'm a fan of the 2-group approach" statement
above.
This is my proposed shorthand for the following three commands:

neutron grouppolicy-contract-create --poilcy-rule rule contract
neutron grouppolicy-endpoint-group-update --provides-contract contract
group2
neutron grouppolicy-endpoint-group-update --consumes-contract contract
group1

see:
http://lists.openstack.org/pipermail/openstack-dev/2014-July/041467.html

> One major difference between GP and current neutron (other than
> security groups) is in your last statement about tying things to
> networks.  Like security groups, groups in GP can have ports that
> span networks.
>
> So are you saying that group policy doesn't allow us to do different
> enforcement points on the same network as we do today?

That was not my intent.  I was only trying to point out that these groups
(like security groups) are not scoped by networks.

>  Unlike security groups, GP allows more rich policy than just allow.
>
> Right - I can see that GP does provide an allow/deny construct,
> though as you pointed out we can extend the current security group
> concept in neutron to provide allow/deny semantics to achieve the
> same thing no?

My only comments is that GP has been in the pipeline longer that the BP I
cite below...

> That of course begs the question of extending security groups (think
> this blueprint https://review.openstack.org/#/c/93112/ ) but that
> approach may have its own issues.___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [git-review] Supporting development in local branches

2014-08-06 Thread Chris Friesen


On 08/06/2014 01:14 PM, Yuriy Taraday wrote:

On Wed, Aug 6, 2014 at 7:23 PM, Ben Nemec mailto:openst...@nemebean.com>> wrote:




Again, this is why the tests should pass against all of your commits.
If that's the case, you can verify your changes as you rebase before you
update the commit.


Ok, one more time. You don't need to do rebase. You merge master with
one local commit resolving dependencies in the process and then fix
tests and everything with the second one. It's really simple.


Personally I find rebasing my current changes onto the latest upstream 
to be an intuitive way to work.  That way it's obvious what changes I'm 
making on top of the current upstream codebase.


In my view merging the upstream onto my dev branch only makes sense if 
I've published my dev branch to someone and don't want to break their 
history the next time they try to pull.


I would far rather rebase my current changes and explicitly resolve 
conflicts than merge upstream on top of my changes and then fix 
conflicts in merge commits.


Chris

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

2014-08-06 Thread Mohammad Banikazemi

Yes, indeed.
I do not want to be over dramatic but the discussion on the original "Group
Based Policy and the way forward" thread is nothing short of heartbreaking.
After months and months of discussions, three presentations at the past
three summits, a design session at the last summit, and (most relevant to
this thread) the approval of the spec, why are we talking about the merits
of the work now?

I understand if people think this is not a good idea or this is not a good
time. What I do not understand is why these concerns were not raised
clearly and openly earlier.

Best,

Mohammad

From:   Stefano Maffulli 
To: "OpenStack Development Mailing List (not for usage questions)"

Date:   08/06/2014 04:47 PM
Subject:Re: [openstack-dev] How to improve the specs review process
(was Re: [Neutron] Group Based Policy and the way forward)

On Wed 06 Aug 2014 01:21:26 PM PDT, Eugene Nikanorov wrote:
> So I don't think it's fair to blame reviewers here.

Just want to be super clear: there is no 'blaming' here. This is a
request to regroup and look at why we are having this conversation about
GBP now, this late in cycle, and about such fundamental topics (the
choice of 'endpoint' as name is only one of them), after in-person
conversations over more than one release cycle and summits.

I am available for the meeting on Monday, Kyle.

In order to prepare for the meeting we should agree on the scope of the
root cause analysis. I think the problem should be framed around the
message Mark McClain sent, especially the "Why this email" which I quote
below:

> Our community has been discussing and working on Group Based Policy
> (GBP) for many months.  I think the discussion has reached a point
> where we need to openly discuss a few issues before moving forward.
[...]

I think the fact that this very fair question has been raised so late is
the problem we need to find the cause for. Would you agree?

We'll use time during the meeting on Monday to use a simple technique to
investigate this deeply, no need to spend time now and via email.

/stef

--
Ask and answer questions on https://ask.openstack.org

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward


On 08/06/2014 04:51 PM, Prasad Vellanki wrote:

Jay
Doesnt the current plugin model in neutron work the way you are saying.
We have a a core set of APIs that there is a reference model for and
individual vendors have substituted plugins that enhance and sometimes
replace networking component. GBP in that respect does not change. There
is a reference implementation in neutron for declarative model in
neutron and vendors can substitute their implementation to enhance what
is in reference.

But what you need to understand is the declarative model that it
provides which Ryan has elaborated which current neutron does not provide.


Why can't it live outside of the Neutron tree?

-jay


On Wed, Aug 6, 2014 at 1:27 PM, Jay Pipes mailto:jaypi...@gmail.com>> wrote:

On 08/06/2014 04:13 PM, Sumit Naiksatam wrote:

On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton mailto:blak...@gmail.com>> wrote:

I believe the referential security group rules solve
this problem (unless
I'm not understanding):


I think the disconnect is that you are comparing the way to
current mapping
driver implements things for the reference implementation
with the existing
APIs. Under this light, it's not going to look like there is
a point to this
code being in Neutron since, as you said, the abstraction
could happen at a
client. However, this changes once new mapping drivers can
be added that
implement things differently.

Let's take the security groups example. Using the security
groups API
directly is imperative ("put a firewall rule on this port
that blocks this
IP") compared to a higher level declarative abstraction
("make sure these
two endpoints cannot communicate"). With the former, the
ports must support
security groups and there is nowhere except for the firewall
rules on that
port to implement it without violating the user's
expectation. With the
latter, a mapping driver could determine that communication
between these
two hosts can be prevented by using an ACL on a router or a
switch, which
doesn't violate the user's intent and buys a performance
improvement and
works with ports that don't support security groups.

Group based policy is trying to move the requests into the
declarative
abstraction so optimizations like the one above can be made.


Kevin, you have captured the GBP value prop and difference very
succinctly. The major difference is in the declarative (GBP) versus
imperative (current) style of programming.

This has been stated very clearly and explicitly in the blueprint
spec. If one does not appreciate the difference or advantage of one
over the other, then this discussion is pointless.


"One" does appreciate the value of having porcelain APIs overlay a
plumbing API. This discussion was about the proper way and place to
introduce such functionality.

However, it seems to me that the end-goal of the GBP effort is
*actually* to provide a higher-layer API to Neutron that would
essentially enable proprietary vendors to entirely swap out all of
Neutron core for a new set of drivers that spoke proprietary device
APIs.

If this is the end-goal, it should be stated more clearly, IMO.

The classic plumbing versus porcelain API conversation [1] is a good
one, and one worth having in the context of Neutron.

It's almost like some Neutron contributor folks are saying "let's
add a porcelain API so we can ditch all the existing plumbing APIs
and replace with our own stuff". And that's not what the point of
plumbing vs. porcelain is.

-jay

[1]
http://git-scm.com/book/en/__Git-Internals-Plumbing-and-__Porcelain



_
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.__org

http://lists.openstack.org/__cgi-bin/mailman/listinfo/__openstack-dev 





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On Wed, Aug 6, 2014 at 1:52 PM, Jay Pipes  wrote:
> On 08/06/2014 04:36 PM, Sumit Naiksatam wrote:
>>
>> On Wed, Aug 6, 2014 at 1:27 PM, Jay Pipes  wrote:
>>>
>>> On 08/06/2014 04:13 PM, Sumit Naiksatam wrote:


 On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:
>>
>>
>> I believe the referential security group rules solve this problem
>> (unless
>> I'm not understanding):
>
>
>
> I think the disconnect is that you are comparing the way to current
> mapping
> driver implements things for the reference implementation with the
> existing
> APIs. Under this light, it's not going to look like there is a point to
> this
> code being in Neutron since, as you said, the abstraction could happen
> at
> a
> client. However, this changes once new mapping drivers can be added
> that
> implement things differently.
>
> Let's take the security groups example. Using the security groups API
> directly is imperative ("put a firewall rule on this port that blocks
> this
> IP") compared to a higher level declarative abstraction ("make sure
> these
> two endpoints cannot communicate"). With the former, the ports must
> support
> security groups and there is nowhere except for the firewall rules on
> that
> port to implement it without violating the user's expectation. With the
> latter, a mapping driver could determine that communication between
> these
> two hosts can be prevented by using an ACL on a router or a switch,
> which
> doesn't violate the user's intent and buys a performance improvement
> and
> works with ports that don't support security groups.
>
> Group based policy is trying to move the requests into the declarative
> abstraction so optimizations like the one above can be made.
>

 Kevin, you have captured the GBP value prop and difference very
 succinctly. The major difference is in the declarative (GBP) versus
 imperative (current) style of programming.

 This has been stated very clearly and explicitly in the blueprint
 spec. If one does not appreciate the difference or advantage of one
 over the other, then this discussion is pointless.
>>>
>>>
>>>
>>> "One" does appreciate the value of having porcelain APIs overlay a
>>> plumbing
>>> API. This discussion was about the proper way and place to introduce such
>>> functionality.
>>>
>>> However, it seems to me that the end-goal of the GBP effort is *actually*
>>> to
>>> provide a higher-layer API to Neutron that would essentially enable
>>> proprietary vendors to entirely swap out all of Neutron core for a new
>>> set
>>> of drivers that spoke proprietary device APIs.
>>>
>>> If this is the end-goal, it should be stated more clearly, IMO.
>>>
>>
>> The goal and design intent is unambiguously stated in the spec [1];
>>
>> L36-L38
>> "The policy framework described in this blueprint complements the current
>> Neutron model with the notion of policies that can be applied between
>> groups of endpoints."
>>
>> Note the choice of words - "complements". The implementation also has
>> been faithful to this intent.
>>
>> I am not sure why you would draw the conclusion that you did.
>>
>> [1]
>> https://review.openstack.org/#/c/89469/10/specs/juno/group-based-policy-abstraction.rst,cm
>
>
> OK, cool. Then it seems to me that would be a perfect justification for
> having this code and API live outside of the Neutron tree, then?
>
> In other words, what benefit does having the GBP code in the Neutron
> codebase give us?
>

And for that I would again request that you look at the following
stated in the blueprint spec ;-):

L43-48
"This proposal suggests a model that allows application administrators to
express their networking requirements using group and policy
abstractions, with the specifics of policy enforcement and
implementation left to the underlying policy driver. The main
advantage of the extensions described in this blueprint is that they
allow for an application-centric interface to Neutron that complements
the existing network-centric interface."

L52-78 also spell out some of the details of these claims.


> -jay
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Deprecating CONF.block_device_allocate_retries_interval

2014-08-06 Thread Michael Still

Maybe we should change how we wait?

I get that we don't want to sit around forever, but perhaps we should
specify a total maximum time to wait instead of a number of iterations
of a loop? Something like "15 minutes should be long enough for
anyone!". Eventlet sleeps are also pretty cheap, so having a bigger
sleep time inside them just means that we overshoot more than we would
otherwise.

Michael

On Thu, Aug 7, 2014 at 3:54 AM, Jay Pipes  wrote:
> Hi Stackers!
>
> So, Liyi Meng has an interesting patch up for Nova:
>
> https://review.openstack.org/#/c/104876
>
> that changes the way that the interval and number of retries is calculated
> for a piece of code that waits around for a block device mapping to become
> active.
>
> Namely, the patch discards the value of the following configuration options
> *if the volume size is not 0* (which is a typical case):
>
> * CONF.block_device_allocate_retries_interval
> * CONF.block_device_allocate_retries
>
> and in their place, instead uses a hard-coded 60 max number of retries and
> calculates a more "appropriate" interval by looking at the size of the
> volume to be created. The algorithm uses the sensible idea that it will take
> longer to allocate larger volumes than smaller volumes, and therefore the
> interval time for larger volumes should be longer than smaller ones.
>
> So... here's the question: since this code essentially makes the above two
> configuration options obselete for the majority of cases (where volume size
> is not 0), should we do one of the following?
>
> 1) We should just deprecate both the options, with a note in the option help
> text that these options are not used when volume size is not 0, and that the
> interval is calculated based on volume size
>
> or
>
> 2) We should deprecate the CONF.block_device_allocate_retries_interval
> option only, and keep the CONF.block_device_allocate_retries configuration
> option as-is, changing the help text to read something like "Max number of
> retries. We calculate the interval of the retry based on the size of the
> volume."
>
> I bring this up on the mailing list because I think Liyi's patch offers an
> interesting future direction to the way that we think about our retry
> approach in Nova. Instead of having hard-coded or configurable interval
> times, I think Liyi's approach of calculating the interval length based on
> some input values is a good direction to take.
>
> Thoughts?
>
> -jay
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Rackspace Australia

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Edgar Magana

Ivar,

Of course and this is why we are having this conversation, in order to merge 
our different opinions.

Edgar

From: Ivar Lazzaro mailto:ivarlazz...@gmail.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Date: Wednesday, August 6, 2014 at 1:41 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way forward

Hi Edgar,

Actually, I think that other reviewers saw that name clash, and still thought 
it was ok to use the same terminology in such a different context.
BP reviews are a community effort right? So of course someones' idea may be 
different from yours.

Regards,
Ivar.


On Wed, Aug 6, 2014 at 10:29 PM, Edgar Magana 
mailto:edgar.mag...@workday.com>> wrote:
Basically, I am admitting that I did not catch in my review the part of
the endpoint term that Jay was pointing out.

Edgar

On 8/6/14, 11:32 AM, "Sumit Naiksatam" 
mailto:sumitnaiksa...@gmail.com>> wrote:

>Not sure what you are talking about? You claim now that you had
>suggestion which was not considered, yet you +2'ed a patch, by stating
>that "All looks good to me!".
>
>On Wed, Aug 6, 2014 at 11:19 AM, Edgar Magana 
>mailto:edgar.mag...@workday.com>>
>wrote:
>> That is the beauty of the open source projects, there is always a
>>smartest
>> reviewer catching out the facts that you don¹t.
>>
>> Edgar
>>
>> On 8/6/14, 10:55 AM, "Sumit Naiksatam" 
>> mailto:sumitnaiksa...@gmail.com>> wrote:
>>
>>>Edgar, you seemed to have +2'ed this patch on July 2nd [1]:
>>>
>>>"
>>>Edgar Magana
>>>Jul 2 8:42 AM
>>>
>>>Patch Set 13: Code-Review+2
>>>
>>>All looks good to me! I am not approving yet because Nachi was also
>>>reviewing this code and I would like to see his opinion as well.
>>>"
>>>
>>>That would suggest that you were happy with what was in it. I don't
>>>see anything in the review comments that suggests otherwise.
>>>
>>>[1]  https://review.openstack.org/#/c/95900/
>>>
>>>On Wed, Aug 6, 2014 at 10:39 AM, Edgar Magana 
>>>mailto:edgar.mag...@workday.com>>
>>>wrote:
 This is the consequence of a proposal that is not following the
standardized
 terminology (IETF - RFC) for any Policy-based System:
 http://tools.ietf.org/html/rfc3198

 Well, I did bring  this point during the Hong Kong Summit but as you
can see
 my comments were totally ignored:

https://docs.google.com/document/d/1ZbOFxAoibZbJmDWx1oOrOsDcov6Cuom5aaB
Ir
upCD9E/edit

 I clearly saw this kind of issues coming. Let me quote myself what I
 suggested: "For instance: "endpoints" should be "enforcement point"

 I do not understand why GBP did not include this suggestionŠ

 Edgar

 From: Kevin Benton mailto:blak...@gmail.com>>
 Reply-To: "OpenStack Development Mailing List (not for usage
questions)"
 mailto:openstack-dev@lists.openstack.org>>
 Date: Wednesday, August 6, 2014 at 10:22 AM
 To: "OpenStack Development Mailing List (not for usage questions)"
 mailto:openstack-dev@lists.openstack.org>>

 Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way
 forward

 What I was referring to was also not Keystone's definition of an
endpoint.
 It's almost as if the term has many uses and was not invented for
Keystone.
 :-)

 http://www.wireshark.org/docs/wsug_html_chunked/ChStatEndpoints.html

 Did a similar discussion occur when Heat wanted to use the word
'template'
 since this was clearly already in use by Horizon?

 On Aug 6, 2014 9:24 AM, "Jay Pipes" 
 mailto:jaypi...@gmail.com>> wrote:
>
> On 08/06/2014 02:12 AM, Kevin Benton wrote:
>>
>> Given that, pointing to the Nova parity work seems a bit like a red
>> herring. This new API is being developed orthogonally to the
>>existing
>> API endpoints
>
>
> You see how you used the term endpoints there? :P
>
> -jay
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

>>>
>>>___
>>>OpenStack-dev mailing list
>>>OpenStack-dev@lists.openstack.org
>>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://li

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On 08/06/2014 04:36 PM, Sumit Naiksatam wrote:

On Wed, Aug 6, 2014 at 1:27 PM, Jay Pipes wrote:

On 08/06/2014 04:13 PM, Sumit Naiksatam wrote:

On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton wrote:

I believe the referential security group rules solve this problem
(unless
I'm not understanding):

I think the disconnect is that you are comparing the way to current
mapping
driver implements things for the reference implementation with the
existing
APIs. Under this light, it's not going to look like there is a point to
this
code being in Neutron since, as you said, the abstraction could happen at
a
client. However, this changes once new mapping drivers can be added that
implement things differently.

Let's take the security groups example. Using the security groups API
directly is imperative ("put a firewall rule on this port that blocks
this
IP") compared to a higher level declarative abstraction ("make sure these
two endpoints cannot communicate"). With the former, the ports must
support
security groups and there is nowhere except for the firewall rules on
that
port to implement it without violating the user's expectation. With the
latter, a mapping driver could determine that communication between these
two hosts can be prevented by using an ACL on a router or a switch, which
doesn't violate the user's intent and buys a performance improvement and
works with ports that don't support security groups.

Group based policy is trying to move the requests into the declarative
abstraction so optimizations like the one above can be made.

Kevin, you have captured the GBP value prop and difference very
succinctly. The major difference is in the declarative (GBP) versus
imperative (current) style of programming.

This has been stated very clearly and explicitly in the blueprint
spec. If one does not appreciate the difference or advantage of one
over the other, then this discussion is pointless.

"One" does appreciate the value of having porcelain APIs overlay a plumbing
API. This discussion was about the proper way and place to introduce such
functionality.

However, it seems to me that the end-goal of the GBP effort is *actually* to
provide a higher-layer API to Neutron that would essentially enable
proprietary vendors to entirely swap out all of Neutron core for a new set
of drivers that spoke proprietary device APIs.

If this is the end-goal, it should be stated more clearly, IMO.

The goal and design intent is unambiguously stated in the spec [1];

L36-L38
"The policy framework described in this blueprint complements the current
Neutron model with the notion of policies that can be applied between
groups of endpoints."

Note the choice of words - "complements". The implementation also has
been faithful to this intent.

I am not sure why you would draw the conclusion that you did.

[1]
https://review.openstack.org/#/c/89469/10/specs/juno/group-based-policy-abstraction.rst,cm

OK, cool. Then it seems to me that would be a perfect justification for
having this code and API live outside of the Neutron tree, then?

In other words, what benefit does having the GBP code in the Neutron
codebase give us?

-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Prasad Vellanki

Jay
Doesnt the current plugin model in neutron work the way you are saying. We
have a a core set of APIs that there is a reference model for and
individual vendors have substituted plugins that enhance and sometimes
replace networking component. GBP in that respect does not change. There is
a reference implementation in neutron for declarative model in neutron and
vendors can substitute their implementation to enhance what is in
reference.

But what you need to understand is the declarative model that it provides
which Ryan has elaborated which current neutron does not provide.

prasadv


On Wed, Aug 6, 2014 at 1:27 PM, Jay Pipes  wrote:

> On 08/06/2014 04:13 PM, Sumit Naiksatam wrote:
>
>> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:
>>
>>> I believe the referential security group rules solve this problem (unless
 I'm not understanding):

>>>
>>> I think the disconnect is that you are comparing the way to current
>>> mapping
>>> driver implements things for the reference implementation with the
>>> existing
>>> APIs. Under this light, it's not going to look like there is a point to
>>> this
>>> code being in Neutron since, as you said, the abstraction could happen
>>> at a
>>> client. However, this changes once new mapping drivers can be added that
>>> implement things differently.
>>>
>>> Let's take the security groups example. Using the security groups API
>>> directly is imperative ("put a firewall rule on this port that blocks
>>> this
>>> IP") compared to a higher level declarative abstraction ("make sure these
>>> two endpoints cannot communicate"). With the former, the ports must
>>> support
>>> security groups and there is nowhere except for the firewall rules on
>>> that
>>> port to implement it without violating the user's expectation. With the
>>> latter, a mapping driver could determine that communication between these
>>> two hosts can be prevented by using an ACL on a router or a switch, which
>>> doesn't violate the user's intent and buys a performance improvement and
>>> works with ports that don't support security groups.
>>>
>>> Group based policy is trying to move the requests into the declarative
>>> abstraction so optimizations like the one above can be made.
>>>
>>>
>> Kevin, you have captured the GBP value prop and difference very
>> succinctly. The major difference is in the declarative (GBP) versus
>> imperative (current) style of programming.
>>
>> This has been stated very clearly and explicitly in the blueprint
>> spec. If one does not appreciate the difference or advantage of one
>> over the other, then this discussion is pointless.
>>
>
> "One" does appreciate the value of having porcelain APIs overlay a
> plumbing API. This discussion was about the proper way and place to
> introduce such functionality.
>
> However, it seems to me that the end-goal of the GBP effort is *actually*
> to provide a higher-layer API to Neutron that would essentially enable
> proprietary vendors to entirely swap out all of Neutron core for a new set
> of drivers that spoke proprietary device APIs.
>
> If this is the end-goal, it should be stated more clearly, IMO.
>
> The classic plumbing versus porcelain API conversation [1] is a good one,
> and one worth having in the context of Neutron.
>
> It's almost like some Neutron contributor folks are saying "let's add a
> porcelain API so we can ditch all the existing plumbing APIs and replace
> with our own stuff". And that's not what the point of plumbing vs.
> porcelain is.
>
> -jay
>
> [1] http://git-scm.com/book/en/Git-Internals-Plumbing-and-Porcelain
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

2014-08-06 Thread Pedro Marques

On Aug 6, 2014, at 1:27 PM, Jay Pipes  wrote:
> 
> However, it seems to me that the end-goal of the GBP effort is *actually* to 
> provide a higher-layer API to Neutron that would essentially enable 
> proprietary vendors to entirely swap out all of Neutron core for a new set of 
> drivers that spoke proprietary device APIs.
> 
> If this is the end-goal, it should be stated more clearly, IMO.

I believe that people should be considered innocent until proven otherwise. Is 
there a reason to believe there is some hidden reason behind this proposal ? It 
seems to me that this is uncalled for.

Neutron allows vendors to speak to proprietary device APIs, it was designed to 
do so, AFAIK. It is also possibly to "entirely swap out all of the Neutron 
core"... the proponents of the group based policy didn't have to go through so 
much trouble if that was their intent. As far as i know most plugins talk to a 
proprietary API.

I happen to disagree technically with a couple of choices made by this 
proposal; but the blueprint was approved. Which means that i lost the argument, 
or didn't raise it on time, or didn't argue convincingly... regardless of the 
reason, the time to argue about the goal has passed. The decision of the 
community was to approve the spec and that decision should be respected.

  Pedro.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] How to improve the specs review process (was Re: [Neutron] Group Based Policy and the way forward)

On Wed 06 Aug 2014 01:21:26 PM PDT, Eugene Nikanorov wrote:
> So I don't think it's fair to blame reviewers here.

Just want to be super clear: there is no 'blaming' here. This is a
request to regroup and look at why we are having this conversation about
GBP now, this late in cycle, and about such fundamental topics (the
choice of 'endpoint' as name is only one of them), after in-person
conversations over more than one release cycle and summits.

I am available for the meeting on Monday, Kyle.

In order to prepare for the meeting we should agree on the scope of the
root cause analysis. I think the problem should be framed around the
message Mark McClain sent, especially the "Why this email" which I quote
below:

> Our community has been discussing and working on Group Based Policy
> (GBP) for many months.  I think the discussion has reached a point
> where we need to openly discuss a few issues before moving forward.
[...]

I think the fact that this very fair question has been raised so late is
the problem we need to find the cause for. Would you agree?

We'll use time during the meeting on Monday to use a simple technique to
investigate this deeply, no need to spend time now and via email.

/stef

-- 
Ask and answer questions on https://ask.openstack.org

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Group Based Policy and the way forward

Hi Edgar,

Actually, I think that other reviewers saw that name clash, and still
thought it was ok to use the same terminology in such a different context.
BP reviews are a community effort right? So of course someones' idea may be
different from yours.

Regards,
Ivar.


On Wed, Aug 6, 2014 at 10:29 PM, Edgar Magana 
wrote:

> Basically, I am admitting that I did not catch in my review the part of
> the endpoint term that Jay was pointing out.
>
> Edgar
>
> On 8/6/14, 11:32 AM, "Sumit Naiksatam"  wrote:
>
> >Not sure what you are talking about? You claim now that you had
> >suggestion which was not considered, yet you +2'ed a patch, by stating
> >that "All looks good to me!".
> >
> >On Wed, Aug 6, 2014 at 11:19 AM, Edgar Magana 
> >wrote:
> >> That is the beauty of the open source projects, there is always a
> >>smartest
> >> reviewer catching out the facts that you don¹t.
> >>
> >> Edgar
> >>
> >> On 8/6/14, 10:55 AM, "Sumit Naiksatam" 
> wrote:
> >>
> >>>Edgar, you seemed to have +2'ed this patch on July 2nd [1]:
> >>>
> >>>"
> >>>Edgar Magana
> >>>Jul 2 8:42 AM
> >>>
> >>>Patch Set 13: Code-Review+2
> >>>
> >>>All looks good to me! I am not approving yet because Nachi was also
> >>>reviewing this code and I would like to see his opinion as well.
> >>>"
> >>>
> >>>That would suggest that you were happy with what was in it. I don't
> >>>see anything in the review comments that suggests otherwise.
> >>>
> >>>[1]  https://review.openstack.org/#/c/95900/
> >>>
> >>>On Wed, Aug 6, 2014 at 10:39 AM, Edgar Magana  >
> >>>wrote:
>  This is the consequence of a proposal that is not following the
> standardized
>  terminology (IETF - RFC) for any Policy-based System:
>  http://tools.ietf.org/html/rfc3198
> 
>  Well, I did bring  this point during the Hong Kong Summit but as you
> can see
>  my comments were totally ignored:
> 
> 
> https://docs.google.com/document/d/1ZbOFxAoibZbJmDWx1oOrOsDcov6Cuom5aaB
> Ir
> upCD9E/edit
> 
>  I clearly saw this kind of issues coming. Let me quote myself what I
>  suggested: "For instance: "endpoints" should be "enforcement point"
> 
>  I do not understand why GBP did not include this suggestionŠ
> 
>  Edgar
> 
>  From: Kevin Benton 
>  Reply-To: "OpenStack Development Mailing List (not for usage
> questions)"
>  
>  Date: Wednesday, August 6, 2014 at 10:22 AM
>  To: "OpenStack Development Mailing List (not for usage questions)"
>  
> 
>  Subject: Re: [openstack-dev] [Neutron] Group Based Policy and the way
>  forward
> 
>  What I was referring to was also not Keystone's definition of an
> endpoint.
>  It's almost as if the term has many uses and was not invented for
> Keystone.
>  :-)
> 
>  http://www.wireshark.org/docs/wsug_html_chunked/ChStatEndpoints.html
> 
>  Did a similar discussion occur when Heat wanted to use the word
> 'template'
>  since this was clearly already in use by Horizon?
> 
>  On Aug 6, 2014 9:24 AM, "Jay Pipes"  wrote:
> >
> > On 08/06/2014 02:12 AM, Kevin Benton wrote:
> >>
> >> Given that, pointing to the Nova parity work seems a bit like a red
> >> herring. This new API is being developed orthogonally to the
> >>existing
> >> API endpoints
> >
> >
> > You see how you used the term endpoints there? :P
> >
> > -jay
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
>  ___
>  OpenStack-dev mailing list
>  OpenStack-dev@lists.openstack.org
>  http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> >>>
> >>>___
> >>>OpenStack-dev mailing list
> >>>OpenStack-dev@lists.openstack.org
> >>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> >> ___
> >> OpenStack-dev mailing list
> >> OpenStack-dev@lists.openstack.org
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >___
> >OpenStack-dev mailing list
> >OpenStack-dev@lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward

On Wed, Aug 6, 2014 at 1:27 PM, Jay Pipes  wrote:
> On 08/06/2014 04:13 PM, Sumit Naiksatam wrote:
>>
>> On Wed, Aug 6, 2014 at 12:46 PM, Kevin Benton  wrote:

 I believe the referential security group rules solve this problem
 (unless
 I'm not understanding):
>>>
>>>
>>> I think the disconnect is that you are comparing the way to current
>>> mapping
>>> driver implements things for the reference implementation with the
>>> existing
>>> APIs. Under this light, it's not going to look like there is a point to
>>> this
>>> code being in Neutron since, as you said, the abstraction could happen at
>>> a
>>> client. However, this changes once new mapping drivers can be added that
>>> implement things differently.
>>>
>>> Let's take the security groups example. Using the security groups API
>>> directly is imperative ("put a firewall rule on this port that blocks
>>> this
>>> IP") compared to a higher level declarative abstraction ("make sure these
>>> two endpoints cannot communicate"). With the former, the ports must
>>> support
>>> security groups and there is nowhere except for the firewall rules on
>>> that
>>> port to implement it without violating the user's expectation. With the
>>> latter, a mapping driver could determine that communication between these
>>> two hosts can be prevented by using an ACL on a router or a switch, which
>>> doesn't violate the user's intent and buys a performance improvement and
>>> works with ports that don't support security groups.
>>>
>>> Group based policy is trying to move the requests into the declarative
>>> abstraction so optimizations like the one above can be made.
>>>
>>
>> Kevin, you have captured the GBP value prop and difference very
>> succinctly. The major difference is in the declarative (GBP) versus
>> imperative (current) style of programming.
>>
>> This has been stated very clearly and explicitly in the blueprint
>> spec. If one does not appreciate the difference or advantage of one
>> over the other, then this discussion is pointless.
>
>
> "One" does appreciate the value of having porcelain APIs overlay a plumbing
> API. This discussion was about the proper way and place to introduce such
> functionality.
>
> However, it seems to me that the end-goal of the GBP effort is *actually* to
> provide a higher-layer API to Neutron that would essentially enable
> proprietary vendors to entirely swap out all of Neutron core for a new set
> of drivers that spoke proprietary device APIs.
>
> If this is the end-goal, it should be stated more clearly, IMO.
>

The goal and design intent is unambiguously stated in the spec [1];

L36-L38
"The policy framework described in this blueprint complements the current
Neutron model with the notion of policies that can be applied between
groups of endpoints."

Note the choice of words - "complements". The implementation also has
been faithful to this intent.

I am not sure why you would draw the conclusion that you did.

[1] 
https://review.openstack.org/#/c/89469/10/specs/juno/group-based-policy-abstraction.rst,cm

> The classic plumbing versus porcelain API conversation [1] is a good one,
> and one worth having in the context of Neutron.
>
> It's almost like some Neutron contributor folks are saying "let's add a
> porcelain API so we can ditch all the existing plumbing APIs and replace
> with our own stuff". And that's not what the point of plumbing vs. porcelain
> is.
>
> -jay
>
> [1] http://git-scm.com/book/en/Git-Internals-Plumbing-and-Porcelain
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Fwd: FW: [Neutron] Group Based Policy and the way forward