As Daya suggested, I will be using single priority per SG to handle this issue 
in Boron.

Sequencing at SG level and SR level needs to be handled later.






[] On Behalf Of Aswin 
Sent: Monday, October 17, 2016 12:59 PM
To: Dayavanti Gopal Kamath <>
Subject: Re: [netvirt-dev] [openflowplugin-dev] Need inputs on handling 
deletion of flows which have common attributes



The priorities of flows need not be across the tenant space I think  as there 
will not be any successful match in Tenant A SG rules for Tenant B traffic. So 
to make it deterministic it will be sufficient if we make the priories of SG/SR 
different with in a tenant and thus we can reuse the priority  values across 
tenants . 

But I am not sure about the real world numbers. If 255 group and 255 rules are 
too low within a tenant, then as you mentioned we think of solving the Som's 
issue first by making use of SG priority for group now.


We delete rules individually now, and having a cookie per group should help us 
in deleting all the rules in a group with a single request. We can do this an 
optimization, but this will not help us in making the implementation 
deterministic right? Am I missing anything?




On Sun, Oct 16, 2016 at 4:01 PM, Dayavanti Gopal Kamath 
<> > wrote:

Hi vishal,

Probably too advanced at this point to get quota params for each tenant, also I 
don’t think we can do much with them. since the priorities of the flows are 
across tenant space, we can only have 1 such limit, total number of groups 
across all tenants, and max number of rules per group across all groups. 
however, priority is only 16 bits, if we split up the space in half, we get 255 
rules per group, and 255 groups, which seems way too less number of groups. We 
probably have to limit to about 31 rules per group, and a total of 2k groups.  
This is also a bit low, anyone has ideas on typical deployment numbers? 


I’m re-thinking if prio space should be used for both, from a scale perspective.

Alternatively, just do unordered list within groups, and add a single priority 
per group. Then we support ordered lists whenever such a contract is needed 
from the NB separately. This will still solve som’s original problem.





From: Vishal Thapar 
Sent: Saturday, October 15, 2016 1:50 PM
To: Aswin Suryanarayanan < <> >; 
Dayavanti Gopal Kamath < 
<> >
Cc: <> ; 
<> ; <> 

Subject: RE: [netvirt-dev] [openflowplugin-dev] Need inputs on handling 
deletion of flows which have common attributes


Hi Aswin,


Yes, We can leverage IdManager for this, by creating a pool for every SG and 
getting Ids for individual SRs.


Limits on no. of SGs per project and SRs per SGs are configurable parameters. 
If we can figure out a way to get this information from neutron conf files, it 
would make it a better solution. [1]


I would still prefer considering cookies as a means to identify which flows to 
delete. I might be remembering incorrectly but I believe it performs faster 
because of smaller match [cookie field only] and wildcard support means we can 
add bulk deletes like ‘delete all flows for SG1’ or ‘delete all flows of tenant 







[] On Behalf Of Aswin 
Sent: 14 October 2016 19:08
To: Dayavanti Gopal Kamath < 
<> >
Cc: <> ; 
<> ; <> 
Subject: Re: [netvirt-dev] [openflowplugin-dev] Need inputs on handling 
deletion of flows which have common attributes



Yes having different set of priority for SG and SR was my thought as well. Do 
the ID manager or any component in genius can help in creating these priorities 
?. I think we need think of making these changes in the NeutronVpnListener side 
and AclService should be just using the values  as an offset to compute  the 
final flow priority.

It affects the no of groups/rules, but I think already there is a default limit 
for the tenant in the number of SG that can be created, so if we choose 
sufficiently large data type  for priority it should addressed to an extent.




On Fri, Oct 14, 2016 at 4:55 PM, Dayavanti Gopal Kamath 
<> > wrote:

Hi Ashwin,

So, basically, to get the ordered list within the SG, we can apply diff 
priorities for the rules also for the dataplane, right? At the same time, each 
SG will need a diff prio range, so that all rules within a given SG can be in 
consecutive priorities, whereas all rules within a diff SG, can be in a diff 
set of prios. This means we could break up the prio space into 2 sets, with 
MSBs getting assigned to SGs, and LSBs getting assigned to the rules 

It reduces the number of rules/groups we can support overall though, to achieve 
this correctness. 





From: Aswin Suryanarayanan [ 
<> ] 
Sent: Friday, October 14, 2016 2:22 PM
To: Dayavanti Gopal Kamath < 
<> >
Cc: Somashekar Byrappa < 
<> >; 
<> ; 
<> ; 

Subject: Re: [netvirt-dev] [openflowplugin-dev] Need inputs on handling 
deletion of flows which have common attributes



We can think of deterministic implementation as well since it will solve 
problem we are facing now.  As mentioned in one of the solution of changing the 
priority based on SG can make it more deterministic. But we need to have 
priority associated with the SG itself as otherwise it will be challenging to 
identify first and last SG as we can add and remove them randomly. Since 
openstack does not provide any sequencing , we  require a logic which can 
associate the priority to the SG in netvirt.

This will not solve the issue Vishal mentioned where two SR( security rule )in 
a SG with one rule being the subset of other, we will not able to predict which 
will be hit first as all SR in a SG still have same priority.






On Thu, Oct 13, 2016 at 6:00 PM, Dayavanti Gopal Kamath 
<> > wrote:

Hi som,

while SG may work this way, and ordering might not matter, as a service ACL 
should definitely implement it as an ordered list, to be more generic, and be 
able to support other classifier based features with minimal changes. There was 
goal for SGs at some point (long back, I think during G or H ), to add deny 
acl’s, but I don’t know the latest status. 


Question for all, how does the netvirt community feel about changing SG impl to 
be an ordered list for Carbon. 


Prio-wise, we do get some uncertainty between different SGs applied on a port, 
since Openstack does not provide any sequencing between SGs but that’s a 
problem for the operator to solve I think,the code should only ensure default 
SG has the lowest priority, and we could apply SGs in reduced priority order as 
and when they are applied, ie first group gets highest prio, and last group 
gets lowest prio.







From: Somashekar B [ 
<> ] 
Sent: Thursday, October 13, 2016 4:53 PM
To: Dayavanti Gopal Kamath < 
<> >; 
<> ; 
<> ; 

Subject: RE: [netvirt-dev] [openflowplugin-dev] Need inputs on handling 
deletion of flows which have common attributes


Hi Daya,


Currently, all the flows corresponding to SG rules (irrespective of SG) have 
same priority.

One basic question here, does ordering at a SG level or SG rules level really 
matter? Becoz whatever we specify in SG rules are for allow traffic only. There 
are no deny rules. So I believe sequencing shouldn't matter unless if we are 
specifically looking for flow statistics.  Also there are no attributes in 
openstack which specifically mention about the sequence numbers. SG and rules 
are just passed as an array to ODL. So in case if we have to maintain 
sequencing, we will be following the array indexing.


With your inputs on making use of priority, I think we can have different 
priorities based on port + SG. This would create multiple flows on switch. So 
the issue doesn’t arise.

Hope SG priorities doesn’t matter.


Please share your thoughts.






From: Dayavanti Gopal Kamath [] 
Sent: Thursday, October 13, 2016 2:18 PM
To: Somashekar Byrappa < 
<> >; 
<> ; 
<> ; 
Subject: RE: [netvirt-dev] [openflowplugin-dev] Need inputs on handling 
deletion of flows which have common attributes


Hi som,

I would question the need for this usecase itself.


sg rules would be an ordered list, which need to be applied in sequence, so 
when these rules are programmed in the tables, we cannot have any interleaving 
between rules from different SGs. I think this is a contract that cannot be 
violated. For e.g

if SG1 has rule1, rule2, rule3 and SG2 has rule4,rule2, rule5. We need to make 
sure the tables contain 2 instances of rule 2, in both these ordered lists. The 
relative priority between all rules of SG1 and all rules of SG2 is a separate 
issue of course, and a separate discussion on whether these prios are 

Basic point is, we need to ensure the sequencing is intact within a group, 
either by adjusting the priorities or adding more specific match criteria, for 
different SGs. 


2ndly, and more broadly, this does not sound typical to apply multiple SGs to 
the same VM, and additionally have the same rule in each such SG. This would 
call for some re-organization of the SG rules themselves, so from my 
perspective, we need not have huge design changes in the code to support such a 
use case.





[] On Behalf Of Somashekar B
Sent: Thursday, October 13, 2016 12:49 PM
To: <> ; <> 
Subject: Re: [netvirt-dev] [openflowplugin-dev] Need inputs on handling 
deletion of flows which have common attributes


Thanks Jozef for your inputs.

Maybe a common module needs to be written which can handle this scenario 
instead of every application module handling on their own.


Anymore inputs from others? Or else for time being, I will handle this in 
security groups module itself.





From: Jozef Bacigál [] 
Sent: Friday, October 7, 2016 3:58 PM
To: Somashekar B < 
<> >; <> 
Subject: RE: [openflowplugin-dev] Need inputs on handling deletion of flows 
which have common attributes


Hi Somashekar,


from Plugin POV it is quite impossible to handle this use case without a 
performance impact. On the device you can’t store two identical flows and you 
can’t store flow id as you mentioned below. But the plugins reconciliation 
working on change event so if you delete one flow on configuration it will be 
deleted on switch and reconciliation won’t start check all configuration unless 
you disconnect device. So yes, it would be better approach to handle this use 
case from the application side instead to let the plugin always check 
everything on each event.




From: Somashekar B [ <>] 
Sent: Thursday, October 6, 2016 1:09 PM
To:  <>;  
Subject: [openflowplugin-dev] Need inputs on handling deletion of flows which 
have common attributes


Hi All,


I am looking for all your inputs for the below issue.


Issue: Create multiple flows with different flowId, keeping rest of the 
attributes same.

In this case, there will be multiple flow entries in config DS, but a single 
flow on the switch.

When we delete one of this overlapping flow, flow gets deleted on the switch.


Use case:


We can associate multiple security groups to a VM from Openstack. 

Each rule in the security group (SG) generally gets translated to a flow in one 
of the ACL tables (41/252) based on the direction (ingress/egress).

Let’s say we create multiple SGs having few overlapping rules (highlighted in 
yellow) among each other like below:


| sg1     | ingress, IPv4, 22/tcp, remote_ip_prefix: 
<>                |

|            | ingress, IPv4, icmp, remote_ip_prefix: 
<>                   |

| sg2     | ingress, IPv4, 22/tcp, remote_ip_prefix: 
<>                |



When we associate both SGs (sg1 and sg2) to a VM, ACL module creates three flow 
entries totally in config DS. 

For the above two overlapping rules, two flow entries are created in config DS 
with different flowId, rest all the other attributes remains same.

This would result in a single flow on the switch as flowId is not a flow 
attribute on the switch.


cookie=0x6900000, duration=11.895s, table=252, n_packets=0, n_bytes=0, 
 <> ,tp_dst=22 actions=ct(commit,zone=5000),resubmit(,220)

cookie=0x6900000, duration=75.237s, table=252, n_packets=0, n_bytes=0, 
 <>  actions=ct(commit,zone=5000),resubmit(,220)


Problem arises when we dissociate one of the SGs (let’s say, sg2) from the VM. 
In this case, we delete one of the overlapping flow from config DS.

This would result in deletion of flow from the switch. Even though there is 
another flow entry (having same data with different flow ID) in the config DS, 
this flow gets deleted from the switch.

This is how we generally handle all DCN/DTCN.


This is a common problem which might occur in other situations as well.


Question is how to handle this kind of scenario?

Which module should handle this? Plugin or applications?


In case, if we need to handle it from the applications side, we could think of 
below option:


1.       When adding a flow to config DS,

a.       Construct the flow id using the match criteria which would generate 
same flow ID for all the overlapping rules. So that only one flow will be 
created for all overlapping rules.

b.       With the constructed flowId, query config DS to check if flow already 

                                                               i.      If no, 
add this flow with flow name something like: ACL-<counter>. Counter would 
signify the number of overlapping flows configured.

                                                             ii.      If yes,  
update the flow name by incrementing the counter. Eg: ACL-2

2.       In case of deleting flow from config DS,

a.       Construct the flow ID as mentioned in 1.a

b.        With the constructed flowId, query config DS to check if flow already 

                                                               i.      If no, 

                                                             ii.      If yes,  
check the counter value in the flow name (ACL-2), if its value is 1, delete the 
flow else decrement the counter and update the flow in config DS.


Cons of this approach: 

                One additional read call to config DS during addition/deletion 
of every flow.


Please share your thoughts to handle it optimally. 






Software Engineer

Sídlo / Mlynské Nivy 56 / 821 05 Bratislava / Slovakia
R&D centrum / Janka Kráľa 9 /  974 01 Banská Bystrica / Slovakia
+421 908 766 972 /  <>
reception: +421 2 206 65 114 /  <>


netvirt-dev mailing list <>




openflowplugin-dev mailing list

Reply via email to