Re: [openstack-dev] [neutron][FWaaS] __init__ arguments issue status

[Frances, Margaret]

Hi Doug.

The old and new MROs are both pretty complicated, and it’s not entirely clear 
to me yet why the original one worked. (The MROs are included below for reading 
pleasure; they're embellished to show the incoming args to self’s init and 
outgoing args to super’s init in each case.)

I’m fairly sure the APIs for the mixins can be made the same, and I’ll try 
that.  But I still wonder if in fact the problem is a base class ordering issue.

The error that 223343 produced occurs in method call #6 in the "AFTER" MRO, 
where we get the following trace:

super(PeriodicTasks, self).__init__()
TypeError: __init__() takes exactly 2 arguments (1 given)


For grins, we changed PeriodicTasks’s call to super init as suggested by the 
trace:

super(PeriodicTasks, self).__init__(conf)


At this point FWaaSAgentRpcCallbackMixin (AFTER, #8) complained:

super(FWaaSAgentRpcCallbackMixin, self).__init__(host)
TypeError: object.__init__() takes no parameters


Changing *that* class as suggested elicited the following (to me baffling) 
result:

super(FWaaSAgentRpcCallbackMixin, self).__init__()
TypeError: __init__() takes exactly 2 arguments (1 given)


I find it baffling because FWaaSAgentRpcCallbackMixin is the end of the line, 
it’s a subclass of object, and object doesn’t allow arguments to init (so whose 
init is that? that’s the next thing I’m going to look at).  (It’s for these 
same reasons that I don’t understand why things worked before the 223343 
change.)

I’m still looking at things.  (And learning about MRO, which I’ve never really 
dealt with before.)  Will run pdb and see what surfaces.

Thanks for your help.  Thoughts, comments, suggestions all welcome.
Margaret


BEFORE 223343
 1. varmour_router_vArmourL3NATAgent(host, conf)-->(host, conf)
 2. agent_L3NATAgent(host, conf)-->(conf)
 3. firewall_l3_agent_FWaaSL3AgentRpcCallback   (conf)-->(host)
 4. api_FWaaSAgentRpcCallbackMixin  (host)-->(host)
 5. ha_AgentMixin   (host)-->(host)
 6. dvr_AgentMixin  (host)-->(host)
 7. manager_Manager (host)-->(conf)
 8. periodic_task_PeriodicTasks (conf)-->()
 9. firewall_l3_agent_FWaaSL3AgentRpcCallback   (conf)-->(host)
10. api_FWaaSAgentRpcCallbackMixin  (host)-->(host)
11. object

AFTER 223343
 1. varmour_router_vArmourL3NATAgent(host, conf)-->(host, conf)
 2. agent_L3NATAgent(host, conf)-->(host)
 3. ha_AgentMixin   (host)-->(host)
 4. dvr_AgentMixin  (host)-->(host)
 5. manager_Manager (host)-->(conf)
 6. periodic_task_PeriodicTasks (conf)-->()
 7. firewall_l3_agent_FWaaSL3AgentRpcCallback   (conf)-->(host)
 8. api_FWaaSAgentRpcCallbackMixin  (host)-->(host)
 9. object

--
Margaret Frances
Eng 4, Prodt Dev Engineering



> On May 5, 2016, at 7:06 PM, Doug Hellmann  wrote:
> 
> Excerpts from Nate Johnston's message of 2016-05-05 17:40:13 -0400:
>> FWaaS team,
>> 
>> After a day of looking at the tests currently failing in the FWaaS repo, I
>> believe I have the issue narrowed down considerably. First, to restate what
>> is going on.  If you check out the neutron-fwaas repository and run `tox -e
>> py27` in it, you will get six errors all in the
>> neutron_fwaas.tests.unit.services.firewall.agents.varmour.test_varmour_router.TestVarmourRouter
>> section.
>> Running the py34 tests results in similar problems.  The failures follow
>> the following form:
>> 
>> Captured traceback:
>> 
>> ~~~
>> 
>>Traceback (most recent call last):
>> 
>>  File
>> "neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
>> line 190, in test_agent_external_gateway
>> 
>>router = self._create_router()
>> 
>>  File
>> "neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
>> line 87, in _create_router
>> 
>>router = varmour_router.vArmourL3NATAgent(HOSTNAME, self.conf)
>> 
>>  File
>> "neutron_fwaas/services/firewall/agents/varmour/varmour_router.py", line
>> 54, in __init__
>> 
>>super(vArmourL3NATAgent, self).__init__(host, conf)
>> 
>>  File "/home/njohns002/
>> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py",
>> line 244, in __init__
>> 
>>super(L3NATAgent, self).__init__(host=self.conf.host)
>> 
>>  File "/home/njohns002/
>> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/ha.py",
>> line 93, in __init__
>> 
>>super(AgentMixin, self).__init__(host)
>> 
>>  File "/home/njohns002/
>> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/dvr.py",
>> line 30, in __init__
>> 
>>super(AgentMixin, self).__init__(host)
>> 
>> 

Re: [openstack-dev] [neutron] [qos] gathering Friday 9:30

[Frances, Margaret]

Hi Miguel,

Nate and I have a conflicting meeting, about FWaaS, at 9:30.  I’m reaching out 
to him now to see how we might fan out to get coverage.

Margaret

--
Margaret Frances
Eng 4, Prodt Dev Engineering



> On Apr 28, 2016, at 6:02 PM, Miguel Angel Ajo Pelayo  
> wrote:
> 
> Does governors ballroom in Hilton sound ok?
> 
> We can move to somewhere else if necessary.
> 



signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Sender Auth Failure] Re: [Sender Auth Failure] Re: [neutron] How could an L2 agent extension access agent methods ?

[Frances, Margaret]

Hello Ihar,

On 12/17/15, 11:00 AM, "Ihar Hrachyshka"  wrote:

>Margaret  wrote:
>
>>Hello Ihar,
>>
>>I have some comments and questions about your proposal.  My apologies if
>>any of what I say here results from misunderstandings on my part.
>
>Thanks a lot for the reply. I will try to clear up below.
>
>>
>>1. I believe there are two sorts of redirection at play here.  The first
>>involves inter-table traversal while the second allows a frame to exit
>>the
>>OF pipeline either by being sent to a different port or by being dropped.
>>Some of what I say next makes use of this distinction.
>>
>
>I am not sure I understand what is inter-table traversal. My assumption
>was  
>that extension tables defined for modification actions don’t redirect
>anywhere except back into table0 (using default flow rules we add to each
> 
>table on its bootstrapping).

OpenFlow’s Goto instruction and OvS’ resubmit action both result in
inter-table traversal in the sense I meant above.  This is as opposed to
what you've called a 'redirection flow table,' which sends the packet out
of the pipeline altogether.  I wanted to distinguish these two things in
part because the proposal describes table 0 flow entries ‘redirecting'
frames to another OF table and vice versa.

>
>>2. OpenFlow¹s Goto instruction directs a frame from one table to the
>>next.
>>  A redirection in this sense must be to a higher-numbered table, which
>>is
>>to say that OF pipeline processing can only go forward (see p.18, para.2
>>of the 1.4.1 spec
>>>f-
>>specifications/openflow/openflow-switch-v1.4.1.pdf>).  However, OvS (at
>>least v2.0.2) implements a resubmit action, which re-searches another
>>table‹higher-, lower-, or even same-numbered‹and executes any actions
>>found there in addition to any subsequent actions in the current flow
>>entry.  It is by using resubmit that the proposed design could work, as
>>shown in the ovs-ofctl command posted here
>>>y>
>>.  (Maybe there are other ways, too.)  The resubmit action is a Nicira
>>vendor extension that at least at one point, and maybe still, was known
>>to
>>be implemented only by OvS.  I mention this because I wonder if the
>>proposed design (and my sample command) calls for flow traversal in a
>>manner not explicitly supported by OpenFlow and so may not work in future
>>versions of OvS.
>>
>
>Thanks for directions to specific OF features we can utilize! I believe
>we  
>may assume some implementation of resubmit can be safely expected to be
>present in the OVS. We may refine API we rely on later if we see it
>deprecated.
>
>>3. Regarding the idea of sorting feature flows by kind: I believe that
>>what is meant by a 'redirection flow table' is a table that could
>>possibly
>>remove the frame from OF pipeline processing‹i.e., by forwarding or
>>dropping it.  Can you correct/confirm?
>>
>
>Yes, that’s the intent. I feel I need to dig OF documentation a bit so
>that  
>I make myself more in line with terminology used there. Sorry for
>misunderstandings occurring due to vague terms used.
>
>>4. Even though the design promotes playing nice by means of feature flow
>>kinds, I think that features might nevertheless still step on each
>>others¹
>>toes due to assumptions made about field content.  I¹m thinking, for
>>instance, of two features whose in-place frame modifications should be
>>done in a particular order.  Because of this, I¹m not sure that the
>>granularity of the proposed design can guarantee feature cooperation.
>>Maybe it would help to prioritize feature flows as ingress-processing
>>(that is, the flow should be exercised as early as possible in the
>>pipeline) versus egress-processing (the opposite) in addition to kind‹or
>>maybe that is just what  the notion of feature flow kind calls for, at
>>least in part.  Tied (tangential?) to this is the distinction that
>>OpenFlow makes between an action list and an action set: the former is a
>>series of actions that is applied to the frame immediately and in the
>>order specified in the flow entry; the latter is a proper set of actions
>>that is applied to the frame only upon its exit from the OF pipeline and
>>in an order specified by protocol.  (Action set content is modified as
>>the
>>frame traverses the OF pipeline.)  Should action sets be disallowed?
>
>I learned a bit more from you, again. :) Thanks!
>
>I am not sure I completely follow the suggestion with prioritizing as
>ingress-processing. Can you elaborate?

You and Rosella have had a subsequent conversation that I think addresses
my concerns.  But to continue my earlier thought: if multiple extensions
actually might modify the same frame elements, and in a way that would
change behavior based on ordering: I was thinking that within the category
of frame-modifying flows, for example, feature 

Re: [openstack-dev] [Sender Auth Failure] Re: [neutron] How could an L2 agent extension access agent methods ?

[Frances, Margaret]

Hello Ihar,

I have some comments and questions about your proposal.  My apologies if
any of what I say here results from misunderstandings on my part.

1. I believe there are two sorts of redirection at play here.  The first
involves inter-table traversal while the second allows a frame to exit the
OF pipeline either by being sent to a different port or by being dropped.
Some of what I say next makes use of this distinction.

2. OpenFlow¹s Goto instruction directs a frame from one table to the next.
 A redirection in this sense must be to a higher-numbered table, which is
to say that OF pipeline processing can only go forward (see p.18, para.2
of the 1.4.1 spec 
).  However, OvS (at
least v2.0.2) implements a resubmit action, which re-searches another
table‹higher-, lower-, or even same-numbered‹and executes any actions
found there in addition to any subsequent actions in the current flow
entry.  It is by using resubmit that the proposed design could work, as
shown in the ovs-ofctl command posted here

.  (Maybe there are other ways, too.)  The resubmit action is a Nicira
vendor extension that at least at one point, and maybe still, was known to
be implemented only by OvS.  I mention this because I wonder if the
proposed design (and my sample command) calls for flow traversal in a
manner not explicitly supported by OpenFlow and so may not work in future
versions of OvS.

3. Regarding the idea of sorting feature flows by kind: I believe that
what is meant by a 'redirection flow table' is a table that could possibly
remove the frame from OF pipeline processing‹i.e., by forwarding or
dropping it.  Can you correct/confirm?

4. Even though the design promotes playing nice by means of feature flow
kinds, I think that features might nevertheless still step on each others¹
toes due to assumptions made about field content.  I¹m thinking, for
instance, of two features whose in-place frame modifications should be
done in a particular order.  Because of this, I¹m not sure that the
granularity of the proposed design can guarantee feature cooperation.
Maybe it would help to prioritize feature flows as ingress-processing
(that is, the flow should be exercised as early as possible in the
pipeline) versus egress-processing (the opposite) in addition to kind‹or
maybe that is just what  the notion of feature flow kind calls for, at
least in part.  Tied (tangential?) to this is the distinction that
OpenFlow makes between an action list and an action set: the former is a
series of actions that is applied to the frame immediately and in the
order specified in the flow entry; the latter is a proper set of actions
that is applied to the frame only upon its exit from the OF pipeline and
in an order specified by protocol.  (Action set content is modified as the
frame traverses the OF pipeline.)  Should action sets be disallowed?

5. Is it a correct rephrasing of the third bullet of the high-level design
to say: each feature-specific flow entry in table 0 would be triggered
only if the frame's relevant OF metadata has not already been updated as a
result of the frame's previous traversal of the feature table.  I
apologize if I¹m suggesting something here that you didn¹t mean.

Hope this is helpful.
Margaret
--
Margaret Frances
Eng 4, Prodt Dev Engineering



On 12/3/15, 11:29 AM, "Johnston, Nate" 
wrote:

>Ihar,
>
>This is brilliant.  The complexity of doing graceful CRUD on the OVS flow
>table especially when other features are active is so complex, that
>abstracting the management of it into functionality optimized for that
>task is an incredibly good idea, especially for those of us like me who
>are not experts in OVS, and thus have a hard time seeing the edge cases.
>
>Thanks very much for this.
>
>‹N.
>
>> On Dec 3, 2015, at 10:46 AM, Ihar Hrachyshka 
>>wrote:
>> 
>> Hi,
>> 
>> Small update on the RFE. It was approved for Mitaka, assuming we come
>>up with proper details upfront thru neutron-specs process.
>> 
>> In the meantime, we have found more use cases for flow management among
>>features in development: QoS DSCP, also the new OF based firewall
>>driver. Both authors for those new features independently realized that
>>agent does not currently play nice with flows set by external code due
>>to its graceful restart behaviour when rules with unknown cookies are
>>cleaned up. [The agent uses a random session uuid() to mark rules that
>>belong to its current run.]
>> 
>> Before I proceed, full disclosure: I know almost nothing about OpenFlow
>>capabilities, so some pieces below may make no sense. I tried to come up
>>with high level model first and then try to map it to available OF
>>features. Please don¹t hesitate to comment, I like to learn new stuff! ;)
>> 
>> I am