Re: [openstack-dev] [neutron][FWaaS] __init__ arguments issue status

[Sridar Kandaswamy (skandasw)]

Hi All:

In digging thru this, yes with the neutron change, it changed the MRO as below 
and thus the issue.


(,
 , , , 
, ,

<—— the issue is at this point where we have a mismatch with args

,
 ,
 )


Nate, Margaret – thanks for digging thru this – lets get together during the 
day to discuss this more. Margaret, to answer ur question – it worked before 
due to a favorable ordering with the older hacked inheritance relationship. We 
can find a way to fix this in fwaas but more importantly need to get some 
missing pieces in to the Observer Hierarchy patch set as well.


Thanks


Sridar

From: Doug Wiegley 
<doug...@parksidesoftware.com<mailto:doug...@parksidesoftware.com>>
Reply-To: OpenStack List 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Date: Thursday, May 5, 2016 at 9:40 PM
To: OpenStack List 
<openstack-dev@lists.openstack.org<mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [neutron][FWaaS] __init__ arguments issue status

This break is almost certainly because of the following neutron change, to 
unwind the incestuous inheritance that was in neutron (dependency arrow was 
circular):

https://review.openstack.org/#/c/223343/

I don’t expect there will be a lot of appetite to revert that, so it will need 
to be addressed in neutron-fwaas. Likely it should’ve had an ML warning first, 
sorry about that, this has been a longstanding issue.

doug



On May 5, 2016, at 7:00 PM, Frances, Margaret 
<margaret_fran...@cable.comcast.com<mailto:margaret_fran...@cable.comcast.com>> 
wrote:

Hi Doug.

The old and new MROs are both pretty complicated, and it’s not entirely clear 
to me yet why the original one worked. (The MROs are included below for reading 
pleasure; they're embellished to show the incoming args to self’s init and 
outgoing args to super’s init in each case.)

I’m fairly sure the APIs for the mixins can be made the same, and I’ll try 
that.  But I still wonder if in fact the problem is a base class ordering issue.

The error that 223343 produced occurs in method call #6 in the "AFTER" MRO, 
where we get the following trace:

super(PeriodicTasks, self).__init__()
TypeError: __init__() takes exactly 2 arguments (1 given)


For grins, we changed PeriodicTasks’s call to super init as suggested by the 
trace:

super(PeriodicTasks, self).__init__(conf)


At this point FWaaSAgentRpcCallbackMixin (AFTER, #8) complained:

super(FWaaSAgentRpcCallbackMixin, self).__init__(host)
TypeError: object.__init__() takes no parameters


Changing *that* class as suggested elicited the following (to me baffling) 
result:

super(FWaaSAgentRpcCallbackMixin, self).__init__()
TypeError: __init__() takes exactly 2 arguments (1 given)


I find it baffling because FWaaSAgentRpcCallbackMixin is the end of the line, 
it’s a subclass of object, and object doesn’t allow arguments to init (so whose 
init is that? that’s the next thing I’m going to look at).  (It’s for these 
same reasons that I don’t understand why things worked before the 223343 
change.)

I’m still looking at things.  (And learning about MRO, which I’ve never really 
dealt with before.)  Will run pdb and see what surfaces.

Thanks for your help.  Thoughts, comments, suggestions all welcome.
Margaret


BEFORE 223343
 1. varmour_router_vArmourL3NATAgent (host, conf)-->(host, conf)
 2. agent_L3NATAgent  (host, conf)-->(conf)
 3. firewall_l3_agent_FWaaSL3AgentRpcCallback (conf)-->(host)
 4. api_FWaaSAgentRpcCallbackMixin (host)-->(host)
 5. ha_AgentMixin (host)-->(host)
 6. dvr_AgentMixin (host)-->(host)
 7. manager_Manager (host)-->(conf)
 8. periodic_task_PeriodicTasks(conf)-->()
 9. firewall_l3_agent_FWaaSL3AgentRpcCallback(conf)-->(host)
10. api_FWaaSAgentRpcCallbackMixin(host)-->(host)
11. object

AFTER 223343
 1. varmour_router_vArmourL3NATAgent (host, conf)-->(host, conf)
 2. agent_L3NATAgent  (host, conf)-->(host)
 3. ha_AgentMixin (host)-->(host)
 4. dvr_AgentMixin (host)-->(host)
 5. manager_Manager (host)-->(conf)
 6. periodic_task_PeriodicTasks(conf)-->()
 7. firewall_l3_agent_FWaaSL3AgentRpcCallback (conf)-->(host)
 8. api_FWaaSAgentRpcCallbackMixin(host)-->(host)
 9. object

--
Margaret Frances
Eng 4, Prodt Dev Engineering



On May 5, 2016, at 7:06 PM, Doug Hellmann 
<d...@doughellmann.com<mailto:d...@doughellmann.com>> wrote:

Excerpts from Nate Johnston's message of 2016-05-05 17:40:13 -0400:
FWaaS team,

After a day of looking at the tests currently failing in the FWaaS repo, I
believe I have the issue narrowed down considerably. First, to restate what
is going on.  If you check out the neutron-fwaas repository and run `tox -e
py27` in it, you will get six errors all in the
neutron_fwaas.tests.unit.services.firewall.agents.varmour.test_varmour_router.TestVarmourRouter
section.
Running the py34 tests results in similar problems.  The failures follow
the follow

Re: [openstack-dev] [neutron][FWaaS] __init__ arguments issue status

[Doug Wiegley]

This break is almost certainly because of the following neutron change, to 
unwind the incestuous inheritance that was in neutron (dependency arrow was 
circular):

https://review.openstack.org/#/c/223343/ 


I don’t expect there will be a lot of appetite to revert that, so it will need 
to be addressed in neutron-fwaas. Likely it should’ve had an ML warning first, 
sorry about that, this has been a longstanding issue.

doug



> On May 5, 2016, at 7:00 PM, Frances, Margaret 
>  wrote:
> 
> Hi Doug.  
> 
> The old and new MROs are both pretty complicated, and it’s not entirely clear 
> to me yet why the original one worked. (The MROs are included below for 
> reading pleasure; they're embellished to show the incoming args to self’s 
> init and outgoing args to super’s init in each case.)
> 
> I’m fairly sure the APIs for the mixins can be made the same, and I’ll try 
> that.  But I still wonder if in fact the problem is a base class ordering 
> issue.  
> 
> The error that 223343 produced occurs in method call #6 in the "AFTER" MRO, 
> where we get the following trace:
> 
>   super(PeriodicTasks, self).__init__()
> TypeError: __init__() takes exactly 2 arguments (1 given)
> 
> 
> For grins, we changed PeriodicTasks’s call to super init as suggested by the 
> trace:
> 
>   super(PeriodicTasks, self).__init__(conf)
> 
> 
> At this point FWaaSAgentRpcCallbackMixin (AFTER, #8) complained:
> 
>   super(FWaaSAgentRpcCallbackMixin, self).__init__(host)
> TypeError: object.__init__() takes no parameters
> 
> 
> Changing *that* class as suggested elicited the following (to me baffling) 
> result:
> 
>   super(FWaaSAgentRpcCallbackMixin, self).__init__()
> TypeError: __init__() takes exactly 2 arguments (1 given)
> 
> 
> I find it baffling because FWaaSAgentRpcCallbackMixin is the end of the line, 
> it’s a subclass of object, and object doesn’t allow arguments to init (so 
> whose init is that? that’s the next thing I’m going to look at).  (It’s for 
> these same reasons that I don’t understand why things worked before the 
> 223343 change.)
> 
> I’m still looking at things.  (And learning about MRO, which I’ve never 
> really dealt with before.)  Will run pdb and see what surfaces.  
> 
> Thanks for your help.  Thoughts, comments, suggestions all welcome.
> Margaret
> 
> 
> BEFORE 223343
>  1. varmour_router_vArmourL3NATAgent  (host, conf)-->(host, conf)
>  2. agent_L3NATAgent  (host, conf)-->(conf)
>  3. firewall_l3_agent_FWaaSL3AgentRpcCallback (conf)-->(host)
>  4. api_FWaaSAgentRpcCallbackMixin(host)-->(host)
>  5. ha_AgentMixin (host)-->(host)
>  6. dvr_AgentMixin(host)-->(host)
>  7. manager_Manager   (host)-->(conf)
>  8. periodic_task_PeriodicTasks   (conf)-->()
>  9. firewall_l3_agent_FWaaSL3AgentRpcCallback (conf)-->(host)
> 10. api_FWaaSAgentRpcCallbackMixin(host)-->(host)
> 11. object
> 
> AFTER 223343
>  1. varmour_router_vArmourL3NATAgent  (host, conf)-->(host, conf)
>  2. agent_L3NATAgent  (host, conf)-->(host)
>  3. ha_AgentMixin (host)-->(host)
>  4. dvr_AgentMixin(host)-->(host)
>  5. manager_Manager   (host)-->(conf)
>  6. periodic_task_PeriodicTasks   (conf)-->()
>  7. firewall_l3_agent_FWaaSL3AgentRpcCallback (conf)-->(host)
>  8. api_FWaaSAgentRpcCallbackMixin(host)-->(host)
>  9. object
> 
> --
> Margaret Frances
> Eng 4, Prodt Dev Engineering
> 
> 
> 
>> On May 5, 2016, at 7:06 PM, Doug Hellmann > > wrote:
>> 
>> Excerpts from Nate Johnston's message of 2016-05-05 17:40:13 -0400:
>>> FWaaS team,
>>> 
>>> After a day of looking at the tests currently failing in the FWaaS repo, I
>>> believe I have the issue narrowed down considerably. First, to restate what
>>> is going on.  If you check out the neutron-fwaas repository and run `tox -e
>>> py27` in it, you will get six errors all in the
>>> neutron_fwaas.tests.unit.services.firewall.agents.varmour.test_varmour_router.TestVarmourRouter
>>> section.
>>> Running the py34 tests results in similar problems.  The failures follow
>>> the following form:
>>> 
>>> Captured traceback:
>>> 
>>> ~~~
>>> 
>>>Traceback (most recent call last):
>>> 
>>>  File
>>> "neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
>>> line 190, in test_agent_external_gateway
>>> 
>>>router = self._create_router()
>>> 
>>>  File
>>> "neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
>>> line 87, in _create_router
>>> 
>>>router = varmour_router.vArmourL3NATAgent(HOSTNAME, self.conf)
>>> 
>>>  File
>>> 

Re: [openstack-dev] [neutron][FWaaS] __init__ arguments issue status

[Frances, Margaret]

Hi Doug.

The old and new MROs are both pretty complicated, and it’s not entirely clear 
to me yet why the original one worked. (The MROs are included below for reading 
pleasure; they're embellished to show the incoming args to self’s init and 
outgoing args to super’s init in each case.)

I’m fairly sure the APIs for the mixins can be made the same, and I’ll try 
that.  But I still wonder if in fact the problem is a base class ordering issue.

The error that 223343 produced occurs in method call #6 in the "AFTER" MRO, 
where we get the following trace:

super(PeriodicTasks, self).__init__()
TypeError: __init__() takes exactly 2 arguments (1 given)


For grins, we changed PeriodicTasks’s call to super init as suggested by the 
trace:

super(PeriodicTasks, self).__init__(conf)


At this point FWaaSAgentRpcCallbackMixin (AFTER, #8) complained:

super(FWaaSAgentRpcCallbackMixin, self).__init__(host)
TypeError: object.__init__() takes no parameters


Changing *that* class as suggested elicited the following (to me baffling) 
result:

super(FWaaSAgentRpcCallbackMixin, self).__init__()
TypeError: __init__() takes exactly 2 arguments (1 given)


I find it baffling because FWaaSAgentRpcCallbackMixin is the end of the line, 
it’s a subclass of object, and object doesn’t allow arguments to init (so whose 
init is that? that’s the next thing I’m going to look at).  (It’s for these 
same reasons that I don’t understand why things worked before the 223343 
change.)

I’m still looking at things.  (And learning about MRO, which I’ve never really 
dealt with before.)  Will run pdb and see what surfaces.

Thanks for your help.  Thoughts, comments, suggestions all welcome.
Margaret


BEFORE 223343
 1. varmour_router_vArmourL3NATAgent(host, conf)-->(host, conf)
 2. agent_L3NATAgent(host, conf)-->(conf)
 3. firewall_l3_agent_FWaaSL3AgentRpcCallback   (conf)-->(host)
 4. api_FWaaSAgentRpcCallbackMixin  (host)-->(host)
 5. ha_AgentMixin   (host)-->(host)
 6. dvr_AgentMixin  (host)-->(host)
 7. manager_Manager (host)-->(conf)
 8. periodic_task_PeriodicTasks (conf)-->()
 9. firewall_l3_agent_FWaaSL3AgentRpcCallback   (conf)-->(host)
10. api_FWaaSAgentRpcCallbackMixin  (host)-->(host)
11. object

AFTER 223343
 1. varmour_router_vArmourL3NATAgent(host, conf)-->(host, conf)
 2. agent_L3NATAgent(host, conf)-->(host)
 3. ha_AgentMixin   (host)-->(host)
 4. dvr_AgentMixin  (host)-->(host)
 5. manager_Manager (host)-->(conf)
 6. periodic_task_PeriodicTasks (conf)-->()
 7. firewall_l3_agent_FWaaSL3AgentRpcCallback   (conf)-->(host)
 8. api_FWaaSAgentRpcCallbackMixin  (host)-->(host)
 9. object

--
Margaret Frances
Eng 4, Prodt Dev Engineering



> On May 5, 2016, at 7:06 PM, Doug Hellmann  wrote:
> 
> Excerpts from Nate Johnston's message of 2016-05-05 17:40:13 -0400:
>> FWaaS team,
>> 
>> After a day of looking at the tests currently failing in the FWaaS repo, I
>> believe I have the issue narrowed down considerably. First, to restate what
>> is going on.  If you check out the neutron-fwaas repository and run `tox -e
>> py27` in it, you will get six errors all in the
>> neutron_fwaas.tests.unit.services.firewall.agents.varmour.test_varmour_router.TestVarmourRouter
>> section.
>> Running the py34 tests results in similar problems.  The failures follow
>> the following form:
>> 
>> Captured traceback:
>> 
>> ~~~
>> 
>>Traceback (most recent call last):
>> 
>>  File
>> "neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
>> line 190, in test_agent_external_gateway
>> 
>>router = self._create_router()
>> 
>>  File
>> "neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
>> line 87, in _create_router
>> 
>>router = varmour_router.vArmourL3NATAgent(HOSTNAME, self.conf)
>> 
>>  File
>> "neutron_fwaas/services/firewall/agents/varmour/varmour_router.py", line
>> 54, in __init__
>> 
>>super(vArmourL3NATAgent, self).__init__(host, conf)
>> 
>>  File "/home/njohns002/
>> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py",
>> line 244, in __init__
>> 
>>super(L3NATAgent, self).__init__(host=self.conf.host)
>> 
>>  File "/home/njohns002/
>> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/ha.py",
>> line 93, in __init__
>> 
>>super(AgentMixin, self).__init__(host)
>> 
>>  File "/home/njohns002/
>> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/dvr.py",
>> line 30, in __init__
>> 
>>super(AgentMixin, self).__init__(host)
>> 
>> 

Re: [openstack-dev] [neutron][FWaaS] __init__ arguments issue status

[Doug Hellmann]

Excerpts from Nate Johnston's message of 2016-05-05 17:40:13 -0400:
> FWaaS team,
> 
> After a day of looking at the tests currently failing in the FWaaS repo, I
> believe I have the issue narrowed down considerably. First, to restate what
> is going on.  If you check out the neutron-fwaas repository and run `tox -e
> py27` in it, you will get six errors all in the
> neutron_fwaas.tests.unit.services.firewall.agents.varmour.test_varmour_router.TestVarmourRouter
> section.
> Running the py34 tests results in similar problems.  The failures follow
> the following form:
> 
> Captured traceback:
> 
> ~~~
> 
> Traceback (most recent call last):
> 
>   File
> "neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
> line 190, in test_agent_external_gateway
> 
> router = self._create_router()
> 
>   File
> "neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
> line 87, in _create_router
> 
> router = varmour_router.vArmourL3NATAgent(HOSTNAME, self.conf)
> 
>   File
> "neutron_fwaas/services/firewall/agents/varmour/varmour_router.py", line
> 54, in __init__
> 
> super(vArmourL3NATAgent, self).__init__(host, conf)
> 
>   File "/home/njohns002/
> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py",
> line 244, in __init__
> 
> super(L3NATAgent, self).__init__(host=self.conf.host)
> 
>   File "/home/njohns002/
> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/ha.py",
> line 93, in __init__
> 
> super(AgentMixin, self).__init__(host)
> 
>   File "/home/njohns002/
> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/dvr.py",
> line 30, in __init__
> 
> super(AgentMixin, self).__init__(host)
> 
>   File "/home/njohns002/
> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py",
> line 70, in __init__
> 
> super(FWaaSAgentRpcCallbackMixin, self).__init__(host)
> 
>   File "/home/njohns002/
> github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/manager.py",
> line 44, in __init__
> 
> super(Manager, self).__init__(conf)
> 
>   File "/home/njohns002/
> github.com/openstack/neutron-fwaas-311159/.tox/py27/lib/python2.7/site-packages/oslo_service/periodic_task.py",
> line 177, in __init__
> 
> super(PeriodicTasks, self).__init__()
> 
> TypeError: __init__() takes exactly 2 arguments (1 given)
> 
> I pinged the Oslo folks, and they were able to help me look into the issue,
> which cleared oslo.service of any role.  The change that introduced this is
> actually a neutron change - https://review.openstack.org/#/c/223343/ - and
> I could experimentally test for it by doing the following, which checks out
> the change before the problem one, "Remove BGP code from neutron".  At that
> point `tox -e py27` could complete successfully.
> 
> Everything works with the older commit:   cd .tox/py27/src/neutron && git
> checkout fe702f8f2af265554c7ff6f464b99562f8c54254 && cd - && tox -e py27
> Things break with commit 223343:  cd .tox/py27/src/neutron && git
> checkout f31861843d90e013d31fb76fc576b49a35e218aa4  && cd - && tox -e py27
> 
> My guess on this is that the reason for the breakage is due to multiple
> inheritance and the changing of the ancestry for the L3NATAgent object.  So
> the focus of my effort (with Margaret Frances providing crucial insight) is
> determining what precisely needs to be fixed or reverted to make this work,
> while in keeping with the removal of FWaaS code from Neutron.  I shall
> continue to look at this tomorrow, but if anyone wishes to pick up the
> torch and figure this out then you should feel free to do so.  If not, I
> shall resume tomorrow.
> 
> Thanks,
> 
> --N.

Based on looking at the class hierarchy for vArmourL3NATAgent, I think
the problem is that the mixin classes up and down that hierarchy don't
take the same arguments for __init__(). That's going to make using super()
difficult. Normally one would just need to switch the order of the base
classes so all of the mixins are initialized before the Manager and
PeriodicTask base classes, but doing that doesn't fix the problem in
this case.

Is it possible to make all of the mixin APIs the same?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][FWaaS] __init__ arguments issue status

[Nate Johnston]

FWaaS team,

After a day of looking at the tests currently failing in the FWaaS repo, I
believe I have the issue narrowed down considerably. First, to restate what
is going on.  If you check out the neutron-fwaas repository and run `tox -e
py27` in it, you will get six errors all in the
neutron_fwaas.tests.unit.services.firewall.agents.varmour.test_varmour_router.TestVarmourRouter
section.
Running the py34 tests results in similar problems.  The failures follow
the following form:

Captured traceback:

~~~

Traceback (most recent call last):

  File
"neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
line 190, in test_agent_external_gateway

router = self._create_router()

  File
"neutron_fwaas/tests/unit/services/firewall/agents/varmour/test_varmour_router.py",
line 87, in _create_router

router = varmour_router.vArmourL3NATAgent(HOSTNAME, self.conf)

  File
"neutron_fwaas/services/firewall/agents/varmour/varmour_router.py", line
54, in __init__

super(vArmourL3NATAgent, self).__init__(host, conf)

  File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py",
line 244, in __init__

super(L3NATAgent, self).__init__(host=self.conf.host)

  File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/ha.py",
line 93, in __init__

super(AgentMixin, self).__init__(host)

  File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/dvr.py",
line 30, in __init__

super(AgentMixin, self).__init__(host)

  File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/agent/l3/agent.py",
line 70, in __init__

super(FWaaSAgentRpcCallbackMixin, self).__init__(host)

  File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/src/neutron/neutron/manager.py",
line 44, in __init__

super(Manager, self).__init__(conf)

  File "/home/njohns002/
github.com/openstack/neutron-fwaas-311159/.tox/py27/lib/python2.7/site-packages/oslo_service/periodic_task.py",
line 177, in __init__

super(PeriodicTasks, self).__init__()

TypeError: __init__() takes exactly 2 arguments (1 given)

I pinged the Oslo folks, and they were able to help me look into the issue,
which cleared oslo.service of any role.  The change that introduced this is
actually a neutron change - https://review.openstack.org/#/c/223343/ - and
I could experimentally test for it by doing the following, which checks out
the change before the problem one, "Remove BGP code from neutron".  At that
point `tox -e py27` could complete successfully.

Everything works with the older commit:   cd .tox/py27/src/neutron && git
checkout fe702f8f2af265554c7ff6f464b99562f8c54254 && cd - && tox -e py27
Things break with commit 223343:  cd .tox/py27/src/neutron && git
checkout f31861843d90e013d31fb76fc576b49a35e218aa4  && cd - && tox -e py27

My guess on this is that the reason for the breakage is due to multiple
inheritance and the changing of the ancestry for the L3NATAgent object.  So
the focus of my effort (with Margaret Frances providing crucial insight) is
determining what precisely needs to be fixed or reverted to make this work,
while in keeping with the removal of FWaaS code from Neutron.  I shall
continue to look at this tomorrow, but if anyone wishes to pick up the
torch and figure this out then you should feel free to do so.  If not, I
shall resume tomorrow.

Thanks,

--N.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev