Re: [openstack-dev] [neutron] Question on functional tests
Unit tests should run successfully in a very limited environment, with no sudo, namespaces etc. Some packagers even run unit tests as part of their build process in hardened environment (I know Debian does, and some teams from Red Hat consider it too, like Neutron). So if it really needs to interact with outside world like that, please implement it in functional test namespace. On 01/20/2015 09:02 PM, Kevin Benton wrote: I don't believe we have any unit tests that create namespaces or veth pairs. This sounds like it belongs with functional tests. On Tue, Jan 20, 2015 at 10:20 AM, Numan Siddique numan.siddi...@enovance.com mailto:numan.siddi...@enovance.com wrote: Hello, I am working on a bug [1] on neutron vpnaas and submitted the patch here [2]. The test code to test the fix does the following - creates a namespace - creates a veth pair and add one interface into the namespace - configures the interface with an ip address and - adds a default gateway - and of course tests the code. This test code only tests a specific function (OpenSwanProcess._get_nexthop()) Reviewers of this patch are not clear if this should be part of functional tests or unit tests. Can unit tests create linux namespaces, interfaces etc or it falls under functional tests? Please let me know your thoughts on this. [1] - https://bugs.launchpad.net/neutron/+bug/1405413 [2] - https://review.openstack.org/#/c/145005/5 Regards Numan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Question on functional tests
Yes. That's right. Also there is a unit test here [1] which mocks ip netns exec. [1] - https://review.openstack.org/#/c/145005/5/neutron_vpnaas/tests/unit/services/vpn/device_drivers/test_ipsec.py Thanks Numan On 01/21/2015 02:22 PM, Kevin Benton wrote: So the test wouldn't make much sense then without the creation of the namespace, right? If that's the case, it sounds like it is a very low level functional test making sure that routes can be installed into namespaces. On Wed, Jan 21, 2015 at 12:19 AM, Numan Siddique numan.siddi...@enovance.com mailto:numan.siddi...@enovance.com wrote: It is asserting the return value of ip netns exec ns ip route get ip_address. Thanks Numan On 01/21/2015 12:34 PM, Kevin Benton wrote: Is the test asserting things about interactions with the system, or does it just happen to use a system call as a side effect of one of the setups? On Tue, Jan 20, 2015 at 1:24 PM, Paul Michali p...@michali.net mailto:p...@michali.net wrote: My question is whether the tests proposed should be unit tests or functional tests. They only test one method, and it's not a complete piece of functionality - like creating a VPN connection. If that one system call is mocked, these could all be treated as unit tests. So I'm wondering if there is an advantage in actually testing the system call (getaddrinfo), as part of this work? Thoughts? PCM (Paul Michali) IRC pc_m (irc.freenode.com http://irc.freenode.com) Twitter... @pmichali On Tue, Jan 20, 2015 at 3:02 PM, Kevin Benton blak...@gmail.com mailto:blak...@gmail.com wrote: I don't believe we have any unit tests that create namespaces or veth pairs. This sounds like it belongs with functional tests. On Tue, Jan 20, 2015 at 10:20 AM, Numan Siddique numan.siddi...@enovance.com mailto:numan.siddi...@enovance.com wrote: Hello, I am working on a bug [1] on neutron vpnaas and submitted the patch here [2]. The test code to test the fix does the following - creates a namespace - creates a veth pair and add one interface into the namespace - configures the interface with an ip address and - adds a default gateway - and of course tests the code. This test code only tests a specific function (OpenSwanProcess._get_nexthop()) Reviewers of this patch are not clear if this should be part of functional tests or unit tests. Can unit tests create linux namespaces, interfaces etc or it falls under functional tests? Please let me know your thoughts on this. [1] - https://bugs.launchpad.net/neutron/+bug/1405413 [2] - https://review.openstack.org/#/c/145005/5 Regards Numan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe mailto:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __
Re: [openstack-dev] [neutron] Question on functional tests
It is asserting the return value of ip netns exec ns ip route get ip_address. Thanks Numan On 01/21/2015 12:34 PM, Kevin Benton wrote: Is the test asserting things about interactions with the system, or does it just happen to use a system call as a side effect of one of the setups? On Tue, Jan 20, 2015 at 1:24 PM, Paul Michali p...@michali.net mailto:p...@michali.net wrote: My question is whether the tests proposed should be unit tests or functional tests. They only test one method, and it's not a complete piece of functionality - like creating a VPN connection. If that one system call is mocked, these could all be treated as unit tests. So I'm wondering if there is an advantage in actually testing the system call (getaddrinfo), as part of this work? Thoughts? PCM (Paul Michali) IRC pc_m (irc.freenode.com http://irc.freenode.com) Twitter... @pmichali On Tue, Jan 20, 2015 at 3:02 PM, Kevin Benton blak...@gmail.com mailto:blak...@gmail.com wrote: I don't believe we have any unit tests that create namespaces or veth pairs. This sounds like it belongs with functional tests. On Tue, Jan 20, 2015 at 10:20 AM, Numan Siddique numan.siddi...@enovance.com mailto:numan.siddi...@enovance.com wrote: Hello, I am working on a bug [1] on neutron vpnaas and submitted the patch here [2]. The test code to test the fix does the following - creates a namespace - creates a veth pair and add one interface into the namespace - configures the interface with an ip address and - adds a default gateway - and of course tests the code. This test code only tests a specific function (OpenSwanProcess._get_nexthop()) Reviewers of this patch are not clear if this should be part of functional tests or unit tests. Can unit tests create linux namespaces, interfaces etc or it falls under functional tests? Please let me know your thoughts on this. [1] - https://bugs.launchpad.net/neutron/+bug/1405413 [2] - https://review.openstack.org/#/c/145005/5 Regards Numan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Question on functional tests
So the test wouldn't make much sense then without the creation of the namespace, right? If that's the case, it sounds like it is a very low level functional test making sure that routes can be installed into namespaces. On Wed, Jan 21, 2015 at 12:19 AM, Numan Siddique numan.siddi...@enovance.com wrote: It is asserting the return value of ip netns exec ns ip route get ip_address. Thanks Numan On 01/21/2015 12:34 PM, Kevin Benton wrote: Is the test asserting things about interactions with the system, or does it just happen to use a system call as a side effect of one of the setups? On Tue, Jan 20, 2015 at 1:24 PM, Paul Michali p...@michali.net wrote: My question is whether the tests proposed should be unit tests or functional tests. They only test one method, and it's not a complete piece of functionality - like creating a VPN connection. If that one system call is mocked, these could all be treated as unit tests. So I'm wondering if there is an advantage in actually testing the system call (getaddrinfo), as part of this work? Thoughts? PCM (Paul Michali) IRC pc_m (irc.freenode.com) Twitter... @pmichali On Tue, Jan 20, 2015 at 3:02 PM, Kevin Benton blak...@gmail.com wrote: I don't believe we have any unit tests that create namespaces or veth pairs. This sounds like it belongs with functional tests. On Tue, Jan 20, 2015 at 10:20 AM, Numan Siddique numan.siddi...@enovance.com wrote: Hello, I am working on a bug [1] on neutron vpnaas and submitted the patch here [2]. The test code to test the fix does the following - creates a namespace - creates a veth pair and add one interface into the namespace - configures the interface with an ip address and - adds a default gateway - and of course tests the code. This test code only tests a specific function ( OpenSwanProcess. _get_nexthop()) Reviewers of this patch are not clear if this should be part of functional tests or unit tests. Can unit tests create linux namespaces, interfaces etc or it falls under functional tests? Please let me know your thoughts on this. [1] - https://bugs.launchpad.net/neutron/+bug/1405413 [2] - https://review.openstack.org/#/c/145005/5 Regards Numan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Question on functional tests
I don't believe we have any unit tests that create namespaces or veth pairs. This sounds like it belongs with functional tests. On Tue, Jan 20, 2015 at 10:20 AM, Numan Siddique numan.siddi...@enovance.com wrote: Hello, I am working on a bug [1] on neutron vpnaas and submitted the patch here [2]. The test code to test the fix does the following - creates a namespace - creates a veth pair and add one interface into the namespace - configures the interface with an ip address and - adds a default gateway - and of course tests the code. This test code only tests a specific function ( OpenSwanProcess. _get_nexthop()) Reviewers of this patch are not clear if this should be part of functional tests or unit tests. Can unit tests create linux namespaces, interfaces etc or it falls under functional tests? Please let me know your thoughts on this. [1] - https://bugs.launchpad.net/neutron/+bug/1405413 [2] - https://review.openstack.org/#/c/145005/5 Regards Numan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Question on functional tests
My question is whether the tests proposed should be unit tests or functional tests. They only test one method, and it's not a complete piece of functionality - like creating a VPN connection. If that one system call is mocked, these could all be treated as unit tests. So I'm wondering if there is an advantage in actually testing the system call (getaddrinfo), as part of this work? Thoughts? PCM (Paul Michali) IRC pc_m (irc.freenode.com) Twitter... @pmichali On Tue, Jan 20, 2015 at 3:02 PM, Kevin Benton blak...@gmail.com wrote: I don't believe we have any unit tests that create namespaces or veth pairs. This sounds like it belongs with functional tests. On Tue, Jan 20, 2015 at 10:20 AM, Numan Siddique numan.siddi...@enovance.com wrote: Hello, I am working on a bug [1] on neutron vpnaas and submitted the patch here [2]. The test code to test the fix does the following - creates a namespace - creates a veth pair and add one interface into the namespace - configures the interface with an ip address and - adds a default gateway - and of course tests the code. This test code only tests a specific function ( OpenSwanProcess. _get_nexthop()) Reviewers of this patch are not clear if this should be part of functional tests or unit tests. Can unit tests create linux namespaces, interfaces etc or it falls under functional tests? Please let me know your thoughts on this. [1] - https://bugs.launchpad.net/neutron/+bug/1405413 [2] - https://review.openstack.org/#/c/145005/5 Regards Numan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [neutron] Question on functional tests
Is the test asserting things about interactions with the system, or does it just happen to use a system call as a side effect of one of the setups? On Tue, Jan 20, 2015 at 1:24 PM, Paul Michali p...@michali.net wrote: My question is whether the tests proposed should be unit tests or functional tests. They only test one method, and it's not a complete piece of functionality - like creating a VPN connection. If that one system call is mocked, these could all be treated as unit tests. So I'm wondering if there is an advantage in actually testing the system call (getaddrinfo), as part of this work? Thoughts? PCM (Paul Michali) IRC pc_m (irc.freenode.com) Twitter... @pmichali On Tue, Jan 20, 2015 at 3:02 PM, Kevin Benton blak...@gmail.com wrote: I don't believe we have any unit tests that create namespaces or veth pairs. This sounds like it belongs with functional tests. On Tue, Jan 20, 2015 at 10:20 AM, Numan Siddique numan.siddi...@enovance.com wrote: Hello, I am working on a bug [1] on neutron vpnaas and submitted the patch here [2]. The test code to test the fix does the following - creates a namespace - creates a veth pair and add one interface into the namespace - configures the interface with an ip address and - adds a default gateway - and of course tests the code. This test code only tests a specific function ( OpenSwanProcess. _get_nexthop()) Reviewers of this patch are not clear if this should be part of functional tests or unit tests. Can unit tests create linux namespaces, interfaces etc or it falls under functional tests? Please let me know your thoughts on this. [1] - https://bugs.launchpad.net/neutron/+bug/1405413 [2] - https://review.openstack.org/#/c/145005/5 Regards Numan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Kevin Benton __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
