Re: [openstack-dev] [gantt] How to include nova modules in unit tests
I agree, the gantt tree during the interim will be tightly coupled with the nova tree, any changes to those internal nova APIs will have to be reflected back into the gantt tree. Ultimately we want to completely decouple gantt from nova but that is a harder problem than just moving the scheduler specific code into a separate tree. When I said `cut the cord from nova it'll be easy' I was referring to identifying what imports need to be addressed. Actually fixing gantt to not use that nova code will certainly be a big job. -- Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale Ph: 303/443-3786 From: Joe Gordon [mailto:joe.gord...@gmail.com] Sent: Wednesday, January 22, 2014 2:31 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit tests On Tue, Jan 21, 2014 at 7:35 PM, Dugger, Donald D mailto:donald.d.dug...@intel.com>> wrote: Well, the first goal is to get the scheduler code into a separate tree, even though that code is still utilizing common code from nova. Right now just about every scheduler file includes some nova modules. Ultimately yes, we want to remove the depency on nova but that is a future effort and would create way too many changes for the immediate future. The nova code you are trying to use isn't a public API and can change at any time. Before considering using gantt we would have to fully remove any nova imports in gantt. When we want to cut the cord from nova it'll be easy, just remove that line from the `test-requirements.txt' file and we'll be forced to replace all of the nova code. I'm not sure it will actually be that easy. -- Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale Ph: 303/443-3786 -Original Message- From: Robert Collins [mailto:robe...@robertcollins.net<mailto:robe...@robertcollins.net>] Sent: Tuesday, January 21, 2014 5:16 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit tests On 22 January 2014 11:57, Dugger, Donald D mailto:donald.d.dug...@intel.com>> wrote: > I almost have the unit tests for gantt working except for one problem > - is there a way to have the test infrastructure allow the gantt tree > to import objects from the nova tree. > > > > The problem is that we want to break out just the scheduler code into > the gantt tree without duplicating all of nova. The current scheduler > has many imports of nova objects, which is not a problem except for the unit > tests. > The unit tests run in an environment that doesn't include the nova > tree so all of those imports wind up failing. The goal though is to have an independent system; perhaps marking all the tests that still depend on tendrils of nova 'skipped' and then work on burning down the skips to 0 is a better approach than making it easy to have such dependencies? -Rob -- Robert Collins mailto:rbtcoll...@hp.com>> Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [gantt] How to include nova modules in unit tests
2014/1/22 Joe Gordon > > > > On Tue, Jan 21, 2014 at 7:35 PM, Dugger, Donald D < > donald.d.dug...@intel.com> wrote: > >> Well, the first goal is to get the scheduler code into a separate tree, >> even though that code is still utilizing common code from nova. Right now >> just about every scheduler file includes some nova modules. Ultimately >> yes, we want to remove the depency on nova but that is a future effort and >> would create way too many changes for the immediate future. >> > > The nova code you are trying to use isn't a public API and can change at > any time. Before considering using gantt we would have to fully remove any > nova imports in gantt. > > >> >> When we want to cut the cord from nova it'll be easy, just remove that >> line from the `test-requirements.txt' file and we'll be forced to replace >> all of the nova code. >> > > I'm not sure it will actually be that easy. > > > Well, I think we should review all the imports in order to make sure all of the needs can be addressed thanks to the Nova API (flavors, instances and others). If yes, instead of importing classes, Gantt should only have dependency on novaclient so it would use it to get objects. The main problem is about objects synchronization, if Gantt does have its own models for the objects it has to handle. That's why I think the most difficult thing for Gantt is to play with abstract concepts on its side. My 2 cents, -Sylvain ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [gantt] How to include nova modules in unit tests
On Tue, Jan 21, 2014 at 7:35 PM, Dugger, Donald D wrote: > Well, the first goal is to get the scheduler code into a separate tree, > even though that code is still utilizing common code from nova. Right now > just about every scheduler file includes some nova modules. Ultimately > yes, we want to remove the depency on nova but that is a future effort and > would create way too many changes for the immediate future. > The nova code you are trying to use isn't a public API and can change at any time. Before considering using gantt we would have to fully remove any nova imports in gantt. > > When we want to cut the cord from nova it'll be easy, just remove that > line from the `test-requirements.txt' file and we'll be forced to replace > all of the nova code. > I'm not sure it will actually be that easy. > > -- > Don Dugger > "Censeo Toto nos in Kansa esse decisse." - D. Gale > Ph: 303/443-3786 > > -Original Message- > From: Robert Collins [mailto:robe...@robertcollins.net] > Sent: Tuesday, January 21, 2014 5:16 PM > To: OpenStack Development Mailing List (not for usage questions) > Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit > tests > > On 22 January 2014 11:57, Dugger, Donald D > wrote: > > I almost have the unit tests for gantt working except for one problem > > - is there a way to have the test infrastructure allow the gantt tree > > to import objects from the nova tree. > > > > > > > > The problem is that we want to break out just the scheduler code into > > the gantt tree without duplicating all of nova. The current scheduler > > has many imports of nova objects, which is not a problem except for the > unit tests. > > The unit tests run in an environment that doesn't include the nova > > tree so all of those imports wind up failing. > > The goal though is to have an independent system; perhaps marking all the > tests that still depend on tendrils of nova 'skipped' and then work on > burning down the skips to 0 is a better approach than making it easy to > have such dependencies? > > -Rob > > -- > Robert Collins > Distinguished Technologist > HP Converged Cloud > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [gantt] How to include nova modules in unit tests
Le 22/01/2014 01:37, Dugger, Donald D a écrit : Sylvain- Tnx, that worked great. (Now if I can just find a way to get the affinity tests working, all the other tests pass. I only have 17 tests failing out of 254.) I'm pretty busy these days with Climate 0.1 to deliver, but if I find some time, I will take a look on these. -Sylvain ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [gantt] How to include nova modules in unit tests
Sylvain- Tnx, that worked great. (Now if I can just find a way to get the affinity tests working, all the other tests pass. I only have 17 tests failing out of 254.) -- Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale Ph: 303/443-3786 From: Sylvain Bauza [mailto:sylvain.ba...@gmail.com] Sent: Tuesday, January 21, 2014 4:25 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit tests Hi Don, Look at how Climate is doing this [1]. The idea is to pull the master tarball as an eggfile. That works fine, the only issues we have is when we register opts to CONF where we could get DuplicateOpt if we try to register opt with same name when importing. -Sylvain [1] : https://github.com/stackforge/climate-nova/blob/master/test-requirements.txt#L15 2014/1/21 Dugger, Donald D mailto:donald.d.dug...@intel.com>> I almost have the unit tests for gantt working except for one problem - is there a way to have the test infrastructure allow the gantt tree to import objects from the nova tree. The problem is that we want to break out just the scheduler code into the gantt tree without duplicating all of nova. The current scheduler has many imports of nova objects, which is not a problem except for the unit tests. The unit tests run in an environment that doesn't include the nova tree so all of those imports wind up failing. Adding `nova' to the `test-requirements.txt' file doesn't work because this causes the test system to look for a tar ball at `http://pypi.python.org/simple/nova/' and there isn't a tarball there, just a link to the openstack home page. I can get the unit tests working in my local tree by putting in a symlink to the nova directory but that doesn't seem like a real solution for Jenkins. Anyone have an ideas on how to address this? -- Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale Ph: 303/443-3786 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org<mailto:OpenStack-dev@lists.openstack.org> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [gantt] How to include nova modules in unit tests
Well, the first goal is to get the scheduler code into a separate tree, even though that code is still utilizing common code from nova. Right now just about every scheduler file includes some nova modules. Ultimately yes, we want to remove the depency on nova but that is a future effort and would create way too many changes for the immediate future. When we want to cut the cord from nova it'll be easy, just remove that line from the `test-requirements.txt' file and we'll be forced to replace all of the nova code. -- Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale Ph: 303/443-3786 -Original Message- From: Robert Collins [mailto:robe...@robertcollins.net] Sent: Tuesday, January 21, 2014 5:16 PM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit tests On 22 January 2014 11:57, Dugger, Donald D wrote: > I almost have the unit tests for gantt working except for one problem > - is there a way to have the test infrastructure allow the gantt tree > to import objects from the nova tree. > > > > The problem is that we want to break out just the scheduler code into > the gantt tree without duplicating all of nova. The current scheduler > has many imports of nova objects, which is not a problem except for the unit > tests. > The unit tests run in an environment that doesn't include the nova > tree so all of those imports wind up failing. The goal though is to have an independent system; perhaps marking all the tests that still depend on tendrils of nova 'skipped' and then work on burning down the skips to 0 is a better approach than making it easy to have such dependencies? -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [gantt] How to include nova modules in unit tests
On 22 January 2014 11:57, Dugger, Donald D wrote: > I almost have the unit tests for gantt working except for one problem – is > there a way to have the test infrastructure allow the gantt tree to import > objects from the nova tree. > > > > The problem is that we want to break out just the scheduler code into the > gantt tree without duplicating all of nova. The current scheduler has many > imports of nova objects, which is not a problem except for the unit tests. > The unit tests run in an environment that doesn’t include the nova tree so > all of those imports wind up failing. The goal though is to have an independent system; perhaps marking all the tests that still depend on tendrils of nova 'skipped' and then work on burning down the skips to 0 is a better approach than making it easy to have such dependencies? -Rob -- Robert Collins Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [gantt] How to include nova modules in unit tests
Hi Don, Look at how Climate is doing this [1]. The idea is to pull the master tarball as an eggfile. That works fine, the only issues we have is when we register opts to CONF where we could get DuplicateOpt if we try to register opt with same name when importing. -Sylvain [1] : https://github.com/stackforge/climate-nova/blob/master/test-requirements.txt#L15 2014/1/21 Dugger, Donald D > I almost have the unit tests for gantt working except for one problem – > is there a way to have the test infrastructure allow the gantt tree to > import objects from the nova tree. > > > > The problem is that we want to break out just the scheduler code into the > gantt tree without duplicating all of nova. The current scheduler has many > imports of nova objects, which is not a problem except for the unit tests. > The unit tests run in an environment that doesn’t include the nova tree so > all of those imports wind up failing. > > > > Adding `nova’ to the `test-requirements.txt’ file doesn’t work because > this causes the test system to look for a tar ball at ` > http://pypi.python.org/simple/nova/’ and there isn’t a tarball there, > just a link to the openstack home page. > > > > I can get the unit tests working in my local tree by putting in a symlink > to the nova directory but that doesn’t seem like a real solution for > Jenkins. > > > > Anyone have an ideas on how to address this? > > > > -- > > Don Dugger > > "Censeo Toto nos in Kansa esse decisse." - D. Gale > > Ph: 303/443-3786 > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [gantt] How to include nova modules in unit tests
I almost have the unit tests for gantt working except for one problem - is there a way to have the test infrastructure allow the gantt tree to import objects from the nova tree. The problem is that we want to break out just the scheduler code into the gantt tree without duplicating all of nova. The current scheduler has many imports of nova objects, which is not a problem except for the unit tests. The unit tests run in an environment that doesn't include the nova tree so all of those imports wind up failing. Adding `nova' to the `test-requirements.txt' file doesn't work because this causes the test system to look for a tar ball at `http://pypi.python.org/simple/nova/' and there isn't a tarball there, just a link to the openstack home page. I can get the unit tests working in my local tree by putting in a symlink to the nova directory but that doesn't seem like a real solution for Jenkins. Anyone have an ideas on how to address this? -- Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale Ph: 303/443-3786 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
