Re: [openstack-dev] [gantt] How to include nova modules in unit tests

2014-01-22 Thread Sylvain Bauza

Le 22/01/2014 01:37, Dugger, Donald D a écrit :


Sylvain-

Tnx, that worked great.

(Now if I can just find a way to get the affinity tests working, all 
the other tests pass.  I only have 17 tests failing out of 254.)





I'm pretty busy these days with Climate 0.1 to deliver, but if I find 
some time, I will take a look on these.


-Sylvain
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [gantt] How to include nova modules in unit tests

2014-01-22 Thread Joe Gordon
On Tue, Jan 21, 2014 at 7:35 PM, Dugger, Donald D donald.d.dug...@intel.com
 wrote:

 Well, the first goal is to get the scheduler code into a separate tree,
 even though that code is still utilizing common code from nova.  Right now
 just about every scheduler file includes some nova modules.  Ultimately
 yes, we want to remove the depency on nova but that is a future effort and
 would create way too many changes for the immediate future.


The nova code you are trying to use isn't a public API and can change at
any time. Before considering using gantt we would have to fully remove any
nova imports in gantt.



 When we want to cut the cord from nova it'll be easy, just remove that
 line from the `test-requirements.txt' file and we'll be forced to replace
 all of the nova code.


I'm not sure it will actually be that easy.



 --
 Don Dugger
 Censeo Toto nos in Kansa esse decisse. - D. Gale
 Ph: 303/443-3786

 -Original Message-
 From: Robert Collins [mailto:robe...@robertcollins.net]
 Sent: Tuesday, January 21, 2014 5:16 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit
 tests

 On 22 January 2014 11:57, Dugger, Donald D donald.d.dug...@intel.com
 wrote:
  I almost have the unit tests for gantt working except for one problem
  - is there a way to have the test infrastructure allow the gantt tree
  to import objects from the nova tree.
 
 
 
  The problem is that we want to break out just the scheduler code into
  the gantt tree without duplicating all of nova.  The current scheduler
  has many imports of nova objects, which is not a problem except for the
 unit tests.
  The unit tests run in an environment that doesn't include the nova
  tree so all of those imports wind up failing.

 The goal though is to have an independent system; perhaps marking all the
 tests that still depend on tendrils of nova 'skipped' and then work on
 burning down the skips to 0 is a better approach than making it easy to
 have such dependencies?

 -Rob

 --
 Robert Collins rbtcoll...@hp.com
 Distinguished Technologist
 HP Converged Cloud

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [gantt] How to include nova modules in unit tests

2014-01-22 Thread Sylvain Bauza
2014/1/22 Joe Gordon joe.gord...@gmail.com




 On Tue, Jan 21, 2014 at 7:35 PM, Dugger, Donald D 
 donald.d.dug...@intel.com wrote:

 Well, the first goal is to get the scheduler code into a separate tree,
 even though that code is still utilizing common code from nova.  Right now
 just about every scheduler file includes some nova modules.  Ultimately
 yes, we want to remove the depency on nova but that is a future effort and
 would create way too many changes for the immediate future.


 The nova code you are trying to use isn't a public API and can change at
 any time. Before considering using gantt we would have to fully remove any
 nova imports in gantt.



 When we want to cut the cord from nova it'll be easy, just remove that
 line from the `test-requirements.txt' file and we'll be forced to replace
 all of the nova code.


 I'm not sure it will actually be that easy.



Well, I think we should review all the imports in order to make sure all of
the needs can be addressed thanks to the Nova API (flavors, instances and
others). If yes, instead of importing classes, Gantt should only have
dependency on novaclient so it would use it to get objects.

The main problem is about objects synchronization, if Gantt does have its
own models for the objects it has to handle. That's why I think the most
difficult thing for Gantt is to play with abstract concepts on its side.

My 2 cents,
-Sylvain
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [gantt] How to include nova modules in unit tests

2014-01-22 Thread Dugger, Donald D
I agree, the gantt tree during the interim will be tightly coupled with the 
nova tree, any changes to those internal nova APIs will have to be reflected 
back into the gantt tree.  Ultimately we want to completely decouple gantt from 
nova but that is a harder problem than just moving the scheduler specific code 
into a separate tree.

When I said `cut the cord from nova it'll be easy' I was referring to 
identifying what imports need to be addressed.  Actually fixing gantt to not 
use that nova code will certainly be a big job.

--
Don Dugger
Censeo Toto nos in Kansa esse decisse. - D. Gale
Ph: 303/443-3786

From: Joe Gordon [mailto:joe.gord...@gmail.com]
Sent: Wednesday, January 22, 2014 2:31 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit tests



On Tue, Jan 21, 2014 at 7:35 PM, Dugger, Donald D 
donald.d.dug...@intel.commailto:donald.d.dug...@intel.com wrote:
Well, the first goal is to get the scheduler code into a separate tree, even 
though that code is still utilizing common code from nova.  Right now just 
about every scheduler file includes some nova modules.  Ultimately yes, we want 
to remove the depency on nova but that is a future effort and would create way 
too many changes for the immediate future.

The nova code you are trying to use isn't a public API and can change at any 
time. Before considering using gantt we would have to fully remove any nova 
imports in gantt.



When we want to cut the cord from nova it'll be easy, just remove that line 
from the `test-requirements.txt' file and we'll be forced to replace all of the 
nova code.

I'm not sure it will actually be that easy.


--
Don Dugger
Censeo Toto nos in Kansa esse decisse. - D. Gale
Ph: 303/443-3786tel:303%2F443-3786
-Original Message-
From: Robert Collins 
[mailto:robe...@robertcollins.netmailto:robe...@robertcollins.net]
Sent: Tuesday, January 21, 2014 5:16 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit tests

On 22 January 2014 11:57, Dugger, Donald D 
donald.d.dug...@intel.commailto:donald.d.dug...@intel.com wrote:
 I almost have the unit tests for gantt working except for one problem
 - is there a way to have the test infrastructure allow the gantt tree
 to import objects from the nova tree.



 The problem is that we want to break out just the scheduler code into
 the gantt tree without duplicating all of nova.  The current scheduler
 has many imports of nova objects, which is not a problem except for the unit 
 tests.
 The unit tests run in an environment that doesn't include the nova
 tree so all of those imports wind up failing.

The goal though is to have an independent system; perhaps marking all the tests 
that still depend on tendrils of nova 'skipped' and then work on burning down 
the skips to 0 is a better approach than making it easy to have such 
dependencies?

-Rob

--
Robert Collins rbtcoll...@hp.commailto:rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


[openstack-dev] [gantt] How to include nova modules in unit tests

2014-01-21 Thread Dugger, Donald D
I almost have the unit tests for gantt working except for one problem - is 
there a way to have the test infrastructure allow the gantt tree to import 
objects from the nova tree.

The problem is that we want to break out just the scheduler code into the gantt 
tree without duplicating all of nova.  The current scheduler has many imports 
of nova objects, which is not a problem except for the unit tests.  The unit 
tests run in an environment that doesn't include the nova tree so all of those 
imports wind up failing.

Adding `nova' to the `test-requirements.txt' file doesn't work because this 
causes the test system to look for a tar ball at 
`http://pypi.python.org/simple/nova/' and there isn't a tarball there, just a 
link to the openstack home page.

I can get the unit tests working in my local tree by putting in a symlink to 
the nova directory but that doesn't seem like a real solution for Jenkins.

Anyone have an ideas on how to address this?

--
Don Dugger
Censeo Toto nos in Kansa esse decisse. - D. Gale
Ph: 303/443-3786

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [gantt] How to include nova modules in unit tests

2014-01-21 Thread Sylvain Bauza
Hi Don,

Look at how Climate is doing this [1]. The idea is to pull the master
tarball as an eggfile. That works fine, the only issues we have is when we
register opts to CONF where we could get DuplicateOpt if we try to register
opt with same name when importing.

-Sylvain


[1] :
https://github.com/stackforge/climate-nova/blob/master/test-requirements.txt#L15


2014/1/21 Dugger, Donald D donald.d.dug...@intel.com

  I almost have the unit tests for gantt working except for one problem –
 is there a way to have the test infrastructure allow the gantt tree to
 import objects from the nova tree.



 The problem is that we want to break out just the scheduler code into the
 gantt tree without duplicating all of nova.  The current scheduler has many
 imports of nova objects, which is not a problem except for the unit tests.
 The unit tests run in an environment that doesn’t include the nova tree so
 all of those imports wind up failing.



 Adding `nova’ to the `test-requirements.txt’ file doesn’t work because
 this causes the test system to look for a tar ball at `
 http://pypi.python.org/simple/nova/’ and there isn’t a tarball there,
 just a link to the openstack home page.



 I can get the unit tests working in my local tree by putting in a symlink
 to the nova directory but that doesn’t seem like a real solution for
 Jenkins.



 Anyone have an ideas on how to address this?



 --

 Don Dugger

 Censeo Toto nos in Kansa esse decisse. - D. Gale

 Ph: 303/443-3786



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [gantt] How to include nova modules in unit tests

2014-01-21 Thread Robert Collins
On 22 January 2014 11:57, Dugger, Donald D donald.d.dug...@intel.com wrote:
 I almost have the unit tests for gantt working except for one problem – is
 there a way to have the test infrastructure allow the gantt tree to import
 objects from the nova tree.



 The problem is that we want to break out just the scheduler code into the
 gantt tree without duplicating all of nova.  The current scheduler has many
 imports of nova objects, which is not a problem except for the unit tests.
 The unit tests run in an environment that doesn’t include the nova tree so
 all of those imports wind up failing.

The goal though is to have an independent system; perhaps marking all
the tests that still depend on tendrils of nova 'skipped' and then
work on burning down the skips to 0 is a better approach than making
it easy to have such dependencies?

-Rob

-- 
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [gantt] How to include nova modules in unit tests

2014-01-21 Thread Dugger, Donald D
Well, the first goal is to get the scheduler code into a separate tree, even 
though that code is still utilizing common code from nova.  Right now just 
about every scheduler file includes some nova modules.  Ultimately yes, we want 
to remove the depency on nova but that is a future effort and would create way 
too many changes for the immediate future.

When we want to cut the cord from nova it'll be easy, just remove that line 
from the `test-requirements.txt' file and we'll be forced to replace all of the 
nova code.

--
Don Dugger
Censeo Toto nos in Kansa esse decisse. - D. Gale
Ph: 303/443-3786

-Original Message-
From: Robert Collins [mailto:robe...@robertcollins.net] 
Sent: Tuesday, January 21, 2014 5:16 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit tests

On 22 January 2014 11:57, Dugger, Donald D donald.d.dug...@intel.com wrote:
 I almost have the unit tests for gantt working except for one problem 
 - is there a way to have the test infrastructure allow the gantt tree 
 to import objects from the nova tree.



 The problem is that we want to break out just the scheduler code into 
 the gantt tree without duplicating all of nova.  The current scheduler 
 has many imports of nova objects, which is not a problem except for the unit 
 tests.
 The unit tests run in an environment that doesn't include the nova 
 tree so all of those imports wind up failing.

The goal though is to have an independent system; perhaps marking all the tests 
that still depend on tendrils of nova 'skipped' and then work on burning down 
the skips to 0 is a better approach than making it easy to have such 
dependencies?

-Rob

--
Robert Collins rbtcoll...@hp.com
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [gantt] How to include nova modules in unit tests

2014-01-21 Thread Dugger, Donald D
Sylvain-

Tnx, that worked great.

(Now if I can just find a way to get the affinity tests working, all the other 
tests pass.  I only have 17 tests failing out of 254.)

--
Don Dugger
Censeo Toto nos in Kansa esse decisse. - D. Gale
Ph: 303/443-3786

From: Sylvain Bauza [mailto:sylvain.ba...@gmail.com]
Sent: Tuesday, January 21, 2014 4:25 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [gantt] How to include nova modules in unit tests

Hi Don,

Look at how Climate is doing this [1]. The idea is to pull the master tarball 
as an eggfile. That works fine, the only issues we have is when we register 
opts to CONF where we could get DuplicateOpt if we try to register opt with 
same name when importing.

-Sylvain


[1] : 
https://github.com/stackforge/climate-nova/blob/master/test-requirements.txt#L15

2014/1/21 Dugger, Donald D 
donald.d.dug...@intel.commailto:donald.d.dug...@intel.com
I almost have the unit tests for gantt working except for one problem - is 
there a way to have the test infrastructure allow the gantt tree to import 
objects from the nova tree.

The problem is that we want to break out just the scheduler code into the gantt 
tree without duplicating all of nova.  The current scheduler has many imports 
of nova objects, which is not a problem except for the unit tests.  The unit 
tests run in an environment that doesn't include the nova tree so all of those 
imports wind up failing.

Adding `nova' to the `test-requirements.txt' file doesn't work because this 
causes the test system to look for a tar ball at 
`http://pypi.python.org/simple/nova/' and there isn't a tarball there, just a 
link to the openstack home page.

I can get the unit tests working in my local tree by putting in a symlink to 
the nova directory but that doesn't seem like a real solution for Jenkins.

Anyone have an ideas on how to address this?

--
Don Dugger
Censeo Toto nos in Kansa esse decisse. - D. Gale
Ph: 303/443-3786tel:303%2F443-3786


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev