Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-21 Thread Yair Zaslavsky



On 08/21/2012 11:59 AM, Allon Mureinik wrote:



- Original Message -

From: "Mike Kolesnik" 
To: "Shireesh Anjal" 
Cc: engine-devel@ovirt.org, "infra" 
Sent: Tuesday, August 21, 2012 9:43:23 AM
Subject: Re: [Engine-devel] Any way to break the dao_unit_tests up?

- Original Message -

On Monday 20 August 2012 11:16 AM, Mike Kolesnik wrote:

- Original Message -

The DAO unit tests take twice as long as the rest of the test to
run
is
there any way to break them up into two pieces?


It will not be easy..

The way the tests are built today is with DB-unit.
DB-unit allows to have an XML file with predefined data (called
fixtures) which is used to recreate the DB data each time a
test-class is run.

This is all fine, except that in our tests there are (at least) 2
issues:
   1. The same fixtures.xml file is used in all DAO tests.
   2. Some DAOs require fixtures for several tables.

Now, we could fix issue #1 by splitting the fixtures file into
smaller files, each relating to only one table, which would allow
us to run these tests in parallel on the same DB.
Issue #2 would require to figure out which tests require several
fixtures, and have them run isolated from the other tests which
require only a single table.

A simpler solution could be to have the tests run each on it's
own
db schema (or it's own db) which eliminates the dependencies and
allows to run all in parallel, but is a bit more complicated to
maintain (we would need some script that generates these
schemas/dbs for tests automatically) and keeping multiple schemas
up to date would also require CPU time.

This is speaking in terms of the tests themselves, without
considering the build process itself.


There are two issues here:

1) DB connection is created during initialization of every test
case,
and destroyed at the end of each test case execution
2) The fixtures data is inserted during initialization of every
test
case

I think both of these can be resolved by
   - creating the test data only during initialization of the first
   test
case, which will include creating the connection (with auto-commit
=
false), inserting fixtures data and committing it
   - rolling back any changes done to the database during test case
execution in the tearDown method

I just tried this in two phases. Using the same connection across
all
test cases brought down the dao unit tests run time from 4:42.683s
to
1:07.628s, and inserting the fixtures data only once further
brought
it
down to just 22.295s ! (on my local development machine)

I've just sent a patch with these changes:
http://gerrit.ovirt.org/7336


Patch merged,

Thanks Shireesh for the contribution, now the DAO tests are super
fast!

20-something seconds to run DAO tests?
awesome!
Kudos, Shireesh!


+100 on this, good work!








--
Thanks
Robert Middleswarth
@rmiddle (twitter/IRC)

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel




___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-21 Thread Allon Mureinik


- Original Message -
> From: "Mike Kolesnik" 
> To: "Shireesh Anjal" 
> Cc: engine-devel@ovirt.org, "infra" 
> Sent: Tuesday, August 21, 2012 9:43:23 AM
> Subject: Re: [Engine-devel] Any way to break the dao_unit_tests up?
> 
> - Original Message -
> > On Monday 20 August 2012 11:16 AM, Mike Kolesnik wrote:
> > > - Original Message -
> > >> The DAO unit tests take twice as long as the rest of the test to
> > >> run
> > >> is
> > >> there any way to break them up into two pieces?
> > >>
> > > It will not be easy..
> > >
> > > The way the tests are built today is with DB-unit.
> > > DB-unit allows to have an XML file with predefined data (called
> > > fixtures) which is used to recreate the DB data each time a
> > > test-class is run.
> > >
> > > This is all fine, except that in our tests there are (at least) 2
> > > issues:
> > >   1. The same fixtures.xml file is used in all DAO tests.
> > >   2. Some DAOs require fixtures for several tables.
> > >
> > > Now, we could fix issue #1 by splitting the fixtures file into
> > > smaller files, each relating to only one table, which would allow
> > > us to run these tests in parallel on the same DB.
> > > Issue #2 would require to figure out which tests require several
> > > fixtures, and have them run isolated from the other tests which
> > > require only a single table.
> > >
> > > A simpler solution could be to have the tests run each on it's
> > > own
> > > db schema (or it's own db) which eliminates the dependencies and
> > > allows to run all in parallel, but is a bit more complicated to
> > > maintain (we would need some script that generates these
> > > schemas/dbs for tests automatically) and keeping multiple schemas
> > > up to date would also require CPU time.
> > >
> > > This is speaking in terms of the tests themselves, without
> > > considering the build process itself.
> > 
> > There are two issues here:
> > 
> > 1) DB connection is created during initialization of every test
> > case,
> > and destroyed at the end of each test case execution
> > 2) The fixtures data is inserted during initialization of every
> > test
> > case
> > 
> > I think both of these can be resolved by
> >   - creating the test data only during initialization of the first
> >   test
> > case, which will include creating the connection (with auto-commit
> > =
> > false), inserting fixtures data and committing it
> >   - rolling back any changes done to the database during test case
> > execution in the tearDown method
> > 
> > I just tried this in two phases. Using the same connection across
> > all
> > test cases brought down the dao unit tests run time from 4:42.683s
> > to
> > 1:07.628s, and inserting the fixtures data only once further
> > brought
> > it
> > down to just 22.295s ! (on my local development machine)
> > 
> > I've just sent a patch with these changes:
> > http://gerrit.ovirt.org/7336
> 
> Patch merged,
> 
> Thanks Shireesh for the contribution, now the DAO tests are super
> fast!
20-something seconds to run DAO tests?
awesome!
Kudos, Shireesh!
> 
> > 
> > >
> > >> --
> > >> Thanks
> > >> Robert Middleswarth
> > >> @rmiddle (twitter/IRC)
> > >>
> > >> ___
> > >> Engine-devel mailing list
> > >> Engine-devel@ovirt.org
> > >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>
> > > ___
> > > Engine-devel mailing list
> > > Engine-devel@ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> > 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-20 Thread Mike Kolesnik
- Original Message -
> On Monday 20 August 2012 11:16 AM, Mike Kolesnik wrote:
> > - Original Message -
> >> The DAO unit tests take twice as long as the rest of the test to
> >> run
> >> is
> >> there any way to break them up into two pieces?
> >>
> > It will not be easy..
> >
> > The way the tests are built today is with DB-unit.
> > DB-unit allows to have an XML file with predefined data (called
> > fixtures) which is used to recreate the DB data each time a
> > test-class is run.
> >
> > This is all fine, except that in our tests there are (at least) 2
> > issues:
> >   1. The same fixtures.xml file is used in all DAO tests.
> >   2. Some DAOs require fixtures for several tables.
> >
> > Now, we could fix issue #1 by splitting the fixtures file into
> > smaller files, each relating to only one table, which would allow
> > us to run these tests in parallel on the same DB.
> > Issue #2 would require to figure out which tests require several
> > fixtures, and have them run isolated from the other tests which
> > require only a single table.
> >
> > A simpler solution could be to have the tests run each on it's own
> > db schema (or it's own db) which eliminates the dependencies and
> > allows to run all in parallel, but is a bit more complicated to
> > maintain (we would need some script that generates these
> > schemas/dbs for tests automatically) and keeping multiple schemas
> > up to date would also require CPU time.
> >
> > This is speaking in terms of the tests themselves, without
> > considering the build process itself.
> 
> There are two issues here:
> 
> 1) DB connection is created during initialization of every test case,
> and destroyed at the end of each test case execution
> 2) The fixtures data is inserted during initialization of every test
> case
> 
> I think both of these can be resolved by
>   - creating the test data only during initialization of the first
>   test
> case, which will include creating the connection (with auto-commit =
> false), inserting fixtures data and committing it
>   - rolling back any changes done to the database during test case
> execution in the tearDown method
> 
> I just tried this in two phases. Using the same connection across all
> test cases brought down the dao unit tests run time from 4:42.683s to
> 1:07.628s, and inserting the fixtures data only once further brought
> it
> down to just 22.295s ! (on my local development machine)
> 
> I've just sent a patch with these changes:
> http://gerrit.ovirt.org/7336

Patch merged, 

Thanks Shireesh for the contribution, now the DAO tests are super fast!

> 
> >
> >> --
> >> Thanks
> >> Robert Middleswarth
> >> @rmiddle (twitter/IRC)
> >>
> >> ___
> >> Engine-devel mailing list
> >> Engine-devel@ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> > ___
> > Engine-devel mailing list
> > Engine-devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-20 Thread Yair Zaslavsky



On 08/20/2012 01:30 PM, Shireesh Anjal wrote:

On Monday 20 August 2012 11:16 AM, Mike Kolesnik wrote:

- Original Message -

The DAO unit tests take twice as long as the rest of the test to run
is
there any way to break them up into two pieces?


It will not be easy..

The way the tests are built today is with DB-unit.
DB-unit allows to have an XML file with predefined data (called
fixtures) which is used to recreate the DB data each time a test-class
is run.

This is all fine, except that in our tests there are (at least) 2 issues:
  1. The same fixtures.xml file is used in all DAO tests.
  2. Some DAOs require fixtures for several tables.

Now, we could fix issue #1 by splitting the fixtures file into smaller
files, each relating to only one table, which would allow us to run
these tests in parallel on the same DB.
Issue #2 would require to figure out which tests require several
fixtures, and have them run isolated from the other tests which
require only a single table.

A simpler solution could be to have the tests run each on it's own db
schema (or it's own db) which eliminates the dependencies and allows
to run all in parallel, but is a bit more complicated to maintain (we
would need some script that generates these schemas/dbs for tests
automatically) and keeping multiple schemas up to date would also
require CPU time.

This is speaking in terms of the tests themselves, without considering
the build process itself.


There are two issues here:

1) DB connection is created during initialization of every test case,
and destroyed at the end of each test case execution
2) The fixtures data is inserted during initialization of every test case

I think both of these can be resolved by
  - creating the test data only during initialization of the first test
case, which will include creating the connection (with auto-commit =
false), inserting fixtures data and committing it
  - rolling back any changes done to the database during test case
execution in the tearDown method

I just tried this in two phases. Using the same connection across all
test cases brought down the dao unit tests run time from 4:42.683s to
1:07.628s, and inserting the fixtures data only once further brought it
down to just 22.295s ! (on my local development machine)

I've just sent a patch with these changes: http://gerrit.ovirt.org/7336


Hi Shireesh, I am also providing some input on some missing functionality







--
Thanks
Robert Middleswarth
@rmiddle (twitter/IRC)

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-20 Thread Allon Mureinik


- Original Message -
> From: "Shireesh Anjal" 
> To: "Mike Kolesnik" 
> Cc: engine-devel@ovirt.org, "infra" 
> Sent: Monday, August 20, 2012 1:30:43 PM
> Subject: Re: [Engine-devel] Any way to break the dao_unit_tests up?
> 
> On Monday 20 August 2012 11:16 AM, Mike Kolesnik wrote:
> > - Original Message -
> >> The DAO unit tests take twice as long as the rest of the test to
> >> run
> >> is
> >> there any way to break them up into two pieces?
> >>
> > It will not be easy..
> >
> > The way the tests are built today is with DB-unit.
> > DB-unit allows to have an XML file with predefined data (called
> > fixtures) which is used to recreate the DB data each time a
> > test-class is run.
> >
> > This is all fine, except that in our tests there are (at least) 2
> > issues:
> >   1. The same fixtures.xml file is used in all DAO tests.
> >   2. Some DAOs require fixtures for several tables.
> >
> > Now, we could fix issue #1 by splitting the fixtures file into
> > smaller files, each relating to only one table, which would allow
> > us to run these tests in parallel on the same DB.
> > Issue #2 would require to figure out which tests require several
> > fixtures, and have them run isolated from the other tests which
> > require only a single table.
> >
> > A simpler solution could be to have the tests run each on it's own
> > db schema (or it's own db) which eliminates the dependencies and
> > allows to run all in parallel, but is a bit more complicated to
> > maintain (we would need some script that generates these
> > schemas/dbs for tests automatically) and keeping multiple schemas
> > up to date would also require CPU time.
> >
> > This is speaking in terms of the tests themselves, without
> > considering the build process itself.
> 
> There are two issues here:
> 
> 1) DB connection is created during initialization of every test case,
> and destroyed at the end of each test case execution
> 2) The fixtures data is inserted during initialization of every test
> case
> 
> I think both of these can be resolved by
>   - creating the test data only during initialization of the first
>   test
> case, which will include creating the connection (with auto-commit =
> false), inserting fixtures data and committing it
>   - rolling back any changes done to the database during test case
> execution in the tearDown method
> 
> I just tried this in two phases. Using the same connection across all
> test cases brought down the dao unit tests run time from 4:42.683s to
> 1:07.628s, and inserting the fixtures data only once further brought
> it
> down to just 22.295s ! (on my local development machine)
> 
> I've just sent a patch with these changes:
> http://gerrit.ovirt.org/7336

Beat me to the punch :-)
(BTW, I have some implementation details - see inline in gerrit)
> 
> >
> >> --
> >> Thanks
> >> Robert Middleswarth
> >> @rmiddle (twitter/IRC)
> >>
> >> ___
> >> Engine-devel mailing list
> >> Engine-devel@ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> > ___
> > Engine-devel mailing list
> > Engine-devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-20 Thread Shireesh Anjal

On Monday 20 August 2012 11:16 AM, Mike Kolesnik wrote:

- Original Message -

The DAO unit tests take twice as long as the rest of the test to run
is
there any way to break them up into two pieces?


It will not be easy..

The way the tests are built today is with DB-unit.
DB-unit allows to have an XML file with predefined data (called fixtures) which 
is used to recreate the DB data each time a test-class is run.

This is all fine, except that in our tests there are (at least) 2 issues:
  1. The same fixtures.xml file is used in all DAO tests.
  2. Some DAOs require fixtures for several tables.

Now, we could fix issue #1 by splitting the fixtures file into smaller files, 
each relating to only one table, which would allow us to run these tests in 
parallel on the same DB.
Issue #2 would require to figure out which tests require several fixtures, and 
have them run isolated from the other tests which require only a single table.

A simpler solution could be to have the tests run each on it's own db schema 
(or it's own db) which eliminates the dependencies and allows to run all in 
parallel, but is a bit more complicated to maintain (we would need some script 
that generates these schemas/dbs for tests automatically) and keeping multiple 
schemas up to date would also require CPU time.

This is speaking in terms of the tests themselves, without considering the 
build process itself.


There are two issues here:

1) DB connection is created during initialization of every test case, 
and destroyed at the end of each test case execution

2) The fixtures data is inserted during initialization of every test case

I think both of these can be resolved by
 - creating the test data only during initialization of the first test 
case, which will include creating the connection (with auto-commit = 
false), inserting fixtures data and committing it
 - rolling back any changes done to the database during test case 
execution in the tearDown method


I just tried this in two phases. Using the same connection across all 
test cases brought down the dao unit tests run time from 4:42.683s to 
1:07.628s, and inserting the fixtures data only once further brought it 
down to just 22.295s ! (on my local development machine)


I've just sent a patch with these changes: http://gerrit.ovirt.org/7336




--
Thanks
Robert Middleswarth
@rmiddle (twitter/IRC)

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-20 Thread Eyal Edri


- Original Message -
> From: "Itamar Heim" 
> To: "Eyal Edri" 
> Cc: "Yaniv Kaul" , engine-devel@ovirt.org, "Robert 
> Middleswarth" 
> Sent: Monday, August 20, 2012 10:31:15 AM
> Subject: Re: [Engine-devel] Any way to break the dao_unit_tests up?
> 
> On 08/20/2012 10:30 AM, Eyal Edri wrote:
> >
> >
> > - Original Message -
> >> From: "Itamar Heim" 
> >> To: "Robert Middleswarth" 
> >> Cc: "Yaniv Kaul" , engine-devel@ovirt.org, "Eyal
> >> Edri" 
> >> Sent: Monday, August 20, 2012 10:26:10 AM
> >> Subject: Re: [Engine-devel] Any way to break the dao_unit_tests
> >> up?
> >>
> >> On 08/20/2012 09:40 AM, Robert Middleswarth wrote:
> >>> On 08/20/2012 02:02 AM, Yaniv Kaul wrote:
> >>>> On 08/19/2012 11:03 PM, Robert Middleswarth wrote:
> >>>>> The DAO unit tests take twice as long as the rest of the test
> >>>>> to
> >>>>> run
> >>>>> is there any way to break them up into two pieces?
> >>>>>
> >>>>>
> >>>>
> >>>> Can they run in parallel to the rest of the tests?
> >>>> May be a KISS solution for this problem.
> >>>> Y.
> >>>>
> >>> Yaniv,
> >>>
> >>> That is what I am doing but the current test can't be ran in
> >>> parallel on
> >>> the same host and the jobs backup several hours and none of the
> >>> results
> >>
> >> actually, why can't the test run in parallel on same host?
> >
> > i think we run into errors when trying to do it..
> > it's worth trying to do again and see if the errors are due to
> > environment issues or problems in the tests themselfs.
> 
> if you try to use same schema name, you will fail, but i think we
> fixed
> the dao tests to run in parallel in the past by setting the schema
> name
> per run, etc.
> 

we're using a diff db name, if that's what you mean.
i meant some random tests were failing, but lets check it again and verify it 
is the case.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-20 Thread Itamar Heim

On 08/20/2012 10:30 AM, Eyal Edri wrote:



- Original Message -

From: "Itamar Heim" 
To: "Robert Middleswarth" 
Cc: "Yaniv Kaul" , engine-devel@ovirt.org, "Eyal Edri" 

Sent: Monday, August 20, 2012 10:26:10 AM
Subject: Re: [Engine-devel] Any way to break the dao_unit_tests up?

On 08/20/2012 09:40 AM, Robert Middleswarth wrote:

On 08/20/2012 02:02 AM, Yaniv Kaul wrote:

On 08/19/2012 11:03 PM, Robert Middleswarth wrote:

The DAO unit tests take twice as long as the rest of the test to
run
is there any way to break them up into two pieces?




Can they run in parallel to the rest of the tests?
May be a KISS solution for this problem.
Y.


Yaniv,

That is what I am doing but the current test can't be ran in
parallel on
the same host and the jobs backup several hours and none of the
results


actually, why can't the test run in parallel on same host?


i think we run into errors when trying to do it..
it's worth trying to do again and see if the errors are due to environment 
issues or problems in the tests themselfs.


if you try to use same schema name, you will fail, but i think we fixed 
the dao tests to run in parallel in the past by setting the schema name 
per run, etc.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-20 Thread Eyal Edri


- Original Message -
> From: "Itamar Heim" 
> To: "Robert Middleswarth" 
> Cc: "Yaniv Kaul" , engine-devel@ovirt.org, "Eyal Edri" 
> 
> Sent: Monday, August 20, 2012 10:26:10 AM
> Subject: Re: [Engine-devel] Any way to break the dao_unit_tests up?
> 
> On 08/20/2012 09:40 AM, Robert Middleswarth wrote:
> > On 08/20/2012 02:02 AM, Yaniv Kaul wrote:
> >> On 08/19/2012 11:03 PM, Robert Middleswarth wrote:
> >>> The DAO unit tests take twice as long as the rest of the test to
> >>> run
> >>> is there any way to break them up into two pieces?
> >>>
> >>>
> >>
> >> Can they run in parallel to the rest of the tests?
> >> May be a KISS solution for this problem.
> >> Y.
> >>
> > Yaniv,
> >
> > That is what I am doing but the current test can't be ran in
> > parallel on
> > the same host and the jobs backup several hours and none of the
> > results
> 
> actually, why can't the test run in parallel on same host?

i think we run into errors when trying to do it.. 
it's worth trying to do again and see if the errors are due to environment 
issues or problems in the tests themselfs.

> 
> > get written  to Gerrit until all the jobes finishes.
> >
> 
> 
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-20 Thread Itamar Heim

On 08/20/2012 09:40 AM, Robert Middleswarth wrote:

On 08/20/2012 02:02 AM, Yaniv Kaul wrote:

On 08/19/2012 11:03 PM, Robert Middleswarth wrote:

The DAO unit tests take twice as long as the rest of the test to run
is there any way to break them up into two pieces?




Can they run in parallel to the rest of the tests?
May be a KISS solution for this problem.
Y.


Yaniv,

That is what I am doing but the current test can't be ran in parallel on
the same host and the jobs backup several hours and none of the results


actually, why can't the test run in parallel on same host?


get written  to Gerrit until all the jobes finishes.




___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-19 Thread Robert Middleswarth

On 08/20/2012 02:02 AM, Yaniv Kaul wrote:

On 08/19/2012 11:03 PM, Robert Middleswarth wrote:
The DAO unit tests take twice as long as the rest of the test to run 
is there any way to break them up into two pieces?





Can they run in parallel to the rest of the tests?
May be a KISS solution for this problem.
Y.


Yaniv,

That is what I am doing but the current test can't be ran in parallel on 
the same host and the jobs backup several hours and none of the results 
get written  to Gerrit until all the jobes finishes.


--
Thanks
Robert Middleswarth
@rmiddle (twitter/Freenode IRC)
@RobertM (OFTC IRC)

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-19 Thread Yaniv Kaul

On 08/19/2012 11:03 PM, Robert Middleswarth wrote:
The DAO unit tests take twice as long as the rest of the test to run 
is there any way to break them up into two pieces?





Can they run in parallel to the rest of the tests?
May be a KISS solution for this problem.
Y.

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


Re: [Engine-devel] Any way to break the dao_unit_tests up?

2012-08-19 Thread Mike Kolesnik
- Original Message -
> The DAO unit tests take twice as long as the rest of the test to run
> is
> there any way to break them up into two pieces?
> 

It will not be easy..

The way the tests are built today is with DB-unit.
DB-unit allows to have an XML file with predefined data (called fixtures) which 
is used to recreate the DB data each time a test-class is run.

This is all fine, except that in our tests there are (at least) 2 issues:
 1. The same fixtures.xml file is used in all DAO tests.
 2. Some DAOs require fixtures for several tables.

Now, we could fix issue #1 by splitting the fixtures file into smaller files, 
each relating to only one table, which would allow us to run these tests in 
parallel on the same DB.
Issue #2 would require to figure out which tests require several fixtures, and 
have them run isolated from the other tests which require only a single table.

A simpler solution could be to have the tests run each on it's own db schema 
(or it's own db) which eliminates the dependencies and allows to run all in 
parallel, but is a bit more complicated to maintain (we would need some script 
that generates these schemas/dbs for tests automatically) and keeping multiple 
schemas up to date would also require CPU time.

This is speaking in terms of the tests themselves, without considering the 
build process itself.

> 
> --
> Thanks
> Robert Middleswarth
> @rmiddle (twitter/IRC)
> 
> ___
> Engine-devel mailing list
> Engine-devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


[Engine-devel] Any way to break the dao_unit_tests up?

2012-08-19 Thread Robert Middleswarth
The DAO unit tests take twice as long as the rest of the test to run is 
there any way to break them up into two pieces?



--
Thanks
Robert Middleswarth
@rmiddle (twitter/IRC)

___
Engine-devel mailing list
Engine-devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel