Hi Yves,
On IRC, we were talking about making the BDD tests be executed in parallel.
At the time, I objected to declare them 'parallel' as of now, because there
are interactions between the feature files. (Mind you that I'm deliberately
not saying that the scenarios can be parallellized at all, which is a
completely different problem.)
Let me take this opportunity to explain what the problem for the BDD tests
and parallellism is: all feature files use a "template" database to speed
up the execution of the scenarios: for each scenario a new database is
created, unless specified otherwise. However, database creation takes a
*lot* of time. So, instead of creating a new database from scratch, each
feature creates a template database once and copies that database later.
Which is *much* faster. Also, each feature file creates the template to
ensure tests are being run against the latest version of the schema.
Especially when running these tests on a developer's local system, this is
important. It's much less important on Travis, as that starts out with a
clean system anyway.
Taking a tour around the code, here is where the name of the template
database is being set:
https://github.com/ledgersmb/LedgerSMB/blob/master/t/lib/Pherkin/Extension/LedgerSMB.pm#L32
This is the code to create the template database (and an initial user):
https://github.com/ledgersmb/LedgerSMB/blob/master/t/lib/Pherkin/Extension/LedgerSMB.pm#L117
This is where the template creation is triggered, unless the template has
been created already:
https://github.com/ledgersmb/LedgerSMB/blob/master/t/lib/Pherkin/Extension/LedgerSMB.pm#L173
And this is the step definition which triggers the creation of the template
from the feature files:
https://github.com/ledgersmb/LedgerSMB/blob/master/t/lib/Pherkin/Extension/ledgersmb_steps/common_steps.pl#L13
When the template exists and a new test requires a new company database,
it's created through this step implementation:
https://github.com/ledgersmb/LedgerSMB/blob/master/t/lib/Pherkin/Extension/ledgersmb_steps/common_steps.pl#L13
Note that it uses the global variable `$company_seq` right above the step
definition to create sequentially numbered (unique) test databases. These
test databases are all called "standard-<sequence-number>". This is a
problem under the `prove` execution of the feature files, because the
features are executed as separate processes (forks) and by consequence the
sequence numbers will be re-used.
Today I was thinking what we can do to speed up the testing of the feature
files under `prove` with parallellism. This is what I came up with:
* If we change the 'ensure_template` function to create a template unless
the environment variable LSMB_NEW_DB has a value, in which case we assume
that to be a valid template.
* If we change the name of the test database instances to include the $$
(process id) (next to the sequence number, that is), we will have unique
test database names, at least for the duration of the test run.
The second point should probably include removing the database when the
test completes successfully (because otherwise the local developer's
database gets poluted with loads of test remnants).
Comments?
--
Bye,
Erik.
http://efficito.com -- Hosted accounting and ERP.
Robust and Flexible. No vendor lock-in.
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Ledger-smb-devel mailing list
Ledger-smb-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ledger-smb-devel
