Am 30.09.2013 um 20:03 schrieb Tim Bunce <[email protected]>:
> On Mon, Sep 30, 2013 at 08:52:04AM +0200, Jens Rehsack wrote:
>>
>> Am 29.09.2013 um 21:26 schrieb Tim Bunce <[email protected]>:
>>
>>> The DBI adopted an approach towards testing 'variants' by creating extra
>>> test files that act as wrappers around existing test files.
>>>
>>> The wrappers modify the process in some way, such as setting environment
>>> variables, before executing the original test file.
>>>
>>> It works well, and the DBIT has adopted the same approach.
>>>
>>> I think the approach is good and more widely applicable than just the DBI.
>>>
>>> I'd like to propose that we factor it out into a separate distribution.
>>>
>>> Thoughts?
>>
>>
>> I had the same idea - even several separate distributions, but I rejected
>> myself there for following reasons:
>>
>> 1) The variants of the driver result in an n choose k combination
>> 2) The driver attributes (eg. list of dbm backends, list of mldbm
>> serializers)
>> result in a cross product of those vectors.
>>
>> When making separate distributions, 1st approach would be to create a really
>> generic combinatorics distribution. There are some, but all currently have
>> drawbacks. This distribution would likely develop an own life (combinatoric
>> problems tend to become complex and require lot of performance speed ups).
>>
>> Based on that Math::Fast::Combi (*gg*) Test::MakeVariantTesTFiles would
>> require exactly to know what shall be done where.
>
> I think it just requires a callback to prune combinations that aren't wanted.
That callback can be complicated and the decision can be made at several points:
* configuration definition
* test case
* datasource configurator (DSN::Provider, Test::Database)
The routing of those decision callbacks could be somewhat complicated but
with some effort reasonable.
>> I think - from the current point of view, this would result in an over
>> complex API. Maybe sourcing out the combinatoric algorithms would clean
>> up the code a bit and later when we know better we can additionally create
>> a distributions for computing variants (combinations, variations,
>> permutations,
>> cross products, …) and based on that ?
>
> I think a clean simple API could be designed. Keeping the test rewriting
> logic separate simplifies the rest of DBIT. It's a Separation Of Concerns.
>
> Here's a rough suggestion for an API:
>
> write_variants(
> # required:
> test_files => [
> 't/foo.t', ...
> ],
> variants => {
> wibble => { ... },
> wobble => { ... },
> },
> # optional:
> filter => sub {
> my ($test_file, $variant_set) = @_;
> delete $variant_set->{wibble} if ...;
> return [ %$variant_set ]
> }
> );
>
> The write_variants sub would call the filter callback for every combination
> of variants (in this example: wibble alone, wobble alone, wibble + wobble).
And for "none" ;)
> The filter can return zero or more array refs where each array contains
> the variants to be applied, in order. Returning nothing means don't
> create variants for that test. Returning multiple items makes it easy
> for the filter to generate extra permutations if appropriate for some tests.
>
> What's missing?
Short answer: More levels and driver tunable settings. I give you a longer
example call (of the currently supported and implemented variants - beside
the filter callback - DBI::Test::Conf can populate):
write_variants(
# required:
test_cases => {
foo => '@BEGIN@
do "t/foo.t"
@END@',
bar => '@BEGIN@
do "t/bar.t"
@END@',
},
variants => {
dbi => {
pureperl => {
name => "DBI::PurePerl",
category => "dbi",
cat_abbrev => "z",
abbrev => "p",
init_stub => [
'$ENV{DBI_PUREPERL} = 2', ],
cleanup_stub => ['delete
$ENV{DBI_PUREPERL};'],
},
},
dialect => {
ansi_sql => {
name => "ANSI SQL",
category => "ansi",
cat_abbrev => "l",
abbrev => "a",
init_stub => [ '...', ],
cleanup_stub => ['...'],
},
},
proxy => {
gofer => {
name => "DBD::Gofer",
category => "dbi",
cat_abbrev => "z",
abbrev => "g",
init_stub =>
[ q{$ENV{DBI_AUTOPROXY} =
'dbi:Gofer:transport=null;policy=pedantic'}, ],
cleanup_stub => [q|delete $ENV{DBI_AUTOPROXY};|],
},
},
sql_engine => {
nano => {
name => "DBI::SQL::Nano",
category => "dbi",
cat_abbrev => "z",
abbrev => "n",
init_stub => [
q{$ENV{DBI_SQL_NANO} = 1}, ],
cleanup_stub => [q|delete
$ENV{DBI_SQL_NANO};|],
},
}
},
drivers => {
dbm => {
category => "driver",
cat_abbrev => "d",
abbrev => "d",
driver => "dbi:DBM:",
variants => {
mldbm => {
f => { dbm_mldbm =>
'FreezeThaw' },
d => { dbm_mldbm =>
'Data::Dumper' },
s => { dbm_mldbm =>
'Storable' },
# … JSON, YAML
},
type => {
s => { dbm_type =>
'SDBM_File' },
g => { dbm_type =>
'GDBM_File' },
d => { dbm_type =>
'DB_File' },
b => {
dbm_type
=> 'BerkeleyDB',
dbm_berkeley_flags
=> '...'
}
},
},
name => "DSN for DBM",
},
csv => {
category => "driver",
cat_abbrev => "d",
abbrev => "c",
driver => "dbi:CSV:",
variants => {
type => {
p => { csv_class =>
'Text::CSV' },
x => { csv_class =>
'Text::CSV_XS' },
},
},
name => "DSN for CSV",
},
},
# optional:
filter => sub {
my ( $test_file, $variant_set ) = @_;
delete $variant_set->{wibble} if ...;
return [%$variant_set];
}
);
Cheers
--
Jens Rehsack
pkgsrc, Perl5
[email protected]
