On Wed, Jan 27, 2016 at 03:02:15PM +0000, Morton, Derek J wrote: > > > > > >-----Original Message----- > >From: Ville Syrjälä [mailto:[email protected]] > >Sent: Wednesday, January 27, 2016 2:31 PM > >To: Morton, Derek J > >Cc: [email protected] > >Subject: Re: [Intel-gfx] [PATCH i-g-t] lib/igt_core.c: Expand --run-subtest > >functionality. > > > >On Wed, Jan 27, 2016 at 01:30:01PM +0000, Morton, Derek J wrote: > >> > > >> > > >> >-----Original Message----- > >> >From: Ville Syrjälä [mailto:[email protected]] > >> >Sent: Wednesday, January 27, 2016 12:33 PM > >> >To: Morton, Derek J > >> >Cc: [email protected] > >> >Subject: Re: [Intel-gfx] [PATCH i-g-t] lib/igt_core.c: Expand > >> >--run-subtest functionality. > >> > > >> >On Wed, Jan 27, 2016 at 10:05:56AM +0000, Derek Morton wrote: > >> >> Added support for specifying arbitary lists of subtests to run or > >> >> to exclude from being run by using : or ^ as a seperator. > >> >> > >> >> :subtest1:subtest2: Will run subtest1 and subtest2 > >> >> ^subtest1^subtest2^ will run all subtests except subtest1 and > >> >> subtest2 > >> > > >> >Hmm. Getting a bit complicated perhaps. Would it be simpler to just allow > >> >specifying the --r option multiple times? So we'd start with the full > >> >list of subtests, and each --r option would filter the list in some way? > >> > >> By -r I assume you mean --run-subtest? Running a test with -r <subtest> > >> just prints the usage options at the moment. > > > >--r not -r ;) > > > >> Allowing it to be added multiple times could be a way of building up a > >> list of subtests to run, but a completely new command line option would > >> need to then be added (--skip-subtest?) to allow building up a list of > >> subtests to be skipped. > > > >Or could be something like --r !subtest-name > > > >> With my change as is, it allows a string to be passed to the test which > >> details which subtests should be run. If a new parameter such as > >> --skip-subtest is added then it would require knowledge of whether the > >> string contains subtests to run or not to run. That would complicate any > >> scripts that use this to automate testing. > >> Allowing --run-subtest (and --skip-subtest) to be specified multiple times > >> will also complicate the code in igt_core.c. Currently there is a simple > >> call to strdup(). If --run-subtest can be specified multiple times the > >> code will have to deal with concatenating strings and any memory > >> reallocation that needs. Also how to deal with the possibility of multiple > >> wildcard expressions? > > > >Hmm. I suppoose my original idea of start with full list and filter stuff > >out doesn't work entirely well. Eg. --r foo --r bar would run nothing. So I > >guess the option would have to be able to add to the list as well. > > > >So I guess we could make it so that '!' removes the specified test(s), > >no-'!' adds them, and if this is the first --r option and there's no '!' > >we'd clear the list entirely before adding the specified test(s) to it. > > > >> > >> I think that will just end up more complicated than the simple separated > >> list solution this patch introduces. > > > >I suppose. But it would avoid adding a new language which can look a bit > >like a weird regexp but isn't. > > > >Maybe if you just use ',' to separate the subtests specifications, and '!' > >to specify the not condition things would look a bit more standardish. > > I can't use ! as the shell uses it for some command history substitution. > That is why I chose ^. There are very few special character the shell does > not mess up :-(.
A bunch of stuff uses ! (eg. test, find, etc.). The shell won't eat it if there's a space after it, or you could always escape it. getopt doesn't support it natively however so it could a pain to use the space trick. > > How about the following: > > Comma separated list of subtests to specify a list of subtests to run. > Prepend ^ to the list to specify a list of subtests to exclude. > > I would then have to use strstr to look for a comma in what is specified to > --run-subtests. Is there any risk of a comma in a wildcard expression? I > found some code for fnmatch with google and it looks like it is simply ?*[]- > which is treated as special characters. > > //Derek > > > > >> > >> //Derek > >> > >> > > >> >> > >> >> Any subtest string not starting : or ^ is treated as a normal > >> >> wildcard expression. > >> >> > >> >> This is required mainly on android to exclude subtests that test > >> >> features that do not exist in the android driver while still being > >> >> able to run other subtests in the binary when a wildcard expression > >> >> is insufficient. > >> >> > >> >> Signed-off-by: Derek Morton <[email protected]> > >> >> --- > >> >> lib/igt_core.c | 42 ++++++++++++++++++++++++++++++++++++++++-- > >> >> 1 file changed, 40 insertions(+), 2 deletions(-) > >> >> > >> >> diff --git a/lib/igt_core.c b/lib/igt_core.c index 6b69bb7..b9e7470 > >> >> 100644 > >> >> --- a/lib/igt_core.c > >> >> +++ b/lib/igt_core.c > >> >> @@ -207,7 +207,15 @@ > >> >> * To do that obtain the lists of subtests with "--list-subtests", > >> >> which can be > >> >> * run as non-root and doesn't require the i915 driver to be loaded > >> >> (or any > >> >> * intel gpu to be present). Then individual subtests can be run > >> >> with > >> >> - * "--run-subtest". Usage help for tests with subtests can be > >> >> obtained with the > >> >> + * "--run-subtest". --run-subtest accepts wildcard characters. A > >> >> + list of > >> >> + * subtests to run may be specified by using : as a seperator. A > >> >> + list of > >> >> + * subtests to exclude may be specified using ^ as a seperator. > >> >> + * > >> >> + * - --run-subtest basic* will run all subtests starting basic. > >> >> + * - --run-subtest :subtest1:subtest2: will run only subtest1 and > >> >> + subtest2 > >> >> + * - --run-subtest ^subtest1^subtest2^ will run all except > >> >> + subtest1 and subtest2 > >> >> + * > >> >> + * Usage help for tests with subtests can be obtained with the > >> >> * "--help" command line option. > >> >> */ > >> >> > >> >> @@ -786,6 +794,35 @@ void igt_simple_init_parse_opts(int *argc, char > >> >> **argv, > >> >> extra_opt_handler, handler_data); } > >> >> > >> >> +static bool check_testlist(const char *subtest_name) { > >> >> + char *p; > >> >> + > >> >> + /* Run subtests in list > >> >> + * Look for subtest_name in list of form > >> >> :subtest1:subtest2:subtest3: > >> >> + * return true if found. > >> >> + */ > >> >> + if (run_single_subtest[0] == ':') { > >> >> + p = strstr(run_single_subtest, subtest_name); > >> >> + if ((p) && (*(p-1) == ':') && > >> >> (*(p+strlen(subtest_name)) == ':' )) > >> >> + return true; > >> >> + } > >> >> + /* Run subtests not in list > >> >> + * Look for subtest_name in list of form > >> >> ^test1^subtest2^subtest3^ > >> >> + * return true if not found. > >> >> + */ > >> >> + else if (run_single_subtest[0] == '^') { > >> >> + p = strstr(run_single_subtest, subtest_name); > >> >> + if (!((p) && (*(p-1) == '^') && > >> >> (*(p+strlen(subtest_name)) == '^' ))) > >> >> + return true; > >> >> + } > >> >> + /* Run subtests that match shell wildcard */ > >> >> + else if (fnmatch(run_single_subtest, subtest_name, 0) == 0) > >> >> + return true; > >> >> + > >> >> + return false; > >> >> +} > >> >> + > >> >> /* > >> >> * Note: Testcases which use these helpers MUST NOT output anything to > >> >> stdout > >> >> * outside of places protected by igt_run_subtest checks - the > >> >> piglit @@ -814,7 +851,8 @@ bool __igt_run_subtest(const char > >> >> *subtest_name) > >> >> } > >> >> > >> >> if (run_single_subtest) { > >> >> - if (fnmatch(run_single_subtest, subtest_name, 0) != 0) > >> >> + > >> >> + if (check_testlist(subtest_name) == false) > >> >> return false; > >> >> else > >> >> run_single_subtest_found = true; > >> >> -- > >> >> 1.9.1 > >> >> > >> >> _______________________________________________ > >> >> Intel-gfx mailing list > >> >> [email protected] > >> >> http://lists.freedesktop.org/mailman/listinfo/intel-gfx > >> > > >> >-- > >> >Ville Syrjälä > >> >Intel OTC > >> > > > > >-- > >Ville Syrjälä > >Intel OTC > > -- Ville Syrjälä Intel OTC _______________________________________________ Intel-gfx mailing list [email protected] http://lists.freedesktop.org/mailman/listinfo/intel-gfx
