Re: [CMake] Running test who have two labels?

2017-03-25 Thread Eric Noulard 
2017-03-24 21:17 GMT+01:00 David Cole :

> Here's a bash script wrapper you could use with existing ctest. Save
> it out to a file named ctest-two-labels.sh and then call it with bash
> on Mac or Linux:
>
>
> label1=$1
> label2=$2
>
> if [ -z "$label1" ]; then
> echo "script takes two label arguments as input, missing arg 1"
> exit 1
> fi
>
> if [ -z "$label2" ]; then
> echo "script takes two label arguments as input, missing arg 2"
> exit 2
> fi
>
> tmpfile=./ctest-two-labels-sh-tmp.txt
> testlistFile=./ctest-two-labels-sh-testlist.txt
> testnumsFile=./ctest-two-labels-sh-testnums.txt
>
> ctest -L "^($label1)$" -N > "$tmpfile"
> ctest -L "^($label2)$" -N >> "$tmpfile"
>
> cat "$tmpfile" | grep "  Test #" | sort | uniq -d > "$testlistFile"
>

Strange enough I got the very same idea to dump and "uniqify" the list I
want.


>
> cat "$testlistFile" | awk -F "#" '{ print $2; }' | awk -F ":" '{ print
> $1; }' > "$testnumsFile"
>
> testnums=$(cat "$testnumsFile" | paste -s -d, -)
>
> #echo Tests with both labels $label1 and $label2:
> #cat "$testlistFile"
> #
> #echo Just the test numbers:
> #cat "$testnumsFile"
> #
> #echo The test numbers, assembled into a ctest -I string to run just
> those numbered tests:
> #echo $testnums
>
> echo Running command line:
> echo ""
> echo "  ctest -I \"0,0,0,$testnums\" -N"
> echo ""
> echo Run it without the -N to actually execute the tests...
> echo ""
>
> ctest -I "0,0,0,$testnums" -N
>
> It's a "back of the envelope / proof of concept" script. Polish it up
> and make it nice if the approach seems reasonable to you.
>

The approach is reasonable. I was simply afraid that dump + filtering of
tests and/or labels
wouldn't be efficient with 600++ tests. That said it's working. Thank you
very much for the proof of concept.
I guess that may be dumping whole tests names (with ctest -N ) and labels
(with --print-labels) would make it
simple for me to craft a script to easily select tests subset with a ctest
wrapper.

It would even be simpler if ctest could dump all tests with associated
labels.
I'll think about it.


-- 
Eric
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] Running test who have two labels?

2017-03-24 Thread David Cole via CMake 
Here's a bash script wrapper you could use with existing ctest. Save
it out to a file named ctest-two-labels.sh and then call it with bash
on Mac or Linux:


label1=$1
label2=$2

if [ -z "$label1" ]; then
echo "script takes two label arguments as input, missing arg 1"
exit 1
fi

if [ -z "$label2" ]; then
echo "script takes two label arguments as input, missing arg 2"
exit 2
fi

tmpfile=./ctest-two-labels-sh-tmp.txt
testlistFile=./ctest-two-labels-sh-testlist.txt
testnumsFile=./ctest-two-labels-sh-testnums.txt

ctest -L "^($label1)$" -N > "$tmpfile"
ctest -L "^($label2)$" -N >> "$tmpfile"

cat "$tmpfile" | grep "  Test #" | sort | uniq -d > "$testlistFile"

cat "$testlistFile" | awk -F "#" '{ print $2; }' | awk -F ":" '{ print
$1; }' > "$testnumsFile"

testnums=$(cat "$testnumsFile" | paste -s -d, -)

#echo Tests with both labels $label1 and $label2:
#cat "$testlistFile"
#
#echo Just the test numbers:
#cat "$testnumsFile"
#
#echo The test numbers, assembled into a ctest -I string to run just
those numbered tests:
#echo $testnums

echo Running command line:
echo ""
echo "  ctest -I \"0,0,0,$testnums\" -N"
echo ""
echo Run it without the -N to actually execute the tests...
echo ""

ctest -I "0,0,0,$testnums" -N



It's a "back of the envelope / proof of concept" script. Polish it up
and make it nice if the approach seems reasonable to you.


HTH,
David C.





On Fri, Mar 24, 2017 at 8:08 AM, Eric Noulard  wrote:
>
>
> 2017-03-24 12:30 GMT+01:00 Nils Gladitz :
>>
>> On 03/24/2017 11:50 AM, Eric Noulard wrote:
>>
>>> Hi there,
>>>
>>> I'm playing with ctest LABELS and I wanted to know whether if it is
>>> possible
>>> to run the set of tests which have 2 labels ?
>>>
>>> I manage to have all tests which have **either** L1 or L2:
>>>
>>> ctest -L "L1|L2"
>>>
>>> but how can I write a proper command line for both L1 and L2 ?
>>>
>>> apparently
>>>
>>> ctest -L "L1" -L "L2"
>>>
>>> only takes into account the last -L argument.
>>>
>>
>> I don't think it is possible currently.
>> To a degree you might be able to work around it by creating additional
>> labels that combine the existing ones e.g. "L1", "L2", "L1-L2".
>
>
> Yes of course.
> I do test name mangling but I was hoping for more "classifying" feature with
> labels and was expecting a simple
> OR and AND operation with labels. This is not the case.
>
>
>>
>>
>> I was pondering trying to implement test filtering based on the condition
>> evaluator CMake uses for if()/while() at some point
>> (cmConditionEvaluator.cxx).
>> e.g. --run-tests-if "L1 IN_LIST TEST_LABELS AND L2 IN_LIST TEST_LABELS"
>
>
> It could have been nice but may be overkill my needs.
> A repetable ANDing -L or -LE will do the job.
>
> That said since test filtering is regex-based I simply rediscovered that
> ANDing or negating regex is not easy.
>
> --
> Eric
>
> --
>
> Powered by www.kitware.com
>
> Please keep messages on-topic and check the CMake FAQ at:
> http://www.cmake.org/Wiki/CMake_FAQ
>
> Kitware offers various services to support the CMake community. For more
> information on each offering, please visit:
>
> CMake Support: http://cmake.org/cmake/help/support.html
> CMake Consulting: http://cmake.org/cmake/help/consulting.html
> CMake Training Courses: http://cmake.org/cmake/help/training.html
>
> Visit other Kitware open-source projects at
> http://www.kitware.com/opensource/opensource.html
>
> Follow this link to subscribe/unsubscribe:
> http://public.kitware.com/mailman/listinfo/cmake
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] Running test who have two labels?

2017-03-24 Thread Eric Noulard 
2017-03-24 12:30 GMT+01:00 Nils Gladitz :

> On 03/24/2017 11:50 AM, Eric Noulard wrote:
>
> Hi there,
>>
>> I'm playing with ctest LABELS and I wanted to know whether if it is
>> possible
>> to run the set of tests which have 2 labels ?
>>
>> I manage to have all tests which have **either** L1 or L2:
>>
>> ctest -L "L1|L2"
>>
>> but how can I write a proper command line for both L1 and L2 ?
>>
>> apparently
>>
>> ctest -L "L1" -L "L2"
>>
>> only takes into account the last -L argument.
>>
>>
> I don't think it is possible currently.
> To a degree you might be able to work around it by creating additional
> labels that combine the existing ones e.g. "L1", "L2", "L1-L2".
>

Yes of course.
I do test name mangling but I was hoping for more "classifying" feature
with labels and was expecting a simple
OR and AND operation with labels. This is not the case.



>
> I was pondering trying to implement test filtering based on the condition
> evaluator CMake uses for if()/while() at some point
> (cmConditionEvaluator.cxx).
> e.g. --run-tests-if "L1 IN_LIST TEST_LABELS AND L2 IN_LIST TEST_LABELS"


It could have been nice but may be overkill my needs.
A repetable ANDing -L or -LE will do the job.

That said since test filtering is regex-based I simply rediscovered that
ANDing or negating regex is not easy.

-- 
Eric
-- 

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake

Re: [CMake] Running test who have two labels?

2017-03-24 Thread Nils Gladitz 

On 03/24/2017 11:50 AM, Eric Noulard wrote:


Hi there,

I'm playing with ctest LABELS and I wanted to know whether if it is 
possible

to run the set of tests which have 2 labels ?

I manage to have all tests which have **either** L1 or L2:

ctest -L "L1|L2"

but how can I write a proper command line for both L1 and L2 ?

apparently

ctest -L "L1" -L "L2"

only takes into account the last -L argument.



I don't think it is possible currently.
To a degree you might be able to work around it by creating additional 
labels that combine the existing ones e.g. "L1", "L2", "L1-L2".


I was pondering trying to implement test filtering based on the 
condition evaluator CMake uses for if()/while() at some point 
(cmConditionEvaluator.cxx).

e.g. --run-tests-if "L1 IN_LIST TEST_LABELS AND L2 IN_LIST TEST_LABELS"

Nils
--

Powered by www.kitware.com

Please keep messages on-topic and check the CMake FAQ at: 
http://www.cmake.org/Wiki/CMake_FAQ

Kitware offers various services to support the CMake community. For more 
information on each offering, please visit:

CMake Support: http://cmake.org/cmake/help/support.html
CMake Consulting: http://cmake.org/cmake/help/consulting.html
CMake Training Courses: http://cmake.org/cmake/help/training.html

Visit other Kitware open-source projects at 
http://www.kitware.com/opensource/opensource.html

Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/cmake