1- Calling clear-system from within perform is wrong. 2- I still recommend redirecting the output-translations for that particular tree around the load-system with coverage, and dropping the :force t.
—♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• http://fare.tunes.org "I believe that sex is one of the most beautiful, natural, wholesome things that money can buy." — Steve Martin On Tue, Oct 18, 2016 at 6:21 PM, Alexey Veretennikov <txm.four...@gmail.com> wrote: > Hi, > > I've done something slightly more brute-force and probably not elegant. > I've modified the test asd file for my library in the following way (see > below) > > Basically I've tried to introduce the same operation as test-op. This > helps me to keep dependencies loaded/compiled via quicklisp. But it will > 1) Turn on the coverage > 2) Force reload of test subject with load-op and :force t > 3) Run tests > 4) Dump coverage results and [optionally] open browser with them. > > Please let me know if it could be done in an easier/more elegant way or > if there are any drawbacks of this approach. > > #| > Test package. > Usage: > (ql:quickload :mylib-api-test) > (asdf/operate:test-system :mylib-api) > > In order to perform code coverage with these tests (currenty supported > only on LispWorks 7), run the following: > (asdf/operate:operate 'mylib-api-test-asd::coverage-op :mylib-api-test) > |# > > > > (in-package :cl-user) > (defpackage mylib-api-test-asd > (:use :cl :asdf) > (:export coverage-op)) > > (in-package :mylib-api-test-asd) > > > (defclass coverage-op (selfward-operation) > ((selfward-operation :initform 'load-op :allocation :class)) > (:documentation "Test coverage operation")) > > > (defsystem mylib-api-test > :author "Alexey Veretennikov" > :license "BSD" > :depends-on (:mylib-api > :cl-fad > :flexi-streams > :prove) > :components ((:module "t" > :components > ((:file "base") > (:test-file "pack-test") > (:test-file "utils-test")))) > :description "Test system for mylib-api" > > :defsystem-depends-on (:prove-asdf) > :perform (test-op :after (op c) > (funcall (intern #.(string :run-test-system) :prove-asdf) > c) > (asdf:clear-system c)) > :perform (coverage-op (op c) > (run-tests-with-coverage) > (asdf:clear-system c))) > > > #+lispworks7 > (defun generate-coverage-output-path () > (multiple-value-bind (second minute hour date month year);; day) > (get-decoded-time) > (let ((results-directory-name > (pathname > (format nil > "mylib-api-coverage_~4,'0d-~2,'0d-~2,'0d_~2,'0d_~2,'0d_~2,'0d/index.html" > year month date hour minute second)))) > > (merge-pathnames results-directory-name (hcl:get-temp-directory))))) > > > #+lispworks7 > (defun run-lw-test-coverage () > (hcl:clear-code-coverage) > (hcl:with-code-coverage-generation () > (asdf/operate:load-system :mylib-api :force t)) > (asdf/operate:test-system :mylib-api-test) > (let ((output-file (generate-coverage-output-path))) > (hcl:code-coverage-data-generate-coloring-html output-file) > (format *standard-output* "Generated coverage report to ~a" output-file) > #+macosx > (objc:invoke (objc:invoke "NSWorkspace" "sharedWorkspace") "openURL:" > (objc:invoke "NSURL" "URLWithString:" > (concatenate 'string "file://" (namestring > output-file)))))) > > > (defun run-tests-with-coverage () > #+lispworks7 > (run-lw-test-coverage) > #-lispworks7 > (error "Code coverage generation currently supported only on LispWorks 7 > and above")) > > > > Faré <fah...@gmail.com> writes: > >> I recommend that you write run tests, etc., in a separate process, as >> orchestrated by a script that just after it loads ASDF >> 1- loads all the library code for which you do NOT want test coverage >> 2- turns on coverage >> 3- configures the asdf-output-translations to redirect object files >> for those systems that you DO want coverage (and only those) to an >> alternate location >> 4- load the rest of the code >> 5- runs the test >> >> —♯ƒ • François-René ÐVB Rideau •Reflection&Cybernethics• >> http://fare.tunes.org >> Reasons for existence are usually provided for things that don't exist; >> they would be wasted on things which do. — Saul Gorn >> >> >> On Sat, Oct 15, 2016 at 2:52 AM, Alexey Veretennikov >> <txm.four...@gmail.com> wrote: >>> Anyone ? Is impossible to do or too hard? >>> I see it as following: >>> make a new operation which will: >>> 1) compile the test cases >>> 1) clean the test object (remove fasls of target system to test but not >>> dependencies) >>> 2) execute necessary startup code (set up the code coverage) >>> 3) run testcases >>> 4) perform teardown code (turn off the code coverage and collect stats etc) >>> >>> How could I achieve at least this? >>> >>> Alexey Veretennikov <txm.four...@gmail.com> writes: >>> >>>> Hi all, >>>> >>>> Right now I'm running unit tests using Fukamachi's prove library: >>>> (asdf/operate:test-system 'my-system). >>>> >>>> I want to run my tests generating the tests coverage of my system. >>>> >>>> For this I would like to have similar operation, but which will: >>>> 1) turn on the code coverage in LispWorks (just call to some global >>>> function) >>>> 2) rebuild system which I want to test (in this case my-system), but >>>> only (!) this system (not dependencies) >>>> 3) run tests >>>> 4) call coverage results processing function. >>>> 5) on normal run (asdf/operate:test-system 'my-system) rebuild the >>>> system without coverage. >>>> >>>> How could I proceed with this task? >>> >>> -- >>> Br, >>> /Alexey >>> > > -- > Br, > /Alexey