Here is an idea I was considering implementing if I ever get time to do it.
I am interested in hearing other's thoughts on the usefulness and
implementation of it.

The main point is to integrate a "test report" (or "status board" as it is
called in sqlite) into fossil. The rationale being that just like docs or
tickets are versioned along with the code since they are essentially part
of the state of the code at any specific revision (e.g. if I go back to an
older version, I would want the docs for that version and a list of known
issues with it, too), also tests passing/failing are part of this state.
Note that I am not suggesting adding a test runner that will automatically
fill the report, only the report itself.

My simplest idea for implementing this is to use the existing tags
mechanism. For the sake of the example I will use tag names test-*, but
this might not be the best choice in reality.
A new test would be added as a propagating tag such as "test-name=Compiles
without warnings". A link on the check-in page (maybe in the "Other Links"
line) will link to a list of tests (maybe like this
https://www.sqlite.org/checklists/3081100/index) with a list of all tests
(taken from the test-name tags) and an option to mark each as
passed/failed. A passed/failed test will be indicated as a non-propagating
tag such as "test-Compiles-without-warnings=pass".
A test-runner or CI system could write a simple file that is a control
artifact that adds the pass/fail tags as needed to the specific check-in
and adds this file to the fossil repo.
A drawback of this approach is that tags do not propagate across merges,
and I think it makes sense that if a test case is added to a branch, it
should propagate branches that it is merged into.

Another more complex approach would be to add a new type of artifact -  a
"Test artifact". Or maybe have the test cases as a new artifact but the
results as regular tags?

As you can see, this is only a half-baked idea. I would appreciate comments
both on the idea itself and on the implementation of it.

Thanks,
Baruch

-- 
˙uʍop-ǝpısdn sı ɹoʇıuoɯ ɹnoʎ 'sıɥʇ pɐǝɹ uɐɔ noʎ ɟı
_______________________________________________
fossil-dev mailing list
fossil-dev@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/fossil-dev

Reply via email to