Am 18.09.2020 um 23:56 schrieb Jean Abou Samra:
Hi,
Le 18/09/2020 à 11:15, Phil Holmes a écrit :
I don't know if this isn't clear, but just to state the original
point of verifying issues.
If the change is claimed to fix a bug, we compile the previously
buggy code with the release in which the bug is claimed fixed and
check that the bug is no longer there. It it has really disappeared,
we change the status of the issue from Fixed to Verified. i.e. we
are certain that the bug is no longer there.
If the change is to provide updated functionality, then it can be
really quite hard to verify the new functionality, and in any case
the patch review system should do that. So we simply check the patch
was pushed into the claimed build. If it's clear that it was, we
mark the status as Verified.
That was the original intention of verifying issues.
Thanks for the insight.
So, Michael and I verified nearly all issues in Status::Fixed.
Actually, Jean did the vast majority, because I had to leave on Friday
afternoon for a rehearsal weekend.
The ones remaining are listed here:
https://gitlab.com/lilypond/lilypond/-/issues?scope=all=%E2%9C%93=closed_name[]=Status%3A%3AFixed[milestone_title]=2.21.7
Among these,
https://gitlab.com/lilypond/lilypond/-/issues/6027
which I don't know how to test (Jonas?).
The others (5 items) are Windows-related. Maybe Michael or someone
else with access to a Windows machine can go over them?
I verified them and noticed a couple of another issues.
(#6044-#6046)
Lilypond-book on Windows seems broken to me, because
the correct way to run it is neither obvious nor documented.
The script lacks a file ending (2.20.0 had lilypond-book.py, 2.21.6 only
lilypond-book) and can only be run like 'python lilypond-book foo.lytex'
Jean, what would you propose as a good way to ship these python scripts
for the Windows releases?
Otherwise, part of the work was extremely simple, namely check that
the commits had made it into some branch under release/, which is fast;
and part of it was much harder, when trying to reproduce memory-related
bugs, for example. I reopened a few of these issues.
This doesn't answer Jonas' initial question: what should be done,
in terms of policies, with regard to issue verification? I don't
know. Personally I think it's not really needed as long as every
bug fix contains a regression test.
Do we agree that 'verification' in the sense of
'checking whether a particular commit made it into a release x.y.z'
is not necessary anymore, because attached merge requests show that
the commit has gone in?
Speaking of regression tests, I also went through comparison pages
from "2.21.6 vs. 2.21.5" to "2.21.2 vs. 2.21.1" included. Honestly,
I've had my share of this; could somebody jump in to verify the
last two pages as well? These are:
lilypond.org/test/v2.21.1-1/compare-v2.21.0-1/index.html
http://lilypond.org/test/v2.21.0-1/compare-v2.20.0-1/index.html
I find that very difficult, because some changes may be intended, others
not...
Also there are often slight spacing changes that are probably "Heisenbugs",
like James mentioned.
Depending on your time and energy available, you may even go
farther. Or simply (I mean simple, not costless) verify the
entire test suite:
http://lilypond.org/doc/v2.21/input/regression/collated-files.html
That'll be all for today.
Thank you very much, Jean!
Best,
Jean
