Re: 2.21.0 Issues all verified!

[Carl Sorensen]

On Sat, May 16, 2020 at 4:28 PM David Kastrup  wrote:

>
> Uh, I think it's the wrong point of time to feel discouraged: we are at
> the current point of time figuring out what the tool does well and not
> so well under which usage.

I completely agree with this.  I miss the Rietveld review process, but
mostly I'm not familiar at all with the GitLab one.

>
> Personally, so far my main issue is, well, the workflow bypassing
> issues.  Changes that are only presented as merge requests without any
> accompanying issue are just too intangible for my taste: those
> correspond more to what we previously said "things like that you can
> push directly to staging".  I find it awkward to find my way around
> them, and after pushing there does not seem an obvious reverse relation
> to a tangible report that one could easily derive from seeing the commit
> in the repository.
>
> I prefer having an actual issue number for the details for anything of
> non-trivial nature.

+1

I believe we should declare (and try to enforce) a policy that the
name of a branch in a merge request should include an issue number.

With git-cl upload, an issue was automatically created if it didn't
already exist.  I really liked that functionality.  If we can't do
such a thing automatically in GitLab, I think we should do it by
policy.  As you say, it's very hard to track merge requests without a
tie to the issue tracker.

Thanks,

Carl

Re: Final resolution of issue 4751

[Carl Sorensen]

On Sat, May 16, 2020 at 6:43 PM Carl Sorensen  wrote:
>
> On Sat, May 16, 2020 at 4:20 PM Valentin Villenave
>  wrote:
> >
> > On 5/16/20, David Kastrup  wrote:
> > > I don't really have a good idea here.
> >
> > Looks like quite a mess.  I’ve re-categorized it as Status::Started,
> > and removed the Fixed_2_19_44 label for now.  I‘ve also linked it with
> > issue #4781 since it seems there’s quite a bit of overlap.
>
> I think Status::Started is wrong, since it has been pushed in some form.
>
> I think I will check through it again and see if I can make a better
> determination, instead of having it go to Started.
>
> I suspect the best thing to do is to close this issue, since it's so
> messed up (a push, and closed, and reopened, and closed again, even
> before today).
>
> If there are problems with musicxml2ly, I think it deserves a new issue.
>

After more review, and seeing how the commits are all messed up with a
huge variety of other changes, the best thing to do is close this
issue and call it Verified.  IMO it's better to start over than try to
fix this one (and we do hav 4781 which came after and claims to fix
it.

Carl

Re: Final resolution of issue 4751

[Carl Sorensen]

On Sat, May 16, 2020 at 4:20 PM Valentin Villenave
 wrote:
>
> On 5/16/20, David Kastrup  wrote:
> > I don't really have a good idea here.
>
> Looks like quite a mess.  I’ve re-categorized it as Status::Started,
> and removed the Fixed_2_19_44 label for now.  I‘ve also linked it with
> issue #4781 since it seems there’s quite a bit of overlap.

I think Status::Started is wrong, since it has been pushed in some form.

I think I will check through it again and see if I can make a better
determination, instead of having it go to Started.

I suspect the best thing to do is to close this issue, since it's so
messed up (a push, and closed, and reopened, and closed again, even
before today).

If there are problems with musicxml2ly, I think it deserves a new issue.

>
> Jean, since you wanted to dive into musicxml stuff, perhaps you could
> investigate exactly what parts of John Gourlay’s work got merged
> (and/or possibly un-merged) and whether there remain some potentially
> interesting leftovers? There’s a good chance that was the last time
> anyone did any substantial work on musicxml import, so perhaps that
> would be as good an entry point as any (though, I’m afraid, not very
> glamorous).

If Jean wants to do it, I think it should follow after a resolution of
this issue.

Thanks,

Carl

>
> Cheers,
> -- V.

Re: 2.21.0 Issues all verified!

[Karlin High]

On 5/16/2020 5:28 PM, David Kastrup wrote:

I prefer having an actual issue number for the details for anything of
non-trivial nature.


I don't know how related this is, but GitLab has "Description templates."



It looks like that could prefill descriptions for issues and merge 
requests with common questions, like does this need a regtest, does it 
need documentation, does it need a "Changes" entry, etc.

--
Karlin High
Missouri, USA

Re: 2.21.0 Issues all verified!

[David Kastrup]

Valentin Villenave  writes:

> On 5/16/20, Carl Sorensen  wrote:
>> Thanks to all who have helped us get
>> back to a more regular release schedule!
>
> Well, thanks to you!  And to everybody who gave a hand (Federico,
> Karlin, and a few others?).  For a little while I was reminded of the
> Google tracker era, and of all the great work Graham contributed to.
> (Though today some of us evidently have a different approach in mind
> -- and didn’t seem to be impressed by this collective effort, which
> Jonas criticized as favoring "quantity over quality".)
>
> We’ll see what the issue-tracking policy becomes in the future
> (marking issues as "Verified" may well go the way of the dodo, since
> from what Jonas told me GitLab pushes us to only have two states: Open
> and Closed).  My main concern is that merge requests shoudn’t get
> prioritized over code consistency and QA.  (From a personal point of
> view, I am feeling quite discouraged but that’s my sort-of default
> mode so it doesn’t carry much meaning.)

Uh, I think it's the wrong point of time to feel discouraged: we are at
the current point of time figuring out what the tool does well and not
so well under which usage.

Personally, so far my main issue is, well, the workflow bypassing
issues.  Changes that are only presented as merge requests without any
accompanying issue are just too intangible for my taste: those
correspond more to what we previously said "things like that you can
push directly to staging".  I find it awkward to find my way around
them, and after pushing there does not seem an obvious reverse relation
to a tangible report that one could easily derive from seeing the commit
in the repository.

I prefer having an actual issue number for the details for anything of
non-trivial nature.

But there is no point in being discouraged as long as we haven't even
decided on particular workflows: instead one should bring up problems
one sees.

-- 
David Kastrup

Re: Final resolution of issue 4751

[Valentin Villenave]

On 5/16/20, David Kastrup  wrote:
> I don't really have a good idea here.

Looks like quite a mess.  I’ve re-categorized it as Status::Started,
and removed the Fixed_2_19_44 label for now.  I‘ve also linked it with
issue #4781 since it seems there’s quite a bit of overlap.

Jean, since you wanted to dive into musicxml stuff, perhaps you could
investigate exactly what parts of John Gourlay’s work got merged
(and/or possibly un-merged) and whether there remain some potentially
interesting leftovers? There’s a good chance that was the last time
anyone did any substantial work on musicxml import, so perhaps that
would be as good an entry point as any (though, I’m afraid, not very
glamorous).

Cheers,
-- V.

Re: 2.21.0 Issues all verified!

[Karlin High]

On 5/16/2020 5:05 PM, Valentin Villenave wrote:

Well, thanks to you!  And to everybody who gave a hand (Federico,
Karlin, and a few others?).


Thanks also, Carl! I verified exactly one issue, slowly and carefully 
just to learn the process. Last week didn't give me capacity to get much 
beyond that. I expect that an effort like this one is easier to 
crowd-source than things like the Python3 upgrade.


I'm increasingly convinced that group efforts are best raised by giving 
step-by-step instructions. As was seen here, or in Knut Petersen's 
"Please Test GUB" work.

--
Karlin High
Missouri, USA

Re: 2.21.0 Issues all verified!

[Valentin Villenave]

On 5/16/20, Carl Sorensen  wrote:
> Thanks to all who have helped us get
> back to a more regular release schedule!

Well, thanks to you!  And to everybody who gave a hand (Federico,
Karlin, and a few others?).  For a little while I was reminded of the
Google tracker era, and of all the great work Graham contributed to.
(Though today some of us evidently have a different approach in mind
-- and didn’t seem to be impressed by this collective effort, which
Jonas criticized as favoring "quantity over quality".)

We’ll see what the issue-tracking policy becomes in the future
(marking issues as "Verified" may well go the way of the dodo, since
from what Jonas told me GitLab pushes us to only have two states: Open
and Closed).  My main concern is that merge requests shoudn’t get
prioritized over code consistency and QA.  (From a personal point of
view, I am feeling quite discouraged but that’s my sort-of default
mode so it doesn’t carry much meaning.)

Cheers,
-- V.

Re: Final resolution of issue 4751

[David Kastrup]

Carl Sorensen  writes:

> Trying to verify closed issues, I came across #4751
>
> https://gitlab.com/lilypond/lilypond/-/issues/4751
>
> Apparently it was pushed, and then changes were made, and people
> didn't like it, and so there is no final commit.  Yet it is still
> tagged as fixed,
>
> Can anybody give me guidance as to how I should proceed?  Should I
> just verify the issue and call it good?
>
> Thanks for the input.

Huh, that's a tricky one.  I don't remember the details any more, but it
appears like the imported changes possibly caused some regressions (but
there are no hard verifiable statements in this report itself?) but were
kept in.  I don't really have a good idea here.

-- 
David Kastrup

Re: 2.21.0 Issues all verified!

[Carl Sorensen]

On Sat, May 16, 2020 at 2:12 PM Carl Sorensen  wrote:
>
> I just verified the last of the issues with Fixed_2_21_0.  Thanks everybody 
> for all the work!
>
> I found a few issues that had misspelled labels, and relabeled them to the 
> proper spelling.
>
> It looks like we still need to verify issues for 2.20, as well as for 2.21.1

I've now verified all of the outstanding issues for 2.20 (some of them
had to be moved to 2.21.0, as they had not been placed into 2.20.

It looks like we have 33 issues to be verified for 2.21.1.  Muuch
better than 522+ for 2.21.0!  Thanks to all who have helped us get
back to a more regular release schedule!

Carl

Final resolution of issue 4751

[Carl Sorensen]

Trying to verify closed issues, I came across #4751

https://gitlab.com/lilypond/lilypond/-/issues/4751

Apparently it was pushed, and then changes were made, and people
didn't like it, and so there is no final commit.  Yet it is still
tagged as fixed,

Can anybody give me guidance as to how I should proceed?  Should I
just verify the issue and call it good?

Thanks for the input.

Carl

2.21.0 Issues all verified!

[Carl Sorensen]

I just verified the last of the issues with Fixed_2_21_0.  Thanks everybody
for all the work!

I found a few issues that had misspelled labels, and relabeled them to the
proper spelling.

It looks like we still need to verify issues for 2.20, as well as for 2.21.1

There are also a few unverified issues from older releases, but I'll work
through those this afternoon.

Thanks,

Carl

Re: Another round of questions

[Jean Abou Samra]

Hi,
Le 16/05/2020 à 20:02, David Kastrup a écrit :

Valentin Villenave  writes:


On 5/16/20, Werner LEMBERG  wrote:

IMHO only bugfixes should be applied to `musicxml2ly` since `xml2ly`
covers much more of MusixXML (and will, AFAIK, also eventually support
its successor, MNX).

Huh.  xml2ly has very different requirements than musicxml2ly and is
(AFAICS) unlikely to ever be integrated into LilyPond.  Anecdotally, I
have encountered several cases where musicxml2ly turned to provide
much more useful output than xml2ly (but that was a couple of years
ago, maybe its development has sped up since then).

Plus, since we’re discussing python-based tools, there are a few
useful functions being developed in python-ly (as part of
Frescobaldi), which share a bit more DNA with musicxml2ly than with
anything else.  So I wouldn’t count musicxml2ly out just yet.

Personally, I see nothing wrong with maintaining tools while they are
used and distributed.  In a corporate setting, alloting large amounts of
resources into a part of the tool set that has a perspective to be
replaced in the course of a corporate development makes of course little
sense.

We aren't there.  We have no timetable for a replacement or its
viability, and so I don't see the point in discouraging contributors
from making fixes to what will continue for an indeterminate time to be
part of the tool set useful for achieving objectives.


Thank you all for your replies. Based on that information, I decide to 
dive into the code, and we'll see what I can improve in it. My concern 
was about the possible replacement being planned and pending (I told 
myself it could have been delayed by the transition to GitLab), but if 
it is still to be discussed wether a replacement is tangible, then I 
will step into musicxml2ly with the rest. I'm unlikely to implement 
stunning features anyway. My goal is rather to adapt it to Python 3, and 
thus make it more maintainable by simplifying many parts of the code 
that can be rewritten elegantly using constructs and standard modules 
appeared in later versions of Python. Also, I consider it an investment 
for a future when I would be done with the very exacting part of my 
curriculum I'm currently in, and depending on many factors, I might 
start contributing more seriously (but that's not for right now, really).


Best,

Jean Abou Samra

Re: Another round of questions

[David Kastrup]

Valentin Villenave  writes:

> On 5/16/20, Werner LEMBERG  wrote:
>> IMHO only bugfixes should be applied to `musicxml2ly` since `xml2ly`
>> covers much more of MusixXML (and will, AFAIK, also eventually support
>> its successor, MNX).
>
> Huh.  xml2ly has very different requirements than musicxml2ly and is
> (AFAICS) unlikely to ever be integrated into LilyPond.  Anecdotally, I
> have encountered several cases where musicxml2ly turned to provide
> much more useful output than xml2ly (but that was a couple of years
> ago, maybe its development has sped up since then).
>
> Plus, since we’re discussing python-based tools, there are a few
> useful functions being developed in python-ly (as part of
> Frescobaldi), which share a bit more DNA with musicxml2ly than with
> anything else.  So I wouldn’t count musicxml2ly out just yet.

Personally, I see nothing wrong with maintaining tools while they are
used and distributed.  In a corporate setting, alloting large amounts of
resources into a part of the tool set that has a perspective to be
replaced in the course of a corporate development makes of course little
sense.

We aren't there.  We have no timetable for a replacement or its
viability, and so I don't see the point in discouraging contributors
from making fixes to what will continue for an indeterminate time to be
part of the tool set useful for achieving objectives.

-- 
David Kastrup

Re: Another round of questions

[Valentin Villenave]

On 5/16/20, Werner LEMBERG  wrote:
> IMHO only bugfixes should be applied to `musicxml2ly` since `xml2ly`
> covers much more of MusixXML (and will, AFAIK, also eventually support
> its successor, MNX).

Huh.  xml2ly has very different requirements than musicxml2ly and is
(AFAICS) unlikely to ever be integrated into LilyPond.  Anecdotally, I
have encountered several cases where musicxml2ly turned to provide
much more useful output than xml2ly (but that was a couple of years
ago, maybe its development has sped up since then).

Plus, since we’re discussing python-based tools, there are a few
useful functions being developed in python-ly (as part of
Frescobaldi), which share a bit more DNA with musicxml2ly than with
anything else.  So I wouldn’t count musicxml2ly out just yet.

Cheers,
-- V.

Re: Building the website

[David Kastrup]

Jonas Hahnfeld  writes:

> To make this more explicit: I don't think we should do this. Mirroring
> to Savannah and pulling from there is just fine and, as others have
> expressed, probably the wiser decision long-term.

With regard to "long-term": updating the pull location of the website
server after a wholesale server migration should be the least of our
worries.  While it makes strategic sense sticking with Savannah for
links we publicly advertise (in the Internet community, every
recommendation tends to stick around for years after you want it gone),
the site the web server pulls from is not really public.

So giving this have one hop less to work with is reasonable in my book
for the sake of ongoing operations.

-- 
David Kastrup

Re: Building the website

[Jonas Hahnfeld]

To make this more explicit: I don't think we should do this. Mirroring
to Savannah and pulling from there is just fine and, as others have
expressed, probably the wiser decision long-term.

Am Samstag, den 16.05.2020, 12:22 +0200 schrieb Han-Wen Nienhuys:
> I did this. For reference, you can find out what is running as follows:
> 
> $ crontab -l
> # from website-rebuild.cron
> LILYPOND_GIT=/home/graham/lilypond/lilypond-git/
> LILYPOND_WEB_MEDIA_GIT=/home/graham/lilypond/lilypond-extra/
> PATH=/home/graham/usr/bin/:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
> 
> 
> 11 * * * * $HOME/lilypond/bin/update-git.sh > /dev/null 2>&1
> 37 * * * * $HOME/lilypond/bin/make-website.sh > /dev/null 2>&1
> 
> $ cat lilypond/bin/update-git.sh
> ### update-git.sh
> #!/bin/sh
> resultFile=/home/graham/lilypond/gitpull`date +%H`.txt
> cd $LILYPOND_GIT
> git fetch origin  > $resultFile 2>&1
> git merge origin/master >> $resultFile 2>&1
> cd $LILYPOND_WEB_MEDIA_GIT
> git fetch origin >> $resultFile 2>&1
> git merge origin/master >> $resultFile 2>&1
> 
> I've updated the git config under /home/graham/lilypond/lilypond-git/
> to point to gitlab now.
> 
> On Sat, May 16, 2020 at 12:11 PM Jonas Hahnfeld <
> hah...@hahnjo.de
> > wrote:
> > Am Samstag, den 16.05.2020, 11:02 +0100 schrieb Phil Holmes:
> > > Currently I expect that the website is still being built from Savannah.  
> > > It
> > > is updated by 2 cron jobs that run every hour, 30 minutes apart.
> > > update-git.sh updates its repository and make-website.sh makes the 
> > > website.
> > > These scripts are shown on
> > > http://lilypond.org/doc/v2.21/Documentation/contributor/uploading-and-security
> > > 
> > > 
> > > I assume we should plan to change the server to use gitlab.  I can access
> > > the server but don't feel confident to update its git repository.  Could
> > > Jonas (or anyone else) confirm that we should change the server to use the
> > > new location and give me a step-by-step on how to do this?
> > 
> > Not necessarily, we can continue to update from Savannah as long as we
> > push master from GitLab to Savannah (see other thread). I've now pushed
> > manually, so the website should be updated according to the established
> > scripts.
> > 
> > Jonas
> 
> 
> 
> 


signature.asc
Description: This is a digitally signed message part

Re: Building the website

[Han-Wen Nienhuys]

I did this. For reference, you can find out what is running as follows:

$ crontab -l
# from website-rebuild.cron
LILYPOND_GIT=/home/graham/lilypond/lilypond-git/
LILYPOND_WEB_MEDIA_GIT=/home/graham/lilypond/lilypond-extra/
PATH=/home/graham/usr/bin/:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin


11 * * * * $HOME/lilypond/bin/update-git.sh > /dev/null 2>&1
37 * * * * $HOME/lilypond/bin/make-website.sh > /dev/null 2>&1

$ cat lilypond/bin/update-git.sh
### update-git.sh
#!/bin/sh
resultFile=/home/graham/lilypond/gitpull`date +%H`.txt
cd $LILYPOND_GIT
git fetch origin  > $resultFile 2>&1
git merge origin/master >> $resultFile 2>&1
cd $LILYPOND_WEB_MEDIA_GIT
git fetch origin >> $resultFile 2>&1
git merge origin/master >> $resultFile 2>&1

I've updated the git config under /home/graham/lilypond/lilypond-git/
to point to gitlab now.

On Sat, May 16, 2020 at 12:11 PM Jonas Hahnfeld  wrote:
>
> Am Samstag, den 16.05.2020, 11:02 +0100 schrieb Phil Holmes:
> > Currently I expect that the website is still being built from Savannah.  It
> > is updated by 2 cron jobs that run every hour, 30 minutes apart.
> > update-git.sh updates its repository and make-website.sh makes the website.
> > These scripts are shown on
> > http://lilypond.org/doc/v2.21/Documentation/contributor/uploading-and-security
> >
> > I assume we should plan to change the server to use gitlab.  I can access
> > the server but don't feel confident to update its git repository.  Could
> > Jonas (or anyone else) confirm that we should change the server to use the
> > new location and give me a step-by-step on how to do this?
>
> Not necessarily, we can continue to update from Savannah as long as we
> push master from GitLab to Savannah (see other thread). I've now pushed
> manually, so the website should be updated according to the established
> scripts.
>
> Jonas



-- 
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen

Re: Building the website

[Jonas Hahnfeld]

Am Samstag, den 16.05.2020, 11:02 +0100 schrieb Phil Holmes:
> Currently I expect that the website is still being built from Savannah.  It 
> is updated by 2 cron jobs that run every hour, 30 minutes apart. 
> update-git.sh updates its repository and make-website.sh makes the website. 
> These scripts are shown on 
> http://lilypond.org/doc/v2.21/Documentation/contributor/uploading-and-security
> 
> I assume we should plan to change the server to use gitlab.  I can access 
> the server but don't feel confident to update its git repository.  Could 
> Jonas (or anyone else) confirm that we should change the server to use the 
> new location and give me a step-by-step on how to do this?

Not necessarily, we can continue to update from Savannah as long as we
push master from GitLab to Savannah (see other thread). I've now pushed
manually, so the website should be updated according to the established
scripts.

Jonas


signature.asc
Description: This is a digitally signed message part

Building the website

[Phil Holmes]

Currently I expect that the website is still being built from Savannah.  It 
is updated by 2 cron jobs that run every hour, 30 minutes apart. 
update-git.sh updates its repository and make-website.sh makes the website. 
These scripts are shown on 
http://lilypond.org/doc/v2.21/Documentation/contributor/uploading-and-security


I assume we should plan to change the server to use gitlab.  I can access 
the server but don't feel confident to update its git repository.  Could 
Jonas (or anyone else) confirm that we should change the server to use the 
new location and give me a step-by-step on how to do this?


Thanks.

--
Phil Holmes

Re: PATCHES - Countdown for May 15th

['James']

On 15/05/2020 15:25, Jonas Hahnfeld wrote:

I cannot really tell easily a real new patch and one that is rebased
ready for pushing.

What do you think?

That the script is doing exactly what I told it to do: The diff between
the previous and the rebased commit is not empty. Therefore it adds the
Patch::new label, removing Patch::push.
While correct in theory (new diff = new testing), I think we should
ignore updates once in Patch::push. If the author changes something
which was not in previous versions of the patch, we need to reset to
Patch::new manually. Does this sound acceptable?

Jonas


I'll let you and the developers figure that out, I can work with 
whatever I need to do. If it means I occasionally retest a patch twice, 
then so be it. No big deal.


James

Re: mirror GitLab -> Savannah

[Jonas Hahnfeld]

Am Donnerstag, den 14.05.2020, 20:31 +0200 schrieb Jonas Hahnfeld:
> To keep this short: I'd like to enable push mirroring from GitLab to
> Savannah for the following branches [1]:
>  - master
>  - release/unstable
>  - stable/*
>  - translation*
> This is a feature of GitLab and runs automatically in the background:
> https://gitlab.com/help/user/project/repository/repository_mirroring.md
> 
> I've already created a new lilypod_bot user and requested access to the
> Savannah project. Unless somebody objects, I'll configure GitLab
> accordingly and paste the generated SSH key for the Savannah user.

Update on this one: I've tried to configure the necessary options, but
apparently it's broken on gitlab.com since this week:
https://gitlab.com/gitlab-org/gitlab/-/issues/216619

For now, I've manually pushed master to Savannah in order to update the
webpage with the link to the new issue tracker.

Jonas


signature.asc
Description: This is a digitally signed message part

Re: PATCHES - Countdown for May 15th

[Jonas Hahnfeld]

Am Freitag, den 15.05.2020, 21:32 +0200 schrieb Jonas Hahnfeld:
> Am Freitag, den 15.05.2020, 16:57 +0100 schrieb Kevin Barry:
> > On Fri, May 15, 2020 at 04:25:52PM +0200, Jonas Hahnfeld wrote:
> > > That the script is doing exactly what I told it to do: The diff between
> > > the previous and the rebased commit is not empty. Therefore it adds the
> > > Patch::new label, removing Patch::push.
> > 
> > Shouldn't `diff staging...HEAD` be the same before and after a rebase
> > (three dots)?
> 
> Yes, I think this works as long as staging not already includes some
> (rebased) commits that were previously part of the merge request. We
> could of course determine the merge base between the old and new commit
> which should also avoid that case.

My thinking was wrong here: The common merge base of the two commits
results in a larger diff for the new commit because it now includes all
commits since the merge base, which is not what we want.
So I went with Kevin's original suggestion with one modification: The
staging branch is a moving target. Depending on when the webhook runs,
the rebased commits might have already landed which makes the diff
empty. Instead I ask GitLab for the diffs relative to the respective
base commits. I think this does the right thing (tm), at least in my
testing.

I already deployed the change to my server, here for reference: 
https://gitlab.com/lilypond/infrastructure/-/merge_requests/4
Testing in the repo and feedback would be welcome!

Jonas


signature.asc
Description: This is a digitally signed message part