Re: Old build environments
On Fri, Oct 16, 2020 at 1:02 PM Michael Käppler wrote: > Hi all, > a few days ago I wanted to try how some functionality in LilyPond > worked, when it was added long > time ago. (about 15 years ago, around 2.7.x or 2.8.x) > Not very surprisingly I could not even get the code to compile with my > current build setup. > > That made we wonder if it would be a good idea to store a build > environment that is proven to work > with the code base every, say, minor version bump. > I actually think that is the motivation for GUB. You could checkout the GUB repository for the appropriate date, and the LilyPond code should build under GUB. Have you tried that? Carl > >
Re: stable/2.22 has branched; master is 2.23.0
On Fri, Oct 16, 2020 at 11:41 AM Jonas Hahnfeld wrote: > > So master is 2.23.0 and usual development can resume, with the pleading > to avoid disruptive changes that make picking harder than necessary. > Fixes should also go to master and I'll pick them to the branch. If you > really need to do that, please use 'git cherry-pick -x' to record the > original hash. > > When do we stop worrying about cherry-picking and feel free to make disruptive changes again? I'm not asking because I have something in the queue; I'm just trying to understand the policy. Thanks, Carl
Old build environments
Hi all, a few days ago I wanted to try how some functionality in LilyPond worked, when it was added long time ago. (about 15 years ago, around 2.7.x or 2.8.x) Not very surprisingly I could not even get the code to compile with my current build setup. That made we wonder if it would be a good idea to store a build environment that is proven to work with the code base every, say, minor version bump. This could be a container (Docker? QEMU?) or an ISO image that can be run inside a VM. If the container formats are changing as quickly as the build environments, this does not make sense, however. But surely there are concepts out there for long-term archivation (or better, long-term reproducibility) of software. What do you think? Michael
Re: @url vs. @uref
Am 16.10.2020 um 15:52 schrieb Jonas Hahnfeld: Am Freitag, den 16.10.2020, 15:46 +0200 schrieb Michael Käppler: Am 15.10.2020 um 23:03 schrieb Werner LEMBERG: we're using both `@url` and `@uref` commands in our documentation, the overwhelming majority is `@uref`. michael] ~/lilypond/Documentation/en (master)]> git grep '@url' | wc -l 9 michael] ~/lilypond/Documentation/en (master)]> git grep '@uref' | wc -l 622 The functionality is exactly the same: https://www.gnu.org/software/texinfo/manual/texinfo/html_node/_0040url.html I'd like to unify this. Shall we go with `@uref` to avoid big diffs and because it is already the majority, or use `@url` what seems more intuitive to me? Since this is a pure mechanical change that can be done by a little script I favour a change to `@url` everywhere. Here you are: https://gitlab.com/lilypond/lilypond/-/merge_requests/462 As commented there, this is the absolutely worst moment to post such mechanical change with no merit (as far as I understand): This will be a real nightmare of merge conflicts when picking commits between master and the to-be-created stable/2.22. You're absolutely right, Jonas, I should have thought about that. I closed the MR. Michael
Re: stable/2.22 has branched; master is 2.23.0
Am Freitag, den 16.10.2020, 20:01 +0200 schrieb Jean-Charles Malahieude: > Le 16/10/2020 à 19:41, Jonas Hahnfeld a écrit : > > Hi all, > > > > I created stable/2.22 as discussed and the situation is as follows: > > * b7c2bc5feb (origin/master) Reset Changes document for next cycle > > * 7f0bb931c3 Bump VERSION to 2.23.0 for next cycle > > > * 8a58e932f8 (origin/stable/2.22) ci: Build branch on push > > > * b63a51f52f (origin/translation) Bump VERSION to 2.21.80 for release > > > candidates > > > / > > * b5de5719b7 Merge branch 'translation' into master > > > > So master is 2.23.0 and usual development can resume, with the pleading > > to avoid disruptive changes that make picking harder than necessary. > > Fixes should also go to master and I'll pick them to the branch. If you > > really need to do that, please use 'git cherry-pick -x' to record the > > original hash. > > > > Thanks, Jonas! > > I should have finished and sent fr.po to the FTP over the week-end. > I'll group the fetching once a week in a MR to master. I wonder if the translated po files should go via master or directly to stable/2.22: lilypond.pot will be updated with the next unstable releases (should that be necessary). But I don't mind either way, shouldn't pose big problems... > > translation builds on stable/2.22 and must only be merged there. > > Attempting a merge into master should give conflicts in VERSION, but > > please don't try at home ;-) > > > > In the past releases, the Changes document was grouped into subheadings > > instead of a list. If anyone volunteers this time, please speak up. > > > > I've tried to keep this grouping for the French version, hoping to have > correctly qualified the entries… Your chance to group the English version the same and make it "correct" by definition :-D Jonas signature.asc Description: This is a digitally signed message part
Re: stable/2.22 has branched; master is 2.23.0
Le 16/10/2020 à 19:41, Jonas Hahnfeld a écrit : Hi all, I created stable/2.22 as discussed and the situation is as follows: * b7c2bc5feb (origin/master) Reset Changes document for next cycle * 7f0bb931c3 Bump VERSION to 2.23.0 for next cycle | * 8a58e932f8 (origin/stable/2.22) ci: Build branch on push | * b63a51f52f (origin/translation) Bump VERSION to 2.21.80 for release candidates |/ * b5de5719b7 Merge branch 'translation' into master So master is 2.23.0 and usual development can resume, with the pleading to avoid disruptive changes that make picking harder than necessary. Fixes should also go to master and I'll pick them to the branch. If you really need to do that, please use 'git cherry-pick -x' to record the original hash. Thanks, Jonas! I should have finished and sent fr.po to the FTP over the week-end. I'll group the fetching once a week in a MR to master. translation builds on stable/2.22 and must only be merged there. Attempting a merge into master should give conflicts in VERSION, but please don't try at home ;-) In the past releases, the Changes document was grouped into subheadings instead of a list. If anyone volunteers this time, please speak up. I've tried to keep this grouping for the French version, hoping to have correctly qualified the entries… Cheers, -- Jean-Charles
stable/2.22 has branched; master is 2.23.0
Hi all, I created stable/2.22 as discussed and the situation is as follows: * b7c2bc5feb (origin/master) Reset Changes document for next cycle * 7f0bb931c3 Bump VERSION to 2.23.0 for next cycle | * 8a58e932f8 (origin/stable/2.22) ci: Build branch on push | * b63a51f52f (origin/translation) Bump VERSION to 2.21.80 for release candidates |/ * b5de5719b7 Merge branch 'translation' into master So master is 2.23.0 and usual development can resume, with the pleading to avoid disruptive changes that make picking harder than necessary. Fixes should also go to master and I'll pick them to the branch. If you really need to do that, please use 'git cherry-pick -x' to record the original hash. translation builds on stable/2.22 and must only be merged there. Attempting a merge into master should give conflicts in VERSION, but please don't try at home ;-) In the past releases, the Changes document was grouped into subheadings instead of a list. If anyone volunteers this time, please speak up. Thanks, Jonas signature.asc Description: This is a digitally signed message part
strange formatting with involved `\break`
Folks,
can someone please explain to me why the following code
{
\compressEmptyMeasures
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 | \break
R4*120 | \break
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 |
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 |
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 |
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 |
}
causes such an uneven formatting after the multi-measure? And is
there a work-around available?
Werner
Hi,
(Excuse the bad email threading, which will get fixed next time I reply.)
Interesting. The example can be made even more minimal, without
\compressEmptyMeasures:
{
c'1
\break
\repeat unfold 35 { c'1 }
}
This produces the same bad output in 2.20. In 2.18, the output
is still suboptimal, but different (files attached).
A workaround would be
\paper {
page-breaking = #ly:minimal-breaking
}
Both ly:optimal-page-breaking and ly:page-turn-breaking are faulty;
ly:one-page-breaking is not.
This issue could be related:
https://gitlab.com/lilypond/lilypond/-/issues/5488
Regards,
Jean
mwe-2.18.pdf
Description: Adobe PDF document
mwe-2.20.pdf
Description: Adobe PDF document
Re: @url vs. @uref
Am Freitag, den 16.10.2020, 15:46 +0200 schrieb Michael Käppler: > Am 15.10.2020 um 23:03 schrieb Werner LEMBERG: > > > we're using both `@url` and `@uref` commands in our documentation, > > > the overwhelming majority is `@uref`. > > > > > > michael] ~/lilypond/Documentation/en (master)]> git grep '@url' | wc > > > -l > > > 9 > > > michael] ~/lilypond/Documentation/en (master)]> git grep '@uref' | wc > > > -l > > > 622 > > > > > > The functionality is exactly the same: > > > https://www.gnu.org/software/texinfo/manual/texinfo/html_node/_0040url.html > > > > > > I'd like to unify this. Shall we go with `@uref` to avoid big diffs > > > and because it is already the majority, or use `@url` what seems > > > more intuitive to me? > > Since this is a pure mechanical change that can be done by a little > > script I favour a change to `@url` everywhere. > > Here you are: > https://gitlab.com/lilypond/lilypond/-/merge_requests/462 As commented there, this is the absolutely worst moment to post such mechanical change with no merit (as far as I understand): This will be a real nightmare of merge conflicts when picking commits between master and the to-be-created stable/2.22. quote: "note that it would be helpful to avoid large-scale changes" Jonas signature.asc Description: This is a digitally signed message part
Re: @url vs. @uref
Am 15.10.2020 um 23:03 schrieb Werner LEMBERG: we're using both `@url` and `@uref` commands in our documentation, the overwhelming majority is `@uref`. michael] ~/lilypond/Documentation/en (master)]> git grep '@url' | wc -l 9 michael] ~/lilypond/Documentation/en (master)]> git grep '@uref' | wc -l 622 The functionality is exactly the same: https://www.gnu.org/software/texinfo/manual/texinfo/html_node/_0040url.html I'd like to unify this. Shall we go with `@uref` to avoid big diffs and because it is already the majority, or use `@url` what seems more intuitive to me? Since this is a pure mechanical change that can be done by a little script I favour a change to `@url` everywhere. Here you are: https://gitlab.com/lilypond/lilypond/-/merge_requests/462 A nice weekend everyone, Michael Werner
strange formatting with involved `\break`
[lilypond git 647e127c07a794d087c5e39bf23c0d4a7d66a957 from Oct. 6th]
Folks,
can someone please explain to me why the following code
{
\compressEmptyMeasures
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 | \break
R4*120 | \break
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 |
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 |
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 |
c'4 4 4 4 | 4 4 4 4 | 4 4 4 4 | 4 4 4 4 |
}
causes such an uneven formatting after the multi-measure? And is
there a work-around available?
Werner
Re: LilyPond disabled on Wikimedia
On Fri, Oct 16, 2020 at 3:32 AM Carl Sorensen wrote: > Unfortunately, I’m not capable of handling this problem right now. > > > > But it’s hard for me to imagine that something that would require a major > version bump is only a week’s worth of work. I’ve copied Han-Wen to try to > understand more about why the change would require a major version bump. > > The one week estimate is not from me, or I am starting to forget things. To make a program secure, you have to introduce a security boundary, from where you control what the program can do. If you look in the archives, you can find https://gitlab.com/lilypond/lilypond/-/merge_requests/310 and its discussion. This tried to introduce a security boundary at the system call level, but this is impractical for various reasons outlined in the MR. The other obvious boundary is at the Scheme interpretation: if we can control what Scheme code does, the threat surface is reduced to C++ errors (eg. use-after-free, buffer overflow), which is a drastic reduction relative to escaping to the system through #(system "naughty-command") The mechanism to do this is through GUILE modules: a GUILE module introduces a namespace, and a GUILE module can only see symbols explicitly imported with (use-modules). So if we do evalation within an empty or safe module, we prevent unsafe code from executing. Or differently expressed: we prevent people from running the 'system function by declining to make the function value available to untrusted Scheme code. So far so good. Unfortunately, LilyPond parser scopes (eg \header, \paper) etc are implemented as modules as well, so the following works sym = 2% toplevel scope #(set! sym (1+ sym)) In order to get access to interesting functions, all scopes import GUILE and lilypond bindings, see https://github.com/lilypond/lilypond/blob/d99780e93bfeafbcafce1c2653eac8e294057e84/lily/ly-module.cc#L53 These bindings bring in a number of unsafe functions. In particular, you can do myvar = \system to store the unsafe 'system function into myvar in any given scope. Then, you can use one of the many callback mechanisms to invoke the 'system function to do mischief. The solution would be to ly_make_module always create a "safe" module (the code is just below), but I couldn't get this to work easily: the obvious solution triggers obscure errors in the GUILE module system. I do not know if fixing is possible and if this might take 2 hours or 2 weeks. When the GUILE problem is fixed, the system interface will have to be revisited too: the PS -> PDF/PNG conversions are partially driven from LilyPond parser scopes (have a look at init.ly), so they will need significant rearrangement. Finally, such a change will invalidate any .ly file that does anything remotely interesting (such as interact with the file system directly). I don't know how serious the impact to complicated extensions is (orchestral lily, XML/SVG/speech synth output etc.), hence the reason that this will require a 3.0 version bump. WMF could actually run LilyPond in a seccomp-style sandbox to secure it for Wikimedia itself, but IIUC, they decided LilyPond files pose an unacceptable risk to their users, hence their insistence on a better containment story for .ly files. Thanks, > > > Carl > > > > > > *From: *Tim Starling > *Date: *Thursday, October 15, 2020 at 6:17 PM > *To: *Étienne Beaulé , Carl Sorensen < > c_soren...@byu.edu> > *Cc: *Daniel Benjamin Miller , " > lilypond-devel@gnu.org" > *Subject: *Re: LilyPond disabled on Wikimedia > > > > A number of safe mode escape vulnerabilities were discovered. One of them, > tracked internally as T260225, was discovered by Han-Wen and has not been > rectified after two months. > > > > I discussed a plan for rectifying it with Han-Wen, and suggested that we > could contribute funding towards fixing it. However, I was not able to get > approval for funding it. So the task remains open for volunteers to > address. Of course, it is difficult to recruit volunteers when it is a > private security issue. > > > > Han-Wen commented that the rectification we discussed would require a > major version bump to 3.0. I don't consider that to be a blocker. I think > security hardening would make a good headline improvement for a 3.0 release. > > > > I would estimate it as approximately one week of work. If you're willing > to put that kind of time in, I can forward you the previous communications > on this issue. > > > > -- Tim Starling > > > > On 16/10/20 10:46 am, Étienne Beaulé wrote: > > Hello, I’m the maintainer of the Score extension. > > > > There is also https://nvd.nist.gov/vuln/detail/CVE-2020-17353 which > affects LilyPond through PostScript code injection. We’ve also done a > security audit. I’ve CC’d Tim Starling who performed the audit to this > thread, and he’s be in a better position to responsibly disclose problems. > > > > We hope to get LilyPond back on the Wikis, and that vulnerabilities get > fixed well
Re: plan for branching
Am Freitag, den 16.10.2020, 08:17 +0200 schrieb Michael Käppler:
> Am 16.10.2020 um 01:41 schrieb Dan Eble:
> > On Oct 15, 2020, at 09:26, Jonas Hahnfeld wrote:
> > > Am Dienstag, den 13.10.2020, 15:11 +0200 schrieb Jonas Hahnfeld:
> > > > After creating the branch, I will (unless somebody doesn't want me to
> > > > do this and volunteers to do the work; see also below):
> > > > - bump master to 2.23.0, in preparation for the next cycle;
> > > > - bump stable/2.22 to 2.21.80, in preparation for the first release
> > > > candidate;
> > > > - merge and / or forward the translation branch to stable/2.22.
> > > I'd like to do this tomorrow or Saturday at the latest, unless I hear
> > > objections before that.
> > >
> > > Jonas
> > Are there any concerns about merging
> > https://gitlab.com/lilypond/lilypond/-/merge_requests/458
> > ("Sequential_iterator garbage collection") ahead of schedule? That's the
> > last thing I've got for the stable release.
> I do not have any concerns. But if I understood Jonas correctly, there
> will be
> some release candidates before 2.22, anyway and Jonas would cherry-pick
> fixes like yours into stable/2.22.
> So IMHO no urgent need for fast-tracking.
Yes, that MR was already at the top of my list for backporting.
Jonas
signature.asc
Description: This is a digitally signed message part
Re: plan for branching
Am 16.10.2020 um 01:41 schrieb Dan Eble:
On Oct 15, 2020, at 09:26, Jonas Hahnfeld wrote:
Am Dienstag, den 13.10.2020, 15:11 +0200 schrieb Jonas Hahnfeld:
After creating the branch, I will (unless somebody doesn't want me to
do this and volunteers to do the work; see also below):
- bump master to 2.23.0, in preparation for the next cycle;
- bump stable/2.22 to 2.21.80, in preparation for the first release
candidate;
- merge and / or forward the translation branch to stable/2.22.
I'd like to do this tomorrow or Saturday at the latest, unless I hear
objections before that.
Jonas
Are there any concerns about merging
https://gitlab.com/lilypond/lilypond/-/merge_requests/458 ("Sequential_iterator
garbage collection") ahead of schedule? That's the last thing I've got for the
stable release.
I do not have any concerns. But if I understood Jonas correctly, there
will be
some release candidates before 2.22, anyway and Jonas would cherry-pick
fixes like yours into stable/2.22.
So IMHO no urgent need for fast-tracking.
Cheers,
Michael
