Re: [RFC] Moving to Guile 2.2 and away from GUB

[Han-Wen Nienhuys]

On Thu, Nov 25, 2021 at 7:49 PM Jonas Hahnfeld via Discussions on
LilyPond development  wrote:

> > Check out 
> > if you can trigger this with some frequency.
>
> Nope, still crashes in the same place.

Would be interesting to see if you can trigger it on a compile without
-O2. If not, that might point to a place where we forgot to remember a
value on the stack.

> > Also it may be worth
> > figuring out whether the crash occurs in connection with frenched
> > systems (which are explicitly removed before the end of typesetting).
>
> How do I check this? The System that I'm marking still looks fine, as
> far as I can tell, but the problem with GC issues is that it can also
> be any other Grob running havoc (even though it's suspicious that I
> only see it crashing in System::derived_mark...)

IIRC, the whole business with grob-array skipping marking was an
attempt at cutting a corner for performance and/or limiting stack
depth when doing GC marking. It may very well not be relevant today,
especially for BGC, which has an explicitly allocated mark stack.

Does it help if you mark X/Y parents in Grob::mark_smob?

-- 
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen

Re: ANN: Pygments support for LilyPond

[Jean Abou Samra]

Le 26/11/2021 à 07:09, Werner LEMBERG a écrit :

According to Gavin, Texinfo has no way to print text monospace text
in bold.

Here's the thing: The CM family doesn't contain a font for monospace
bold.  Since `texinfo.tex` essentially only uses this font family,
there is no support for it.



Ah! OK, of course. Given your help offer below,
I suppose you have another font in mind. What
would that be?




[...] Doing

\input{texinfo}

@tex
\textbf{ABC}DEF
@end tex

XXX

@bye

I get all of "ABCDEF" bold. I don't understand why.

This I can explain.  Contrary to LaTeX, `\textbf` doesn't take an
argument in texinfo.  It simply switches to a bold font, and you have
to limit its effect with a group:

   {\textbf ABC}DEF


Do you happen to have solutions for the whole TeX thing?  Otherwise
I think I'll give up on that and only implement HTML for now.  I
admire your TeX wizardry, but as to me personally, the *TeX suite
just makes me tear my hair out...

:-)

To be serious: Is the use of bold typewriter mandatory?  Is there a
solution that only uses italics and colours?  Do we also need bold
italic typewriter?



Well, the style uses bold for keywords like \score.
It's taken long to stabilize (not that I consider it
good, my judgement is simply not brilliant with colors)
so I'm not very keen on changing it again.




If we really need that, I can set up a solution – please give me one
day.  For XeTeX (and luatex), it's almost trivial.  For pdfTeX,
however, it needs more efforts.  I guess it makes sense for testing
purposes to have support for both pdfTeX and XeTeX.



Would be great. Thanks for your help.

Jean



 Werner


PS: Since my patch to `@setcolor` is probably not going upstream, I
 will also set up a solution without modifying `texinfo.tex`.

Re: ANN: Pygments support for LilyPond

[Werner LEMBERG]

> According to Gavin, Texinfo has no way to print text monospace text
> in bold.

Here's the thing: The CM family doesn't contain a font for monospace
bold.  Since `texinfo.tex` essentially only uses this font family,
there is no support for it.

> [...] Doing
> 
> \input{texinfo}
> 
> @tex
> \textbf{ABC}DEF
> @end tex
> 
> XXX
> 
> @bye
> 
> I get all of "ABCDEF" bold. I don't understand why.

This I can explain.  Contrary to LaTeX, `\textbf` doesn't take an
argument in texinfo.  It simply switches to a bold font, and you have
to limit its effect with a group:

  {\textbf ABC}DEF

> Do you happen to have solutions for the whole TeX thing?  Otherwise
> I think I'll give up on that and only implement HTML for now.  I
> admire your TeX wizardry, but as to me personally, the *TeX suite
> just makes me tear my hair out...

:-)

To be serious: Is the use of bold typewriter mandatory?  Is there a
solution that only uses italics and colours?  Do we also need bold
italic typewriter?

If we really need that, I can set up a solution – please give me one
day.  For XeTeX (and luatex), it's almost trivial.  For pdfTeX,
however, it needs more efforts.  I guess it makes sense for testing
purposes to have support for both pdfTeX and XeTeX.


Werner


PS: Since my patch to `@setcolor` is probably not going upstream, I
will also set up a solution without modifying `texinfo.tex`.

Re: ANN: Pygments support for LilyPond

[Jean Abou Samra]

Le 24/11/2021 à 14:52, Jean Abou Samra a écrit :

Le 24/11/2021 à 14:36, Werner LEMBERG a écrit :

This doesn't print any color, with or without your patch:  [...]

Interesting, for me it works, see attached file.

D'oh, yesterday I used the wrong incantation, sorry.  I can reproduce
the problem; attached is a new patch that also takes care of xetex.


Thanks! This works great. It might be time to submit it upstream.



OK, what do I do here? According to Gavin, Texinfo
has no way to print text monospace text in bold.
Do you know if there is a way around in TeX? Doing

\input{texinfo}

@tex
\textbf{ABC}DEF
@end tex

XXX

@bye

I get all of "ABCDEF" bold. I don't understand why.

Do you happen to have solutions for the whole
TeX thing? Otherwise I think I'll give up on
that and only implement HTML for now. I admire
your TeX wizardry, but as to me personally,
the *TeX suite just makes me tear my hair out...

Jean

Re: [RFC] Moving to Guile 2.2 and away from GUB

[Han-Wen Nienhuys]

On Thu, Nov 25, 2021 at 10:08 PM Jonas Hahnfeld via Discussions on
LilyPond development  wrote:

> Fun fact: Did you try placing the compiled file (with only the
> extension .go) into out/lib/lilypond/current/ccache/lily and make
> lilypond load it? It results in "Unbound variable: all-grob-
> descriptions", likely because the compiler doesn't understand define-
> session-public which is defined in lily. Actually, this points to yet
> another problem where some files (to become modules) use the module
> (lily) which in turn will have to load all other modules. So we likely
> have to re-think the entire layering of the scm/ directory...

semi related - in order to fix the --safe mode, we'll probably have to
reorganize how things depend on each other anyway (with potential
incompatibilities).

-- 
Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen

Re: [RFC] Moving to Guile 2.2 and away from GUB

[Jonas Hahnfeld via Discussions on LilyPond development]

Am Donnerstag, dem 25.11.2021 um 20:51 +0100 schrieb Jean Abou Samra:
> Le 25/11/2021 à 20:06, Jonas Hahnfeld a écrit :
> > Am Donnerstag, dem 25.11.2021 um 19:51 +0100 schrieb Jean Abou Samra:
> > > Le 24/11/2021 à 22:41, Jonas Hahnfeld via Discussions on LilyPond
> > > development a écrit :
> > > > This improves with compiled bytecode, as Jean replied, but this won't
> > > > be included initially (I
> > > > forgot to mention this). We should definitely have it for the next
> > > > stable release, but I'm optimistic that this is possible.
> > > 
> > > In my opinion, while not a requirement for switching
> > > to Guile 2, it would be really nicer (and wise to implement
> > > before removing Guile 1 support entirely)
> > That's exactly the thing that is *not* going to happen in that order,
> > at least not by me: Making the files byte-compilable will require many
> > changes (making all files modules on their own, as you write, plus
> > making sure that the different modules correctly depend on each other,
> > plus moving all calls to functions defined in C out of the parts that
> > are interpreted by the compiler, because they're not there unless we
> > run the full environment)
> 
> Could you explain the last part? I don't understand what you mean.

I can't reproduce right now, but I remember that when I tried I quickly
ran into errors that the compiler could not find functions that are
defined in C. My hypothesis is that the compiler tries to evaluate
"constants" to store only the result in the bytecode...

> 
> > which I think will require constructs that
> > are simply not available in Guile 1.8.
> 
> Not sure. Only actually trying will tell though.
> 
> > [...] On top of all that, I'm not actually
> > sure it's a good idea to byte-compile during development because it's
> > really, really slow - I currently estimate that compiling all *.scm
> > files can easily take as long as compiling all C++ files.
> 
> 
> Well, for me, it's proving quite fast:
> 
> $ rm -rf out/share/lilypond/current/guile/ccache/
> $ time out/bin/lilypond
> [Zillions of warnings...]
> real    0m58,060s
> user    0m57,757s
> sys    0m0,125s
> 
> (Compiling all C++ files without parallelism
> takes about 6 minutes.)
> 
> I also have good hope with optimization levels
> (only available with guild).
> 
> $ time guild-2.2 compile scm/define-grobs.scm
> wrote 
> `/home/jean/.cache/guile/ccache/2.2-LE-8-3.A/home/jean/repos/lilypond/scm/define-grobs.scm.go'
>  
> 
> 
> real    0m3,167s
> user    0m3,617s
> sys    0m0,040s
> 
> $ time guild-2.2 compile -O0 scm/define-grobs.scm
> wrote 
> `/home/jean/.cache/guile/ccache/2.2-LE-8-3.A/home/jean/repos/lilypond/scm/define-grobs.scm.go'
>  
> 
> 
> real    0m1,392s
> user    0m1,678s
> sys    0m0,025s

Fun fact: Did you try placing the compiled file (with only the
extension .go) into out/lib/lilypond/current/ccache/lily and make
lilypond load it? It results in "Unbound variable: all-grob-
descriptions", likely because the compiler doesn't understand define-
session-public which is defined in lily. Actually, this points to yet
another problem where some files (to become modules) use the module
(lily) which in turn will have to load all other modules. So we likely
have to re-think the entire layering of the scm/ directory...

Jonas


signature.asc
Description: This is a digitally signed message part

Re: spacing/breaking with header considered separate from music

[Werner LEMBERG]

> The page breaker is the single one area I've always been afraid of
> looking at :-)  It's all C++, the algorithms are not entirely
> trivial, and it is very performance-sensitive.

What about slurs?  This is really black art IMHO :-)


Werner

Re: \tweak Score.MetronomeMark.X-offset

[Werner LEMBERG]

>> If I want to offset a `\tempo` mark horizontally, only
>>
>>\override Score.MetronomeMark.X-offset = #3
>>  \tempo ...
>>
>> works.  Writing
>>
>>\tweak MetronomeMark.X-offset #3 \tempo ...
>> has no effect.
> 
> For reference, this is
> https://gitlab.com/lilypond/lilypond/-/issues/6142.

D'oh.  It's my own report and I can't remember this six months later.
I'm getting old.  Thanks for the link.

>> I now wonder whether it would be possible to extend
>> the `\tweak` command to accept also the context:
>>
>>\tweak Score.MetronomeMark.X-offset #3 \tempo ...
> 
> It doesn't make sense in the logic of \tweak.  [...]

Thanks for the explanation!


Werner

Re: spacing/breaking with header considered separate from music

[David Kastrup]

Jean Abou Samra  writes:

> The page breaker is the single one area I've
> always been afraid of looking at :-)  It's all
> C++, the algorithms are not entirely trivial,
> and it is very performance-sensitive.

It would go a long way if the page _builder_ was moved from C++ to
Scheme and got under programmer control, making things like footnotes
and similar stuff in the page layout programmable rather than hardwired.

Similar to how TeX has an output routine assembling page material under
control of its insertion mechanism (insertion boxes take up some
flexible distance on a page when they are non-empty as well as a given
fraction of their content size to make it possible to account for, say,
two-column footnotes).

The page _breaker_ can then still continue being lowlevel.

-- 
David Kastrup

Re: spacing/breaking with header considered separate from music

[Jean Abou Samra]

Hi Kieren,

Le 25/11/2021 à 16:02, Kieren MacMillan a écrit :

Hi all,

I've always wanted Lilypond to have the following (related) abilities:
   1. Consider header and music separately in the spacing/breaking 
calculations; and



Could you say a bit more? In what respect would
they differ? What kind of settings do you want
to achieve?



   2. Be able to force systems-on-first-page separately from systems-per-page 
[et al.].


Sounds like a reasonable feature request. Alternatively:
a setting to define how many systems the global title
counts for.



Obviously the equivalent behaviour can be accomplished "manually", as I've done 
in my scores through a tedious combination of bookparts, edition-engraver injections, 
etc. I'm just inquiring about whether an automagic solution is possible.

In very high-level terms (e.g. "it would touch these general areas of the code; 
difficulty = x/100"), what would be the scope of the work involved to make this kind 
of control possible?


The page breaker is the single one area I've
always been afraid of looking at :-)  It's all
C++, the algorithms are not entirely trivial,
and it is very performance-sensitive.

Regards,
Jean

Re: [RFC] Moving to Guile 2.2 and away from GUB

[Jean Abou Samra]

Le 25/11/2021 à 20:06, Jonas Hahnfeld a écrit :

Am Donnerstag, dem 25.11.2021 um 19:51 +0100 schrieb Jean Abou Samra:

Le 24/11/2021 à 22:41, Jonas Hahnfeld via Discussions on LilyPond
development a écrit :

This improves with compiled bytecode, as Jean replied, but this won't
be included initially (I
forgot to mention this). We should definitely have it for the next
stable release, but I'm optimistic that this is possible.


In my opinion, while not a requirement for switching
to Guile 2, it would be really nicer (and wise to implement
before removing Guile 1 support entirely)

That's exactly the thing that is *not* going to happen in that order,
at least not by me: Making the files byte-compilable will require many
changes (making all files modules on their own, as you write, plus
making sure that the different modules correctly depend on each other,
plus moving all calls to functions defined in C out of the parts that
are interpreted by the compiler, because they're not there unless we
run the full environment)



Could you explain the last part? I don't
understand what you mean.



which I think will require constructs that
are simply not available in Guile 1.8.



Not sure. Only actually trying will tell though.



[...] On top of all that, I'm not actually
sure it's a good idea to byte-compile during development because it's
really, really slow - I currently estimate that compiling all *.scm
files can easily take as long as compiling all C++ files.




Well, for me, it's proving quite fast:

$ rm -rf out/share/lilypond/current/guile/ccache/
$ time out/bin/lilypond
[Zillions of warnings...]
real    0m58,060s
user    0m57,757s
sys    0m0,125s

(Compiling all C++ files without parallelism
takes about 6 minutes.)

I also have good hope with optimization levels
(only available with guild).

$ time guild-2.2 compile scm/define-grobs.scm
wrote 
`/home/jean/.cache/guile/ccache/2.2-LE-8-3.A/home/jean/repos/lilypond/scm/define-grobs.scm.go' 



real    0m3,167s
user    0m3,617s
sys    0m0,040s

$ time guild-2.2 compile -O0 scm/define-grobs.scm
wrote 
`/home/jean/.cache/guile/ccache/2.2-LE-8-3.A/home/jean/repos/lilypond/scm/define-grobs.scm.go' 



real    0m1,392s
user    0m1,678s
sys    0m0,025s


Best,
Jean

Re: \tweak Score.MetronomeMark.X-offset

[Jean Abou Samra]

Le 25/11/2021 à 20:15, Werner LEMBERG a écrit :

If I want to offset a `\tempo` mark horizontally, only

   \override Score.MetronomeMark.X-offset = #3
 \tempo ...

works.  Writing

   \tweak MetronomeMark.X-offset #3 \tempo ...
has no effect.


For reference, this is
https://gitlab.com/lilypond/lilypond/-/issues/6142.


I now wonder whether it would be possible to extend
the `\tweak` command to accept also the context:

   \tweak Score.MetronomeMark.X-offset #3 \tempo ...


It doesn't make sense in the logic of \tweak.
The problem is how to let the user select a
grob on the page from the text input. \override
and \tweak take two different approaches:
while \override applies to all grobs created
at the same moment in a certain scope given
by a context, \tweak looks at the cause of
a grob in the music stream. The cause does
not have anything to do with a context. The
problem with \tweak on \tempo specifications
containing "duration = bpm" is that they are
represented as SequentialMusic containing a
TempoChangeEvent and a \set for tempoWholesPerMinutes
(try \displayMusic \tempo 4 = 80). \tweak should
affect the TempoChangeEvent in this case. I
have not investigated whether it would be saner
to change \tweak's logic or to refactor
\tempo to emit one single event, where
tempoWholePerMinutes would be set by an engraver
listening to this event.



In case this isn't possible it would be nice to document it in the NR
sections about `\tweak` and `\override`– only stems, automatic beams,
accidentals, clefs, and time signatures are mentioned as not working
with `\tweak`.


You can \tweak accidentals, stems and beams with
the \tweak Accidental.blabla syntax (read the next
paragraph in the documentation :-). The problem with
\tweak on clefs or time signatures is exactly the same
as the one with \tempo.

Best,
Jean

\tweak Score.MetronomeMark.X-offset

[Werner LEMBERG]

If I want to offset a `\tempo` mark horizontally, only

  \override Score.MetronomeMark.X-offset = #3
\tempo ...

works.  Writing

  \tweak MetronomeMark.X-offset #3 \tempo ...

has no effect.  I now wonder whether it would be possible to extend
the `\tweak` command to accept also the context:

  \tweak Score.MetronomeMark.X-offset #3 \tempo ...

In case this isn't possible it would be nice to document it in the NR
sections about `\tweak` and `\override`– only stems, automatic beams,
accidentals, clefs, and time signatures are mentioned as not working
with `\tweak`.


Werner

Re: [RFC] Moving to Guile 2.2 and away from GUB

[Jonas Hahnfeld via Discussions on LilyPond development]

Am Donnerstag, dem 25.11.2021 um 19:51 +0100 schrieb Jean Abou Samra:
> Le 24/11/2021 à 22:41, Jonas Hahnfeld via Discussions on LilyPond 
> development a écrit :
> > This improves with compiled bytecode, as Jean replied, but this won't 
> > be included initially (I
> > forgot to mention this). We should definitely have it for the next
> > stable release, but I'm optimistic that this is possible.
> 
> 
> In my opinion, while not a requirement for switching
> to Guile 2, it would be really nicer (and wise to implement
> before removing Guile 1 support entirely)

That's exactly the thing that is *not* going to happen in that order,
at least not by me: Making the files byte-compilable will require many
changes (making all files modules on their own, as you write, plus
making sure that the different modules correctly depend on each other,
plus moving all calls to functions defined in C out of the parts that
are interpreted by the compiler, because they're not there unless we
run the full environment) which I think will require constructs that
are simply not available in Guile 1.8.

> to byte-compile scm/*.scm in the build system with 'guild compile'.

... plus 'guild compile' wasn't a thing in Guile 1.8, so we'd have to
do different things depending on which version of Guile we are
compiling for (probably doable). On top of all that, I'm not actually
sure it's a good idea to byte-compile during development because it's
really, really slow - I currently estimate that compiling all *.scm
files can easily take as long as compiling all C++ files.

> It currently fails because we load the files by just
> reading them without making them modules and the
> compiler doesn't know about our macros at compilation
> time. I haven't followed everything related to Guile 2,
> is there prior discussion of this?
> 
> Best,
> Jean



signature.asc
Description: This is a digitally signed message part

Re: [RFC] Moving to Guile 2.2 and away from GUB

[David Kastrup]

Jonas Hahnfeld  writes:

> Am Donnerstag, dem 25.11.2021 um 16:18 +0100 schrieb David Kastrup:

>> > System::init_elements has
>> > 
>> >   set_object (this, "all-elements", scm_arr);
>> > 
>> > So *all_elements_ is protected as part of being a Spanner.  As long as
>> > the spanner has not committed suicide (when it drops its object list).
>> > If it has committed suicide but not been collected yet, *all_elements_
>> > may be collected before the System itself.
>
> Ah ok, otherwise I would have argued that all_elements_ not being
> initialized could only happen during construction of Systems, which is
> not what I'm seeing in the backtrace of the crash...
>
>> > This is garbage.
>> 
>> Check out 
>> if you can trigger this with some frequency.
>
> Nope, still crashes in the same place.

Huh.  That would have been a rather obvious candidate.  Still think this
change is definitely warranted.

>> Also it may be worth figuring out whether the crash occurs in
>> connection with frenched systems (which are explicitly removed before
>> the end of typesetting).
>
> How do I check this? The System that I'm marking still looks fine, as
> far as I can tell, but the problem with GC issues is that it can also
> be any other Grob running havoc (even though it's suspicious that I
> only see it crashing in System::derived_mark...)

Could be something that happens in the course of line breaking plus
frenching?  The original system may get frenched and the broken parts
miss out on an all_elements_scm_ of their own as their savior?
Something like that?

Just stabbing around in the dark.  The whole Grob_array thing is an
abomination.

-- 
David Kastrup

Re: [RFC] Moving to Guile 2.2 and away from GUB

[Jean Abou Samra]

Le 24/11/2021 à 22:41, Jonas Hahnfeld via Discussions on LilyPond 
development a écrit :
This improves with compiled bytecode, as Jean replied, but this won't 
be included initially (I

forgot to mention this). We should definitely have it for the next
stable release, but I'm optimistic that this is possible.



In my opinion, while not a requirement for switching
to Guile 2, it would be really nicer (and wise to implement
before removing Guile 1 support entirely) to byte-compile
scm/*.scm in the build system with 'guild compile'. It
currently fails because we load the files by just
reading them without making them modules and the
compiler doesn't know about our macros at compilation
time. I haven't followed everything related to Guile 2,
is there prior discussion of this?

Best,
Jean

Re: [RFC] Moving to Guile 2.2 and away from GUB

[Jonas Hahnfeld via Discussions on LilyPond development]

Am Donnerstag, dem 25.11.2021 um 16:18 +0100 schrieb David Kastrup:
> David Kastrup  writes:
> > David Kastrup  writes:
> > 
> > > David Kastrup  writes:
> > > > > It's relatively reproducible, I get it every 2nd-3rd time for full
> > > > > 'make doc's. Looking at the core dump, it's crashing in
> > > > > System::derived_mark when marking all_elements_ because the 
> > > > > std::vector
> > > > > has garbage in its size field (it was probably freed). What I don't
> > > > > understand is how that can happen since the only way of setting
> > > > > all_elements_ is in System::init_elements where the originating SCM is
> > > > > also protected...
> > > > 
> > > > My guess is that std::vector is not freed but rather not initialised.
> > > > That kind of thing is what the Preinit class is supposed to prevent, but
> > > > System does not use it.
> > > > 
> > > > Note also that all_elements_ is initialised to nullptr only _after_
> > > > garbage collection has already been activated: the in-class
> > > > initialisation syntax makes it easier to gloss over initialisation order
> > > > (which is not changed).
> > > > 
> > > > I'll try GC-proofing System.  Only time will tell whether this makes a
> > > > difference and/or whether other problems remain.
> > > 
> > > Somewhat embarrassingly, it would not appear that anything protects
> > > *all_elements_ as such (only its elements are protected in
> > > System::derived_mark).  That would be kind of a WTF even with Guile-1.8
> > > .
> > 
> > Well...
> > 
> > System::init_elements has
> > 
> >   set_object (this, "all-elements", scm_arr);
> > 
> > So *all_elements_ is protected as part of being a Spanner.  As long as
> > the spanner has not committed suicide (when it drops its object list).
> > If it has committed suicide but not been collected yet, *all_elements_
> > may be collected before the System itself.

Ah ok, otherwise I would have argued that all_elements_ not being
initialized could only happen during construction of Systems, which is
not what I'm seeing in the backtrace of the crash...

> > This is garbage.
> 
> Check out 
> if you can trigger this with some frequency.

Nope, still crashes in the same place.

> Also it may be worth
> figuring out whether the crash occurs in connection with frenched
> systems (which are explicitly removed before the end of typesetting).

How do I check this? The System that I'm marking still looks fine, as
far as I can tell, but the problem with GC issues is that it can also
be any other Grob running havoc (even though it's suspicious that I
only see it crashing in System::derived_mark...)


signature.asc
Description: This is a digitally signed message part

Re: [RFC] Moving to Guile 2.2 and away from GUB

[David Kastrup]

David Kastrup  writes:

> David Kastrup  writes:
>
>> David Kastrup  writes:
>>
>>> Jonas Hahnfeld via Discussions on LilyPond development
>>>  writes:
>>>
 Am Donnerstag, dem 25.11.2021 um 08:40 +0100 schrieb Jonas Hahnfeld via
 Discussions on LilyPond development:
> Am Mittwoch, dem 24.11.2021 um 23:01 +0100 schrieb Han-Wen Nienhuys:
> > On Wed, Nov 24, 2021 at 8:13 PM Jonas Hahnfeld via Discussions on
> > LilyPond development  wrote:
> > > Assuming no major problems are found, we could then move completely to
> > > Guile 2.2 and do releases without GUB. Both steps still require a bit
> > > of work; I'm relatively sure there is still a GC related issue causing
> > > (very rare) crashes with Guile 2.2
> > 
> > can you say more about this?
> 
> Not really: I remember weird crashes with dumps that were not
> immediately obvious.

 It's relatively reproducible, I get it every 2nd-3rd time for full
 'make doc's. Looking at the core dump, it's crashing in
 System::derived_mark when marking all_elements_ because the std::vector
 has garbage in its size field (it was probably freed). What I don't
 understand is how that can happen since the only way of setting
 all_elements_ is in System::init_elements where the originating SCM is
 also protected...
>>>
>>> My guess is that std::vector is not freed but rather not initialised.
>>> That kind of thing is what the Preinit class is supposed to prevent, but
>>> System does not use it.
>>>
>>> Note also that all_elements_ is initialised to nullptr only _after_
>>> garbage collection has already been activated: the in-class
>>> initialisation syntax makes it easier to gloss over initialisation order
>>> (which is not changed).
>>>
>>> I'll try GC-proofing System.  Only time will tell whether this makes a
>>> difference and/or whether other problems remain.
>>
>> Somewhat embarrassingly, it would not appear that anything protects
>> *all_elements_ as such (only its elements are protected in
>> System::derived_mark).  That would be kind of a WTF even with Guile-1.8
>> .
>
> Well...
>
> System::init_elements has
>
>   set_object (this, "all-elements", scm_arr);
>
> So *all_elements_ is protected as part of being a Spanner.  As long as
> the spanner has not committed suicide (when it drops its object list).
> If it has committed suicide but not been collected yet, *all_elements_
> may be collected before the System itself.
>
> This is garbage.

Check out 
if you can trigger this with some frequency.  Also it may be worth
figuring out whether the crash occurs in connection with frenched
systems (which are explicitly removed before the end of typesetting).

-- 
David Kastrup

spacing/breaking with header considered separate from music

[Kieren MacMillan]

Hi all,

I've always wanted Lilypond to have the following (related) abilities:
  1. Consider header and music separately in the spacing/breaking calculations; 
and
  2. Be able to force systems-on-first-page separately from systems-per-page 
[et al.].

Obviously the equivalent behaviour can be accomplished "manually", as I've done 
in my scores through a tedious combination of bookparts, edition-engraver 
injections, etc. I'm just inquiring about whether an automagic solution is 
possible.

In very high-level terms (e.g. "it would touch these general areas of the code; 
difficulty = x/100"), what would be the scope of the work involved to make this 
kind of control possible?

Thanks,
Kieren.

Re: [RFC] Moving to Guile 2.2 and away from GUB

[David Kastrup]

David Kastrup  writes:

> David Kastrup  writes:
>
>> Jonas Hahnfeld via Discussions on LilyPond development
>>  writes:
>>
>>> Am Donnerstag, dem 25.11.2021 um 08:40 +0100 schrieb Jonas Hahnfeld via
>>> Discussions on LilyPond development:
 Am Mittwoch, dem 24.11.2021 um 23:01 +0100 schrieb Han-Wen Nienhuys:
 > On Wed, Nov 24, 2021 at 8:13 PM Jonas Hahnfeld via Discussions on
 > LilyPond development  wrote:
 > > Assuming no major problems are found, we could then move completely to
 > > Guile 2.2 and do releases without GUB. Both steps still require a bit
 > > of work; I'm relatively sure there is still a GC related issue causing
 > > (very rare) crashes with Guile 2.2
 > 
 > can you say more about this?
 
 Not really: I remember weird crashes with dumps that were not
 immediately obvious.
>>>
>>> It's relatively reproducible, I get it every 2nd-3rd time for full
>>> 'make doc's. Looking at the core dump, it's crashing in
>>> System::derived_mark when marking all_elements_ because the std::vector
>>> has garbage in its size field (it was probably freed). What I don't
>>> understand is how that can happen since the only way of setting
>>> all_elements_ is in System::init_elements where the originating SCM is
>>> also protected...
>>
>> My guess is that std::vector is not freed but rather not initialised.
>> That kind of thing is what the Preinit class is supposed to prevent, but
>> System does not use it.
>>
>> Note also that all_elements_ is initialised to nullptr only _after_
>> garbage collection has already been activated: the in-class
>> initialisation syntax makes it easier to gloss over initialisation order
>> (which is not changed).
>>
>> I'll try GC-proofing System.  Only time will tell whether this makes a
>> difference and/or whether other problems remain.
>
> Somewhat embarrassingly, it would not appear that anything protects
> *all_elements_ as such (only its elements are protected in
> System::derived_mark).  That would be kind of a WTF even with Guile-1.8
> .

Well...

System::init_elements has

  set_object (this, "all-elements", scm_arr);

So *all_elements_ is protected as part of being a Spanner.  As long as
the spanner has not committed suicide (when it drops its object list).
If it has committed suicide but not been collected yet, *all_elements_
may be collected before the System itself.

This is garbage.

-- 
David Kastrup

Re: [RFC] Moving to Guile 2.2 and away from GUB

[David Kastrup]

David Kastrup  writes:

> Jonas Hahnfeld via Discussions on LilyPond development
>  writes:
>
>> Am Donnerstag, dem 25.11.2021 um 08:40 +0100 schrieb Jonas Hahnfeld via
>> Discussions on LilyPond development:
>>> Am Mittwoch, dem 24.11.2021 um 23:01 +0100 schrieb Han-Wen Nienhuys:
>>> > On Wed, Nov 24, 2021 at 8:13 PM Jonas Hahnfeld via Discussions on
>>> > LilyPond development  wrote:
>>> > > Assuming no major problems are found, we could then move completely to
>>> > > Guile 2.2 and do releases without GUB. Both steps still require a bit
>>> > > of work; I'm relatively sure there is still a GC related issue causing
>>> > > (very rare) crashes with Guile 2.2
>>> > 
>>> > can you say more about this?
>>> 
>>> Not really: I remember weird crashes with dumps that were not
>>> immediately obvious.
>>
>> It's relatively reproducible, I get it every 2nd-3rd time for full
>> 'make doc's. Looking at the core dump, it's crashing in
>> System::derived_mark when marking all_elements_ because the std::vector
>> has garbage in its size field (it was probably freed). What I don't
>> understand is how that can happen since the only way of setting
>> all_elements_ is in System::init_elements where the originating SCM is
>> also protected...
>
> My guess is that std::vector is not freed but rather not initialised.
> That kind of thing is what the Preinit class is supposed to prevent, but
> System does not use it.
>
> Note also that all_elements_ is initialised to nullptr only _after_
> garbage collection has already been activated: the in-class
> initialisation syntax makes it easier to gloss over initialisation order
> (which is not changed).
>
> I'll try GC-proofing System.  Only time will tell whether this makes a
> difference and/or whether other problems remain.

Somewhat embarrassingly, it would not appear that anything protects
*all_elements_ as such (only its elements are protected in
System::derived_mark).  That would be kind of a WTF even with Guile-1.8
.

-- 
David Kastrup

Re: [RFC] Moving to Guile 2.2 and away from GUB

[David Kastrup]

Jonas Hahnfeld via Discussions on LilyPond development
 writes:

> Am Donnerstag, dem 25.11.2021 um 08:40 +0100 schrieb Jonas Hahnfeld via
> Discussions on LilyPond development:
>> Am Mittwoch, dem 24.11.2021 um 23:01 +0100 schrieb Han-Wen Nienhuys:
>> > On Wed, Nov 24, 2021 at 8:13 PM Jonas Hahnfeld via Discussions on
>> > LilyPond development  wrote:
>> > > Assuming no major problems are found, we could then move completely to
>> > > Guile 2.2 and do releases without GUB. Both steps still require a bit
>> > > of work; I'm relatively sure there is still a GC related issue causing
>> > > (very rare) crashes with Guile 2.2
>> > 
>> > can you say more about this?
>> 
>> Not really: I remember weird crashes with dumps that were not
>> immediately obvious.
>
> It's relatively reproducible, I get it every 2nd-3rd time for full
> 'make doc's. Looking at the core dump, it's crashing in
> System::derived_mark when marking all_elements_ because the std::vector
> has garbage in its size field (it was probably freed). What I don't
> understand is how that can happen since the only way of setting
> all_elements_ is in System::init_elements where the originating SCM is
> also protected...

My guess is that std::vector is not freed but rather not initialised.
That kind of thing is what the Preinit class is supposed to prevent, but
System does not use it.

Note also that all_elements_ is initialised to nullptr only _after_
garbage collection has already been activated: the in-class
initialisation syntax makes it easier to gloss over initialisation order
(which is not changed).

I'll try GC-proofing System.  Only time will tell whether this makes a
difference and/or whether other problems remain.

-- 
David Kastrup

Re: [RFC] Moving to Guile 2.2 and away from GUB

[Jonas Hahnfeld via Discussions on LilyPond development]

Am Donnerstag, dem 25.11.2021 um 08:40 +0100 schrieb Jonas Hahnfeld via
Discussions on LilyPond development:
> Am Mittwoch, dem 24.11.2021 um 23:01 +0100 schrieb Han-Wen Nienhuys:
> > On Wed, Nov 24, 2021 at 8:13 PM Jonas Hahnfeld via Discussions on
> > LilyPond development  wrote:
> > > Assuming no major problems are found, we could then move completely to
> > > Guile 2.2 and do releases without GUB. Both steps still require a bit
> > > of work; I'm relatively sure there is still a GC related issue causing
> > > (very rare) crashes with Guile 2.2
> > 
> > can you say more about this?
> 
> Not really: I remember weird crashes with dumps that were not
> immediately obvious.

It's relatively reproducible, I get it every 2nd-3rd time for full
'make doc's. Looking at the core dump, it's crashing in
System::derived_mark when marking all_elements_ because the std::vector
has garbage in its size field (it was probably freed). What I don't
understand is how that can happen since the only way of setting
all_elements_ is in System::init_elements where the originating SCM is
also protected...

> There's also this in a recent CI run:
> https://gitlab.com/lilypond/lilypond/-/jobs/1775303890

And Jean just hit it (or another crash) again:
https://gitlab.com/lilypond/lilypond/-/jobs/1819543552


signature.asc
Description: This is a digitally signed message part

PATCHES - Countdown for November 25th

[James]

Hello,

Here is the current patch countdown list. The next countdown will be on 
November 27th.


A list of all merge requests can be found here:
https://gitlab.com/lilypond/lilypond/-/merge_requests?sort=label_priority


 Push:

!1015 Replace some unnecessary non-ASCII characters ASCII equivalents - 
Werner Lemberg

https://gitlab.com/lilypond/lilypond/-/merge_requests/1015

!1014 Use normal methods for getting staff symbol height in bar line 
code - Jean Abou Samra

https://gitlab.com/lilypond/lilypond/-/merge_requests/1014

!1013 Remove grob-elts::X-extent function - Jean Abou Samra
https://gitlab.com/lilypond/lilypond/-/merge_requests/1013

!1011 Limit float printing precision in warning for \markup \left-brace 
- Jean Abou Samra

https://gitlab.com/lilypond/lilypond/-/merge_requests/1011

!1008 scm/editor.scm: Avoid character ranges in regex - Jonas Hahnfeld
https://gitlab.com/lilypond/lilypond/-/merge_requests/1008

!1007 mf: More work on accidentals - Werner Lemberg
https://gitlab.com/lilypond/lilypond/-/merge_requests/1007


 Countdown:

!1020 mf: Avoid FontForge overlap removal bug - Werner Lemberg
https://gitlab.com/lilypond/lilypond/-/merge_requests/1020

!1016 Store NoteColumn main extent in a property - Jean Abou Samra
https://gitlab.com/lilypond/lilypond/-/merge_requests/1016


 Review:

!1023 Docfix: Learning Manual, sec 4.2.1, correct number of contexts - 
John Wheeler

https://gitlab.com/lilypond/lilypond/-/merge_requests/1023

!1022 \segnoMark command - Dan Eble
https://gitlab.com/lilypond/lilypond/-/merge_requests/1022

!1021 Repeat_styler: issue AlternativeEvents - Dan Eble
https://gitlab.com/lilypond/lilypond/-/merge_requests/1021

!1019 Draft: RFC: Add syntax highlighting to documentation - Jean Abou 
Samra

https://gitlab.com/lilypond/lilypond/-/merge_requests/1019

!1017 lilypond-book: Fix include handling by `kpsewhich` - Michael 
Käppler

https://gitlab.com/lilypond/lilypond/-/merge_requests/1017


 New:

No patches in New at this time.


 Waiting:

!913 release: binaries with cairo - Han-Wen Nienhuys
https://gitlab.com/lilypond/lilypond/-/merge_requests/913

--
Regards

James