Re: learning (names of) markup commands in scheme: documentation

[Jean Abou Samra]

[Berhard]
Actually, the lilypond.org web server was very insistent that even 
when I googled English, I should read German whenever it was 
available; therefore I assumed the versions were on par.  I will think 
about configuring one browser / profile without language preferences 
so that I can access the English version directly.


(For documentation with a specialised audience, I always wonder if 
such divergence between versions would be better avoided with 
linguistic monoculture, but I really appreciate the effort! [And I 
hope this does not sound deprecating – it's not meant to be!, it's 
just a difficult issue.]).




Divergence depends on the state of maintenance of the
respective translation. The Italian and French ones are
completely up-to-date. I started using LilyPond aged
13 or so and barely able to make sense of the English
words like \header in the LilyPond input, let alone to
read the documentation in English. I don't think our
audience, however specialized, is so much more able
to speak English than with other software. The translations
do bring enormous value.

What would be ideal would be translating the documentation
via Gettext-based tools so the maintenance would be
automatic and English text would be displayed in case
the translation is outdated on a certain paragraph.
I dream of that, but the thing is, tools to do that only
emerged after the main body of translation was done,
and there would be an enormous lot of material to
convert by now ...



Le 22/01/2022 à 10:47, Bernhard Fisseni a écrit :

Good morning, Jean,

thank you very much for your detailed explanation!  This helps me to 
understands things a bit better.


Jean Abou Samra schrieb am 21.01.22 um 19:30:

Le 21/01/2022 à 08:57, Bernhard Fisseni a écrit :






- There is a point in processing when one has to convert every "line"
(in my example) to markup, although the interpretation happens
automatically in other places (see interpret-markup-list in
apply-last-strut).



I'm not sure what you mean here. Can you clarify?


What I meant was that while in the Lilypond source, one can have a top 
\markup, which makes more or less all its argument a markup, when 
writing the markup commands, one occasionally has to apply 
(interpret-markup) or (interpret-markup-list) to inner arguments to 
return the right type, e.g. in the recursive function I wrote.  If I 
understand correctly, one is doing piecewise  evaluation "by hand".  
It feels rather natural if one understands the general system, I 
suppose, as there are these different explicit and implicit 
conversions of syntax into markup.  (If that is too cryptic, I am 
sorry; it is probably not such an important point.)



Well, the "right type" here is not a markup, but a
stencil, namely a device-independent representation of
graphical output. A markup is not a stencil, but merely
the promise of a stencil that will be parameterized by
properties. So when you do interpret-markup, this is not
converting some kind of syntax into a markup, but a markup
into a stencil. When you write \markup in your source file,
you enter markup mode. You can then compose your markup
using commands. The application of a command as
\command [first-arg] [second-arg] [etc.] yields a markup.
Many markup commands take one or several markup arguments,
so when you write \markup \bold \italic a, this ends
up as a markup made from the command \bold, with an argument
that is \markup \italic a (no need to repeat \markup within
the nested construction as you are already in markup mode).
At this point, the compound \markup gets interpreted. That's
the entry point. Then, it is up to the markup command
of that top-level markup to interpret the markup sub-arguments.
Usually it will interpret them (it doesn't have to), but
it is intentional to leave it to the command to do it and
not to do it automatically. Because the command may want to
interpret its markup subarguments with different parameters.
A typical example: \bold is defined as

(define-markup-command (bold layout props arg)
  (markup?)
  (interpret-markup layout (prepend-alist-chain 'font-series 'bold 
props) arg))


which interprets the markup argument with a modified
"props" alist chain (list of alists) that adds settings
so that any string inside "arg" gets interpreted in
a bold font.

Is it clearer to you now?



[David]

Yes, and not only make-CMD-markup, but more importantly
CMD-markup



CMD-markup-function 



Hm?

#(write #{ \markup \bold a #})
=> (# "a")



[Me]

Homework for me: figure out why this does not result
in constant spacing regardless of the heights of
the markups (that part of the code is rather complicated).



Cough. It does. The default baseline-skip just turns
out to be about what you want. If you add more lines,
it starts colliding. How could that have worked? Silly me.

That suggestion is moot: at best it helps you reduce
the minimum distance (baseline-skip) without padding
interfering. You still need manual adjustments.

I 

Re: learning (names of) markup commands in scheme: documentation

[Bernhard Fisseni]

Good morning, Jean,

thank you very much for your detailed explanation!  This helps me to 
understands things a bit better.


Jean Abou Samra schrieb am 21.01.22 um 19:30:

Le 21/01/2022 à 08:57, Bernhard Fisseni a écrit :






- There is a point in processing when one has to convert every "line"
(in my example) to markup, although the interpretation happens
automatically in other places (see interpret-markup-list in
apply-last-strut).



I'm not sure what you mean here. Can you clarify?


What I meant was that while in the Lilypond source, one can have a top 
\markup, which makes more or less all its argument a markup, when 
writing the markup commands, one occasionally has to apply 
(interpret-markup) or (interpret-markup-list) to inner arguments to 
return the right type, e.g. in the recursive function I wrote.  If I 
understand correctly, one is doing piecewise  evaluation "by hand".  It 
feels rather natural if one understands the general system, I suppose, 
as there are these different explicit and implicit conversions of syntax 
into markup.  (If that is too cryptic, I am sorry; it is probably not 
such an important point.)







I would agree that the explanations regarding markups vs. markup
lists, markup vs. stencils, markup in scheme etc. in the Documentation
might be improved.



For what it's worth, a step has been made
just a few weeks ago with

https://gitlab.com/lilypond/lilypond/-/merge_requests/1089

That merge request revised the Notation Reference
material about markups (though not the Extending
Manual material) so that the interplay between
markups and markup lists would be clearer. Part
of my motivation for doing this is that it took
myself ages even as a developer to discover the
way applying a markup command taking a markup as
last argument to a markup list as last arguments
"maps" it as in your examples.


Thank you,  very much, I will have a look at it!




Regarding the problem at hand, I believe it can
be solved by defining a slightly tweaked version
of \column that allows turning off the behavior
that avoids overlap between lines:


[...]

Thank you, that is also an interesting approach, and feels as if the 
problem is resolved is "at a better place" in the command.


Again, thank you for your patience,
best regards,
  Bernhard

Re: learning (names of) markup commands in scheme: documentation

[Bernhard Fisseni]

Hi Lukas,

Lukas-Fabian Moser schrieb am 21.01.22 um 17:15:



Also note

[...]

The standard use case for make-XXX-markup is explained in:
http://lilypond.org/doc/v2.23/Documentation/extending/markup-construction-in-scheme.html
under "Known issues and warnings".


I had read that, and it seemed easier to use the "full version", and 
more consistent, too.  But I understand that the data version using 
#:keywords is a little more easy to read once you are used to it



I would agree that the explanations regarding markups vs. markup lists,
markup vs. stencils, markup in scheme etc. in the Documentation might be
improved. We always welcome contributions!


Thank you – I'll see if I understand enough to help ;-).


(And, your probably noticed
this: The German documentation is in an effectively unmaintained state
at the moment. But it seems there's hope this might change soonish.


Actually, the lilypond.org web server was very insistent that even when 
I googled English, I should read German whenever it was available; 
therefore I assumed the versions were on par.  I will think about 
configuring one browser / profile without language preferences so that I 
can access the English version directly.


(For documentation with a specialised audience, I always wonder if such 
divergence between versions would be better avoided with linguistic 
monoculture, but I really appreciate the effort! [And I hope this does 
not sound deprecating – it's not meant to be!, it's just a difficult 
issue.]).


Thanks,
best regards,
  Bernhard

Re: learning (names of) markup commands in scheme: documentation

[David Kastrup]

Jean Abou Samra  writes:

> Le 21/01/2022 à 08:57, Bernhard Fisseni a écrit :
>> Good morning,
>>
>>   Consequence: There is no collision between an auxiliary function
>> CMD and a homonymous markup command \CMD, as they are (CMD ...) and
>> (make-CMD-markup ...), respectively, in scheme.
>
>
> Yes, and not only make-CMD-markup, but more importantly
> CMD-markup

CMD-markup-function

> which is the function into which your definition gets turned into, in
> charge of doing the markup interpretation.

-- 
David Kastrup

Re: learning (names of) markup commands in scheme: documentation

[Jean Abou Samra]

plans of similar scope, and [I] work on LilyPond as a
hobby in otherwise already LilyPond development
is for me a hobby in already otherwise busy weeks.


Talk about busy weeks...

Re: learning (names of) markup commands in scheme: documentation

[Jean Abou Samra]

Le 21/01/2022 à 08:57, Bernhard Fisseni a écrit :

Good morning,

trying to understand the programming a bit better, I managed to 
transform the mixed command definitions to scheme code (see below).


I've learnt the following in the process; should some of it be made 
more explicit in the manual?  (If so, I could try to think of 
suggestions.)


- In my opinion writing the scheme code is much easier than mixing 
Lilypond and Scheme.


- For every markup command and every markup list command \CMD, there 
is a corresponding scheme function make-CMD-markup.



 
uses this implicitly in the last paragraphs, but an explicit remark 
might be helpful.



 
explains the correspondence with one example, but does not mention 
markup list commands; furthermore, the heading and the wording mislead 
me so that I only understood the generality of this paragraph 
afterwards. Maybe "How Markups Work Internally: Lilypond Commands and 
Corresponding Scheme Commands" might be more explicit?


  Consequence: There is no collision between an auxiliary function CMD 
and a homonymous markup command \CMD, as they are (CMD ...) and 
(make-CMD-markup ...), respectively, in scheme.



Yes, and not only make-CMD-markup, but more importantly
CMD-markup which is the function into which your definition
gets turned into, in charge of doing the markup interpretation.
make-CMD-markup is then essentially defined as

(define (make-CMD-markup . args)
  (cons CMD-markup args))

A simplified version of interpret-markup would be:

(define (interpret-markup layout props mkup)
  (apply (car mkup)
 layout
 props
 (cdr mkup)))


  Question: Do similar correspondences exist for music functions and 
event functions? It looks as if they might me make-CMD-music and 
make-CMD-event, but I find no dokumentation for this. (May be my fault.)



No, there is no such thing. A music function is just
a music function, called by its name, which is the same
in the LilyPond namespace and in the Scheme namespace
-- these are actually the same. Try

#(display relative)

So music functions are first-class objects. Same with
other syntax functions (all those you create with
define-{music,event,scheme,void}-function). You can
even call them in Scheme:

$(relative #{ c' #} #{ c'1 d e #})

(about # vs $ see
https://lilypond.org/doc/v2.23/Documentation/extending/lilypond-scheme-syntax
and
https://extending-lilypond.readthedocs.io/en/latest/lily-and-scheme.html#hash-vs-dollar).

Markups are very special. They are not implemented
as a type of objets, but as mere lists whose car
is a markup command, namely a procedure (the CMD-markup
one) that bears a bunch of object properties
("attributes") like its signature. The \markup syntax
does not look up objects in the namespace normally,
but transforms the names to add "-markup" at the
end.

If you want my opinion, markup internals are not the
best-designed part of LilyPond. I have plans to
change that, but I also have uncountably many
plans of similar scope, and work on LilyPond as a
hobby in otherwise already LilyPond development
is for me a hobby in already otherwise busy weeks.



- A (list ...) of markups is indeed a markup list, and markup lists 
can be treated like just a list (map, length, car, cdr etc.).  
(Actually, I had expected that there were specialised conversions and 
accessor functions.)


  Given that lists are so fundamental, this might be stated somewhere.



Yes, but careful: not every markup list is a list
of markups, even though it is true that a list of
markups constitutes a markup list. The other way to
have a markup list is to apply a markup list command.
For example: \table-of-contents.

#(display #{ \markup \table-of-contents #})

Markup list commands are listed at
https://lilypond.org/doc/v2.23/Documentation/notation/text-markup-list-commands



- There is a point in processing when one has to convert every "line" 
(in my example) to markup, although the interpretation happens 
automatically in other places (see interpret-markup-list in 
apply-last-strut).



I'm not sure what you mean here. Can you clarify?





[snipped]




[Lukas]

Isn't this just

#(define-markup-command (strut-line layout props line)
  (markup?)
  "add strut to the end of a line to ensure correct line spacing"
  (interpret-markup layout props
   (markup #:combine line #:transparent "Ij")))

? 



Well, we have four ways to create markups in Scheme.

#{ \markup \bold "a" #}
(make-bold-markup "a")
(markup #:bold "a")
(list bold-markup "a")

The fourth one relies on markup internals, and I would
recommend against using it since, as said above, we
could want to move away from this representation at
some point.

Between the three former ones, the choice is
mostly a matter of taste. The markup macro has
the caveats mentioned in the 

Re: learning (names of) markup commands in scheme: documentation

[Lukas-Fabian Moser]

Also note that your function, if called with \strut-line { hello world 
}, actually adds your invisible strut to every word of the line. (Just 
remove the make-transparent-markup in your function to see this, or, 
more easuily, remove the #:transparent in my version). Compare:


\markup \strut-line { hello world }
\markup \strut-line \line { hello world }

This difference (I think) originates in the way LilyPond applies 
markup functions to explicit markups of the form { some word and some 
other }, or put differently: Where the implicit \line is put, and how 
a function expecting a single markup is applied to a list of markups.


But note what changes if you replace the markup? predicate in your 
function definition by markup-list?.


I think that was a bit cryptic :-).

Here's a (hopefully) clearer example:

\version "2.22"

#(define-markup-command (demo-eating-markup layout props stuff) (markup?)
   (format #t "Processing single markup: \"~a\"\n" (markup->string stuff))
   (interpret-markup layout props stuff))

#(define-markup-command (demo-eating-markup-list layout props stuff) 
(markup-list?)

   (format #t "Processing markup list: ~a\n" stuff)
   (interpret-markup layout props (make-column-markup stuff)))

\markup \demo-eating-markup { one two }
\markup \demo-eating-markup \line { eins zwei }
\markup \demo-eating-markup-list { oans zwoa }
\markup \demo-eating-markup-list { \line { un deux } }

The standard use case for make-XXX-markup is explained in: 
http://lilypond.org/doc/v2.23/Documentation/extending/markup-construction-in-scheme.html 
under "Known issues and warnings".


I would agree that the explanations regarding markups vs. markup lists, 
markup vs. stencils, markup in scheme etc. in the Documentation might be 
improved. We always welcome contributions! (And, your probably noticed 
this: The German documentation is in an effectively unmaintained state 
at the moment. But it seems there's hope this might change soonish.


Lukas

Re: learning (names of) markup commands in scheme: documentation

[Lukas-Fabian Moser]

Hi Bernhard,

unfortunately I don't have much time at the moment (and there's always 
the chance that Jean or someone else more knowledgeable than me is 
already working on an exhaustive answer), but there's one thing I'd like 
to point out:



#(define-markup-command (strut-line layout props line)
  (markup?)
  "add strut to the end of a line to ensure correct line spacing"
  (interpret-markup layout props
   (markup
    (make-combine-markup
 (make-line-markup (list line))
 (make-transparent-markup "Ij")


Isn't this just

#(define-markup-command (strut-line layout props line)
  (markup?)
  "add strut to the end of a line to ensure correct line spacing"
  (interpret-markup layout props
   (markup #:combine line #:transparent "Ij")))

?

Also note that your function, if called with \strut-line { hello world 
}, actually adds your invisible strut to every word of the line. (Just 
remove the make-transparent-markup in your function to see this, or, 
more easuily, remove the #:transparent in my version). Compare:


\markup \strut-line { hello world }
\markup \strut-line \line { hello world }

This difference (I think) originates in the way LilyPond applies markup 
functions to explicit markups of the form { some word and some other }, 
or put differently: Where the implicit \line is put, and how a function 
expecting a single markup is applied to a list of markups.


But note what changes if you replace the markup? predicate in your 
function definition by markup-list?.


Lukas

learning (names of) markup commands in scheme: documentation

[Bernhard Fisseni]

Good morning,

trying to understand the programming a bit better, I managed to 
transform the mixed command definitions to scheme code (see below).


I've learnt the following in the process; should some of it be made more 
explicit in the manual?  (If so, I could try to think of suggestions.)


- In my opinion writing the scheme code is much easier than mixing 
Lilypond and Scheme.


- For every markup command and every markup list command \CMD, there is 
a corresponding scheme function make-CMD-markup.



 
uses this implicitly in the last paragraphs, but an explicit remark 
might be helpful.



 
explains the correspondence with one example, but does not mention 
markup list commands; furthermore, the heading and the wording mislead 
me so that I only understood the generality of this paragraph 
afterwards. Maybe "How Markups Work Internally: Lilypond Commands and 
Corresponding Scheme Commands" might be more explicit?


  Consequence: There is no collision between an auxiliary function CMD 
and a homonymous markup command \CMD, as they are (CMD ...) and 
(make-CMD-markup ...), respectively, in scheme.


  Question: Do similar correspondences exist for music functions and 
event functions? It looks as if they might me make-CMD-music and 
make-CMD-event, but I find no dokumentation for this. (May be my fault.)


- A (list ...) of markups is indeed a markup list, and markup lists can 
be treated like just a list (map, length, car, cdr etc.).  (Actually, I 
had expected that there were specialised conversions and accessor 
functions.)


  Given that lists are so fundamental, this might be stated somewhere.

- There is a point in processing when one has to convert every "line" 
(in my example) to markup, although the interpretation happens 
automatically in other places (see interpret-markup-list in 
apply-last-strut).



Sorry if this looks like nitpicking, I am just trying to find my way around.

Best regards,
  Bernhard


For reference, this:



#(define-markup-command
(strut-line layout props line)
(markup?)
(interpret-markup layout props #{
  \markup{
\combine #line \transparent "Ij"
  }
#}))

#(define-markup-command
 (stanza-list layout props stanzas)
 (markup-list?)
   (interpret-markup layout props #{
   \markup
   \column {
 #stanzas
   }
  #}))

#(define-markup-list-command
(apply-very-last-strut layout props stanza-lines)
(markup-list?)
(letrec ((apply-last-strut
  (lambda (stanza-lines)
   (if (null? stanza-lines)
stanza-lines
(if (> (length stanza-lines) 1)
 (cons (interpret-markup layout props #{
\markup {#(car stanza-lines)}
  #})
 (apply-last-strut (cdr stanza-lines)))
 (interpret-markup-list layout props #{
   \markuplist {
 \strut-line #stanza-lines
   }
  #}))
  (apply-last-strut stanza-lines)))

#(define-markup-command
(my-stanza layout props number lines)
(markup? markup-list?)
#:properties ((extra-space 1))
(interpret-markup layout props #{
 \markup {
   \combine \null \vspace #extra-space \line {
 \bold #number
 \column {
   \apply-very-last-strut #lines
 }
   }
 }
#}))


became this:

#(define-markup-command (strut-line layout props line)
  (markup?)
  "add strut to the end of a line to ensure correct line spacing"
  (interpret-markup layout props
   (markup
(make-combine-markup
 (make-line-markup (list line))
 (make-transparent-markup "Ij")

#(define-markup-command (stanza-list layout props stanzas)
  (markup-list?)
  "make a column of stanzas (just for explicitness' sake)"
  (interpret-markup layout props
   (markup
(make-column-markup stanzas

#(define-markup-list-command
  (apply-last-strut layout props stanza-lines)
  (markup-list?)
  "append a strut to the last of the stanza-lines to ensure proper spacing"
  (letrec ((apply-last-strut
(lambda (stanza-lines)
 (cond
  ((null? stanza-lines) stanza-lines)
  ((> (length stanza-lines) 1)
   (cons (interpret-markup layout props
  (car stanza-lines))
(apply-last-strut (cdr stanza-lines
  (else
(interpret-markup-list
  layout props
  (map make-strut-line-markup stanza-lines)))
(apply-last-strut stanza-lines)))

#(define-markup-command
   (numbered-stanza layout props number lines)
   (markup? markup-list?)
   #:properties ((extra-space 1))
   "make a numbered stanza with the number to the left of the lyrics"
   (interpret-markup
 layout props
 (markup