This seems like a pretty aggressive ask for AI at this stage,
particularly for AI vision. But I have no doubt it won't be long before
AI is able to do it perfectly. AI progress with various problematic
areas has been palpable. I've noticed that in the past year, AI has gone
from not really knowing LilyPond to writing entire scheme functions that
compile error-free and function as requested. (It's the same for another
esoteric language I use, OpenSCAD, where it used generate wrong code
that looked like C, but now it writes almost correct code every time.)
Below is one example (Gemini 2.5 Pro) where I requested a function that
adds staccato to every note. I could have written it myself in some
hours, but instead I had it in minutes. It's also able to make small
modifications without messing up what's there, such as when I requested
it handle staccatissimo, or to allow putting the dots up or down, etc.
Lately, it's been remembering past conversations and tailoring replies
according to what I'm working on as a whole. It can be really unnerving
when it's right on the money.
Regards,
Curt
%%% Begin AI-generated \addStaccato
#(define (has-articulation? music-list)
"Return #t if any object in music-list already has staccato articulation."
(any (lambda (obj)
(let ((type (ly:music-property obj 'articulation-type)))
(or (eq? type 'staccato)
(eq? type 'staccatissimo))))
music-list))
#(define (add-staccato-recurse m forced-dir)
(let ((name (ly:music-property m 'name)))
(cond
; Case 1: EventChord
((eq? name 'EventChord)
(let ((arts (ly:music-property m 'articulations))
(elts (ly:music-property m 'elements)))
(if (not (or (has-articulation? arts) (has-articulation? elts)))
(let ((new-art (make-music 'ArticulationEvent 'articulation-type
'staccato)))
; Apply direction if specific (not 0)
(if (not (= forced-dir 0))
(set! (ly:music-property new-art 'direction) forced-dir))
(set! (ly:music-property m 'articulations)
(cons new-art arts)))))
m)
; Case 2: NoteEvent
((eq? name 'NoteEvent)
(let ((arts (ly:music-property m 'articulations)))
(if (not (has-articulation? arts))
(let ((new-art (make-music 'ArticulationEvent 'articulation-type
'staccato)))
; Apply direction if specific (not 0)
(if (not (= forced-dir 0))
(set! (ly:music-property new-art 'direction) forced-dir))
(set! (ly:music-property m 'articulations)
(cons new-art arts)))))
m)
; Case 3: Recursion
(else
(let ((elt (ly:music-property m 'element))
(elts (ly:music-property m 'elements)))
(if (ly:music? elt)
(ly:music-set-property! m 'element (add-staccato-recurse elt
forced-dir)))
(if (pair? elts)
(ly:music-set-property! m 'elements
(map (lambda (x) (add-staccato-recurse x forced-dir)) elts)))
m)))))
% Create the user-facing music function with optional direction argument.
addStaccato = #(define-music-function (direction music) ((ly:dir? 0)ly:music?)
"Applies staccato. Usage: \\addStaccato \\music OR \\addStaccato #UP \\music"
(add-staccato-recurse music direction))
%%% End
On 1/6/26 09:52, Matthew Pierce wrote:
I asked Grok to self-design a prompt for accurately converting sheet
music images into Lilypond code. Early results are promising.
My test: I showed Grok a screenshotted page from a cello arrangement I
recently constructed in Lilypond, gave it the prompt below, compiled
the code it suggested, and compared the results
Results: The visual match was astonishingly good, even though Grok's
generated code is different from mine in various ways.
The only major difference was the final system being kicked to the
next page by Grok's code. I suspect this may have been caused by a
slight cropping at the bottom of the page in my screenshotted image.
This prompt might be a great shortcut for importing existing sheet
music into Lilypond code.
Try it on the LLM of your choice. Happy testing!
Prompt follows:
"You are an expert in LilyPond notation and optical music recognition.
Given the attached sheet music image, meticulously reverse-engineer
the LilyPond code that would reproduce it with the highest possible
visual fidelity. Prioritize absolute precision and accuracy in every
aspect of the engraving—including exact note placements, stem
directions, beam groupings, slur shapes and positions, dynamic
markings, articulations, text annotations, staff layout, spacing, and
any special elements like scordatura diagrams or irregular meters—over
any considerations of speed or efficiency. Proceed slowly and
methodically: first, perform a exhaustive layer-by-layer visual
analysis of the image, documenting every observable detail (e.g., clef
type, key signature sharps/flats, time signature symbol, pitch
positions relative to the staff, durations, ties, hairpins, bowings,
and markup coordinates). Cross-reference with music theory and
LilyPond syntax to resolve ambiguities. Only after this thorough
dissection should you generate the complete LilyPond code, including
header, global settings, voices, and layout overrides as needed to
match the image pixel-for-pixel where possible. If uncertainties
arise, note them and propose the most accurate interpretation based on
standard engraving practices."
