Hello everybody,
triggered by some comments on my last post
http://lilypondblog.org/2015/01/introducing-scholarly/ I decided to pick
up a few loose ends that have been lying around for some time now, so I
finally started restructuring openLilyLib, aiming at a quite fundamental
change.
So this message is directed (in descending order of relevance) to
* Those who have already taken part in openLilyLib reorganization
discussions some half a year ago
* Everybody who has contributed to openLilyLib so afr
* All openLilyLib users
* All other LilyPond users/developers
Some motivations to do that now:
* (Valid) Questions were raised if it is a good idea to have many
different "libraries" scattered around.
* Every now and then the idea arose to create some sort of consistent
interface/infrastructure/specifications to create/distribute/use
libraries to extend LilyPond's functionality without having to stuff
everything into Lily herself.
* openLilyLib has become a somewhat bubbly, unconcrete collection of code.
I had ideas about the topic that were partially very clear, partially
rather vague. But now I simply started the task with the initial goal of
moving ScholarLY into the new structure before I had done too much that
would have to be reverted or modified later.
I intended to finish a certain step and announce the project in a blog
post, but I decided to start discussion somewhat earlier and here on the
list for two reasons: I'll be away for a week (with only mobile phone
internet access) and won't be able to reach that state before. And I
think I'd prefer getting feedback before stepping too far. So what
follows is partially an explanation of what I've done, of what I intend
to do next, and it's about opening discussion what direction this should
take. Please understand that it's not about discussing a finished and
polished project.
### Basic structure/concept
Currently openLilyLib is a library of includable LilyPond files,
organized in a directory structure which intends to represent its
conceptional hierarchy. In the new structure openLilyLib will be a
repository of libraries, each one living in a subdirectory of the
top-level /ly directory. While these library are also simple directories
within the repository they are conceptually different in that adding a
library has to be a significantly more deliberate act than just
creating a new directory when one doesn't really know where to put a new
function:
* A library must have a maintainer who is responsible for the
library's target and also for code contributions
* A library must have a clearly visible purpose, and it must have been
discussed that such an addition makes sense (i.e. that the intended
code shouldn't rather be added to other, existing libraries)
* A library must have a certain "weight" that might justify setting up
a subdomain like scholarly.openlilylib.org (doesn't exist, just an
example)
The idea is that openLilyLib becomes a one-stop-shop solution for
LilyPond extension, so it will be easy to share functionality and have
other users easily get hold of it. This will also make it possible for
tools (e.g. LilyPond editors) to depend on external LilyPond code by
simply directing their users to get hold of that one repository as a
dependency.
The hope is that such a set-up will encourage the community to share
code and enable people to use LilyPond without having to reinvent the
wheel each time (just as I wouldn't want to implement regular expression
searching in any programming language but rather use an existing library
for that).
### Accessing openLilyLib and its libraries
"Installing" openLilyLib involves the following steps (also needed if
you want to contribute to the current discussion):
* Downloading/cloning openLilyLib from
https://github.com/openlilylib/openlilylib
(if you haven't done so already)
* Adding its root path to LilyPond's include path
(if you haven't done so already)
* Adding its top-level /ly directory to LilyPond's include path.
Once everything has been migrated the root path include can be
dropped but for now this doubled include is unfortunately necessary.
Step one for using is to activate openLilyLib with
\include "openlilylib"
This will already load some common infrastructure and functionality,
mostly because openLilyLib itself needs it, but it is also available for
users' files. Examples already available are:
* LilyPond version predicates which make it possible for libraries to
support different LilyPond versions by conditionally executing code
through things e.g.
#(if (lilypond-greater-than? "2.19.4") ...
* Logging
Commands like oll:log, oll:warn etc. that trigger messages (and log
them to files) when a certain log-level is set
* alist access
Taken from Jan-Peter's library it greatly simplifies the handling of
(nested) association lists.
This is an example of functionality included because openLilyLib
needs it but that can also be very useful in general contexts.
One of the nicest things so far is global configuration handling.
Libraries can register options with default values that can be set very
simply by end users. One actual example:
% Library code:
\registerOption scholarly.colorize #t
This tells the ScholarLY library to colorize their objects by default.
Users using ScholarLY can later (when the score is going to be published
and coloring isn't wanted anymore) simply write
% User code:
\setOption scholarly.colorize #f
to switch off the coloring.
Another example is changing the color for a certain annotation type in
ScholarLY:
% User code
\setOption scholarly.annotate.colors.critical-remark #blue
So libraries define their option hierarchy, and users can easily
configure the behaviour in their projects.
The best thing is that this isn't restricted to libraries but it's
available as soon as openLilyLib is loaded. So you could for example write
% house library
\registerOption my-house.print-status-footer ##t
% project configuration
\setOption my-house.print-status-footer ##f
A function that is actually responsible for the footer can access the
option's value through
#(if #{ \getOption my-house.print-status-footer #}) ...
Already with annotate this made my library configuration code extremely
shorter because I didn't have to maintain separate configuration
variables and individual access functions for everything.
See
https://github.com/openlilylib/openlilylib/blob/master/ly/scholarly/annotate/config.ily
versus the earlier
https://github.com/openlilylib/openlilylib/blob/dc9fe55adbcb4c065e3451e48b3e6c45b9e4d91e/ly/scholarly/annotate/config.ily
---
Accessing items from one of the libraries is done with the new command
\loadModule "library/path/to/module" or
\loadModule "library/path/to/file.ily"
This command will ensure that any file and library is included only once
and that it is properly initialized. If a library has an init file this
will be parsed once before the first item from the library is loaded.
That makes it possible to have shared code or data in a library, e.g.
common helper routines or a shared data structure, without them having
to be parsed multiple times.
The function is great IMO, but I'm not 100% sure if the term "Module" is
appropriate here. That's one of the points where I think some discussion
would be good. What is a library, what a module, what term should be
used for individual files. Currently a module would be e.g.
contemporary/note-heads, while there could also be individual items such
as contemporary/note-heads/my-fancy-tremoly.ily.
### Documentation
One very important aspect is unfortunately the least thought-through
yet: auto-generated documentation.
In the context of last year's discussion we improved the way openLilyLib
files have to be documented within header fields. This is to become much
more strict because there is /will be an infrastructure to create
consistent usage examples using the data provided in these headers. To
get an idea about that try out
ly/scholarly/usage-examples/diplomatic-line-breaks.ly. The actual code
file ly/scholarly/diplomatic-line-breaks.ily is also a good example of
using the configuration infrastructure.
Discussion on this has to be continued (before too many files have been
included), but basically I think this is a very nice way to create
consistent code and documentation.
Documentation will have to get a second level, though, and this is in a
even more preliminary state. What I've pondered over for quite some time
is a system of API documentation for LilyPond files. What we need is a
consistent syntax to insert documentation comments in (library) LilyPond
files and then a tool/module that creates documentation from this
information. This is intended to create (HTML and other) docs for
libraries or local projects but also a data structure that can be used
in separate tools, e.g. editing environments (to provide e.g. syntax
highlighting or code completion for openLilyLib's libraries).
We had already made some promising experiments, but only using the
header-style documentation and usage examples mentioned above.
###
Please excuse this lengthy post, but I think it is necessary to get the
new openLilyLib up and running. And I think it is worth the effort, as I
have a feeling that this will significantly change the way we can use
LilyPond in the future.
Best
Urs
_______________________________________________
lilypond-user mailing list
[email protected]
https://lists.gnu.org/mailman/listinfo/lilypond-user
