Re: Packages/modules
Am Mittwoch, den 22.01.2020, 11:43 +0100 schrieb Urs Liska: > Am Mittwoch, den 22.01.2020, 11:06 +0100 schrieb David Kastrup: > > Urs Liska writes: > > > > > Am Dienstag, den 21.01.2020, 11:19 +0100 schrieb Urs Liska: > > > > > Ok. One thing to think about is that we want package files > > > > > to > > > > > be > > > > > contributed by "ordinary" users. But something like > > > > > > > > > > \exportSymbols > > > > > transposeSequence,instrumentGroup,scratchMyBack > > > > > > > > > > would be perfectly feasible syntactical sugar. > > > > > > > > > > > > > I'll be more verbose than probably necessary, just to make sure > > > > we're > > > > talking about the same thing. > > > > > > > > ... > > > > > > > > If I got you right then from my experience with openLilyLib and > > > > creating project environments (which basically are the same as > > > > packages), I would say: > > > > > > > > * I'm all for hiding names in packages by default and having > > > > to > > > >explicitly expose/export the package's interface > > > > > > > > > > One more implication: If variables and functions have to be > > > explicitly > > > exported it will be easier for external tools (like Frescobaldi) > > > to > > > add > > > proper support for extensions. > > > > > > I assume that at one point Frescobaldi will > > > > > > * know about available (core and external) extensions > > > * provide ways to "use" an extension (as part of the Score > > > wizard > > > and > > >locally) > > > * at that point know about the options that can be passed to > > > that > > >package > > > * provide autocompletion and highlighting for available symbols > > >exported from extensions > > > * provide actions to generate the code for getting and setting > > > package > > >options > > > > > > So when planning the syntax of that export it would be good to > > > take > > > the > > > needs/interest of IDEs into account that will not work with the > > > result > > > as LilyPond does but that parse the package files themselves. > > > > Maybe we should have single-command exports after all > > You mean that a package has to export every function or variable > separately? I think that would be good wrt self-documentation. > This gave me another idea: How would it be if elements (functions, variables, whatever) exported by packages would have to be addressed through a package namespace: * scholarly.annotate exports \criticalRemark * this can't be used with \criticalRemark but (syntax of course up to the parser maintainer ;-) ) \scholarly.annotate.criticalRemark That way the global namespace would be less pollutable, and identical names in different packages wouldn't be an issue. A user can still do something like criticalRemark = scholarly.annotate.criticalRemark as a local shorthand. Urs
Re: Packages/modules
Urs Liska writes: > Am Mittwoch, den 22.01.2020, 11:43 +0100 schrieb Urs Liska: >> >> You mean that a package has to export every function or variable >> separately? I think that would be good wrt self-documentation. >> > > This gave me another idea: How would it be if elements (functions, > variables, whatever) exported by packages would have to be addressed > through a package namespace: > > * scholarly.annotate exports \criticalRemark > * this can't be used with \criticalRemark but (syntax of course up to > the parser maintainer ;-) ) \scholarly.annotate.criticalRemark The implementation of \x.y.z is not robust enough for that, it's a nuisance, and judging from LaTeX experience, it does not seem to be problem in practice. > That way the global namespace would be less pollutable, and identical > names in different packages wouldn't be an issue. > > A user can still do something like > > criticalRemark = scholarly.annotate.criticalRemark > > as a local shorthand. No, that would be equivalent to criticalRemark = #'(scholarly annotate criticalRemark) -- David Kastrup
Re: Packages/modules
Am Mittwoch, den 22.01.2020, 11:06 +0100 schrieb David Kastrup: > Urs Liska writes: > > > Am Dienstag, den 21.01.2020, 11:19 +0100 schrieb Urs Liska: > > > > Ok. One thing to think about is that we want package files to > > > > be > > > > contributed by "ordinary" users. But something like > > > > > > > > \exportSymbols transposeSequence,instrumentGroup,scratchMyBack > > > > > > > > would be perfectly feasible syntactical sugar. > > > > > > > > > > I'll be more verbose than probably necessary, just to make sure > > > we're > > > talking about the same thing. > > > > > > ... > > > > > > If I got you right then from my experience with openLilyLib and > > > creating project environments (which basically are the same as > > > packages), I would say: > > > > > > * I'm all for hiding names in packages by default and having to > > >explicitly expose/export the package's interface > > > > > > > One more implication: If variables and functions have to be > > explicitly > > exported it will be easier for external tools (like Frescobaldi) to > > add > > proper support for extensions. > > > > I assume that at one point Frescobaldi will > > > > * know about available (core and external) extensions > > * provide ways to "use" an extension (as part of the Score wizard > > and > >locally) > > * at that point know about the options that can be passed to that > >package > > * provide autocompletion and highlighting for available symbols > >exported from extensions > > * provide actions to generate the code for getting and setting > > package > >options > > > > So when planning the syntax of that export it would be good to take > > the > > needs/interest of IDEs into account that will not work with the > > result > > as LilyPond does but that parse the package files themselves. > > Maybe we should have single-command exports after all You mean that a package has to export every function or variable separately? I think that would be good wrt self-documentation. Would it be possible to export some meta information too? e.g. the type of a variable, the signature (including if it is a music-function etc.) of a function? That would be good. The options do something like that already in their current implementation. The \registerOption is used as a guard against trying \setOption with an unregistered option, but the concept can directly be used for package documentation. > and have a > (non-optional ?) documentation string to be used as mouse-over? I > think > a unified approach to documentation might go a long way towards basic > usability. A non-optional docstring sounds nice. This itself doesn't guarantee that the docstrings are actually *helpful*, but if e.g. Frescobaldi provides direct access to them pressure would certainly mount to improve them one by one. Wilbert plans (I don't recall if he said so in a presentation or just while chatting) to use his current rewriting of Frescobaldi's LilyPond parser to provide such "introspection", through a mouse-over effect or by extending the interface of the autocompletion dropdown. This will *certainly* make it more straightforward to get used to LilyPond's internals. Urs >
Re: Packages/modules
Urs Liska writes: > Am Dienstag, den 21.01.2020, 11:19 +0100 schrieb Urs Liska: >> > Ok. One thing to think about is that we want package files to be >> > contributed by "ordinary" users. But something like >> > >> > \exportSymbols transposeSequence,instrumentGroup,scratchMyBack >> > >> > would be perfectly feasible syntactical sugar. >> > >> >> I'll be more verbose than probably necessary, just to make sure we're >> talking about the same thing. >> >> ... >> >> If I got you right then from my experience with openLilyLib and >> creating project environments (which basically are the same as >> packages), I would say: >> >> * I'm all for hiding names in packages by default and having to >>explicitly expose/export the package's interface >> > > One more implication: If variables and functions have to be explicitly > exported it will be easier for external tools (like Frescobaldi) to add > proper support for extensions. > > I assume that at one point Frescobaldi will > > * know about available (core and external) extensions > * provide ways to "use" an extension (as part of the Score wizard and >locally) > * at that point know about the options that can be passed to that >package > * provide autocompletion and highlighting for available symbols >exported from extensions > * provide actions to generate the code for getting and setting package >options > > So when planning the syntax of that export it would be good to take the > needs/interest of IDEs into account that will not work with the result > as LilyPond does but that parse the package files themselves. Maybe we should have single-command exports after all and have a (non-optional ?) documentation string to be used as mouse-over? I think a unified approach to documentation might go a long way towards basic usability. -- David Kastrup
Re: Packages/modules
Am Dienstag, den 21.01.2020, 11:19 +0100 schrieb Urs Liska: > > Ok. One thing to think about is that we want package files to be > > contributed by "ordinary" users. But something like > > > > \exportSymbols transposeSequence,instrumentGroup,scratchMyBack > > > > would be perfectly feasible syntactical sugar. > > > > I'll be more verbose than probably necessary, just to make sure we're > talking about the same thing. > > ... > > If I got you right then from my experience with openLilyLib and > creating project environments (which basically are the same as > packages), I would say: > > * I'm all for hiding names in packages by default and having to >explicitly expose/export the package's interface > One more implication: If variables and functions have to be explicitly exported it will be easier for external tools (like Frescobaldi) to add proper support for extensions. I assume that at one point Frescobaldi will * know about available (core and external) extensions * provide ways to "use" an extension (as part of the Score wizard and locally) * at that point know about the options that can be passed to that package * provide autocompletion and highlighting for available symbols exported from extensions * provide actions to generate the code for getting and setting package options So when planning the syntax of that export it would be good to take the needs/interest of IDEs into account that will not work with the result as LilyPond does but that parse the package files themselves. Urs
Re: Packages/modules
Am Montag, den 20.01.2020, 23:45 +0100 schrieb David Kastrup: > Urs Liska writes: > > > OK. The *current* behaviour of oll-core is: > > > > * loaded packages and modules (let's for now keep the existing > > names) > > are accounted for in an alist. > > * if the requested package/module is already loaded: > > * if options are passed, try setting them (overriding defaults > > and/or values set by an earlier loading) > > => This behaviour has to be discussed > > Sounds to me like overriding defaults is perfectly reasonable (else > specifying options wouldn't work at all) Yes, of course that's the idea behind it. > and overriding values set by an > earlier loading should flag an error. That's probably right. Sometimes giving users too much controls and to many alternatives makes the interface convoluted and as a result confusing. > > Maybe packages should have two ways of overriding a default: set a > default and require a value. > > A request in a user document is always treated as a requirement, two > packages setting defaults leads to an error (or a warning and a > revert > to the original default?) unless some package (or the user) requires > a > particular setting which takes priority. And of course different > requirements cause an error. > > Or is this too contrived, and only requirements should be allowed? I don't find this too contrived in general, but there's one thing where you may be too generic in the description. There are two basic use cases for overriding default values: Loading a package with options and setting an option explicitly after loading the package. The other complication is in packages implicitly loading other packages. The simple case is when a user explicitly sets an option with (current syntax) \setOption package.module.some.option-leaf value. This can happen to change the global behaviour or to change something at some point in the score/music expression (depending on how the value is used). This is what you called a requirement and should always be respected. If *that* triggers an error or produces unwanted results it's clearly the user's responsibility. The other case is when a package depends on another one and loads it, which may result in the depended-on package being loaded more than once, possibly with conflicting options. Say you have a helper package [A] providing a data structure for some use case (for example there is the `breaks` package that provides some alists and accessor functionality to maintain sets of line/page breaks). Such a package may expose an option with a default value (currently the syntax is \registerOption package.option.path default-value (specifying types is on my wish-list)). A client package [B] may just load that while a client package [C] might load it overriding the value, for example because it wants [A] to work in a specific mode. Another client package [D] may override the option with a different value or pass the choice along to the user. This is where conflicts may occur, and where it is not in our reach to know at design time (designing the system, i.e. now) whether these conflicts will be harmless or not. So erroring may be the safest choice. Actually that would be the situation in LaTeX when you have to know that two packages "don't work well" together. After having written this I have the impression that the spot to address your distinction between "setting" defaults and "requiring" values would be: * A package "knows" whether a current value is at its default value or not (either with a "changed" flag or by on-the-fly comparison) * When to the *loading* of a package an option is passed that has already been changed it is considered illegal, and a warning or error is raised (to be decided, probably an error). * When an option is modified explicitly from user code it is simply done. I will raise the issue of how options are handled and stored in a separate thread, we should keep that out of the current discussion. > >>> ... > > > > And of course Guile has modules. Do we already have a thought > > > about > > > how > > > to relate to the module system? > > > > Yes, that's right! \loadModule is definitely a bad name. > > I was not as much worrying about the name, actually, OK, but still that name should be discarded. > but about the > semantics. Seems to me like defaulting to a separate module might be > a > reasonable thing. It would require exporting whatever you want to > use > from outside the module. OK, I *think* I see. I'll comment on that together with the other stuff at the end below. > > > In LilyPond there's a fundamental difference between \include and > > #(use-modules, which is not the case in oll-core. There "modules" > > are > > essentially the same as packages, just used to organize packages in > > a > > hierarchical fashion: > > > > * scholarLY includes modules: > > * annotate > > * choice > > * staff-cancellation > > * ... > > * snippets can be
Re: Packages/modules
Urs Liska writes:
> OK. The *current* behaviour of oll-core is:
>
> * loaded packages and modules (let's for now keep the existing names)
> are accounted for in an alist.
> * if the requested package/module is already loaded:
> * if options are passed, try setting them (overriding defaults
> and/or values set by an earlier loading)
> => This behaviour has to be discussed
Sounds to me like overriding defaults is perfectly reasonable (else
specifying options wouldn't work at all) and overriding values set by an
earlier loading should flag an error.
Maybe packages should have two ways of overriding a default: set a
default and require a value.
A request in a user document is always treated as a requirement, two
packages setting defaults leads to an error (or a warning and a revert
to the original default?) unless some package (or the user) requires a
particular setting which takes priority. And of course different
requirements cause an error.
Or is this too contrived, and only requirements should be allowed?
> * of no options are passed simply continue
> * (exit)
> * look up the file (according to some logic (separate discussion)
> * if the file is found load the package/module
> * otherwise issue a warning, suggesting follow-up errors might come
I think the usual course should be an error. Warning only if using a
special optional load (separate command or option for that).
>> And of course Guile has modules. Do we already have a thought about
>> how
>> to relate to the module system?
>
> Yes, that's right! \loadModule is definitely a bad name.
I was not as much worrying about the name, actually, but about the
semantics. Seems to me like defaulting to a separate module might be a
reasonable thing. It would require exporting whatever you want to use
from outside the module.
> In LilyPond there's a fundamental difference between \include and
> #(use-modules, which is not the case in oll-core. There "modules" are
> essentially the same as packages, just used to organize packages in a
> hierarchical fashion:
>
> * scholarLY includes modules:
> * annotate
> * choice
> * staff-cancellation
> * ...
> * snippets can be addressed like
> \loadModule oll-misc.layout.horizontal-spacing
>
> Actually it would make transition smoother if we choose completely new
> names for the functions.
>
> So:
> * I suggest not to discern between "use/load" and "require",
> just have one command that behaves like require.
> * (caveat: handling of config options when given to a
> secondary call)
> * Is the "\load" prefix the best choice?
> * \loadPackage
> * \usepackage
> * \use
> * \package
> ?
LaTeX has \usepackage and \RequirePackage either of which don't really
match LilyPond naming conventions. A bit of a pity: I'd have opted for
\usepackage otherwise.
> * Do we need a word for the (current) "module" at all?
> What about
> \use scholarly.annotate
> or
> \use \with { subs = annotate.choice } scholarly
> ?
>
>
>> With regard to namespaces, there may be
>> something to be said for requiring explicit export in the long run?
>>
>
> Although I know this is important I don't feel comfortable having an
> opinion about this type of issue.
Ok. One thing to think about is that we want package files to be
contributed by "ordinary" users. But something like
\exportSymbols transposeSequence,instrumentGroup,scratchMyBack
would be perfectly feasible syntactical sugar.
--
David Kastrup
Packages/modules (was: Extension management, first sketch)
Am Montag, den 20.01.2020, 14:14 +0100 schrieb David Kastrup: > Urs Liska writes: > > > Am Montag, den 20.01.2020, 10:27 +0100 schrieb Urs Liska: > > > * A core extension library shipping with LilyPond will be > > > initiated. > > > Extensions that are considered core functionality (prime > > > candidates: > > > edition-engraver, stylesheets) will eventually be moved here > > > from > > > openLilyLib, additionally special functionality (e.g. > > > gregorian.ly, > > > arabic.ly) may over time be moved there to expose the > > > difference > > > between core functionality and use-case specific modules more > > > clearly. These tools will then be called through `\loadModule` > > > instead of `\include`, which will be easy to handle with > > > convert-ly rules. Probably it would be a good idea to > > > eventually > > > expose *all* non-standard notation through explicit packages > > > and have that nicely describe in the LM. This too will not be > > > called openLilyLib. > > > > Thinking about it I would go one step further: > > > > Currently we have a /ly folder in the installation which contains > > core > > files like music-functions-init.ly on the one hand, and on the > > other > > hand optional files that users can \include for specific > > functionality, > > such as the ones I have given as examples above. > > > > I think now that *all* the files that are not included > > unconditionally > > but only upon user request should be treated as packages. This will > > make the code structure clearer, since there is a clear distinction > > between the files LilyPond needs/uses for its startup (then *all* > > files > > in /ly) and optional files at the user's disposition. > > Seconded. > > Regarding \loadModule : LaTeX has \documentclass for defining a > document layout. LilyPond does not have files dedicated to that > purpose > yet so that's something for later. Yes, I think we can ignore that for now. Everything a documentclass does I do with an initial include of the infrastructure, and I think everybody else would do so as well. Conceptually I can see what a document class might be in notation, but I don't see for what a separate construct would be necessary. > > With regard to packages, LaTeX has \usepackage from user documents > and > the sort-of equivalent \RequirePackage from package code (not sure > what > the rationale for the difference is, there may be something related > to > when loading is allowed) and a counterpiece \ProvidePackage. My impression is that - like with \newcommand, \renewcommand and \providecommand they want to give control over the handling of conflicts from already loaded packages. > > Emacs has > require is a built-in function in ‘C source code’. > > (require FEATURE FILENAME NOERROR) > > Probably introduced at or before Emacs version 15. > > If feature FEATURE is not loaded, load it from FILENAME. > If FEATURE is not a member of the list ‘features’, then the > feature is > not loaded; so load the file FILENAME. > > If FILENAME is omitted, the printname of FEATURE is used as the > file > name, and ‘load’ will try to load this name appended with the > suffix > ‘.elc’, ‘.el’, or the system-dependent suffix for dynamic module > files, in that order. The name without appended suffix will not > be > used. See ‘get-load-suffixes’ for the complete list of suffixes. > > The directories in ‘load-path’ are searched when trying to find > the > file name. > > If the optional third argument NOERROR is non-nil, then return > nil if > the file is not found instead of signaling an error. Normally > the > return value is FEATURE. > > The normal messages at start and end of loading FILENAME are > suppressed. > > [back] > > and a matching > > provide is a built-in function in ‘C source code’. > > (provide FEATURE SUBFEATURES) > > Probably introduced at or before Emacs version 15. > > Announce that FEATURE is a feature of the current Emacs. > The optional argument SUBFEATURES should be a list of symbols > listing > particular subfeatures supported in this version of FEATURE. > OK. The *current* behaviour of oll-core is: * loaded packages and modules (let's for now keep the existing names) are accounted for in an alist. * if the requested package/module is already loaded: * if options are passed, try setting them (overriding defaults and/or values set by an earlier loading) => This behaviour has to be discussed * of no options are passed simply continue * (exit) * look up the file (according to some logic (separate discussion) * if the file is found load the package/module * otherwise issue a warning, suggesting follow-up errors might come > And of course Guile has modules. Do we already have a thought about > how > to relate to the module system? Yes, that's right! \loadModule is definitely a bad name. In LilyPond
