Re: Request for Big Time Signature
Hi Han-Wen, Got it, thanks! I will attach the example file here as your reply didn't include it. 发件人: Han-Wen Nienhuys 发送时间: 2020年5月4日 17:21 收件人: Chen Leo ; lilypond-devel 主题: Re: Request for Big Time Signature Hi Leo, thanks for your kind words. I'm focusing my limited time on LilyPond infrastructure, so I can't help you here, but I hope someone else from the development community can. I took the liberty of including them on this reply. On Mon, May 4, 2020 at 6:37 AM Chen Leo wrote: > > Hi Han-Wen, > > I am a new lilypond user and I have been using lilypond for a few months now. > I want to say that I love the experience (especially the way lilypond > automatically handles all the collisions and alignment problems with great > accuracy). I want to thank the development team for creating such amazing > software. > > One personal suggestion would be that I think lilypond can benefit from > implementing more contemporary notations e.g. more symbols and noteheads (I > was happy to hear that lilypond is considering adopting the SMuFL font > format). For most of the notation problems I have encountered, I was able to > solve by copying and modifying different snippets, thanks to the great > lilypond community. However, there’s one specific problem that I can’t seem > to find a solution to, even with the help of several lilypond users. > > The problem is to put large time signatures on top of (or in between) staves, > it is commonly used in a lot of contemporary scores which often feature > constant changes in time structures. In these situations, using this layout > will make the score look a lot clearer and save much horizontal space. > > I found > http://lsr.di.unimi.it/LSR/Snippet?id=272 > which seems like a perfect solution initially. However, soon I was able to > find several problems with this snippet. (I will give examples for all of the > problems in the attached file) > 1a. Time signatures get aligned to cue clefs instead of the bar line whenever > a clef change is made at the end of the bar. > 1b. Time signatures appear twice when one staff has a grace note as the first > note in a bar while other staves do not. > 1c. Space is not allotted for the cautionary time signatures before line > breaks so they always cross the left margin. > 1d. shift-right-at-line-begin has a fixed X-offset value of 3.5 units, this > means that if the user decides to change the clef font size or to use an > alternative font the X-offset value won’t automatically change according to > the width of the clef symbol. > > I tried another approach afterward, which is to use > \override TimeSignature.break-align-symbol = #'staff-bar > This successfully solves problems 1 3 and 4 yet it also starts to cause more > problems since other staves are now aware of the time signature’s horizontal > space. The problems are: > 2a. Parts of hairpins and spanners are missing below the time signatures > after a line break. > 2b. Lilypond pushes everything to the left when \textLengthOn commands are > used on text script objects below the time signature. > 2c. Same as problem 1b from the original snippet. > 2d. Bar numbers are raised to the same height as the time signatures' dummy > staff while it should display on the top music staff. This problem can be > solved by removing "Bar_number_engraver" from the Score context and consists > of it only in the actual top staff’s context. Though this can cause some > problems if the user chooses to hide staves that don’t have a part in a > section of music and the top staff also happens to be one of the hidden > staves. > > After a long discussion, a kind lilypond user named Kevin Barry was able to > provide me with a solution that he thinks is somewhat “hacky” while it solves > problem 2a and 2c quite well. He wrote a custom engraver that replaces all > TimeSignature objects with TextScript objects except at the end of lines so > that space can be allotted for the cautionary time signatures (I’ve also > included his approach in the attached file). Problem 2b and 2d remain > unsolvable. Because the only solution to problem 2b is to set TextScript’s > extra-spacing-width property to #'(+inf.0 . -inf.0) while that means bars are > no longer sensitive to the TextScript objects’ width, it causes collisions > when time signatures exceed the barline. I wasn’t able to solve problem 2d as > well. > > Eventually, I decided to contact someone from the development team and I > found you. > I hope that this can be improved and made into a snippet (or even better if > this gets implemented into lilypond in future versions) as using large time > signatures is really quite common in contemporary ensemble pieces. > > Here are some examples of how this layout can look like in different scores. > > Most scores have the time signatures vertically centered. e.g. > Pierre Boulez - Répons > https://www.youtube.com/watch?v=9aa8ogiwEV4 >
回复: Request for Big Time Signature
Hi Han-Wen, Got it, thanks! I will attach the example file here as your reply didn't include it. 发件人: Han-Wen Nienhuys 发送时间: 2020年5月4日 17:21 收件人: Chen Leo ; lilypond-devel 主题: Re: Request for Big Time Signature Hi Leo, thanks for your kind words. I'm focusing my limited time on LilyPond infrastructure, so I can't help you here, but I hope someone else from the development community can. I took the liberty of including them on this reply. On Mon, May 4, 2020 at 6:37 AM Chen Leo wrote: > > Hi Han-Wen, > > I am a new lilypond user and I have been using lilypond for a few months now. > I want to say that I love the experience (especially the way lilypond > automatically handles all the collisions and alignment problems with great > accuracy). I want to thank the development team for creating such amazing > software. > > One personal suggestion would be that I think lilypond can benefit from > implementing more contemporary notations e.g. more symbols and noteheads (I > was happy to hear that lilypond is considering adopting the SMuFL font > format). For most of the notation problems I have encountered, I was able to > solve by copying and modifying different snippets, thanks to the great > lilypond community. However, there’s one specific problem that I can’t seem > to find a solution to, even with the help of several lilypond users. > > The problem is to put large time signatures on top of (or in between) staves, > it is commonly used in a lot of contemporary scores which often feature > constant changes in time structures. In these situations, using this layout > will make the score look a lot clearer and save much horizontal space. > > I found > http://lsr.di.unimi.it/LSR/Snippet?id=272 > which seems like a perfect solution initially. However, soon I was able to > find several problems with this snippet. (I will give examples for all of the > problems in the attached file) > 1a. Time signatures get aligned to cue clefs instead of the bar line whenever > a clef change is made at the end of the bar. > 1b. Time signatures appear twice when one staff has a grace note as the first > note in a bar while other staves do not. > 1c. Space is not allotted for the cautionary time signatures before line > breaks so they always cross the left margin. > 1d. shift-right-at-line-begin has a fixed X-offset value of 3.5 units, this > means that if the user decides to change the clef font size or to use an > alternative font the X-offset value won’t automatically change according to > the width of the clef symbol. > > I tried another approach afterward, which is to use > \override TimeSignature.break-align-symbol = #'staff-bar > This successfully solves problems 1 3 and 4 yet it also starts to cause more > problems since other staves are now aware of the time signature’s horizontal > space. The problems are: > 2a. Parts of hairpins and spanners are missing below the time signatures > after a line break. > 2b. Lilypond pushes everything to the left when \textLengthOn commands are > used on text script objects below the time signature. > 2c. Same as problem 1b from the original snippet. > 2d. Bar numbers are raised to the same height as the time signatures' dummy > staff while it should display on the top music staff. This problem can be > solved by removing "Bar_number_engraver" from the Score context and consists > of it only in the actual top staff’s context. Though this can cause some > problems if the user chooses to hide staves that don’t have a part in a > section of music and the top staff also happens to be one of the hidden > staves. > > After a long discussion, a kind lilypond user named Kevin Barry was able to > provide me with a solution that he thinks is somewhat “hacky” while it solves > problem 2a and 2c quite well. He wrote a custom engraver that replaces all > TimeSignature objects with TextScript objects except at the end of lines so > that space can be allotted for the cautionary time signatures (I’ve also > included his approach in the attached file). Problem 2b and 2d remain > unsolvable. Because the only solution to problem 2b is to set TextScript’s > extra-spacing-width property to #'(+inf.0 . -inf.0) while that means bars are > no longer sensitive to the TextScript objects’ width, it causes collisions > when time signatures exceed the barline. I wasn’t able to solve problem 2d as > well. > > Eventually, I decided to contact someone from the development team and I > found you. > I hope that this can be improved and made into a snippet (or even better if > this gets implemented into lilypond in future versions) as using large time > signatures is really quite common in contemporary ensemble pieces. > > Here are some examples of how this layout can look like in different scores. > > Most scores have the time signatures vertically centered. e.g. > Pierre Boulez - Répons > https://www.youtube.com/watch?v=9aa8ogiwEV4 >
Re: Almost, but not quite: C++ STL in LilyPond
On 5/5/20, 4:04 PM, "lilypond-devel on behalf of David Kastrup" wrote: David Kastrup writes: If everything can be represented/mapped in similar manner, the Scheme garbage collector does not need any interaction with user-written code for doing its job. And I really think that's where we want to go in order to get memory semantics and behavior that are "Guile orthodox" to a degree that we'll get more attention for problems we are experiencing than what we have been working with so far. CS-> In my opinion, this outcome is worth a lot for our project. We've had lots of garbage-collection-related errors as we have moved to new versions of Guile. And the Guile people (reasonably enough from their point of view) say they can't help us, because our system is too complex. CS -> If we can get "Guile orthodox" on our memory semantics, we'll have much lower barriers to getting help from the Guile crew, IMO. Carl
Re: Almost, but not quite: C++ STL in LilyPond
On May 5, 2020, at 18:03, David Kastrup wrote: > > If everything can be represented/mapped in similar manner, the Scheme > garbage collector does not need any interaction with user-written code > for doing its job. I would not like to see the STL go, but if the getting rid of all the explicit GC code is the other side of it, it might be worth it. — Dan
Re: Almost, but not quite: C++ STL in LilyPond
On May 5, 2020, at 17:48, David Kastrup wrote: > > One thing here is that being under control of a wrapper, one can use > unchecked unsmobs. This is a good idea. I considered this briefly, but I wanted to focus first on making it natural to deal robustly with potentially improper or heterogeneous lists coming from a user, and leave optimizing more restrictive cases for later. — Dan
Re: Almost, but not quite: C++ STL in LilyPond
David Kastrup writes:
> Dan Eble writes:
>
>> On May 5, 2020, at 13:37, David Kastrup wrote:
>>>
>>> What I have ready-to-use is something that stores like an SCM value but
>>> behaves like a Smob pointer with regard to -> and * usage.
>>
>> Oh. I believe I have some of that too. Excerpt:
>>
>> // specialization for pointers
>> template
>> class ly_scm_t
>> {
>> private:
>> using traits = ly_scm_traits;
>>
>> private:
>> SCM scm_ = SCM_UNDEFINED;
>>
>> public:
>> ly_scm_t () = default;
>> ly_scm_t (const ly_scm_t &) = default;
>> ly_scm_t = (const ly_scm_t &) = default;
>> ~ly_scm_t () = default;
>>
>> explicit ly_scm_t (T *v) : scm_ (traits::to_scm (v)) {}
>>
>> explicit operator bool () const { return traits::has_value (scm_); }
>> operator T *() const { return traits::to_value (scm_); }
>> T *operator ->() { return operator T * (); }
>> // TODO: operator *
>> };
>
> Well, mine is a lot less formal I guess. Something akin to
>
> template
> class Scm_ptr
> {
> SCM scm_;
> public:
> Scm_ptr () : scm_ (SCM_UNDEFINED) {}
> Scm_ptr (SCM v) : scm_ (v)
> {
> assert (SCM_UNBNDP (v) || T::is_smob (v));
> }
> Scm_ptr (const T *p) : scm_ (p ? p->self_scm () : SCM_UNDEFINED) {}
> Scm_ptr & operator = (T* p)
> {
> scm_ = p ? p->self_scm () : SCM_UNDEFINED;
> return *this;
> }
> Scm_ptr & operator = (SCM s)
> {
> if (SCM_UNBNDP (s) || LY_ASSERT_SMOB (T, s, 1))
> scm_ = s;
> else
> scm_ = SCM_UNDEFINED;
> return *this;
> }
> T * operator -> () const
> {
> return dynamic_cast (T::unchecked_unsmob (scm_));
> }
> operator T * () const
> {
> return dynamic_cast (T::unchecked_unsmob (scm_));
> }
> operator SCM () const
> {
> return scm_;
> }
> operator bool () const
> {
> return SCM_BNDP (scm_);
> }
> };
>
> One thing here is that being under control of a wrapper, one can use
> unchecked unsmobs. I am not sure about conversion to SCM: this uses a
> conversion operator. I was tempted to use operator & instead. I did
> not have any traits in mind yet, but formalising the allocation/garbage
> protection thing would certainly call for something identifiable with
> regard to types that LilyPond knows how to allocate/protect.
To expound somewhat: a vector class (replete with iterators etc)
implemented through SCM would internally consist of one SCM cell holding
a Scheme vector and one size_t number for its current length. The
capacity would be the actual size of the Scheme vector (with extra slots
filled with SCM_UNBNDP). So the garbage collector footprint is
basically "pouo" (look up Scheme struct vtables for the meaning). A
Smob allocator basically collects the strings of its constituents at
their respective offsets when it is first called and initializes a
Scheme vtable with it, and then uses this vtable to allocate the smob as
an SCM value. Simple_smob can create a "smobbed copy" on demand in a
similar way.
If everything can be represented/mapped in similar manner, the Scheme
garbage collector does not need any interaction with user-written code
for doing its job.
And I really think that's where we want to go in order to get memory
semantics and behavior that are "Guile orthodox" to a degree that we'll
get more attention for problems we are experiencing than what we have
been working with so far.
--
David Kastrup
Re: Almost, but not quite: C++ STL in LilyPond
Dan Eble writes:
> On May 5, 2020, at 13:37, David Kastrup wrote:
>>
>> What I have ready-to-use is something that stores like an SCM value but
>> behaves like a Smob pointer with regard to -> and * usage.
>
> Oh. I believe I have some of that too. Excerpt:
>
> // specialization for pointers
> template
> class ly_scm_t
> {
> private:
> using traits = ly_scm_traits;
>
> private:
> SCM scm_ = SCM_UNDEFINED;
>
> public:
> ly_scm_t () = default;
> ly_scm_t (const ly_scm_t &) = default;
> ly_scm_t = (const ly_scm_t &) = default;
> ~ly_scm_t () = default;
>
> explicit ly_scm_t (T *v) : scm_ (traits::to_scm (v)) {}
>
> explicit operator bool () const { return traits::has_value (scm_); }
> operator T *() const { return traits::to_value (scm_); }
> T *operator ->() { return operator T * (); }
> // TODO: operator *
> };
Well, mine is a lot less formal I guess. Something akin to
template
class Scm_ptr
{
SCM scm_;
public:
Scm_ptr () : scm_ (SCM_UNDEFINED) {}
Scm_ptr (SCM v) : scm_ (v)
{
assert (SCM_UNBNDP (v) || T::is_smob (v));
}
Scm_ptr (const T *p) : scm_ (p ? p->self_scm () : SCM_UNDEFINED) {}
Scm_ptr & operator = (T* p)
{
scm_ = p ? p->self_scm () : SCM_UNDEFINED;
return *this;
}
Scm_ptr & operator = (SCM s)
{
if (SCM_UNBNDP (s) || LY_ASSERT_SMOB (T, s, 1))
scm_ = s;
else
scm_ = SCM_UNDEFINED;
return *this;
}
T * operator -> () const
{
return dynamic_cast (T::unchecked_unsmob (scm_));
}
operator T * () const
{
return dynamic_cast (T::unchecked_unsmob (scm_));
}
operator SCM () const
{
return scm_;
}
operator bool () const
{
return SCM_BNDP (scm_);
}
};
One thing here is that being under control of a wrapper, one can use
unchecked unsmobs. I am not sure about conversion to SCM: this uses a
conversion operator. I was tempted to use operator & instead. I did
not have any traits in mind yet, but formalising the allocation/garbage
protection thing would certainly call for something identifiable with
regard to types that LilyPond knows how to allocate/protect.
--
David Kastrup
Re: Almost, but not quite: C++ STL in LilyPond
On May 5, 2020, at 13:37, David Kastrup wrote:
>
> What I have ready-to-use is something that stores like an SCM value but
> behaves like a Smob pointer with regard to -> and * usage.
Oh. I believe I have some of that too. Excerpt:
// specialization for pointers
template
class ly_scm_t
{
private:
using traits = ly_scm_traits;
private:
SCM scm_ = SCM_UNDEFINED;
public:
ly_scm_t () = default;
ly_scm_t (const ly_scm_t &) = default;
ly_scm_t = (const ly_scm_t &) = default;
~ly_scm_t () = default;
explicit ly_scm_t (T *v) : scm_ (traits::to_scm (v)) {}
explicit operator bool () const { return traits::has_value (scm_); }
operator T *() const { return traits::to_value (scm_); }
T *operator ->() { return operator T * (); }
// TODO: operator *
};
—
Dan
Re: GSoC for LilyPond
On 5/5/20, Owen Lamb wrote: > This is all new for me, so I > appreciate your willingness to help me with newbie questions! I'll probably > have quite a few, especially regarding compiling. As it happens, we do have a “newbie-oriented” document here (which includes a section about compiling), so don’t hesitate to help us clarify and expand it if needed: http://lilypond.org/doc/v2.21/Documentation/contributor/ Cheers, -- V.
Re: Extracting approximate outlines from FT ?
>> Mhmm, you could copy the data stored `face->glyph->outline' into a >> new `FT_Outline` structure and iterate over the outlines, [...] > > I wrote something that seems to be working, but I would like to > cache the contours. For Open_type_font, this is easy, but for > Pango_font, each string can be composed of multiple FT_Faces, and I > suspect they might be deallocated during a program run. Do you know > of a suitable immutable key for FT_Face (eg. full filename) that I > could get out of either Freetype or Pango? FreeType doesn't have such an immutable key – if I understand the issue correctly –, and I don't have sufficient knowledge of Pango to give further advice, sorry. Werner
Re: GSoC for LilyPond
Thank you so much! I'm looking forward to working with you all. I'm not familiar with IRC, although I've heard the name passed around. If this list is more active, I suppose I'll stay here. I do appreciate the ability to spend time crafting a good email response. In the meantime, I have a few days left of finals before my schedule is really free. I've downloaded LilyDev (I mainly use Windows, but I have a couple Raspberry Pi's if it's desirable to test more platforms) and will give compiling a try at the end of this week. This is all new for me, so I appreciate your willingness to help me with newbie questions! I'll probably have quite a few, especially regarding compiling. Thanks, Owen On Tue, May 5, 2020 at 2:50 AM Valentin Villenave wrote: > On 5/5/20, Werner LEMBERG wrote: > > While you have > > me as a personal mentor > > … And a great mentor at that! > > > it is rather the collective wisdom of this > > list that will guide you through your work. > > +1. There are _lots_ of absolutely brilliant people here. > > > We don't have an IRC channel, sorry. > > Er, we used to and we officially still do: > http://lilypond.org/contact.html > mentions #lilyp...@irc.freenode.net > (and even links to a web IRC client). > > That was mostly active when Janneke was around, though. (And it wasn’t > intended specifically for devs, so you’re technically right.) > > > Don't be shy! You should never hesitate to ask even simple things, if > > necessary. LilyPond is a very complex piece of software... > > That it is. > > Cheers, and good luck to you Owen! > > V. >
Re: [trial] current status
Am Dienstag, den 05.05.2020, 19:38 +0200 schrieb David Kastrup: > Jonas Hahnfeld writes: > > > Am Sonntag, den 03.05.2020, 13:51 -0400 schrieb Dan Eble: > > > On May 3, 2020, at 07:00, Jonas Hahnfeld wrote: > > > > As the last two points involve some (short) scripts and I already have > > > > two more for the migration, I'd like to put them into a new repository; > > > > maybe https://gitlab.com/lilypond/infrastructure ? (I think they're > > > > sufficiently orthogonal to LilyPond itself that they deserve their own > > > > place to live.) > > > > > > Sounds good. A reason to put them with LilyPond would be if you > > > wanted them to be branched and versioned with LilyPond, but I don't > > > see that as important. > > > > Right, that's what I wanted to express with "orthogonal" - we do not > > need to version them together with LilyPond. Instead they are tied to > > our decision of the tools we use. > > > > Due to lack of other responses, I went ahead and created the repository > > (see above). I'll still wait a few days for others to voice their > > opinions before posting some final questions and proposing the actual > > migration. > > I think that so far we did this kind of stuff in "lilypond-extra". No > idea what kind of organisation/split ultimately would make sense here. Probably transfer the relevant stuff to proper places? I know it's used to build the website at lilypond.org, but much else of it looks out- dated. signature.asc Description: This is a digitally signed message part
Re: [trial] current status
Jonas Hahnfeld writes: > Am Sonntag, den 03.05.2020, 13:51 -0400 schrieb Dan Eble: >> On May 3, 2020, at 07:00, Jonas Hahnfeld wrote: >> > As the last two points involve some (short) scripts and I already have >> > two more for the migration, I'd like to put them into a new repository; >> > maybe https://gitlab.com/lilypond/infrastructure ? (I think they're >> > sufficiently orthogonal to LilyPond itself that they deserve their own >> > place to live.) >> >> Sounds good. A reason to put them with LilyPond would be if you >> wanted them to be branched and versioned with LilyPond, but I don't >> see that as important. > > Right, that's what I wanted to express with "orthogonal" - we do not > need to version them together with LilyPond. Instead they are tied to > our decision of the tools we use. > > Due to lack of other responses, I went ahead and created the repository > (see above). I'll still wait a few days for others to voice their > opinions before posting some final questions and proposing the actual > migration. I think that so far we did this kind of stuff in "lilypond-extra". No idea what kind of organisation/split ultimately would make sense here. -- David Kastrup
Re: Almost, but not quite: C++ STL in LilyPond
Dan Eble writes: > On May 5, 2020, at 11:09, David Kastrup wrote: >> I'd like to come up with an allocator/container programming model >> comparatively similar to the STL one so that one could mostly steal the >> implementations and "just" add the required Scheme awareness while >> removing of destruction/deallocation tracking. > > It might be difficult for me to devote the time to this discussion > that it deserves, but I want to say that I've been working on C++ > wrappers that allow walking SCM lists (of immediate values or Smobs) > with standard iterator syntax and semantics. I suppose that's > tangential to what you are talking about, but it would be > disappointing if we did significant overlapping work. At the current point of time, I am more concerned with offering container shadows than iterators on existing SCM data structures. The typical SCM list would be somewhat akin to a Forward_list construct. What I have ready-to-use is something that stores like an SCM value but behaves like a Smob pointer with regard to -> and * usage. That would allow to replace a number of Smob members that are now marked with scm_gc_mark (xxx->self_scm ()) with scm_gc_mark (xxx) or scm_gc_mark (xxx.scm ()) which is a first step towards making Smob memory layout be organised where its garbage collection phase can be handled by a struct vtable rather than an explicit hook (this basically shifts the responsibility of whether the whole struct or just individual parts of it participate in the mark phase back to the Guile implementation). But STL containers still require a strategy. -- David Kastrup
Re: Almost, but not quite: C++ STL in LilyPond
> On 5 May 2020, at 18:57, David Kastrup wrote: > > Hans Åberg writes: > >>> On 5 May 2020, at 17:09, David Kastrup wrote: >>> >>> One reason is that we want to get rid of finalisers particularly in >>> connection with the Boehm GC. Finalisers are called when garbage is >>> collected, the collection of garbage is typically indicative of the >>> expected lifetime of our objects (there are a few that might be >>> unambiguously dead before), and thus it will cause C++ destructors to be >>> called. And those trigger, among other things, the deallocation of >>> memory resources, but we'd rather want the deallocation to be determined >>> by Scheme mechanisms and not have any resources in need of destruction >>> other than falling into oblivion. >> >> For the GC objects, I use operator new/delete with an extra argument, >> the latter with empty body, wrapped in structure similar to >> std:shared_ptr. Then the memory resources of those objects are only >> handled by the GC, regardless of finalizers. > > That does not sound like it has anything to do with SCM garbage > collection. Sorry, I sent the message by mistake: I thought I clicked “Delete”, but it was “Send”, so if you so like, just forget about it. > Nor is it phrased in a manner where I would understand what > you are talking about. It is not clear what a "GC object" is supposed > to be, it is not clear what "I use" is supposed to mean with regard to > where and how it is being used, "operator new/delete with an extra > argument, the latter with empty body" presumably means that operator > delete rather than the extra argument have an empty body. It is not > clear what it is supposed to buy us if operator delete has an empty body > when calling operator delete is something triggered by the destructor > and not having to call the destructor is a prerequisite for the life > time being determined by garbage collection without finalisation hook. C++ operator new/delete are called in pairs, so if the GC allocated objects have a pair of their own, then one can set a no-op delete. > It is not clear _what_ is "wrapped in structure similar to > std:shared_ptr" and how that is supposed to interact with Scheme. This class use reference counting; instead use the GC allocating operator new. Don’t know about the Scheme part, though, but it uses something similar, I think. >>> I've tried to come up with a clever callback scheme where the first >>> use of a structure causes fake nodes to be created with a "Pointer" >>> type that actually calls back with its address to its caller that >>> then constructs a layout of SCM elements within its governed class. >>> However, I have found no way to make this "parallel construction" >>> where I could then swap out this specific "Pointer" behavior for >>> regular SCM. >> >> Similarly, it is necessary to call GC_INIT() on the first use on >> MacOS, which can happen before ‘main’ is called. So for that, I let >> operator new/delete call a function pointer which on the first call >> initializes, and also changes the pointer to the ordinary function >> pointer on future calls. > > The ordinary function pointer. I see. Right. GC_malloc and GC_malloc_uncollectable.
Re: [trial] current status
Am Sonntag, den 03.05.2020, 13:51 -0400 schrieb Dan Eble: > On May 3, 2020, at 07:00, Jonas Hahnfeld wrote: > > As the last two points involve some (short) scripts and I already have > > two more for the migration, I'd like to put them into a new repository; > > maybe https://gitlab.com/lilypond/infrastructure ? (I think they're > > sufficiently orthogonal to LilyPond itself that they deserve their own > > place to live.) > > Sounds good. A reason to put them with LilyPond would be if you wanted them > to be branched and versioned with LilyPond, but I don't see that as important. Right, that's what I wanted to express with "orthogonal" - we do not need to version them together with LilyPond. Instead they are tied to our decision of the tools we use. Due to lack of other responses, I went ahead and created the repository (see above). I'll still wait a few days for others to voice their opinions before posting some final questions and proposing the actual migration. Cheers, Jonas signature.asc Description: This is a digitally signed message part
Re: Almost, but not quite: C++ STL in LilyPond
On May 5, 2020, at 11:09, David Kastrup wrote: > I'd like to come up with an allocator/container programming model > comparatively similar to the STL one so that one could mostly steal the > implementations and "just" add the required Scheme awareness while > removing of destruction/deallocation tracking. It might be difficult for me to devote the time to this discussion that it deserves, but I want to say that I've been working on C++ wrappers that allow walking SCM lists (of immediate values or Smobs) with standard iterator syntax and semantics. I suppose that's tangential to what you are talking about, but it would be disappointing if we did significant overlapping work. Regards, — Dan
Re: Almost, but not quite: C++ STL in LilyPond
Hans Åberg writes: >> On 5 May 2020, at 17:09, David Kastrup wrote: >> >> One reason is that we want to get rid of finalisers particularly in >> connection with the Boehm GC. Finalisers are called when garbage is >> collected, the collection of garbage is typically indicative of the >> expected lifetime of our objects (there are a few that might be >> unambiguously dead before), and thus it will cause C++ destructors to be >> called. And those trigger, among other things, the deallocation of >> memory resources, but we'd rather want the deallocation to be determined >> by Scheme mechanisms and not have any resources in need of destruction >> other than falling into oblivion. > > For the GC objects, I use operator new/delete with an extra argument, > the latter with empty body, wrapped in structure similar to > std:shared_ptr. Then the memory resources of those objects are only > handled by the GC, regardless of finalizers. That does not sound like it has anything to do with SCM garbage collection. Nor is it phrased in a manner where I would understand what you are talking about. It is not clear what a "GC object" is supposed to be, it is not clear what "I use" is supposed to mean with regard to where and how it is being used, "operator new/delete with an extra argument, the latter with empty body" presumably means that operator delete rather than the extra argument have an empty body. It is not clear what it is supposed to buy us if operator delete has an empty body when calling operator delete is something triggered by the destructor and not having to call the destructor is a prerequisite for the life time being determined by garbage collection without finalisation hook. It is not clear _what_ is "wrapped in structure similar to std:shared_ptr" and how that is supposed to interact with Scheme. >> I've tried to come up with a clever callback scheme where the first >> use of a structure causes fake nodes to be created with a "Pointer" >> type that actually calls back with its address to its caller that >> then constructs a layout of SCM elements within its governed class. >> However, I have found no way to make this "parallel construction" >> where I could then swap out this specific "Pointer" behavior for >> regular SCM. > > Similarly, it is necessary to call GC_INIT() on the first use on > MacOS, which can happen before ‘main’ is called. So for that, I let > operator new/delete call a function pointer which on the first call > initializes, and also changes the pointer to the ordinary function > pointer on future calls. The ordinary function pointer. I see. -- David Kastrup
Re: Almost, but not quite: C++ STL in LilyPond
> On 5 May 2020, at 17:09, David Kastrup wrote: > > One reason is that we want to get rid of finalisers particularly in > connection with the Boehm GC. Finalisers are called when garbage is > collected, the collection of garbage is typically indicative of the > expected lifetime of our objects (there are a few that might be > unambiguously dead before), and thus it will cause C++ destructors to be > called. And those trigger, among other things, the deallocation of > memory resources, but we'd rather want the deallocation to be determined > by Scheme mechanisms and not have any resources in need of destruction > other than falling into oblivion. For the GC objects, I use operator new/delete with an extra argument, the latter with empty body, wrapped in structure similar to std:shared_ptr. Then the memory resources of those objects are only handled by the GC, regardless of finalizers. > I've tried to come up with a clever callback scheme where the first use > of a structure causes fake nodes to be created with a "Pointer" type > that actually calls back with its address to its caller that then > constructs a layout of SCM elements within its governed class. However, > I have found no way to make this "parallel construction" where I could > then swap out this specific "Pointer" behavior for regular SCM. Similarly, it is necessary to call GC_INIT() on the first use on MacOS, which can happen before ‘main’ is called. So for that, I let operator new/delete call a function pointer which on the first call initializes, and also changes the pointer to the ordinary function pointer on future calls.
Re: Almost, but not quite: C++ STL in LilyPond
Jonas Hahnfeld writes: > Am Dienstag, den 05.05.2020, 17:09 +0200 schrieb David Kastrup: >> I am currently digging back and forth regarding implementation of our >> Smobs (Scheme objects) and garbage collection and STL, and I think I am >> converging on the realisation that we'll have to end up duplicating >> those parts of STL that we are using. > > Please forgive my ignorance, but I'm missing a bit of context. Are we > talking about vector/lists/... of Smobs? Or is the issue that Smobs > contain STL containers? Vectors/lists of Smobs are inherently not protected from garbage protection even when in "auto" stack space or other GC-scanned space (Guile-2 has different options there). That's a problem. One important GC-scanned space that we'd ultimately want to automate scanning of are Smobs. STL containers cannot be automatically scanned even when they are formally located in scanned memory. > In any case I'm not really fond of duplicating code. Given that it > seems to "work" right now, IMHO this needs to give some very clear > advantage over keeping the status quo. GC performance remains a major troublesome point. I want to get the point to where we don't need mark hooks, and where we don't need finalisation hooks. Either are causing performance problems, and with Guile2, they also have a history of causing stability problems as well as "nobody wants to do that, so we are not really interested in your problems" reactions from Guile developers. What I was writing about is to reduce the amount of code duplication by not considering use of STL as free for the taking (which might cause more and more need for reimplementation of stuff). I do not see that at the current point of time regarding both code bases and standards (reflection is not in current C++ standards though it may appear at some point of time) we have a reasonable chance to make STL free for the taking. The underlying dynamic allocation and destruction model is not really a perfect match to what Scheme does and expects, and what the garbage collectors we are dealing with are good at. But I am not arguing API duplication: there is a point in making the semantics of SCM-aware reimplementations follow STL Container semantics. That will allow us at one point in the future when the SCM memory model is a better match to STL allocators, to go back to STL. But the different lifetime models will always mean that we are pressing our luck when using STL under the assumption that running destructors is optional. I might still try coming up with some reflection engine that would work with STL without significant performance impact and some reliability, but I would not expect it to come cost free. -- David Kastrup
Re: Almost, but not quite: C++ STL in LilyPond
Am Dienstag, den 05.05.2020, 17:09 +0200 schrieb David Kastrup: > I am currently digging back and forth regarding implementation of our > Smobs (Scheme objects) and garbage collection and STL, and I think I am > converging on the realisation that we'll have to end up duplicating > those parts of STL that we are using. Please forgive my ignorance, but I'm missing a bit of context. Are we talking about vector/lists/... of Smobs? Or is the issue that Smobs contain STL containers? In any case I'm not really fond of duplicating code. Given that it seems to "work" right now, IMHO this needs to give some very clear advantage over keeping the status quo. Jonas signature.asc Description: This is a digitally signed message part
Almost, but not quite: C++ STL in LilyPond
Hi, I am currently digging back and forth regarding implementation of our Smobs (Scheme objects) and garbage collection and STL, and I think I am converging on the realisation that we'll have to end up duplicating those parts of STL that we are using. One reason is that we want to get rid of finalisers particularly in connection with the Boehm GC. Finalisers are called when garbage is collected, the collection of garbage is typically indicative of the expected lifetime of our objects (there are a few that might be unambiguously dead before), and thus it will cause C++ destructors to be called. And those trigger, among other things, the deallocation of memory resources, but we'd rather want the deallocation to be determined by Scheme mechanisms and not have any resources in need of destruction other than falling into oblivion. Another point is the lack of reflection we have for getting a description of STL nodes (like list nodes) for the sake of knowing which elements may be allocated in what manner where. It is conceivable that Guile-2 will work tolerably just with the hand-waving knowledge "this node may or may not contain pointers to other nodes somewhere" which essentially is the information STL allocators can rely on but having to rely on it does not seem like a good idea with regard to keeping things well-collected. I've tried to come up with a clever callback scheme where the first use of a structure causes fake nodes to be created with a "Pointer" type that actually calls back with its address to its caller that then constructs a layout of SCM elements within its governed class. However, I have found no way to make this "parallel construction" where I could then swap out this specific "Pointer" behavior for regular SCM. Which means that pointer/SCM juggling within the STL framework would always involve some check (empty call or boolean variable) of whether we are still in initialisation phase. Not good. The third reason is that STL allocators just have no knowledge about use patterns and need to be able to allocate single SCM elements and multiple SCM elements at request and keep track of them. That is particularly problematic since SCM allocations are principally per-element and any multiple allocation needs to run through vectors and similar. And of course we'd want to make good use of what we have for generating hash values etc in Scheme already. So that means that I am going to reimplement some STL containers with equal semantics (and hopefully also matching the "Container" definitions so that at least container adaptors from the STL can make use of them) and with more introspection into the Scheme-typical manners of allocation and semantics. In practical terms this means for other developers to avoid using or introducing STL structures into Smobs. Obviously, arrays and vectors will have to get reimplemented in course of my plan, so there is no point in avoiding them. With regard to maps, ordered and unordered sets and similar stuff, the strategy would be to vastly prefer any solution that can be implemented in terms of Scheme functions. To make this less painful, it is suggested that things like Scheme_hash get reimplemented in a manner closely following the STL API so that we can actually program in a C++ style (and encourage other contributors to do so) without thinking too much. I'd like to come up with an allocator/container programming model comparatively similar to the STL one so that one could mostly steal the implementations and "just" add the required Scheme awareness while removing of destruction/deallocation tracking. Using the actual code with trickery around it would be, of course, a lot nicer. But after a lot of deliberation and researching implementation strategies, I am skeptical that making and keeping it work would be robust and dependable (and efficient) enough to make this a really good idea. -- David Kastrup
Re: GSoC for LilyPond
On 5/5/20, Werner LEMBERG wrote: > While you have > me as a personal mentor … And a great mentor at that! > it is rather the collective wisdom of this > list that will guide you through your work. +1. There are _lots_ of absolutely brilliant people here. > We don't have an IRC channel, sorry. Er, we used to and we officially still do: http://lilypond.org/contact.html mentions #lilyp...@irc.freenode.net (and even links to a web IRC client). That was mostly active when Janneke was around, though. (And it wasn’t intended specifically for devs, so you’re technically right.) > Don't be shy! You should never hesitate to ask even simple things, if > necessary. LilyPond is a very complex piece of software... That it is. Cheers, and good luck to you Owen! V.
Re: texi2html <= 1.82 ?
On 05/05/2020 11:14, Jonas Hahnfeld wrote: Hi Dave, Am Dienstag, den 05.05.2020, 11:02 +0200 schrieb Dave Plater: Hi, I maintain lilypond in openSUSE and a user requested 2.20.0 which prompted me to update. I was very disappointed when I received this error from configure: ERROR: Please install required programs: texi2html <= 1.82 (installed: 5.32) Both lilypond 2.18.2 and 2.19.83 (which I built to make sure there were no nasty shocks in 2.20.0) are building happily with texi2html-5.32 and there have been no bug reports. Is this possibly a typo in configure or an m4 macro? yes and no: texihtml-1.82 is really required, but only if you want to build the HTML documentation. This shouldn't be necessary for a distribution package, and there's no influence on the binary. There's a switch --disable-documentation, but this also disables building the info pages which you probably want to have. As such the only possibility for 2.20 is to remove the version constraint, as for example Arch Linux does: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/lilypond#n25 (For 2.22 I tentatively plan to add something like --disable-html- documentation to make texihtml optional for distributions.) Hope this helps Jonas Thanks for the prompt reply, there is: https://bugzilla.suse.com/show_bug.cgi?id=964354 from way back, maybe this is caused by texi2html's version in the doc build. It's a pity that I will have to disable the documentation build, I've discovered I made a texinfo4 package which has a texi2html-1.82 but doesn't build it so I must just see the reason why I disabled it's build and maybe there's hope. Dave
Re: texi2html <= 1.82 ?
Hi Dave, Am Dienstag, den 05.05.2020, 11:02 +0200 schrieb Dave Plater: > Hi, I maintain lilypond in openSUSE and a user requested 2.20.0 which > prompted me to update. I was very disappointed when I received this > error from configure: > ERROR: Please install required programs: texi2html <= 1.82 (installed: > 5.32) > Both lilypond 2.18.2 and 2.19.83 (which I built to make sure there were > no nasty shocks in 2.20.0) are building happily with texi2html-5.32 and > there have been no bug reports. > Is this possibly a typo in configure or an m4 macro? yes and no: texihtml-1.82 is really required, but only if you want to build the HTML documentation. This shouldn't be necessary for a distribution package, and there's no influence on the binary. There's a switch --disable-documentation, but this also disables building the info pages which you probably want to have. As such the only possibility for 2.20 is to remove the version constraint, as for example Arch Linux does: https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/lilypond#n25 (For 2.22 I tentatively plan to add something like --disable-html- documentation to make texihtml optional for distributions.) Hope this helps Jonas signature.asc Description: This is a digitally signed message part
texi2html <= 1.82 ?
Hi, I maintain lilypond in openSUSE and a user requested 2.20.0 which prompted me to update. I was very disappointed when I received this error from configure: ERROR: Please install required programs: texi2html <= 1.82 (installed: 5.32) Both lilypond 2.18.2 and 2.19.83 (which I built to make sure there were no nasty shocks in 2.20.0) are building happily with texi2html-5.32 and there have been no bug reports. Is this possibly a typo in configure or an m4 macro? Thanks Dave
GSoC for LilyPond
Hello Owen, welcome to LilyPond :-) We are glad that you are with us the next few months (and maybe even longer), and thank you in advance for the work on your project. Our main communication is e-mail, which has served us well since the very beginning. Except for really private messages I ask you to always send e-mails to the 'lilypond-devel@gnu.org' mailing list, so please subscribe in case you haven't done so already. While you have me as a personal mentor, it is rather the collective wisdom of this list that will guide you through your work. In particular, you might get an answer earlier because we are distributed all over the world, living in different time zones. We don't have an IRC channel, sorry. Note also that most of us don't have English as our mother tongue, which also means that we are more comfortable with writing e-mails than talking directly. For the former, there is simply more time to formulate words correctly. Don't be shy! You should never hesitate to ask even simple things, if necessary. LilyPond is a very complex piece of software... Werner
