Re: State of LilyPond with Guile 2.2
>> It is very unfortunate that more recent Guile versions cause such a >> serious deterioration for LilyPond. Maybe it makes sense for the >> foreseeable future to stay with the status quo, this is, using >> Guile 1.8 as much as possible, > > For reference: Ubuntu 16.04, the longest supported version to have a > package for Guile 1.8, will go EOL on April 30; Debian Jessie > already is dead since June 2020. Well, Guile 1.8.x served us surprisingly well in the last years. It still has a maintainer, and I'm not aware of serious bugs that are showstoppers for LilyPond. >> and offering support for 2.x (or 3.x) for platforms where version >> 1.8 can't be used. > > The problem I see is that whatever is part of the official binaries > will get the most attention, or we risk giving users a configuration > that developers don't test regularly. Which means the other version > simply won't matter and carry on their own bugs. But we already have this situation: Almost all developers use a Unix-like OS and can be thus served with Guile 1.8.x! Are there actually LilyPond developers who work natively with Windows or MacOS? With 'natively' I mean using a binary specifically compiled for that platform and not a virtual box. Werner
Re: State of LilyPond with Guile 2.2
On Sun, Apr 11, 2021 at 08:11:37PM +0200, Jonas Hahnfeld via Discussions on LilyPond development wrote: > So, that was the main question of the message (sorry if that got hidden > in the lengthy text and many numbers): What is "reasonable"? I think > the numbers I showed are reasonable, but that's certainly subjective. > Yes, they are slower than Guile 1.8 but there'll be a trade-off at some > point. I don't know if my opinion counts, but... Based on the numbers in this thread and the ones from the recent thread with the Guile developer (? whose name escapes me) I think we should move to Guile 2.2 (i.e. make it the default and stop supporting 1.8) as soon as reasonably possible and just accept the deterioration in performance. I don't think we have much to gain by continuing to wait: Guile has already moved on to version 3 so we cannot expect performance problems with 2.2 to change much or go away. The only alternative seems to be Han-wen's suggestion of forking Guile 1.8. I don't know all of the implications of that. Lots of downsides are immediately apparent for developers, but on the other hand everyone is affected by performance so maybe we should give it serious thought. (But again: if there is agreement that this is not the right course then I see no reason not to immediately abandon 1.8.) Kevin
Re: State of LilyPond with Guile 2.2
Am Sonntag, dem 11.04.2021 um 19:55 +0200 schrieb Werner LEMBERG: > > It is very unfortunate that more recent Guile versions cause such a > serious deterioration for LilyPond. Maybe it makes sense for the > foreseeable future to stay with the status quo, this is, using Guile > 1.8 as much as possible, For reference: Ubuntu 16.04, the longest supported version to have a package for Guile 1.8, will go EOL on April 30; Debian Jessie already is dead since June 2020. > and offering support for 2.x (or 3.x) for platforms where version 1.8 > can't be used. The problem I see is that whatever is part of the official binaries will get the most attention, or we risk giving users a configuration that developers don't test regularly. Which means the other version simply won't matter and carry on their own bugs. Jonas signature.asc Description: This is a digitally signed message part
Re: State of LilyPond with Guile 2.2
Am Sonntag, dem 11.04.2021 um 20:04 +0200 schrieb Thomas Morley: > Am So., 11. Apr. 2021 um 19:37 Uhr schrieb Jonas Hahnfeld via > Discussions on LilyPond development : > > > > Am Sonntag, dem 11.04.2021 um 18:04 +0200 schrieb Han-Wen Nienhuys: > > > I wonder if it isn't more practical to fork guile 1.8, and stick it > > > into our tree as a submodule, and always build lilypond against the > > > in-tree version. We'd be up for the maintenance on the 1.8 branch, but > > > it might well be less work than keeping up with the churn that newer > > > GUILE versions bring us. > > > > > > > I had already replied that I don't like that option; it was always a > > given for me that LilyPond would move on. Guile 2.2 also makes binary > > distribution much nicer (because there no more shared srfi libraries, > > so lilypond can be linked as one static executable) and makes it > > possible to offer 64 bit executables for Windows. > > > > But given the reactions, I'll reduce activity on my work towards Guile > > 2.2... > > Jonas, > > once the startup delay and overall lower tempo is reduced to a > reasonable amount I'm for moving on to Guile-2, if not then Guile-3 > (yes, I've read what you wrote about Guile-3). So, that was the main question of the message (sorry if that got hidden in the lengthy text and many numbers): What is "reasonable"? I think the numbers I showed are reasonable, but that's certainly subjective. Yes, they are slower than Guile 1.8 but there'll be a trade-off at some point. Jonas signature.asc Description: This is a digitally signed message part
Re: State of LilyPond with Guile 2.2
Am So., 11. Apr. 2021 um 19:37 Uhr schrieb Jonas Hahnfeld via Discussions on LilyPond development : > > Am Sonntag, dem 11.04.2021 um 18:04 +0200 schrieb Han-Wen Nienhuys: > > I wonder if it isn't more practical to fork guile 1.8, and stick it > > into our tree as a submodule, and always build lilypond against the > > in-tree version. We'd be up for the maintenance on the 1.8 branch, but > > it might well be less work than keeping up with the churn that newer > > GUILE versions bring us. > > > > I had already replied that I don't like that option; it was always a > given for me that LilyPond would move on. Guile 2.2 also makes binary > distribution much nicer (because there no more shared srfi libraries, > so lilypond can be linked as one static executable) and makes it > possible to offer 64 bit executables for Windows. > > But given the reactions, I'll reduce activity on my work towards Guile > 2.2... Jonas, once the startup delay and overall lower tempo is reduced to a reasonable amount I'm for moving on to Guile-2, if not then Guile-3 (yes, I've read what you wrote about Guile-3). Apart from the already mentioned unicode-thingy (admittedly the strongest argument to move up, imho) there are also bug-fixes and new functionality in newer guile-versions. The possibility to have 64 bit executables for Windows is the overall strongest argument, I'd say. I'd love to do my own tests etc, regrettable my spring-break is over. I'll likely will have no time until next weekend. Please, continue the work towards Guile-2/3 Best, Harm
Re: State of LilyPond with Guile 2.2
> I had already replied that I don't like that option; it was always a > given for me that LilyPond would move on. Your arguments are sound, no question, ... > Guile 2.2 also makes binary distribution much nicer (because there > no more shared srfi libraries, so lilypond can be linked as one > static executable) and makes it possible to offer 64 bit executables > for Windows. ... especially to support some 64bit architectures. > But given the reactions, I'll reduce activity on my work towards > Guile 2.2... Hopefully there is not a misunderstanding: I very much appreciate your activities (and I'm quite sure that Han-Wen thinks the same) and invaluable tests! It is very unfortunate that more recent Guile versions cause such a serious deterioration for LilyPond. Maybe it makes sense for the foreseeable future to stay with the status quo, this is, using Guile 1.8 as much as possible, and offering support for 2.x (or 3.x) for platforms where version 1.8 can't be used. And if we go this route it makes sense to do what Han-Wen suggests, namely to add Guile 1.8 as a submodule. Werner
Re: State of LilyPond with Guile 2.2
Am Sonntag, dem 11.04.2021 um 18:04 +0200 schrieb Han-Wen Nienhuys: > On Sun, Apr 11, 2021 at 4:24 PM Jonas Hahnfeld wrote: > > > > Am Sonntag, dem 11.04.2021 um 16:04 +0200 schrieb Han-Wen Nienhuys: > > > On Sun, Apr 11, 2021 at 3:42 PM Jonas Hahnfeld via Discussions on > > > LilyPond development wrote: > > > > > > > > The first set of experiments is about things that developers care > > > > about, namely 'make test' and 'make doc'. For all runs, I used the > > > > options "-j4 CPU_COUNT=4". > > > > > > > > 'make test' when compiled with Guile 1.8.8: > > > > * User time (seconds): 520.68 > > > > * Elapsed (wall clock) time (h:mm:ss or m:ss): 3:59.54 > > > > > > > > > > IIRC, on my 4 CPU machine, the actual time running the regression .ly > > > files is fairly low (about 1 minute). A lot of time is spent running > > > the lilypond-book regression tests, which makes this difference all > > > the more dramatic > > > > True, but the current way of testing lilypond-book is kind of the > > worst-case scenario for LilyPond because all files are compiled > > separately. So 37 times a startup time of 2 seconds is already the > > majority of the slowdown. > > This may warrant some further investigation. Ideally, the shared > snippet database makes it so we have to run lilypond only a few times. Yes, this reminds me that I had an idea how to improve those tests... > > > > > * Maximum resident set size (kbytes): 372560 > > > > 'make test' when compiled with Guile 2.2.6: > > > > * User time (seconds): 710.35 > > > > * Elapsed (wall clock) time (h:mm:ss or m:ss): 5:43.04 > > > > * Maximum resident set size (kbytes): 372344 > > > > > > > > -> 40% slower when using Guile 2.2.6 during development > > > > > > > > > > > > > I didn't see you mention this explicitly: for apples to apples > > > comparisons, you have to disable 'debug-gc-object-lifetimes feature. > > > It causes a significant slowdown with Guile 1.8 for make test/make > > > doc, so the comparison is probably worse. > > > > > > This comes at the expense of increased memory usage (which reduces GC > > > overhead), so you'd have to test with my GUILE patches for a > > > completely honest comparison. I think the advantage of the decreased > > > GC overhead is minor, though. > > > > Right, I didn't think about this. However, I compiled without --enable- > > checking so the actual difference is just garbage collection happening > > between the files, which you think should be minor. Also "apples to > > apples" comparison means without your patches, because that's what > > users get. > > That's not what I meant. The GC between the files is the expensive > part, because it is done unconditionally and has to mark all the > reachable data, and sweep the entire heap. Dropping that (according > to commit e36b7c7ea98) is a 20% speedup, which we already collected > for GUILE 2.2, but not for 1.8. > > Due to the heap scaling bugs, removing the GC between files increases > the heap size with 1.8, and if you don't force GC, for a heap that is > 5x larger, you run GC 5x fewer times. The GC runs themselves involve a > larger heap, so this evens out, but you have to mark the reachable set > 5x fewer times. This is what I meant with a 'minor' advantage. So here are the numbers with debug-gc-object-lifetimes set to #f: 'make test': * User time (seconds): 397.05 * Elapsed (wall clock) time (h:mm:ss or m:ss): 3:22.10 * Maximum resident set size (kbytes): 897048 -> a 15% improvement over the default, at roughly 2.5x the heap times 4 processes (I think, not sure how exactly /usr/bin/time works) 'make doc': * User time (seconds): 2557.86 * Elapsed (wall clock) time (h:mm:ss or m:ss): 16:23.00 * Maximum resident set size (kbytes): 1004776 -> roughly 10% at 2.1x the heap > > > What does "switch to" mean precisely? Do you mean "remove support for > > > guile 1.8", "make 2.2 the default (when given the choice) but keep > > > supporting 1.8" ? > > > > Eventually, I think the target should be to "remove support for Guile > > 1.8". In particular once we provide binaries with Guile 2.2, that will > > be the one version we care about. > > From an engineering perspective, I like the idea of dropping 1.8, > because we can then simplify the C++ GC marking glue which is hard to > get right, but otherwise, it's still not very compelling. > Functionally, the only thing that seems really relevant is Unicode > string handling on the Scheme side. > > I wonder if it isn't more practical to fork guile 1.8, and stick it > into our tree as a submodule, and always build lilypond against the > in-tree version. We'd be up for the maintenance on the 1.8 branch, but > it might well be less work than keeping up with the churn that newer > GUILE versions bring us. > I had already replied that I don't like that option; it was always a given for me that LilyPond would move on. Guile 2.2 also makes binary distribution much nicer (because there no more shared srfi libraries,
Re: State of LilyPond with Guile 2.2
On Sun, Apr 11, 2021 at 4:24 PM Jonas Hahnfeld wrote: > > Am Sonntag, dem 11.04.2021 um 16:04 +0200 schrieb Han-Wen Nienhuys: > > On Sun, Apr 11, 2021 at 3:42 PM Jonas Hahnfeld via Discussions on > > LilyPond development wrote: > > > > > > All numbers are from my laptop running Arch Linux (with pango > > > downgraded to 1:1.48.2-1 to keep out the memory hogging in 1.48.3) > > > and > > > measured with "/usr/bin/time -v". I use commit fce156f219 from > > > https://gitlab.com/lilypond/lilypond/-/merge_requests/723 (see > > > User's > > > View) and the system-provided versions of Guile 1.8.8 and Guile > > > 2.2.6. > > > I don't test Guile 3.0 because it's not available as package, and > > > there > > > seem to be more changes needed to make it even work. > > > I configured the build with --enable-gs-api (because I'm interested > > > in > > > the time spent in LilyPond, not how fast the system can fork gs) > > > > > > > I assume this is also without extractpdfmark, which is also a CPU > > hog. > > Correct, all numbers are without extractpdfmark. > > > > The first set of experiments is about things that developers care > > > about, namely 'make test' and 'make doc'. For all runs, I used the > > > options "-j4 CPU_COUNT=4". > > > > > > 'make test' when compiled with Guile 1.8.8: > > > * User time (seconds): 520.68 > > > * Elapsed (wall clock) time (h:mm:ss or m:ss): 3:59.54 > > > > > > > IIRC, on my 4 CPU machine, the actual time running the regression .ly > > files is fairly low (about 1 minute). A lot of time is spent running > > the lilypond-book regression tests, which makes this difference all > > the more dramatic > > True, but the current way of testing lilypond-book is kind of the > worst-case scenario for LilyPond because all files are compiled > separately. So 37 times a startup time of 2 seconds is already the > majority of the slowdown. This may warrant some further investigation. Ideally, the shared snippet database makes it so we have to run lilypond only a few times. > > > * Maximum resident set size (kbytes): 372560 > > > 'make test' when compiled with Guile 2.2.6: > > > * User time (seconds): 710.35 > > > * Elapsed (wall clock) time (h:mm:ss or m:ss): 5:43.04 > > > * Maximum resident set size (kbytes): 372344 > > > > > > -> 40% slower when using Guile 2.2.6 during development > > > > > > > > > I didn't see you mention this explicitly: for apples to apples > > comparisons, you have to disable 'debug-gc-object-lifetimes feature. > > It causes a significant slowdown with Guile 1.8 for make test/make > > doc, so the comparison is probably worse. > > > > This comes at the expense of increased memory usage (which reduces GC > > overhead), so you'd have to test with my GUILE patches for a > > completely honest comparison. I think the advantage of the decreased > > GC overhead is minor, though. > > Right, I didn't think about this. However, I compiled without --enable- > checking so the actual difference is just garbage collection happening > between the files, which you think should be minor. Also "apples to > apples" comparison means without your patches, because that's what > users get. That's not what I meant. The GC between the files is the expensive part, because it is done unconditionally and has to mark all the reachable data, and sweep the entire heap. Dropping that (according to commit e36b7c7ea98) is a 20% speedup, which we already collected for GUILE 2.2, but not for 1.8. Due to the heap scaling bugs, removing the GC between files increases the heap size with 1.8, and if you don't force GC, for a heap that is 5x larger, you run GC 5x fewer times. The GC runs themselves involve a larger heap, so this evens out, but you have to mark the reachable set 5x fewer times. This is what I meant with a 'minor' advantage. > > > In my opinion, the numbers show that we *must have* compiled > > > bytecode > > > for user installations in order to reach acceptable performance > > > (memory > > > usage seems to be fine). And while compilation by invoking LilyPond > > > is > > > somewhat odd, it works and would be viable for the beginning. > > > > > > For development, I'm less convinced. Sure, 'make test' and 'make > > > doc' > > > get faster but the compilation itself takes a considerable amount > > > of > > > time. Moreover it is my understanding the Guile is notoriously bad > > > at > > > determining which files to recompile, in particular when macros are > > > involved. > > > > > > Personally, I would be ok with the moderate slowdown if that was > > > the > > > only thing preventing a hypothetic switch to Guile 2.2, and the > > > arising > > > > > > > > > What does "switch to" mean precisely? Do you mean "remove support for > > guile 1.8", "make 2.2 the default (when given the choice) but keep > > supporting 1.8" ? > > Eventually, I think the target should be to "remove support for Guile > 1.8". In particular once we provide binaries with Guile 2.2, that will > be the one version we care
Re: State of LilyPond with Guile 2.2
Am Sonntag, dem 11.04.2021 um 16:17 +0200 schrieb Werner LEMBERG: > > I'd like to give an update on the status of Guile 2.2, including > > some benchmark numbers. See the end for my conclusions, but I'd > > welcome comments on your take. > > Thanks for doing the comparison. > > This has certainly be mentioned somewhere, but what exactly is the > advantage for LilyPond of using Guile 2.x or 3.x? That LilyPond compiles again out-of-the-box (or continues to compile for distributions that didn't drop Guile 1.8 yet). And we get 64 bit executables on Windows, which is impossible with Guile 1.8. And in the process, hopefully get 64 bit executables for macOS and maybe even M1. > Your numbers look > as if staying with 1.8 would be the right thing currently, and we > should probably start to ponder how to integrate the Guile 1.8 bundle > into the LilyPond distribution... From a software engineering and maintenance view, this is a very very bad idea. signature.asc Description: This is a digitally signed message part
Re: State of LilyPond with Guile 2.2
Am Sonntag, dem 11.04.2021 um 16:04 +0200 schrieb Han-Wen Nienhuys: > On Sun, Apr 11, 2021 at 3:42 PM Jonas Hahnfeld via Discussions on > LilyPond development wrote: > > > > All numbers are from my laptop running Arch Linux (with pango > > downgraded to 1:1.48.2-1 to keep out the memory hogging in 1.48.3) > > and > > measured with "/usr/bin/time -v". I use commit fce156f219 from > > https://gitlab.com/lilypond/lilypond/-/merge_requests/723 (see > > User's > > View) and the system-provided versions of Guile 1.8.8 and Guile > > 2.2.6. > > I don't test Guile 3.0 because it's not available as package, and > > there > > seem to be more changes needed to make it even work. > > I configured the build with --enable-gs-api (because I'm interested > > in > > the time spent in LilyPond, not how fast the system can fork gs) > > > > I assume this is also without extractpdfmark, which is also a CPU > hog. Correct, all numbers are without extractpdfmark. > > The first set of experiments is about things that developers care > > about, namely 'make test' and 'make doc'. For all runs, I used the > > options "-j4 CPU_COUNT=4". > > > > 'make test' when compiled with Guile 1.8.8: > > * User time (seconds): 520.68 > > * Elapsed (wall clock) time (h:mm:ss or m:ss): 3:59.54 > > > > IIRC, on my 4 CPU machine, the actual time running the regression .ly > files is fairly low (about 1 minute). A lot of time is spent running > the lilypond-book regression tests, which makes this difference all > the more dramatic True, but the current way of testing lilypond-book is kind of the worst-case scenario for LilyPond because all files are compiled separately. So 37 times a startup time of 2 seconds is already the majority of the slowdown. > > > * Maximum resident set size (kbytes): 372560 > > 'make test' when compiled with Guile 2.2.6: > > * User time (seconds): 710.35 > > * Elapsed (wall clock) time (h:mm:ss or m:ss): 5:43.04 > > * Maximum resident set size (kbytes): 372344 > > > > -> 40% slower when using Guile 2.2.6 during development > > > > > I didn't see you mention this explicitly: for apples to apples > comparisons, you have to disable 'debug-gc-object-lifetimes feature. > It causes a significant slowdown with Guile 1.8 for make test/make > doc, so the comparison is probably worse. > > This comes at the expense of increased memory usage (which reduces GC > overhead), so you'd have to test with my GUILE patches for a > completely honest comparison. I think the advantage of the decreased > GC overhead is minor, though. Right, I didn't think about this. However, I compiled without --enable- checking so the actual difference is just garbage collection happening between the files, which you think should be minor. Also "apples to apples" comparison means without your patches, because that's what users get. > > > In my opinion, the numbers show that we *must have* compiled > > bytecode > > for user installations in order to reach acceptable performance > > (memory > > usage seems to be fine). And while compilation by invoking LilyPond > > is > > somewhat odd, it works and would be viable for the beginning. > > > > For development, I'm less convinced. Sure, 'make test' and 'make > > doc' > > get faster but the compilation itself takes a considerable amount > > of > > time. Moreover it is my understanding the Guile is notoriously bad > > at > > determining which files to recompile, in particular when macros are > > involved. > > > > Personally, I would be ok with the moderate slowdown if that was > > the > > only thing preventing a hypothetic switch to Guile 2.2, and the > > arising > > > > > What does "switch to" mean precisely? Do you mean "remove support for > guile 1.8", "make 2.2 the default (when given the choice) but keep > supporting 1.8" ? Eventually, I think the target should be to "remove support for Guile 1.8". In particular once we provide binaries with Guile 2.2, that will be the one version we care about. > > > question is really a matter of prioritizing: There are other items > > that > > need solutions before that switch could happen (in particular the > > release process for binary builds and documentation). What do > > others > > think? Or would you say that proper bytecode compilation is > > required > > before moving to Guile 2.2? (with no clear estimate how feasible > > that > > is and how long it would take) > > > > > How hard is it really to byte-compile the files during the build? With "guild compile"? I don't know. > Could we run lilypond during the compile and collect the .go files? Running the mentioned GUILE_AUTO_COMPILE during a non-cross-build is simple, but you'll need to remove the files yourself because Guile might not pick up changes to macro definitions which will be very odd to debug. You don't even need to collect them, that's only for installation. > Are they architecture independent (could we still cross-compile to > windows?) My understanding is that they
Re: State of LilyPond with Guile 2.2
> I'd like to give an update on the status of Guile 2.2, including > some benchmark numbers. See the end for my conclusions, but I'd > welcome comments on your take. Thanks for doing the comparison. This has certainly be mentioned somewhere, but what exactly is the advantage for LilyPond of using Guile 2.x or 3.x? Your numbers look as if staying with 1.8 would be the right thing currently, and we should probably start to ponder how to integrate the Guile 1.8 bundle into the LilyPond distribution... Werner
Re: State of LilyPond with Guile 2.2
On Sun, Apr 11, 2021 at 3:42 PM Jonas Hahnfeld via Discussions on LilyPond development wrote: > > All numbers are from my laptop running Arch Linux (with pango > downgraded to 1:1.48.2-1 to keep out the memory hogging in 1.48.3) and > measured with "/usr/bin/time -v". I use commit fce156f219 from > https://gitlab.com/lilypond/lilypond/-/merge_requests/723 (see User's > View) and the system-provided versions of Guile 1.8.8 and Guile 2.2.6. > I don't test Guile 3.0 because it's not available as package, and there > seem to be more changes needed to make it even work. > I configured the build with --enable-gs-api (because I'm interested in > the time spent in LilyPond, not how fast the system can fork gs) > I assume this is also without extractpdfmark, which is also a CPU hog. > The first set of experiments is about things that developers care > about, namely 'make test' and 'make doc'. For all runs, I used the > options "-j4 CPU_COUNT=4". > > 'make test' when compiled with Guile 1.8.8: > * User time (seconds): 520.68 > * Elapsed (wall clock) time (h:mm:ss or m:ss): 3:59.54 > IIRC, on my 4 CPU machine, the actual time running the regression .ly files is fairly low (about 1 minute). A lot of time is spent running the lilypond-book regression tests, which makes this difference all the more dramatic > * Maximum resident set size (kbytes): 372560 > 'make test' when compiled with Guile 2.2.6: > * User time (seconds): 710.35 > * Elapsed (wall clock) time (h:mm:ss or m:ss): 5:43.04 > * Maximum resident set size (kbytes): 372344 > > -> 40% slower when using Guile 2.2.6 during development > I didn't see you mention this explicitly: for apples to apples comparisons, you have to disable 'debug-gc-object-lifetimes feature. It causes a significant slowdown with Guile 1.8 for make test/make doc, so the comparison is probably worse. This comes at the expense of increased memory usage (which reduces GC overhead), so you'd have to test with my GUILE patches for a completely honest comparison. I think the advantage of the decreased GC overhead is minor, though. In my opinion, the numbers show that we *must have* compiled bytecode > for user installations in order to reach acceptable performance (memory > usage seems to be fine). And while compilation by invoking LilyPond is > somewhat odd, it works and would be viable for the beginning. > > For development, I'm less convinced. Sure, 'make test' and 'make doc' > get faster but the compilation itself takes a considerable amount of > time. Moreover it is my understanding the Guile is notoriously bad at > determining which files to recompile, in particular when macros are > involved. > > Personally, I would be ok with the moderate slowdown if that was the > only thing preventing a hypothetic switch to Guile 2.2, and the arising > What does "switch to" mean precisely? Do you mean "remove support for guile 1.8", "make 2.2 the default (when given the choice) but keep supporting 1.8" ? > question is really a matter of prioritizing: There are other items that > need solutions before that switch could happen (in particular the > release process for binary builds and documentation). What do others > think? Or would you say that proper bytecode compilation is required > before moving to Guile 2.2? (with no clear estimate how feasible that > is and how long it would take) > How hard is it really to byte-compile the files during the build? Could we run lilypond during the compile and collect the .go files? Are they architecture independent (could we still cross-compile to windows?) Sorry for the long and densely packed message, and thanks for reading > to the end! > no worries, thanks for taking the effort to sort this out! -- Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
State of LilyPond with Guile 2.2
Hi all,
I'd like to give an update on the status of Guile 2.2, including some
benchmark numbers. See the end for my conclusions, but I'd welcome
comments on your take.
Setup
-
All numbers are from my laptop running Arch Linux (with pango
downgraded to 1:1.48.2-1 to keep out the memory hogging in 1.48.3) and
measured with "/usr/bin/time -v". I use commit fce156f219 from
https://gitlab.com/lilypond/lilypond/-/merge_requests/723 (see User's
View) and the system-provided versions of Guile 1.8.8 and Guile 2.2.6.
I don't test Guile 3.0 because it's not available as package, and there
seem to be more changes needed to make it even work.
I configured the build with --enable-gs-api (because I'm interested in
the time spent in LilyPond, not how fast the system can fork gs).
Developer's View
The first set of experiments is about things that developers care
about, namely 'make test' and 'make doc'. For all runs, I used the
options "-j4 CPU_COUNT=4".
'make test' when compiled with Guile 1.8.8:
* User time (seconds): 520.68
* Elapsed (wall clock) time (h:mm:ss or m:ss): 3:59.54
* Maximum resident set size (kbytes): 372560
'make test' when compiled with Guile 2.2.6:
* User time (seconds): 710.35
* Elapsed (wall clock) time (h:mm:ss or m:ss): 5:43.04
* Maximum resident set size (kbytes): 372344
-> 40% slower when using Guile 2.2.6 during development
'make doc' when compiled with Guile 1.8.8:
* User time (seconds): 2915.35
* Elapsed (wall clock) time (h:mm:ss or m:ss): 18:17.24
* Maximum resident set size (kbytes): 467768
'make doc' when compiled with Guile 2.2.6:
* User time (seconds): 3258.72
* Elapsed (wall clock) time (h:mm:ss or m:ss): 20:37.10
* Maximum resident set size (kbytes): 351560
-> around 10% slower when using Guile 2.2.6
Both times can be improved by running
$ touch test.ly
$ GUILE_AUTO_COMPILE=1 ./out/bin/lilypond test.ly
to compile the Guile modules (before you ask: no, guild compile does
not work; see the conclusion). This itself takes 2 minutes and is
unfortunately single-threaded.
Afterwards, 'make test' takes:
* User time (seconds): 494.85
* Elapsed (wall clock) time (h:mm:ss or m:ss): 3:48.45
* Maximum resident set size (kbytes): 372056
and 'make doc' takes:
* User time (seconds): 2929.24
* Elapsed (wall clock) time (h:mm:ss or m:ss): 18:34.07
* Maximum resident set size (kbytes): 351432
which is more or less level with the numbers using Guile 1.8.8.
User's View
---
Most LilyPond users care about installed versions and their own scores,
so I ran 'make install' (with DESTDIR, actually, because I need two
installation with Guile 2.2) and then renamed the build directories for
extra caution. I only give the wall clock time because LilyPond is
single-threaded, so user time is the same.
The first test, simple.ly, is just a "{ c'1 }" to measure startup time.
With Guile 1.8.8:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.90
* Maximum resident set size (kbytes): 132300
With Guile 2.2.6:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:02.85
* Maximum resident set size (kbytes): 102760
Now that's a serious regression and clearly unacceptable for users
trying out LilyPond. To remedy, I install the compiled bytecode from
above (stored in out/share/lilypond/current/guile/) with
$ mkdir -p /path/to/install/lib/lilypond/2.23.2/ccache/lily
$ for f in $(find out/share/lilypond/current/guile/ -name "*.go"); do
cp $f /path/to/install/lib/lilypond/2.23.2/ccache/lily/$(basename $f
.scm.go).go; done
(note: Guile auto-compiles to .scm.go, but only looks for .go files)
Then, I get:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.84
* Maximum resident set size (kbytes): 95628
which is *faster* than with Guile 1.8.8
Some more serious scores:
input/regression/mozart-hrn-3.ly (4 pages, some 200 bars)
With Guile 1.8.8:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:03.13
* Maximum resident set size (kbytes): 276988
With Guile 2.2.6:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.97
* Maximum resident set size (kbytes): 169216
With Guile 2.2.6 and compiled bytecode:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:03.75
* Maximum resident set size (kbytes): 188632
First movement from BWV1060R (one of my entered scores, 7 pages, some
100 bars in two voices)
With Guile 1.8.8:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:04.03
* Maximum resident set size (kbytes): 390452
With Guile 2.2.6:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:07.41
* Maximum resident set size (kbytes): 189364
With Guile 2.2.6 and compiled bytecode:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:05.05
* Maximum resident set size (kbytes): 189336
Missa Dum sacrum mysterium (from
https://lists.gnu.org/archive/html/lilypond-user/2016-11/msg00700.html
and run through convert-ly)
With Guile 1.8.8:
* Elapsed (wall clock) time (h:mm:ss or m:ss): 0:27.15
* Maximum resident set size (kbytes): 969572
With
PATCHES - Countdown for April 11th
Hello, Here is the current patch countdown list. The next countdown will be on April 13th. A list of all merge requests can be found here: https://gitlab.com/lilypond/lilypond/-/merge_requests?sort=label_priority Push: !718 musicxml2ly: sort additional definitions - Han-Wen Nienhuys https://gitlab.com/lilypond/lilypond/-/merge_requests/718 !717 scipts: use basename for program_name - Han-Wen Nienhuys https://gitlab.com/lilypond/lilypond/-/merge_requests/717 !716 configure: Correct and simplify search for guile-devel - Jonas Hahnfeld https://gitlab.com/lilypond/lilypond/-/merge_requests/716 Countdown: No patches in Countdown at this time. Review: !721 Use the format procedure from (ice-9 format) for logging - Jean Abou-Samra https://gitlab.com/lilypond/lilypond/-/merge_requests/721 !720 Mark tremolos on cross-staff beams as cross-staff - Jean Abou-Samra https://gitlab.com/lilypond/lilypond/-/merge_requests/720 !719 assorted lilylib cleanups - Han-Wen Nienhuys https://gitlab.com/lilypond/lilypond/-/merge_requests/719 New: No patches in New at this time. Waiting: No patches in Waiting at this time. *** Regards, James OpenPGP_0x111CA298156CCB85.asc Description: application/pgp-keys OpenPGP_signature Description: OpenPGP digital signature
