Re: Blockers for Guile 2.2
Am Sonntag, dem 27.02.2022 um 01:04 +0100 schrieb Han-Wen Nienhuys: > On Sat, Feb 26, 2022 at 2:02 PM Jonas Hahnfeld wrote: > > > while we make work slower for folks that work on large > > > scores and can afford to side-install Guile 1.8. It also makes > > > development slower for ourselves. Yes, that means some of us will > > > develop on a different platform than many users. This has been the > > > case since we started supporting OSX and Windows. > > > > I slightly disagree, running on a different platform is not the same as > > running with a completely different set of dependencies. And we know > > Guile 2.2 is completely different from 1.8. > > > > Everyone who is passionate about Guile 2.2 can develop against Guile 2.2. > > > > So to be clear here: You would release official binaries with Guile 2.2 > > and rely on a subset of developers to fix the bugs while you make it > > clear that you don't want to do that? Why would anybody accept that > > job? > > I never said I don't want to fix Guile 2.2 bugs, and you should know > as I spent lots and lots of time debugging #6218. I also said I > support moving CI to 2.2, so any MR would pass against 2.2. > > I am just asking to not drop 1.8 support. Ok, so let me try to understand the motivation here: It's not for the official binaries, where you want to move away from GUB so it would be Guile 2.2 only. Neither about distributions, which are looking into or have already switched to Guile 2.2. This leaves "power users" that compile LilyPond on their own, with Guile 1.8. Not sure if it's worth complicating our development lives for what will be a transition period anyway. For your local development, if you care about things working with Guile 2.2, you'll have to test at least the final version of a patch with both versions. So I can only assume that you expect a time saving to come from hacking with Guile 1.8 during prototype? > Most of the work we do isn't actually about Guile anyway, > > $ git log --since 2022/01/01 | grep ^commit|wc > 257 514 12336 > $ git log --grep=[Gg]uile --since 2022/01/01 | grep ^commit|wc > 7 14 336 What you're showing is that most commit messages don't mention Guile. IMO the more relevant metric is how many commits touch scm/, in comparison to those that are not purely updates of the documentation: $ git log --oneline --since 2022-01-01 -- flower/ lily/ ly/ scm/ | wc -l 125 $ git log --oneline --since 2022-01-01 -- scm/ | wc -l 52 One third. Since 2021-01-01 that ratio is closer to every second. signature.asc Description: This is a digitally signed message part
Re: Blockers for Guile 2.2
Le 27/02/2022 à 01:04, Han-Wen Nienhuys a écrit : On Sat, Feb 26, 2022 at 2:02 PM Jonas Hahnfeld wrote: while we make work slower for folks that work on large scores and can afford to side-install Guile 1.8. It also makes development slower for ourselves. Yes, that means some of us will develop on a different platform than many users. This has been the case since we started supporting OSX and Windows. I slightly disagree, running on a different platform is not the same as running with a completely different set of dependencies. And we know Guile 2.2 is completely different from 1.8. Everyone who is passionate about Guile 2.2 can develop against Guile 2.2. So to be clear here: You would release official binaries with Guile 2.2 and rely on a subset of developers to fix the bugs while you make it clear that you don't want to do that? Why would anybody accept that job? I never said I don't want to fix Guile 2.2 bugs, and you should know as I spent lots and lots of time debugging #6218. I also said I support moving CI to 2.2, so any MR would pass against 2.2. I am just asking to not drop 1.8 support. Most of the work we do isn't actually about Guile anyway, $ git log --since 2022/01/01 | grep ^commit|wc 257 514 12336 $ git log --grep=[Gg]uile --since 2022/01/01 | grep ^commit|wc 7 14 336 I agree that the version-specific code we have (cond-expand and GUILEV2) isn't all that much. On the other hand, I would be glad to be able to use Guile 2 features, such as Unicode support, when and unless, (srfi srfi-71) (let and let* accepting multiple values), S-expression comments, scm_c_bind_keyword_arguments, and a few others. Now that my principal concern with the sustainability of Guile 2 binaries (shipping bytecode with them) is cleared, I have mixed feelings about when to leave Guile 1 behind. I understand the desire to keep fast development cycles, but I think the cognitive burden of wondering whether your code will work on both versions is not desirable in the long run regardless of CI issues. In my previous post I showed that compiling all Scheme files can be done in 20s with Guile 2 (so a few seconds for a single file), and 4s with Guile 3 (so near instant for one file). Would that address your concern with compilation speed? Jean
Re: Blockers for Guile 2.2
On Sat, Feb 26, 2022 at 2:02 PM Jonas Hahnfeld wrote: > > while we make work slower for folks that work on large > > scores and can afford to side-install Guile 1.8. It also makes > > development slower for ourselves. Yes, that means some of us will > > develop on a different platform than many users. This has been the > > case since we started supporting OSX and Windows. > > I slightly disagree, running on a different platform is not the same as > running with a completely different set of dependencies. And we know > Guile 2.2 is completely different from 1.8. > > Everyone who is passionate about Guile 2.2 can develop against Guile 2.2. > > So to be clear here: You would release official binaries with Guile 2.2 > and rely on a subset of developers to fix the bugs while you make it > clear that you don't want to do that? Why would anybody accept that > job? I never said I don't want to fix Guile 2.2 bugs, and you should know as I spent lots and lots of time debugging #6218. I also said I support moving CI to 2.2, so any MR would pass against 2.2. I am just asking to not drop 1.8 support. Most of the work we do isn't actually about Guile anyway, $ git log --since 2022/01/01 | grep ^commit|wc 257 514 12336 $ git log --grep=[Gg]uile --since 2022/01/01 | grep ^commit|wc 7 14 336 -- Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
Re: Blockers for Guile 2.2
[David]
I think where we ultimately want to end up is to have Guile use
optimisation for code loaded from .scm files (which should likely use
byte compilation) while not doing so for Guile code and definitions
invoked from .ly files with # and $ because those more likely than not
are not part of performance-relevant inner loops (and if they do, moving
them to .scm or otherwise specifically marking them might be a
reasonable requirement).
Note that "ultimately" does not mean that this may be the best strategy
right now. But LilyPond documents contain a huge amount of ad-hoc
Scheme code introduced with # (often as simple as a numeric constant or
quoted code that may warrant special-casing, and that actually _is_
special-cased within #{ #} passages already to avoid having to form
closures for it).
For one thing I think we definitely do not want to implement
our own caching of byte-compiled code from .ly files. Because
Guile has no notion of dependency tracking, which means that
if a macro changes, it will not detect that it needs to recompile
its call sites as it will just have expanded the macro and
forgot about the origin of the expanded code. Actually I think
it depends on all bindings from the current module, so it would
be problematic even without macros, though I am not sure about that.
Nevertheless, it would be glaringly user-unfriendly to introduce
caching that does not update automatically for our own user base,
in my opinion.
With that in mind, whether to compile the code before executing
it (thus recompiling at every compilation of the .ly file) or to
simply primitive-eval it is a matter of seeing whether the
byte-compilation is fast enough and whether it actually
enhances usability. Speed should really not be an issue for
code from .ly files (whether those from LilyPond or the users').
On the other hand, byte-compiled code tends to give better
error messages when it fails. I have not experimented yet
(usually I try to do the experiments before I write to the
list, but I'm swamped right now), though I think it shouldn't
be too hard a decision.
The status quo for now is that LilyPond's .scm files are
run by the virtual machine whereas Scheme code in # and $
is run through the evaluator (i.e., primitive-eval). That
is in fact exactly what you describe as "where we ultimately
want to end up".
Of course your timings are quite encouraging here for seeing a path
forward but working out the details of what combinations make best sense
both in the short and the long run is likely going to need quite more
experimentation.
At least, I think the status quo is acceptable. Whether other
strategies can turn out better will be interesting to see,
but can be done in a later step.
[Jonas]
He, I always thought auto-compilation didn't optimize! now don't
tell me Guile also applies optimizations while just reading and
supposedly interpreting code...
I don't think it does. At least, you don't usually call eval or
primitive-eval on code to run it repeatedly, just for one-shot
code, so I don't see the sense it would make. Also, it seems
to me that the Guile developers are much more focused on compilation.
[ skipping over the part regarding Guile 3, since I don't think it's
relevant here ]
Perhaps I should have changed the title, but I do think it
is relevant -- it gives hope for a state where development
cycles will be easier. When we want Guile 3 is another question.
I'm in favor of not making the move to Guile 2 wait more
than it already has, but I wonder if at some point in this release
cycle (by which I mean before the next stable release) we will want
to switch to Guile 3. Of course, that will depend on how easy
it turns out to fix issues like
https://gitlab.com/lilypond/lilypond/-/merge_requests/1230#note_855980027
Yes, it looks like we should do this! On the patch, I think it would be
better to apply the strategy from module/scripts/compile.scm and just
get all available-optimizations from the concatenation of tree-il-
default-optimization-options and cps-default-optimization-options
instead of hard-coding the list.
Yes, that is what should be done if we decide to turn off
them all. I still have to do detailed benchmarking to see
which optimizations might save a few percents and which are
expensive, to decide on the final list. Again, that is just
waiting on me having more time (but of course, speaking to
everyone, feel free to beat me to it and experiment by
yourself).
[Luca]
Jean,
how many times did you run these tests?
Eyeballing your numbers it seems there's effectively no difference in
execution time opt/no-opt and 2.2/3.0.
Is the 5% a stable figure, or is it just a one-sample thing?
It's a one-sample thing. I was just trying to get orders
of magnitude (for Guile 3 it's 1m30 vs. 4s, no need for
precise benchmarks to see which is faster :-).
Would it be a passable inference that the reason the optimizer has
effectively no measurable impact in either
Repeat count as "ij" in Gregorian chant
Please see merge request 1235 [1] for an image of a repeat count for Gregorian chant that is generated according to \repeat in the ly code. I learned of this notation only recently and I don't expect to use it myself except as a way to test LilyPond's data flow. Is anyone here interested in specifying the minimum features that this counter must have to deserve a place in LilyPond? I already know about the conflict between using BarLine or BreathingSign for divisions, and I hope to resolve that eventually. What about the counter per se? Thanks and regards, — Dan [1] https://gitlab.com/lilypond/lilypond/-/merge_requests/1235
Re: Blockers for Guile 2.2
Jean, how many times did you run these tests? Eyeballing your numbers it seems there's effectively no difference in execution time opt/no-opt and 2.2/3.0. Is the 5% a stable figure, or is it just a one-sample thing? Would it be a passable inference that the reason the optimizer has effectively no measurable impact in either runtime is that the scheme source code runs are comparatively short and contain large amounts of code the optimizer can't change? I'm imagining that if your source is largely an alternation of scheme language built-ins (or scheme library code) interspersed with fairly frequent calls into lilypond's brain, the optimizer can't do much about the latter. At the same time, you might be sitting in front of gains coming from making these API calls more efficient, which could be interesting (albeit largely orthogonal to the discussion at hand). I'm not sure how division of cost works out, if there is overhead in preparing for invoking the callbacks vs executing them, for example. I guess insight in that could help focus effort, if any was warranted. I thought the -O0 compilation time in Guile 3.0 was _really_ cool, I guess it indicates the front-end of the 3.x compiler is vastly more efficient? Seems like it could be an interesting way forward for the dev group to run 3.x with -O0 for iteration cycles, and then do what David is saying to ship the scm file with optimizations on and have the in-score Scheme be just built -O0. Reading briefly the message you posted, it seems -O1 might be a better way to go still, which might regain a teeny tiny bit of speed at a potentially very modest cost (say even if your 3.5s become 4 or 5, you still come out with a net win compared to 2.2's 20s). I agree these results are a very cool finding. L On Sat, Feb 26, 2022 at 2:47 PM Jean Abou Samra wrote: > Le 26/02/2022 à 13:51, Han-Wen Nienhuys a écrit : > > The Scheme compilation felt much slower, and for C++ ccache takes away > > a lot of the pain of recompiles. It also appears to be > > single-threaded? I admit not having timed it in detail. > > OK, I have very good news regarding compilation speed. > Tests are done with > > rm -rf out/share/lilypond/current/guile/ && time out/bin/lilypond > <...snip...> > For one thing, Guile's optimization make about zero difference for the > speed of the resulting LilyPond executable. For another, disabling > optimizations in Guile 2 already results in a good speedup (1min > to 20s), and while Guile 3 is even slower than Guile 2 at the default > optimization level (1m30), with optimizations disabled it becomes > near instant (3.5s). > > Guile 3 being far better at compilation speed with zero optimizations > apparently comes from what is described in > > http://wingolog.org/archives/2020/06/03/a-baseline-compiler-for-guile > > On the other hand, it does look like Guile 3 is a little slower > than Guile 2 on execution time (5%). > > What do you think? > > Jean >
Re: Blockers for Guile 2.2
Am Samstag, dem 26.02.2022 um 14:47 +0100 schrieb Jean Abou Samra: > Le 26/02/2022 à 13:51, Han-Wen Nienhuys a écrit : > > The Scheme compilation felt much slower, and for C++ ccache takes away > > a lot of the pain of recompiles. It also appears to be > > single-threaded? I admit not having timed it in detail. > > OK, I have very good news regarding compilation speed. > Tests are done with > > rm -rf out/share/lilypond/current/guile/ && time out/bin/lilypond > > (I have GUILE_AUTO_COMPILE=1 in my environment.) > > * master and Guile 2 > > real 0m58,877s > user 0m58,773s > sys 0m0,124s > > Execution time on MSDM.ly: > > real 0m18,870s > user 0m18,727s > sys 0m0,470s > > > * Guile 2, with attached patch disabling all optimizations: He, I always thought auto-compilation didn't optimize! now don't tell me Guile also applies optimizations while just reading and supposedly interpreting code... > real 0m16,791s > user 0m16,700s > sys 0m0,056s > > Execution time on MSDM.ly: > > real 0m18,702s > user 0m18,517s > sys 0m0,509s [ skipping over the part regarding Guile 3, since I don't think it's relevant here ] > For one thing, Guile's optimization make about zero difference for the > speed of the resulting LilyPond executable. For another, disabling > optimizations in Guile 2 already results in a good speedup (1min > to 20s), [...]. > > [...] > > What do you think? Yes, it looks like we should do this! On the patch, I think it would be better to apply the strategy from module/scripts/compile.scm and just get all available-optimizations from the concatenation of tree-il- default-optimization-options and cps-default-optimization-options instead of hard-coding the list. Jonas signature.asc Description: This is a digitally signed message part
Re: Blockers for Guile 2.2
Jean Abou Samra writes:
> For one thing, Guile's optimization make about zero difference for the
> speed of the resulting LilyPond executable. For another, disabling
> optimizations in Guile 2 already results in a good speedup (1min
> to 20s), and while Guile 3 is even slower than Guile 2 at the default
> optimization level (1m30), with optimizations disabled it becomes
> near instant (3.5s).
>
> Guile 3 being far better at compilation speed with zero optimizations
> apparently comes from what is described in
>
> http://wingolog.org/archives/2020/06/03/a-baseline-compiler-for-guile
>
> On the other hand, it does look like Guile 3 is a little slower
> than Guile 2 on execution time (5%).
>
> What do you think?
I think where we ultimately want to end up is to have Guile use
optimisation for code loaded from .scm files (which should likely use
byte compilation) while not doing so for Guile code and definitions
invoked from .ly files with # and $ because those more likely than not
are not part of performance-relevant inner loops (and if they do, moving
them to .scm or otherwise specifically marking them might be a
reasonable requirement).
Note that "ultimately" does not mean that this may be the best strategy
right now. But LilyPond documents contain a huge amount of ad-hoc
Scheme code introduced with # (often as simple as a numeric constant or
quoted code that may warrant special-casing, and that actually _is_
special-cased within #{ #} passages already to avoid having to form
closures for it).
Of course your timings are quite encouraging here for seeing a path
forward but working out the details of what combinations make best sense
both in the short and the long run is likely going to need quite more
experimentation.
--
David Kastrup
Re: Blockers for Guile 2.2
Le 26/02/2022 à 13:51, Han-Wen Nienhuys a écrit :
The Scheme compilation felt much slower, and for C++ ccache takes away
a lot of the pain of recompiles. It also appears to be
single-threaded? I admit not having timed it in detail.
OK, I have very good news regarding compilation speed.
Tests are done with
rm -rf out/share/lilypond/current/guile/ && time out/bin/lilypond
(I have GUILE_AUTO_COMPILE=1 in my environment.)
* master and Guile 2
real 0m58,877s
user 0m58,773s
sys 0m0,124s
Execution time on MSDM.ly:
real 0m18,870s
user 0m18,727s
sys 0m0,470s
* Guile 2, with attached patch disabling all optimizations:
real 0m16,791s
user 0m16,700s
sys 0m0,056s
Execution time on MSDM.ly:
real 0m18,702s
user 0m18,517s
sys 0m0,509s
* Guile 3, with fixes from
https://gitlab.com/lilypond/lilypond/-/merge_requests/1230
real 1m25,302s
user 1m27,644s
sys 0m1,543s
Execution time on MSDM.ly:
real 0m19,533s
user 0m19,268s
sys 0m0,549s
* Guile 3, with https://gitlab.com/lilypond/lilypond/-/merge_requests/1230
plus second attached patch disabling optimizations:
real 0m3,538s
user 0m3,510s
sys 0m0,108s
(!!)
Execution time on MSDM.ly:
real 0m19,571s
user 0m19,387s
sys 0m0,573s
For one thing, Guile's optimization make about zero difference for the
speed of the resulting LilyPond executable. For another, disabling
optimizations in Guile 2 already results in a good speedup (1min
to 20s), and while Guile 3 is even slower than Guile 2 at the default
optimization level (1m30), with optimizations disabled it becomes
near instant (3.5s).
Guile 3 being far better at compilation speed with zero optimizations
apparently comes from what is described in
http://wingolog.org/archives/2020/06/03/a-baseline-compiler-for-guile
On the other hand, it does look like Guile 3 is a little slower
than Guile 2 on execution time (5%).
What do you think?
Jean
From 864143100d232e663b8f0aa8e236b9187d1a48bc Mon Sep 17 00:00:00 2001
From: Jean Abou Samra
Date: Sat, 26 Feb 2022 13:43:37 +0100
Subject: [PATCH] Disable optimizations (Guile 2)
---
lily/guile-init.cc | 18 ++
lily/include/lily-imports.hh | 3 +++
lily/lily-imports.cc | 3 +++
3 files changed, 24 insertions(+)
diff --git a/lily/guile-init.cc b/lily/guile-init.cc
index 2dfae879e4..8d9d1d4bcf 100644
--- a/lily/guile-init.cc
+++ b/lily/guile-init.cc
@@ -59,6 +59,24 @@ ly_init_ly_module ()
for (vsize i = scm_init_funcs_->size (); i--;)
(scm_init_funcs_->at (i)) ();
+#if GUILEV2
+ SCM opts = SCM_EOL;
+ for (const char *name : {
+"precolor-calls?", "rotate-loops?", "licm?", "specialize-numbers?",
+ "resolve-self-references?", "type-fold?", "cse?", "peel-loops?",
+ "prune-bailouts?", "elide-values?", "specialize-primcalls?", "inline-constructors?",
+ "contify?", "prune-top-level-scopes?", "eliminate-dead-code?", "simplify?",
+ "partial-eval?"
+ })
+{
+ SCM kwd = scm_symbol_to_keyword (ly_symbol2scm (name));
+ opts = scm_cons (kwd, scm_cons (SCM_BOOL_F, opts));
+}
+
+ Guile_user::f_auto_compilation_options = opts;
+#endif
+
+
Cpu_timer timer;
if (is_loglevel (LOG_DEBUG))
{
diff --git a/lily/include/lily-imports.hh b/lily/include/lily-imports.hh
index 6b4ad24880..00d2bd5933 100644
--- a/lily/include/lily-imports.hh
+++ b/lily/include/lily-imports.hh
@@ -28,6 +28,9 @@ extern Scm_module module;
typedef Module_variable Variable;
extern Variable apply;
+#if GUILEV2
+extern Variable f_auto_compilation_options;
+#endif
extern Variable equal;
#if GUILEV2
extern Variable f_default_port_encoding;
diff --git a/lily/lily-imports.cc b/lily/lily-imports.cc
index ca4a3aa015..09d4abe6d4 100644
--- a/lily/lily-imports.cc
+++ b/lily/lily-imports.cc
@@ -24,6 +24,9 @@ namespace Guile_user
Scm_module module ("guile-user");
Variable apply ("apply");
+#if GUILEV2
+Variable f_auto_compilation_options ("%auto-compilation-options");
+#endif
Variable equal ("=");
Variable less ("<");
Variable plus ("+");
--
2.32.0
From 0852cc6277a911da4237b2bf013a45988db8b5d0 Mon Sep 17 00:00:00 2001
From: Jean Abou Samra
Date: Sat, 26 Feb 2022 14:20:37 +0100
Subject: [PATCH] Guile 3: set optimization level to 0
---
lily/guile-init.cc | 3 +++
lily/include/lily-imports.hh | 8
lily/lily-imports.cc | 7 +++
3 files changed, 18 insertions(+)
diff --git a/lily/guile-init.cc b/lily/guile-init.cc
index 2dfae879e4..641d0fc657 100644
--- a/lily/guile-init.cc
+++ b/lily/guile-init.cc
@@ -59,6 +59,8 @@ ly_init_ly_module ()
for (vsize i = scm_init_funcs_->size (); i--;)
(scm_init_funcs_->at (i)) ();
+ Compile::default_optimization_level (to_scm (0));
+
Cpu_timer timer;
if (is_loglevel (LOG_DEBUG))
{
@@ -78,6 +80,7 @@ void
ly_c_init_guile ()
{
Guile_user::module.import ();
+ Compile::module.import ();
Lily::module.boot (ly_init_ly_module);
Re: Blockers for Guile 2.2
Am Samstag, dem 26.02.2022 um 13:51 +0100 schrieb Han-Wen Nienhuys: > On Wed, Feb 23, 2022 at 12:36 PM Jonas Hahnfeld wrote: > > > > > * The concern over CI minutes seems like it's the least important: we > > > can buy more computing power (I'm happy to sponsor), and is the > > > duration of CI much of a concern today? > > > > In the past, you've complained the loudest about slow CI builds for > > merging changes... > > A build for 2.2 and 1.8 could be run in parallel, I think? Yes, but it's still going to be slow because we have exactly one very fast runner. > > > I don't think we have to do the doc build across both 1.8 and 2.2, for > > > example. > > > > I think we do because a full doc build provides coverage and stress > > testing that is unachieved by the regression tests. > > The doc build is exactly the same as the regtest: a lot of small files > that are rendered mostly in per-system mode (rather than per page). > It's just more of the same, so you get better coverage for small > windows of time where objects aren't GC protected. > > Also, you make it seem as if perfect coverage is the only alternative > to scrapping the code altogether. I think that is a false dilemma. > > > > * We're talking about the impact of (the lack of) byte compilation on > > > users, but we haven't discussed the impact on ourselves: the startup > > > slowness affects our every debug/edit cycle, and byte compilation is > > > impractical because we change scm files often (by virtue of checking > > > out other branches). > > > > The same happens for C++ files, you also have to recompile. But it's > > true that editing scm files isn't for free anymore. > > The Scheme compilation felt much slower, and for C++ ccache takes away > a lot of the pain of recompiles. It also appears to be > single-threaded? I admit not having timed it in detail. Yes, GUILE_AUTO_COMPILE=1 is single-threaded which is why it will never be the default. > > > If we need to kill 1.8 support because it blocks > > > something important, then so be it, but given the impact on lilypond > > > development, I'd try to postpone it if possible. > > > > So, you propose that development continues on Guile 1.8, but we > > advertise that we consider Guile 2.2 ready for production use? I don't > > think that makes for a great user experience if we let them find > > problems... Also "postpone" up to what point? > > If we drop 1.8 support now, we can drop just a tiny sliver of > complexity (places marked GUILEV2 and guile-2 in C++ and Scheme > respectively), I think you're underestimating the gain here. > while we make work slower for folks that work on large > scores and can afford to side-install Guile 1.8. It also makes > development slower for ourselves. Yes, that means some of us will > develop on a different platform than many users. This has been the > case since we started supporting OSX and Windows. I slightly disagree, running on a different platform is not the same as running with a completely different set of dependencies. And we know Guile 2.2 is completely different from 1.8. > Everyone who is passionate about Guile 2.2 can develop against Guile 2.2. So to be clear here: You would release official binaries with Guile 2.2 and rely on a subset of developers to fix the bugs while you make it clear that you don't want to do that? Why would anybody accept that job? signature.asc Description: This is a digitally signed message part
Re: Blockers for Guile 2.2
On Wed, Feb 23, 2022 at 12:36 PM Jonas Hahnfeld wrote: > > * I'm worried that introducing a new version of lilypond that is > > significantly slower than older versions creates an incentive for > > users to stay on older versions. > > > > * I grepped our source code for "guile-2" (scm) and GUILEV2, but the > > divergence of code paths seems pretty minor. Sure, it's inconvenient > > to have the odd conditional or limit ourselves to what is both in 1.8 > > and 2.2, but I'd rather see us drop 1.8 support once we see an > > obstacle that we cannot paper over. > > The problem is, in order to "see an obstacle", we always have to invest > to test every single change with Guile 1.8 and 2.2 during development, > which will be a significant time sink. > > > * The concern over CI minutes seems like it's the least important: we > > can buy more computing power (I'm happy to sponsor), and is the > > duration of CI much of a concern today? > > In the past, you've complained the loudest about slow CI builds for > merging changes... A build for 2.2 and 1.8 could be run in parallel, I think? > > I don't think we have to do the doc build across both 1.8 and 2.2, for > > example. > > I think we do because a full doc build provides coverage and stress > testing that is unachieved by the regression tests. The doc build is exactly the same as the regtest: a lot of small files that are rendered mostly in per-system mode (rather than per page). It's just more of the same, so you get better coverage for small windows of time where objects aren't GC protected. Also, you make it seem as if perfect coverage is the only alternative to scrapping the code altogether. I think that is a false dilemma. > > * We're talking about the impact of (the lack of) byte compilation on > > users, but we haven't discussed the impact on ourselves: the startup > > slowness affects our every debug/edit cycle, and byte compilation is > > impractical because we change scm files often (by virtue of checking > > out other branches). > > The same happens for C++ files, you also have to recompile. But it's > true that editing scm files isn't for free anymore. The Scheme compilation felt much slower, and for C++ ccache takes away a lot of the pain of recompiles. It also appears to be single-threaded? I admit not having timed it in detail. > > If we need to kill 1.8 support because it blocks > > something important, then so be it, but given the impact on lilypond > > development, I'd try to postpone it if possible. > > So, you propose that development continues on Guile 1.8, but we > advertise that we consider Guile 2.2 ready for production use? I don't > think that makes for a great user experience if we let them find > problems... Also "postpone" up to what point? If we drop 1.8 support now, we can drop just a tiny sliver of complexity (places marked GUILEV2 and guile-2 in C++ and Scheme respectively), while we make work slower for folks that work on large scores and can afford to side-install Guile 1.8. It also makes development slower for ourselves. Yes, that means some of us will develop on a different platform than many users. This has been the case since we started supporting OSX and Windows. Everyone who is passionate about Guile 2.2 can develop against Guile 2.2. I'm not opposed to dropping 1.8, as long as we get something substantial in return for it. For example, we might be able to drop all of the GC marking code if we defer to BDWGC for all memory management. That is an attractive proposition, but I'd like to see a prototype before we actually go there. > > * In the discussion, we've been treating "keeping GUB alive" and > > "supporting GUILE 1.8" as equivalent, but is that really the case? We > > can't have GUILE 1.8 for 64-bit windows, but for OSX/Linux, it would > > be an option with the new release scripts too? > > No, Guile 1.8 has mandatory shared libraries for some srfi modules. > There are tricks you can play to make it work with an otherwise static > build, but they are really ugly and certainly not something I think > should be done for production. ack. -- Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
