Reasoning behind convert-ly rule for stable update?
Hi, I've postponed doing the big convert-ly update beyond 2.18 version numbers since it's bad for merge conflict resolution (there are literally thousands of files affected). Does anybody know _why_ convert-ly updates at least to the last stable version number even if nothing else has been changed? For one thing, it might be a question of efficiency, granted. Going through every conversion rule from history's end to today repeatedly is likely to be expensive. But for another, it loses significant information. Is there some way we can retain this information? Like adding a comment % convert-ly version 2.17.28 or something, and when convert-ly sees this comment, it only deals with that until it does encounter an actually changing conversion? Or make \version accept a range? \version 2.16.0 - 2.17.28 Then LilyPond just needs to check the first number, and convert-ly just deals with the second number as long as it does not need to add another conversion. -- David Kastrup ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: Reasoning behind convert-ly rule for stable update?
Does anybody know _why_ convert-ly updates at least to the last stable version number even if nothing else has been changed? Looks like a bad idea, indeed. Or make \version accept a range? \version 2.16.0 - 2.17.28 Then LilyPond just needs to check the first number, and convert-ly just deals with the second number as long as it does not need to add another conversion. I don't like this. \version should stay unmodified if nothing gets changed, and I favor a change in convert-ly accordingly. However, I can imagine that, completely independent from convert-ly, another script adds something like % tested with version 2.17.95 and this comment gets updated whenever we feel it is the right thing to do. Werner ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: Reasoning behind convert-ly rule for stable update?
Werner LEMBERG w...@gnu.org writes: Does anybody know _why_ convert-ly updates at least to the last stable version number even if nothing else has been changed? Looks like a bad idea, indeed. Or make \version accept a range? \version 2.16.0 - 2.17.28 Then LilyPond just needs to check the first number, and convert-ly just deals with the second number as long as it does not need to add another conversion. I don't like this. \version should stay unmodified if nothing gets changed, and I favor a change in convert-ly accordingly. However, I can imagine that, completely independent from convert-ly, another script adds something like % tested with version 2.17.95 That's nice and all, but irrelevant to the problem of reducing the amount of work convert-ly has to do on unchanged files. Currently convert-ly checks about 30 rules or so at the end of an unstable release cycle. This would easily grow into several hundreds if we don't leave any indication what kind of rules _have_ already been applied without effect. I'm not sure that's actually a problem: at the current point of time, by far the bulk of update-with-convert-ly.sh's runtime is spent running make all. And it's probably also ok to restart with a clean state for LilyPond's core documentation files, and its code files. But for the LSR snippets and the example files and the regression tests, it seems somewhat excessive to update. Now the idea of add a comment is not really useful when the aim is to avoid unnecessary merge conflicts. If we don't want merge conflicts from the main \version, we don't want them from a comment either. Taking a look at our current Documentation/snippets, we have the following statistics: git grep -h '\\version' Documentation/snippets|sort -n|uniq -c 140 \version 2.16.0 26 \version 2.17.11 2 \version 2.17.13 2 \version 2.17.14 9 \version 2.17.15 5 \version 2.17.18 2 \version 2.17.23 4 \version 2.17.24 2 \version 2.17.25 8 \version 2.17.27 4 \version 2.17.28 1 \version 2.17.29 34 \version 2.17.30 2 \version 2.17.5 162 \version 2.17.6 4 \version 2.17.7 So in this cycle, only about a third of the files stayed at 2.16.0. So the average conversion chain length will not actually reach even two stable versions. Of course, it is not random which files get hit: the really short and simple ones are more likely to stick around unchanged for longer. But those are also less costly. The regression tests are quite selective: here we have 718 of 1185 that are at 2.16.0, so about 60%. But that still does not make for much more than a mean delay of 3 stable versions before update. So I lean towards just ripping the respective code out from convert-ly. I _think_ we have options that allow to simulate the update to a particular version if one really wants to. Of course, this will likely make the LSR import slower, too. But I think it can't hold a candle towards the is this safe? checks, either. -- David Kastrup ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: Reasoning behind convert-ly rule for stable update?
David Kastrup d...@gnu.org writes: That's nice and all, but irrelevant to the problem of reducing the amount of work convert-ly has to do on unchanged files. Oh phooey. Of course update-with-convert-ly can generally start with the last stable safely. Nothing in the tree will be older than that. -- David Kastrup ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: Reasoning behind convert-ly rule for stable update?
David Kastrup d...@gnu.org writes: David Kastrup d...@gnu.org writes: That's nice and all, but irrelevant to the problem of reducing the amount of work convert-ly has to do on unchanged files. Oh phooey. Of course update-with-convert-ly can generally start with the last stable safely. Nothing in the tree will be older than that. Ok, here is what _might_ have been the idea from the code. Suppose that I have a file for version 2.15.5 and I run convert-ly 2.17.8 without options on the file, and there is one conversion applied at 2.15.11. Then instead of claiming the file to be 2.15.11, stuff gets rounded up and 2.16.0 is claimed instead, just losing a bit of history within unstable versions. That would make some sense. It would not make things significantly more efficient, but it would throw away old unstable version history which is not all that interesting. But the code currently does not appear to work this way: it seems to promote even an unchanged 2.16.0 to 2.18.0. And besides, updates when there are _no_ changes should be skipped in either case. I'll try to see whether I figure something out in that direction. -- David Kastrup ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: Reasoning behind convert-ly rule for stable update?
2013/11/24 David Kastrup d...@gnu.org: Ok, here is what _might_ have been the idea from the code. Suppose that I have a file for version 2.15.5 and I run convert-ly 2.17.8 without options on the file, and there is one conversion applied at 2.15.11. Then instead of claiming the file to be 2.15.11, stuff gets rounded up and 2.16.0 is claimed instead, just losing a bit of history within unstable versions. That would make some sense. It would not make things significantly more efficient, but it would throw away old unstable version history which is not all that interesting. But the code currently does not appear to work this way: it seems to promote even an unchanged 2.16.0 to 2.18.0. And besides, updates when there are _no_ changes should be skipped in either case. I'll try to see whether I figure something out in that direction. +1 ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: Reasoning behind convert-ly rule for stable update?
On Sun, Nov 24, 2013 at 06:12:16PM +0100, David Kastrup wrote: Does anybody know _why_ convert-ly updates at least to the last stable version number even if nothing else has been changed? Yes, because it's confusing for some users if they've downloaded the latest and greatest lilypond 2.18.0, run convert-ly, and see that their files are 2.17.37. If you check the git history on convert-ly or convert-rules, then check the mailing list archives from a few days before then, you'll see the discussion. Or this might even be in the issue tracker. Also, when dealing with large collections of files, it's reassuring that the files really are current as-of 2.x.0. I mean, if I see input/regression/foo.ly being 2.13.5, does that mean that people forgot to run convert-ly, or does it mean that it really has no syntax changes since then? But for another, it loses significant information. Is there some way we can retain this information? I don't think that's useful info, but I suppose that there's little harm in adding an option to convert-ly which writes a comment with the previous revision. It shouldn't be the default behaviour, though. - Graham ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: Reasoning behind convert-ly rule for stable update?
Graham Percival gra...@percival-music.ca writes: On Sun, Nov 24, 2013 at 06:12:16PM +0100, David Kastrup wrote: Does anybody know _why_ convert-ly updates at least to the last stable version number even if nothing else has been changed? Yes, because it's confusing for some users if they've downloaded the latest and greatest lilypond 2.18.0, run convert-ly, and see that their files are 2.17.37. Well, they can use -c then. If you check the git history on convert-ly or convert-rules, then check the mailing list archives from a few days before then, you'll see the discussion. Or this might even be in the issue tracker. Also, when dealing with large collections of files, it's reassuring that the files really are current as-of 2.x.0. I mean, if I see input/regression/foo.ly being 2.13.5, does that mean that people forgot to run convert-ly, or does it mean that it really has no syntax changes since then? If you want the former, use -c. If I see input/regression/foo.ly being 2.16.0, does that mean that people forgot to run convert-ly, or does it means that it really has no syntax changes since then? The most important thing is that if I see Documentation/snippet/some-file.ly, I can figure out whether it will run with 2.16.2 even when it is contained in 2.18.0. And then there are the merge conflicts from gratuitous changes. I'll take a look whether I can dig up the discussion. -- David Kastrup ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
Re: Reasoning behind convert-ly rule for stable update?
Graham Percival graham at percival-music.ca writes: On Sun, Nov 24, 2013 at 06:12:16PM +0100, David Kastrup wrote: Does anybody know _why_ convert-ly updates at least to the last stable version number even if nothing else has been changed? Yes, because it's confusing for some users if they've downloaded the latest and greatest lilypond 2.18.0, run convert-ly, and see that their files are 2.17.37. If users run convert-ly without the '-d' option, then the version string is updated to the last version considered by convert-ly, and if we have a dummy rule for 2.18.0 (like we have for stable releases since 1.6.0) that version will be 2.18.0 Also, when dealing with large collections of files, it's reassuring that the files really are current as-of 2.x.0. I mean, if I see input/regression/foo.ly being 2.13.5, does that mean that people forgot to run convert-ly, or does it mean that it really has no syntax changes since then? That argument seems to apply to the snippets, reassuring users that they pulled a snippet that should work. For a regression test, if someone forgot to run convert-ly then it will either fail pass its test, or the conversion must not have been important. ___ lilypond-devel mailing list lilypond-devel@gnu.org https://lists.gnu.org/mailman/listinfo/lilypond-devel
