Re: Long Compile time with version 2.19.44

[francisperea]

Hi all,

I'm just writing to confirm that under macOS Sierra (10.12) Lilypond 2.19.48
is extremely slow.

I've downgraded to 2.19.47 and finally to 2.19.46.

2.19.47 was also terribly slow but 2.19.46 worked as usual.

As a sample, a simple file took 1.2 seconds to compile under 2.19.46 and
51.4 seconds under 2.19.48.

Just to provide some reference for those in the same situation and perhaps
to help developers locate the problem.

Regards,

Francis Perea





--
View this message in context: 
http://lilypond.1069038.n5.nabble.com/Long-Compile-time-with-version-2-19-44-tp192343p195236.html
Sent from the User mailing list archive at Nabble.com.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[Andrew Bernard]

Hi Johan,

Th OP is on Windows where there is no strace, I don’t think. However, I
suspect you are on the right track. I attempted to build Frescobaldi
2.19.latest-whatever a while ago on several Linux distros and gave up after
a few weeks of frustration due to the complex Qt code timing out in, sure
enough., network calls to sockets. So something related to this has arisen
in the 2.19 series. Not being particularly fond of Python programming and
knowing nothing at Qt, I abandoned the work.

Andrew


On 9 July 2016 at 4:11:22 AM, Johan Vromans (jvrom...@squirrel.nl) wrote:

On Fri, 8 Jul 2016 08:40:56 -0700 (MST)
BGM  wrote:

> So there is something in my computer that is messing up Lilypond 2.19.44.

Looks like a job for strace to pinpoint the exact place where the process
stalls.

My intuition is that someone is making a network call that times out.
The delay is much too big for caching, even swapping.
Could be DNS, or maybe someone is trying to 'phone home', ...
___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[Johan Vromans]

On Fri, 8 Jul 2016 08:40:56 -0700 (MST)
BGM  wrote:

> So there is something in my computer that is messing up Lilypond 2.19.44.

Looks like a job for strace to pinpoint the exact place where the process
stalls.

My intuition is that someone is making a network call that times out.
The delay is much too big for caching, even swapping.
Could be DNS, or maybe someone is trying to 'phone home', ...


___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[m.tarensk...@zonnet.nl]

Have you tried running different versions of lilypond on different systems, without using frescobaldi? I mean running lilypond directly from the commandline? MT Oorspronkelijk bericht Onderwerp: Re: Long Compile time with version 2.19.44Van: BGM Aan: lilypond-user@gnu.orgCc: I tried this.  I took a Windows 7 computer that never before had Lilypond andI installed Lilypond 2.19.44 and Frescobaldi 2.19 and I had the same compileproblem taking a really long time to compile - and that using the "choirhymn" template that comes with Frescobaldi's "New" menu.  So, to continue the test, I downgraded Frescobaldi to 2.18 and had noproblem at all with Lilypond 2.19.44!So the problem, for me, at least, and on two separate Window 7 computers,was the combination of Frescobaldi 2.19 and Lilypond 2.19.44.Thanks for all your attention with this.  I really do appreciate how activeand helpful this list and its users are.--View this message in context: http://lilypond.1069038.n5.nabble.com/Long-Compile-time-with-version-2-19-44-tp192343p192400.htmlSent from the User mailing list archive at Nabble.com.___lilypond-user mailing listlilypond-user@gnu.orghttps://lists.gnu.org/mailman/listinfo/lilypond-user___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[BGM]

So, on my workstation, I've tried different versions of Lilypond until I
found the most recent version that works correctly.


That version is 2.19.42-1.

Therefore, I deduce that the problem I am having comes from the changes
between version 2.19.42 and 2.19.43.

I'm fetching the Windows binaries from here:
http://download.linuxaudio.org/lilypond/binaries/mingw/

And also for the record, I have no problems with either 2.18 nor 2.19
versions of Frescobaldi.  Both versions work perfectly with Lilypond
2.19.43.



--
View this message in context: 
http://lilypond.1069038.n5.nabble.com/Long-Compile-time-with-version-2-19-44-tp192343p192404.html
Sent from the User mailing list archive at Nabble.com.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[BGM]

Alright, so that was on a fresh system.

So, I completely removed Lilypond and Frescobaldi from my workstation -
preferences and all  - and reinstalled Lilypond 2.19.44 and Frescobaldi 2.18
from scratch.  But I have the same old problem.

So there is something in my computer that is messing up Lilypond 2.19.44.

And in response to David Wright's notions of the cache - when I installed
Lilypond and Frescobaldi on the fresh system, the very first compile/run was
as fast and normal as could be.  So it's not a matter of caching here, I
believe.

And I don't think it's about particular scores either.  I've been running
the speed tests with Frescobaldi's built-in templates, "Choir Hymn" and
"Lead Sheet".  It's all the same.





--
View this message in context: 
http://lilypond.1069038.n5.nabble.com/Long-Compile-time-with-version-2-19-44-tp192343p192403.html
Sent from the User mailing list archive at Nabble.com.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[BGM]

I tried this.  I took a Windows 7 computer that never before had Lilypond and
I installed Lilypond 2.19.44 and Frescobaldi 2.19 and I had the same compile
problem taking a really long time to compile - and that using the "choir
hymn" template that comes with Frescobaldi's "New" menu.  

So, to continue the test, I downgraded Frescobaldi to 2.18 and had no
problem at all with Lilypond 2.19.44!

So the problem, for me, at least, and on two separate Window 7 computers,
was the combination of Frescobaldi 2.19 and Lilypond 2.19.44.

Thanks for all your attention with this.  I really do appreciate how active
and helpful this list and its users are.



--
View this message in context: 
http://lilypond.1069038.n5.nabble.com/Long-Compile-time-with-version-2-19-44-tp192343p192400.html
Sent from the User mailing list archive at Nabble.com.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[David Wright]

On Thu 07 Jul 2016 at 23:17:26 (-0700), Andrew Bernard wrote:
> 
> A long time UNIX user myself (pre 1970’s…), I follow what you are saying,
> but you may want to explain to ordinary folks what clearing the caches does
> and the command 'echo 3 > /proc/sys/vm/drop_caches ‘, especially since the
> OP is on Windows and does not have such nice things available.

OK. The first time you run a program, it has to be loaded into memory
from the disk, along with the various library functions that the
program uses. Once it starts running, the LP source and all its
library calls are similarly loaded. As it runs, any new files created
will have their extents allocated on the disk as they're written.
All in all, a lot of work.

The second time around, much of this work can be avoided because
operating systems don't have to throw away the loaded version of
the program and the buffers containing the LP source files.
Overwriting the output files will also be faster because all the
pointers to the files' locations are already in memory.

How much difference this makes depends on the OS's capabilities.
Linux is very efficient at hanging on to all it can; a common
complaint of new users is that their computer is running out of
memory, which is usually answered with a pointer to
http://www.linuxatemyram.com/

Writing 3 to the pseudofile /proc/sys/vm/drop_caches is linux's
method of instructing the kernel to throw away all the information
that's been cached in memory. (The /proc (likewise /sys) filesystem
is a way of communicating with the OS kernel through reading and
writing what appear to be just ordinary files.)

In the case of the very first run of LP 2.19.44-1, which was
extremely slow, there's also the matter of disk-caching. During
that run, 73 files totalling 1.6MB were stored with names like
~/.lilypond-fonts.cache-2/94f...595-i686-linux.cache-7
(shortening the random part which is a GUID).
Because those files belong to me, the kernel can't throw them
away, so at least the work in generating this cache didn't have
to be repeated.

Windows probably does similar tricks, but less transparently.

Cheers,
David.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[Andrew Bernard]

Hi David,

A long time UNIX user myself (pre 1970’s…), I follow what you are saying,
but you may want to explain to ordinary folks what clearing the caches does
and the command 'echo 3 > /proc/sys/vm/drop_caches ‘, especially since the
OP is on Windows and does not have such nice things available.

OP - on the matter of downgrading, in my personal experience I have seen no
speed difference between the versions you mention.


Andrew
___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[David Wright]

On Thu 07 Jul 2016 at 19:37:54 (-0700), BGM wrote:
> Thanks for you interest, fellas.
> 
> I downgraded lilypond to version 2.19.36-1 and everything works as expected
> - that is, it compiles any of my scores normally in just a second or two.
> 
> My problems were all with the latest lilypond 2.19.44-1 version.

Well, I downloaded 2.19.44-1 (I'm linux x86) and the first run
of the same carol was very slow; the disk was churning throughout.
However, this laptop does have a *very* slow disk, usually being
gently fried at about 65-70°C.

These runs are all the same version, 2.19.44-1. I dropped the
caches halfway through.

real1m17.986s
user0m31.128s
sys 0m1.176s

real0m17.420s
user0m16.948s
sys 0m0.380s

real0m17.368s
user0m16.884s
sys 0m0.400s

# echo 3 > /proc/sys/vm/drop_caches

real0m30.748s
user0m16.960s
sys 0m0.596s

real0m17.565s
user0m16.832s
sys 0m0.500s

So I've gained another second! However, it's always possible the
system load is lower (ie, fewer tabs open in the browser).
But I don't see a problem in this version unless some particular
construction of yours provokes it.

Cheers,
David.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[BGM]

Thanks for you interest, fellas.

I downgraded lilypond to version 2.19.36-1 and everything works as expected
- that is, it compiles any of my scores normally in just a second or two.

My problems were all with the latest lilypond 2.19.44-1 version.

I am using Frescibaldo 2.19 on Windows 7 Pro x64.



--
View this message in context: 
http://lilypond.1069038.n5.nabble.com/Long-Compile-time-with-version-2-19-44-tp192343p192357.html
Sent from the User mailing list archive at Nabble.com.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[Andrew Bernard]

Hi Gabriel-Marie,

What platform are you using? Linux, Windows, Mac? Can you tell us
that, and the OS version? Which Frescobaldi are you running - platform
distribution, or a build from source?

Can you obtain timings from the command line to compare with Frescobaldi?

I have observed the Frescobaldi 2.19 behaves strangely compared to
2.18 in all sorts of ways. Given that there is not a vast set of
improvements to be found in 2.19 when all is said and done. you may be
better off sticking with Frescobaldi 2.18.

You could also try posting to the Frescobaldi group, but it is very
low traffic and not very responsive - but you never know!

Andrew

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[David Wright]

On Thu 07 Jul 2016 at 15:15:42 (-0700), BGM wrote:
> Well, it seems like it hangs on just starting lilypond.  (and, as I've
> mentioned, I didn't have this problem with version 2.18)
> 
> As soon as it gets past this line...
> So, this line takes 99% of the time in the progressbar:
> Starting lilypond-windows.exe 2.19.44 [whatever.ly]...
> 
> then all the rest progresses nice and fast.
> 
> Processing `C:/hymns/whatever.ly'
> Parsing...
> Interpreting music...[8][16]
> Preprocessing graphical objects...
> Interpreting music...
> MIDI output to `whatever.mid'...
> Finding the ideal number of pages...
> Fitting music on 1 page...
> Drawing systems...
> Layout output to `./tmp-lilypond-WYGD9b'...
> Converting to `whatever.pdf'...
> Deleting `./tmp-lilypond-WYGD9b'...
> Success: compilation successfully completed
> Completed successfully in 30.0".

Here are some timings off a reasonably sized carol that generates
seven MIDI files and three PDFs. Old and new versions alternate
(2.18.2 and 2.19.42).

# echo 3 > /proc/sys/vm/drop_caches

real0m33.921s
user0m20.040s
sys 0m0.716s

real0m28.471s
user0m17.696s
sys 0m0.756s

real0m20.586s
user0m19.832s
sys 0m0.556s

real0m18.838s
user0m17.600s
sys 0m0.744s

real0m20.535s
user0m19.800s
sys 0m0.596s

real0m18.677s
user0m17.456s
sys 0m0.844s

So the new version wins by about two seconds.

Note the difference that caching the program makes. You've got to run
a program twice (at least) to get a realistic time.

I always put all the structure of my scores in \score;
ie I don't write   variable = \new ...
so I have no comparative data. However, if you try doing
this, remember again to run both versions twice or you may
only be measuring the efficiency of the operating system.

Cheers,
David.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[BGM]

Well, it seems like it hangs on just starting lilypond.  (and, as I've
mentioned, I didn't have this problem with version 2.18)

As soon as it gets past this line...
So, this line takes 99% of the time in the progressbar:
Starting lilypond-windows.exe 2.19.44 [whatever.ly]...

then all the rest progresses nice and fast.

Processing `C:/hymns/whatever.ly'
Parsing...
Interpreting music...[8][16]
Preprocessing graphical objects...
Interpreting music...
MIDI output to `whatever.mid'...
Finding the ideal number of pages...
Fitting music on 1 page...
Drawing systems...
Layout output to `./tmp-lilypond-WYGD9b'...
Converting to `whatever.pdf'...
Deleting `./tmp-lilypond-WYGD9b'...
Success: compilation successfully completed
Completed successfully in 30.0".



--
View this message in context: 
http://lilypond.1069038.n5.nabble.com/Long-Compile-time-with-version-2-19-44-tp192343p192350.html
Sent from the User mailing list archive at Nabble.com.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[David Kastrup]

"Br. Gabriel-Marie | SSPX"  writes:

> Recently I updated my lilypond from 2.18 to 2.19.44.  I am using
> Frescobaldi for the GUI.
> It used to take only a few seconds to compile a score, but now it
> takes at least 28 seconds every time.
> This is the case even if I don't create a midi.
>
> Am I missing something, or is this the case for everyone?
>
> Even for this minimal example without any compile errors, it took 28.3
> seconds - which is my new average time.

[...]

Nothing unusual here (64bit executable, native Linux compilation).  I
suspect some platform/font/conversion/path/Frescobaldi problem.

Does the Frescobaldi window make it possible to see just after which
console output LilyPond spends most of the time playing dead?

-- 
David Kastrup

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[BGM]

Okay, when I try exactly that, I still get 28.8 seconds for compile time. 
Moving the \new statements does not change anything for me.



--
View this message in context: 
http://lilypond.1069038.n5.nabble.com/Long-Compile-time-with-version-2-19-44-tp192343p192346.html
Sent from the User mailing list archive at Nabble.com.

___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Re: Long Compile time with version 2.19.44

[tisimst]

On Thu, Jul 7, 2016 at 2:25 PM, BGM [via Lilypond] <
ml-node+s1069038n192343...@n5.nabble.com> wrote:

> Recently I updated my lilypond from 2.18 to 2.19.44.  I am using
> Frescobaldi for the GUI.
> It used to take only a few seconds to compile a score, but now it takes at
> least 28 seconds every time.
> This is the case even if I don't create a midi.
>
> Am I missing something, or is this the case for everyone?
>
> Even for this minimal example without any compile errors, it took 28.3
> seconds - which is my new average time.
>
> \version "2.19.44"
> \header {
> title = "Minimal Example"
> }
>
> verses = \new Voice = "verses" \relative c'' {
> \time 4/4
> \key c \major
> a4 b4 c4 d4 | e4 d4 f2  \bar "||"
> }
>
> wordsOne = \new Lyrics  \lyricsto "verses" {
>  Here are some wo -- rds to sing
> }
> wordsTwo = \new Lyrics  \lyricsto "verses" {
> And line up wi -- th them -- selves
> }
> wordsThree = \new Lyrics \lyricsto "verses" {
> See the stanz -- as line up in place
> }
>
> \score {
> <<
> \new Staff{   \verses }
>\wordsOne
> \wordsTwo
>\wordsThree
> >>
> \layout {
>
> }
> }
>
> Here is my output:
>
> Starting lilypond-windows.exe 2.19.44 [Untitled]...
>
> Processing
> `c:/users/brgabr~1/appdata/local/temp/frescobaldi-a08kr0/tmplcpzp0/
> document.ly'
>
> Parsing...
>
> Interpreting music...
>
> Preprocessing graphical objects...
>
> Finding the ideal number of pages...
>
> Fitting music on 1 page...
>
> Drawing systems...
>
> Layout output to `./tmp-lilypond-TVi8uT'...
>
> Converting to `document.pdf'...
>
> Deleting `./tmp-lilypond-TVi8uT'...
>
> Success: compilation successfully completed
>
> Completed successfully in 28.3".
>

I can confirm a very similar looong compilation time on 2.19.36 (over a
minute for me, actually). Re-factoring the code to put all the \new
statements inside the \score block then made the compilation time to 2
seconds:

%%%

\version "2.19.36"
\header {
title = "Minimal Example"
}

verses = \relative c'' {
\time 4/4
\key c \major
a4 b4 c4 d4 | e4 d4 f2  \bar "||"
}

wordsOne = \lyricmode {
 Here are some wo -- rds to sing
}
wordsTwo = \lyricmode {
And line up wi -- th them -- selves
}
wordsThree = \lyricmode {
See the stanz -- as line up in place
}

\score {
<<
\new Staff { \new Voice = "verses" \verses }
\new Lyrics \lyricsto "verses" \wordsOne
\new Lyrics  \lyricsto "verses" \wordsTwo
\new Lyrics  \lyricsto "verses" \wordsThree
>>
\layout { }
}

%%%

So, no explanation from me, only an observation.

HTH,
Abraham




--
View this message in context: 
http://lilypond.1069038.n5.nabble.com/Long-Compile-time-with-version-2-19-44-tp192343p192345.html
Sent from the User mailing list archive at Nabble.com.___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user

Long Compile time with version 2.19.44

[Br. Gabriel-Marie | SSPX]

Recently I updated my lilypond from 2.18 to 2.19.44.  I am 
using Frescobaldi for the GUI.
It used to take only a few seconds to compile a score, but 
now it takes at least 28 seconds every time.

This is the case even if I don't create a midi.

Am I missing something, or is this the case for everyone?

Even for this minimal example without any compile errors, it 
took 28.3 seconds - which is my new average time.


\version "2.19.44"
\header {
title = "Minimal Example"
}

verses = \new Voice = "verses" \relative c'' {
\time 4/4
\key c \major
a4 b4 c4 d4 | e4 d4 f2  \bar "||"
}

wordsOne = \new Lyrics  \lyricsto "verses" {
 Here are some wo -- rds to sing
}
wordsTwo = \new Lyrics  \lyricsto "verses" {
And line up wi -- th them -- selves
}
wordsThree = \new Lyrics \lyricsto "verses" {
See the stanz -- as line up in place
}

\score {
<<
\new Staff{   \verses }
   \wordsOne
\wordsTwo
   \wordsThree
>>
\layout {

}
}

Here is my output:

Starting lilypond-windows.exe 2.19.44 [Untitled]...

Processing 
`c:/users/brgabr~1/appdata/local/temp/frescobaldi-a08kr0/tmplcpzp0/document.ly'


Parsing...

Interpreting music...

Preprocessing graphical objects...

Finding the ideal number of pages...

Fitting music on 1 page...

Drawing systems...

Layout output to `./tmp-lilypond-TVi8uT'...

Converting to `document.pdf'...

Deleting `./tmp-lilypond-TVi8uT'...

Success: compilation successfully completed

Completed successfully in 28.3".




___
lilypond-user mailing list
lilypond-user@gnu.org
https://lists.gnu.org/mailman/listinfo/lilypond-user