Re: LyX 2.2 slowness

[Stephan Witt]

Am 10.01.2017 um 00:24 schrieb Stephan Witt :
> 
> Am 08.01.2017 um 19:09 schrieb Jean-Marc Lasgouttes :
>> 
>> Le 08/01/2017 à 18:59, Stephan Witt a écrit :
>>> Ok, I don’t see this with current 2.2.x. AFAIK, you did the backport and 
>>> applied it there too.
>>> The patch seems to be innocent :)
>> 
>> A bisect would be appreciated :) Note that I have noticed problems with math 
>> previews not updating in master, I do not know whether this is related.
> 
> I think it’s related.
> 
> Bisect is not easy. Many commits are uncompilable because of automake 
> incompatibilities or changes in compiler switches or simply compiler errors.
> 
> Change b917c4e compiles and the icons in text are drawn immediately but 
> included graphics not. HEAD of master seems to handle the icons like included 
> graphics.
> 
> But the current branch 2.2.x has the same problem. So it’s a long standing 
> problem now. I’ll continue my research.

I tried my old disk images and guess what - the same problem is present at 
least in LyX 1.6.7 already and in all subsequent versions I’ve tested.

Stephan

Re: LyX 2.2 slowness

[Stephan Witt]

Am 08.01.2017 um 19:09 schrieb Jean-Marc Lasgouttes :
> 
> Le 08/01/2017 à 18:59, Stephan Witt a écrit :
>> Ok, I don’t see this with current 2.2.x. AFAIK, you did the backport and 
>> applied it there too.
>> The patch seems to be innocent :)
> 
> A bisect would be appreciated :) Note that I have noticed problems with math 
> previews not updating in master, I do not know whether this is related.

I think it’s related.

Bisect is not easy. Many commits are uncompilable because of automake 
incompatibilities or changes in compiler switches or simply compiler errors.

Change b917c4e compiles and the icons in text are drawn immediately but 
included graphics not. HEAD of master seems to handle the icons like included 
graphics.

But the current branch 2.2.x has the same problem. So it’s a long standing 
problem now. I’ll continue my research.

Stephan

PS. To illustrate to problem I attach a screen shot. This is after waiting a 
long time and switching between windows (LyX and Mail) many times. The 
converted image is ready and placed in the cache but not drawn (because of a 
missing redraw event?).

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 08/01/2017 à 18:59, Stephan Witt a écrit :

Ok, I don’t see this with current 2.2.x. AFAIK, you did the backport and 
applied it there too.
The patch seems to be innocent :)


A bisect would be appreciated :) Note that I have noticed problems with 
math previews not updating in master, I do not know whether this is related.


JMarc

Re: LyX 2.2 slowness

[Stephan Witt]

Am 08.01.2017 um 18:42 schrieb Jean-Marc Lasgouttes :
> 
> Le 08/01/2017 à 18:00, Stephan Witt a écrit :
>> I tried the current master (commit 21259b66b5d36913aaf4dcded8aaac3254b04354) 
>> on Mac.
>> It seems to be fast with Qt5 - but I’m not sure how to verify that.
>> 
>> One thing I’ve noticed: the images in text are not shown immediately when 
>> scrolling
>> through the users guide. I didn’t investigate if the mentioned patch is 
>> causing this.
>> To make them appear one has to click in the main area or scroll again after 
>> stop.
> 
> I would be surprised that it is this particular patch, but this is definitely 
> something we need to fix.
> 
> JMarc
> 

Ok, I don’t see this with current 2.2.x. AFAIK, you did the backport and 
applied it there too.
The patch seems to be innocent :)

Stephan

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 08/01/2017 à 18:00, Stephan Witt a écrit :

I tried the current master (commit 21259b66b5d36913aaf4dcded8aaac3254b04354) on 
Mac.
It seems to be fast with Qt5 - but I’m not sure how to verify that.

One thing I’ve noticed: the images in text are not shown immediately when 
scrolling
through the users guide. I didn’t investigate if the mentioned patch is causing 
this.
To make them appear one has to click in the main area or scroll again after 
stop.


I would be surprised that it is this particular patch, but this is 
definitely something we need to fix.


JMarc

Re: LyX 2.2 slowness

[Stephan Witt]

Am 31.12.2016 um 13:16 schrieb Jean-Marc Lasgouttes :
> 
> Le 16/12/2016 à 16:42, Jean-Marc Lasgouttes a écrit :
>> I'd be interested to see other tests, especially on MacOS and Windows.
> 
> Since there not much testing going on, I pushed the patch to master :) 
> Notable differences are:
> * the caching of getLayout is disabled with Qt5
> * the profiling hooks are kept, but profiling is disabled by default
> 
> I am still interested in reports on Mac and Windows

I tried the current master (commit 21259b66b5d36913aaf4dcded8aaac3254b04354) on 
Mac.
It seems to be fast with Qt5 - but I’m not sure how to verify that.

One thing I’ve noticed: the images in text are not shown immediately when 
scrolling
through the users guide. I didn’t investigate if the mentioned patch is causing 
this.
To make them appear one has to click in the main area or scroll again after 
stop.

Stephan 

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 06/01/2017 à 20:16, Richard Heck a écrit :

OK, go ahead and commit then. I will send a note to lyx-devel and
lyx-users announcing that this has happened and encouraging people, as
they are able, to use 2.2.x.


It is in.

JMarc

Re: LyX 2.2 slowness

[Richard Heck]

On 01/06/2017 10:17 AM, Jean-Marc Lasgouttes wrote:
> Le 06/01/2017 à 16:13, Richard Heck a écrit :
>> I guess I'd be inclined to go
>> ahead and commit it, then. Maybe we can just encourage as many people as
>> possible to use the 2.2.x branch (if they're not already using master),
>> so it gets tested.
>
> I would agree with that.
>
>> One question: Do you want to leave all the #ifdefs in the 2.2.x code?
>> They seem to me to be less useful here than in master, but maybe they do
>> no harm. But if they are left, we should probably add something to the
>> release notes about what they do and how to enable/disable them.
>
> The goal is to remain as close as possible to master and to allow
> easily to enable/disable a certain cache in certain conditions (e.g.
> "Qt 5.3.7 with Fedora 21", I don't know) so that the code works well
> for everybody. So actually, it may be very useful in stable.
>
> I removed the pmprof code hooks, though.

OK, go ahead and commit then. I will send a note to lyx-devel and
lyx-users announcing that this has happened and encouraging people, as
they are able, to use 2.2.x.

Richard

Re: LyX 2.2 slowness

[Richard Heck]

On 01/06/2017 04:56 AM, Jean-Marc Lasgouttes wrote:
> Le 31/12/2016 à 15:26, Jean-Marc Lasgouttes a écrit :
>> Le 31/12/2016 à 15:24, Jean-Marc Lasgouttes a écrit :
>>> It is this particular patch. I amended it since the initial July 5
>>> creattion, but apparently git does not update date by default.
>>
>> To be more precise, this is the commit below, pushed on Dec 19. I have
>> prepared a version for stable (for discussion), that I will post later.
>
> Here is the backport for stable. I did not have to change much for the
> backport, which is good in terms of stability.
>
> I used it for a few days, and it seems to work fine.

We have 14 bugs fixed in 2.2.x; 5 that are fixed in master with 2.2.3 or
2.2.x milestones; and another 7 that are fixed in master but without any
milestone. So I was intending, soon, to try to prod for some action on
the latter, looking towards a 2.2.3 release maybe around the end of the
month. This could be delayed a bit, if need be, and I guess this
slowness has been a problem for some people, so it would be good if we
could improve the situation for 2.2.3. I guess I'd be inclined to go
ahead and commit it, then. Maybe we can just encourage as many people as
possible to use the 2.2.x branch (if they're not already using master),
so it gets tested.

One question: Do you want to leave all the #ifdefs in the 2.2.x code?
They seem to me to be less useful here than in master, but maybe they do
no harm. But if they are left, we should probably add something to the
release notes about what they do and how to enable/disable them.

Richard

>
> Thoughts?
>
> JMarc
>
>

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 06/01/2017 à 16:13, Richard Heck a écrit :

I guess I'd be inclined to go
ahead and commit it, then. Maybe we can just encourage as many people as
possible to use the 2.2.x branch (if they're not already using master),
so it gets tested.


I would agree with that.


One question: Do you want to leave all the #ifdefs in the 2.2.x code?
They seem to me to be less useful here than in master, but maybe they do
no harm. But if they are left, we should probably add something to the
release notes about what they do and how to enable/disable them.


The goal is to remain as close as possible to master and to allow easily 
to enable/disable a certain cache in certain conditions (e.g. "Qt 5.3.7 
with Fedora 21", I don't know) so that the code works well for 
everybody. So actually, it may be very useful in stable.


I removed the pmprof code hooks, though.

JMarc

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 31/12/2016 à 15:26, Jean-Marc Lasgouttes a écrit :

Le 31/12/2016 à 15:24, Jean-Marc Lasgouttes a écrit :

It is this particular patch. I amended it since the initial July 5
creattion, but apparently git does not update date by default.


To be more precise, this is the commit below, pushed on Dec 19. I have
prepared a version for stable (for discussion), that I will post later.


Here is the backport for stable. I did not have to change much for the 
backport, which is good in terms of stability.


I used it for a few days, and it seems to work fine.

Thoughts?

JMarc


>From d17de207d85e5b851bde0ddd4289ed290e0576b5 Mon Sep 17 00:00:00 2001
From: Jean-Marc Lasgouttes 
Date: Tue, 5 Jul 2016 14:06:22 +0200
Subject: [PATCH] Add caching for the QTextLayout objects we use

The QTextLayout handling is terribly slow on Qt 4.8.7, but some
caching has been added in Qt5 that makes it much faster. For some
reason, it is not that slow with Qt 4.8.1.

Caches are introduced for the three following methods

* width(doctring), controlled by CACHE_METRICS_WIDTH. This cache already
  existed, but the code has been cleaned up

* getTextLayout, controlled by CACHE_METRICS_QTEXTLAYOUT (disabled by
  default on Qt5, which does its own caching). This is used for pos2x
  and x2pos and now for drawing of text too. The previous code used a
  trivial caching scheme of the last used QTextLayout, but now they
  are properly kept in a QCache. Moreover, the cacheEnabled() property
  is enabled for these QTextLayout object (not sure what this does).

* breakAt, controlled by CACHE_METRICS_BREAKAT. This is the only user
  of QTextLayout which did not have some kind of caching already.

For some weird reasons related to Argument-dependent look-up, the
qHash(docstring) function has to be defined in std namespace, since
lyx::docstring is actually std::basic_string.

(cherry picked from c5119c97fcf84e8)
---
 src/frontends/qt4/GuiFontMetrics.cpp | 158 +--
 src/frontends/qt4/GuiFontMetrics.h   |  52 +---
 src/frontends/qt4/GuiPainter.cpp |  13 ++-
 src/support/convert.cpp  |   7 ++
 4 files changed, 170 insertions(+), 60 deletions(-)

diff --git a/src/frontends/qt4/GuiFontMetrics.cpp b/src/frontends/qt4/GuiFontMetrics.cpp
index a1d5b40..f17ac37 100644
--- a/src/frontends/qt4/GuiFontMetrics.cpp
+++ b/src/frontends/qt4/GuiFontMetrics.cpp
@@ -21,11 +21,33 @@
 
 #include "insets/Inset.h"
 
+#include "support/convert.h"
 #include "support/lassert.h"
 
+#ifdef CACHE_SOME_METRICS
+#include 
+#endif
+
 using namespace std;
 using namespace lyx::support;
 
+#ifdef CACHE_SOME_METRICS
+namespace std {
+
+/*
+ * Argument-dependent lookup implies that this function shall be
+ * declared in the namespace of its argument. But this is std
+ * namespace, since lyx::docstring is just std::basic_string.
+ */
+uint qHash(lyx::docstring const & s)
+{
+	return qHash(QByteArray(reinterpret_cast(s.data()),
+	s.size() * sizeof(lyx::docstring::value_type)));
+}
+
+}
+#endif
+
 namespace lyx {
 namespace frontend {
 
@@ -51,11 +73,23 @@ inline QChar const ucs4_to_qchar(char_type const ucs4)
 } // anon namespace
 
 
-// Limit strwidth_cache_ size to 512kB of string data
+/*
+ * Limit (strwidth|breakat)_cache_ size to 512kB of string data.
+ * Limit qtextlayout_cache_ size to 500 elements (we do not know the
+ * size of the QTextLayout objects anyway).
+ * Note that all these numbers are arbitrary.
+ */
 GuiFontMetrics::GuiFontMetrics(QFont const & font)
-	: font_(font), metrics_(font, 0),
-	  strwidth_cache_(1 << 19),
-	  tl_cache_rtl_(false), tl_cache_wordspacing_(-1.0)
+	: font_(font), metrics_(font, 0)
+#ifdef CACHE_METRICS_WIDTH
+	, strwidth_cache_(1 << 19)
+#endif
+#ifdef CACHE_METRICS_BREAKAT
+	, breakat_cache_(1 << 19)
+#endif
+#ifdef CACHE_METRICS_QTEXTLAYOUT
+	,  qtextlayout_cache_(500)
+#endif
 {
 }
 
@@ -140,14 +174,13 @@ int GuiFontMetrics::rbearing(char_type c) const
 
 int GuiFontMetrics::width(docstring const & s) const
 {
-	QByteArray qba =
-		QByteArray(reinterpret_cast(s.data()),
-		   s.size() * sizeof(docstring::value_type));
-	int * pw = strwidth_cache_[qba];
+#ifdef CACHE_METRICS_WIDTH
+	int * pw = strwidth_cache_[s];
 	if (pw)
 		return *pw;
 	// For some reason QMetrics::width returns a wrong value with Qt5
 	// int w = metrics_.width(toqstr(s));
+#endif
 	QTextLayout tl;
 	tl.setText(toqstr(s));
 	tl.setFont(font_);
@@ -155,8 +188,9 @@ int GuiFontMetrics::width(docstring const & s) const
 	QTextLine line = tl.createLine();
 	tl.endLayout();
 	int w = int(line.naturalTextWidth());
-
-	strwidth_cache_.insert(qba, new int(w), qba.size());
+#ifdef CACHE_METRICS_WIDTH
+	strwidth_cache_.insert(s, new int(w), s.size() * sizeof(char_type));
+#endif
 	return w;
 }
 
@@ -179,58 +213,66 @@ int GuiFontMetrics::signedWidth(docstring const & s) const
 }
 
 
-QTextLayout const &
-GuiFontMetrics::getTextLayout(docstring const & s, QFont font,
-

Re: LyX 2.2 slowness

[Scott Kostyshak]

On Sat, Dec 31, 2016 at 05:36:47PM +0100, Jean-Marc Lasgouttes wrote:
> Le 31/12/2016 à 17:34, Kornel Benko a écrit :
> > > commit c5119c97fcf84e8dd2cfcdd605cc0a9ffa8b5bc4
> > > Author: Jean-Marc Lasgouttes 
> > > Date:   Tue Jul 5 14:06:22 2016 +0200
> > > 
> > >  Add caching for the QTextLayout objects we use
> > 
> > I see. Yes, that was what I have seen, not realizing when it was pushed.
> 
> Does git give us this information, actually?

git log --pretty=fuller

will show both author date and commit date.

Scott


signature.asc
Description: PGP signature

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 31/12/2016 à 17:34, Kornel Benko a écrit :

commit c5119c97fcf84e8dd2cfcdd605cc0a9ffa8b5bc4
Author: Jean-Marc Lasgouttes 
Date:   Tue Jul 5 14:06:22 2016 +0200

 Add caching for the QTextLayout objects we use


I see. Yes, that was what I have seen, not realizing when it was pushed.


Does git give us this information, actually?

JMarc

Re: LyX 2.2 slowness

[Kornel Benko]

Am Samstag, 31. Dezember 2016 um 15:26:47, schrieb Jean-Marc Lasgouttes 

> Le 31/12/2016 à 15:24, Jean-Marc Lasgouttes a écrit :
> > It is this particular patch. I amended it since the initial July 5
> > creattion, but apparently git does not update date by default.
> 
> To be more precise, this is the commit below, pushed on Dec 19. I have 
> prepared a version for stable (for discussion), that I will post later.
> 
> JMarc
> 
> commit c5119c97fcf84e8dd2cfcdd605cc0a9ffa8b5bc4
> Author: Jean-Marc Lasgouttes 
> Date:   Tue Jul 5 14:06:22 2016 +0200
> 
>  Add caching for the QTextLayout objects we use

I see. Yes, that was what I have seen, not realizing when it was pushed.

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 31/12/2016 à 15:24, Jean-Marc Lasgouttes a écrit :

It is this particular patch. I amended it since the initial July 5
creattion, but apparently git does not update date by default.


To be more precise, this is the commit below, pushed on Dec 19. I have 
prepared a version for stable (for discussion), that I will post later.


JMarc

commit c5119c97fcf84e8dd2cfcdd605cc0a9ffa8b5bc4
Author: Jean-Marc Lasgouttes 
Date:   Tue Jul 5 14:06:22 2016 +0200

Add caching for the QTextLayout objects we use

The QTextLayout handling is terribly slow on Qt 4.8.7, but some
caching has been added in Qt5 that makes it much faster. For some
reason, it is not that slow with Qt 4.8.1.

Caches are introduced for the three following methods

* width(doctring), controlled by CACHE_METRICS_WIDTH. This cache 
already

  existed, but the code has been cleaned up

* getTextLayout, controlled by CACHE_METRICS_QTEXTLAYOUT (disabled by
  default on Qt5, which does its own caching). This is used for pos2x
  and x2pos and now for drawing of text too. The previous code used a
  trivial caching scheme of the last used QTextLayout, but now they
  are properly kept in a QCache. Moreover, the cacheEnabled() property
  is enabled for these QTextLayout object (not sure what this does).

* breakAt, controlled by CACHE_METRICS_BREAKAT. This is the only user
  of QTextLayout which did not have some kind of caching already.

For some weird reasons related to Argument-dependent look-up, the
qHash(docstring) function has to be defined in std namespace, since
lyx::docstring is actually std::basic_string.

[NOTE: this version has profiling hooks, enabled by commenting out 
the line

  #define DISABLE_PMPROF
that should eventually be removed.]

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 31/12/2016 à 13:26, Kornel Benko a écrit :

Am Samstag, 31. Dezember 2016 um 13:16:41, schrieb Jean-Marc Lasgouttes 


Le 16/12/2016 à 16:42, Jean-Marc Lasgouttes a écrit :

I'd be interested to see other tests, especially on MacOS and Windows.


Since there not much testing going on, I pushed the patch to master :)


When? I do not see any commit from you on master since Jul 5.


It is this particular patch. I amended it since the initial July 5 
creattion, but apparently git does not update date by default.


JMarc

Re: LyX 2.2 slowness

[Kornel Benko]

Am Samstag, 31. Dezember 2016 um 13:16:41, schrieb Jean-Marc Lasgouttes 

> Le 16/12/2016 à 16:42, Jean-Marc Lasgouttes a écrit :
> > I'd be interested to see other tests, especially on MacOS and Windows.
> 
> Since there not much testing going on, I pushed the patch to master :) 

When? I do not see any commit from you on master since Jul 5.

> Notable differences are:
> * the caching of getLayout is disabled with Qt5
> * the profiling hooks are kept, but profiling is disabled by default
> 
> I am still interested in reports on Mac and Windows
> 
> JMarc

Kornel

signature.asc
Description: This is a digitally signed message part.

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 16/12/2016 à 16:42, Jean-Marc Lasgouttes a écrit :

I'd be interested to see other tests, especially on MacOS and Windows.


Since there not much testing going on, I pushed the patch to master :) 
Notable differences are:

* the caching of getLayout is disabled with Qt5
* the profiling hooks are kept, but profiling is disabled by default

I am still interested in reports on Mac and Windows

JMarc

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 09/12/2016 à 15:58, Jean-Marc Lasgouttes a écrit :

Le 07/12/2016 à 16:10, Jean-Marc Lasgouttes a écrit :

Le 07/12/2016 à 12:15, Jean-Marc Lasgouttes a écrit :

I'll post a new version to try soon.


Here is a patch to play with. It is not perfect, but I would be
interested to know whether it improves performance with Qt4.


I have improved a bit the patch (fixed a couple bugs)  and now it also
comes with built-in profiling.
Just comment-out the
#define DISABLE_PMPROF
at the beginning to enable profiling of the 3 main caches.

Here is what I get on User Guide by using PageDown over the whole file
(not very good for the cache)



Here is the same test with Qt5:

#pmprof# getTextLayout: 19.07usec, count=19115, total=364.61msec
   hit: 53%, 4.17usec, count=10135, total=42.25msec
  miss: 46%, 35.90usec, count=8980, total=322.35msec
#pmprof# breakAt: 42.10usec, count=3981, total=167.60msec
   hit: 69%, 3.44usec, count=2762, total=9.51msec
  miss: 30%, 129.69usec, count=1219, total=158.09msec
#pmprof# width: 6.09usec, count=153658, total=936.00msec
   hit: 89%, 0.47usec, count=137232, total=64.39msec
  miss: 10%, 53.06usec, count=16426, total=871.61msec

Note that these numbers are misleading. Indeed, since Qt5 does its own 
caching, the times in case of "miss" take in account the fact that there 
is a miss also in the lower level Qt caching. Therefore the miss times 
are artificially large.


To have a better view of the effect of the cache, one can run the same 
test with cache disabled (comment out #define CACHE_FONT_METRICS in 
updated patch in attachment).


#pmprof# getTextLayout: 17.09usec, count=19224, total=328.52msec
#pmprof# breakAt: 76.84usec, count=4074, total=313.06msec
#pmprof# width: 17.18usec, count=152237, total=2614.93msec

These numbers show that the effect is actually slightly negative for 
getTextLayout. We might want though to skip the caching of QTextLayout 
on Qt5 (moreover they can use a lot of memory). The two other ones seem 
good to keep.


I'd be interested to see other tests, especially on MacOS and Windows.

JMarc
>From bd638c0c3ced186b81d7161042b62c5439b5382b Mon Sep 17 00:00:00 2001
From: Jean-Marc Lasgouttes 
Date: Tue, 5 Jul 2016 14:06:22 +0200
Subject: [PATCH] Add caching for the QTextLayout objects we use

The QTextLayout handling is terribly slow on Qt 4.8.7, but some
caching has been added in Qt5 that makes it much faster. For some
reason, it is not that slow with Qt 4.8.1.

This commit introduces some caching, which should probably only be
active for Qt 4. This caching only happens when CACHE_FONT_METRICS is
defined.

[NOTE: this version has profiling hooks, enabled by commenting out the line
  #define DISABLE_PMPROF
that should eventually be removed.]

Improve the caching of QTextLayout objects used for pos2x and x2pos
and use them for drawing too. We originally used a trivial caching
of the last QTextLayout, but now they are kept in a QCache.
Moreover, caching is enabled for these QTextLayout object (not sure
what this does).

This patch also adds some caching in the breakAt method, the only user of
QTextLayout which did not have some kind of caching already.

Finally the cached QTextLayout objects are also used by the painter
for drawing text.

For some weird reasons related to Argument-dependent look-up, the
qHash(docstring) function has to be defined in std namespace, since
lyx::docstring is actually std::basic_string.
---
 src/frontends/qt4/GuiFontMetrics.cpp | 156 +--
 src/frontends/qt4/GuiFontMetrics.h   |  33 +---
 src/frontends/qt4/GuiPainter.cpp |  13 ++-
 src/support/convert.cpp  |   7 ++
 4 files changed, 151 insertions(+), 58 deletions(-)

diff --git a/src/frontends/qt4/GuiFontMetrics.cpp b/src/frontends/qt4/GuiFontMetrics.cpp
index d3c89f1..ff61481 100644
--- a/src/frontends/qt4/GuiFontMetrics.cpp
+++ b/src/frontends/qt4/GuiFontMetrics.cpp
@@ -21,11 +21,36 @@
 
 #include "insets/Inset.h"
 
+#include "support/convert.h"
 #include "support/lassert.h"
 
+//#define DISABLE_PMPROF
+#include "support/pmprof.h"
+
+#ifdef CACHE_FONT_METRICS
+#include 
+#endif
+
 using namespace std;
 using namespace lyx::support;
 
+#ifdef CACHE_FONT_METRICS
+namespace std {
+
+/*
+ * Argument-dependent lookup implies that this function shall be
+ * declared in the namespace of its argument. But this is std
+ * namespace, since lyx::docstring is just std::basic_string.
+ */
+uint qHash(lyx::docstring const & s)
+{
+	return qHash(QByteArray(reinterpret_cast(s.data()),
+	s.size() * sizeof(lyx::docstring::value_type)));
+}
+
+}
+#endif
+
 namespace lyx {
 namespace frontend {
 
@@ -51,11 +76,17 @@ inline QChar const ucs4_to_qchar(char_type const ucs4)
 } // anon namespace
 
 
-// Limit strwidth_cache_ size to 512kB of string data
+/*
+ * Limit (strwidth|breakat)_cache_ size to 512kB of string data.
+ * Limit qtextlayout_cache_ size to 500 elements (we do not know the
+ * size of the 

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 07/12/2016 à 16:10, Jean-Marc Lasgouttes a écrit :

Le 07/12/2016 à 12:15, Jean-Marc Lasgouttes a écrit :

I'll post a new version to try soon.


Here is a patch to play with. It is not perfect, but I would be
interested to know whether it improves performance with Qt4.


I have improved a bit the patch (fixed a couple bugs)  and now it also 
comes with built-in profiling.

Just comment-out the
#define DISABLE_PMPROF
at the beginning to enable profiling of the 3 main caches.

Here is what I get on User Guide by using PageDown over the whole file 
(not very good for the cache)


#pmprof# getTextLayout: 11.11usec, count=30382, total=337.39msec
   hit: 62%, 4.41usec, count=18877, total=83.26msec
  miss: 37%, 22.09usec, count=11505, total=254.13msec
#pmprof# breakAt: 21.78usec, count=2247, total=48.94msec
   hit: 80%, 6.36usec, count=1811, total=11.51msec
  miss: 19%, 85.85usec, count=436, total=37.43msec
#pmprof# width: 4.03usec, count=212928, total=857.38msec
   hit: 90%, 1.31usec, count=192906, total=252.11msec
  miss: 9%, 30.23usec, count=20022, total=605.27msec

We can see here that there is a sizeable performance improvement from 
these metrics-related functions (note that getTextLayout is also used by 
the painter).

However, the total of these functions is not very large.

Also Tommaso, did you disable stdlib-debug when building master?

On the same file, if I just keep my finger on CursorRight, I get this 
one instead:

#pmprof# getTextLayout: 14.15usec, count=25433, total=359.96msec
   hit: 99%, 14.13usec, count=25337, total=357.91msec
  miss: 0%, 21.31usec, count=96, total=2.05msec
#pmprof# breakAt: 14.21usec, count=467, total=6.64msec
   hit: 97%, 8.37usec, count=454, total=3.80msec
  miss: 2%, 218.31usec, count=13, total=2.84msec
#pmprof# width: 1.91usec, count=57254, total=109.40msec
   hit: 99%, 1.51usec, count=56962, total=86.22msec
  miss: 0%, 79.41usec, count=292, total=23.19msec

What we can see here is very few calls to breakAt. Nevertheless the 
cache is still efficient.


JMarc
From 233b7954d5877aba58ee987517e326b42d3f796e Mon Sep 17 00:00:00 2001
From: Jean-Marc Lasgouttes 
Date: Tue, 5 Jul 2016 14:06:22 +0200
Subject: [PATCH] Add caching for the QTextLayout objects we use

The QTextLayout handling is terribly slow on Qt 4.8.7, but some
caching has been added in Qt5 that makes it much faster. For some
reason, it is not that slow with Qt 4.8.1.

This commit introduces some caching, which should probably only be
active for Qt 4.

[NOTE: this version has profiling hooks, enabled by commenting out the line
  #define DISABLE_PMPROF
that should eventually be removed.]

Improve the caching of QTextLayout objects used for pos2x and x2pos
and use them for drawing too. We originally used a trivial caching
of the last QTextLayout, but now they are kept in a QCache.
Moreover, caching is enabled for these QTextLayout object (not sure
what this does).

This patch also adds some caching in the breakAt method, the only user of
QTextLayout which did not have some kind of caching already.

Finally the cached QTextLayout objects are also used by the painter
for drawing text.

For some weird reasons related to Argument-dependent look-up, the
qHash(docstring) function has to be defined in std namespace, since
lyx::docstring is actually std::basic_string.
---
 src/frontends/qt4/GuiFontMetrics.cpp |  133 +++---
 src/frontends/qt4/GuiFontMetrics.h   |   27 ---
 src/frontends/qt4/GuiPainter.cpp |   13 +++-
 src/support/convert.cpp  |7 ++
 4 files changed, 123 insertions(+), 57 deletions(-)

diff --git a/src/frontends/qt4/GuiFontMetrics.cpp b/src/frontends/qt4/GuiFontMetrics.cpp
index d3c89f1..d4aabf4 100644
--- a/src/frontends/qt4/GuiFontMetrics.cpp
+++ b/src/frontends/qt4/GuiFontMetrics.cpp
@@ -21,11 +21,33 @@
 
 #include "insets/Inset.h"
 
+#include "support/convert.h"
 #include "support/lassert.h"
 
+#define DISABLE_PMPROF
+#include "support/pmprof.h"
+
+#include 
+
 using namespace std;
 using namespace lyx::support;
 
+namespace std {
+
+/*
+ * Argument-dependent lookup implies that this function shall be
+ * declared in the namespace of its argument. But this is std
+ * namespace, since lyx::docstring is just std::basic_string.
+ */
+uint qHash(lyx::docstring const & s)
+{
+	return qHash(QByteArray(reinterpret_cast(s.data()),
+	s.size() * sizeof(lyx::docstring::value_type)));
+}
+
+}
+
+
 namespace lyx {
 namespace frontend {
 
@@ -51,11 +73,15 @@ inline QChar const ucs4_to_qchar(char_type const ucs4)
 } // anon namespace
 
 
-// Limit strwidth_cache_ size to 512kB of string data
+/*
+ * Limit (strwidth|breakat)_cache_ size to 512kB of string data.
+ * Limit qtextlayout_cache_ size to 500 elements (we do not know the
+ * size of the QTextLayout objects anyway).
+ * Note that all these numbers are arbitrary.
+ */
 GuiFontMetrics::GuiFontMetrics(QFont const & font)
 	: font_(font), metrics_(font, 0),
-	  

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 09/12/2016 à 15:37, Tommaso Cucinotta a écrit :

On 09/12/2016 11:34, Jean-Marc Lasgouttes wrote:

Note though
that with my patch we directly draw the cached QTextLayout object.


your patch turns LyX from a snail to a lightning fast editor :-)!

It's a long time I don't have memory of being able to go through a doc
at such a speed, just keeping the cursor down key pressed.
How big can this new cache grow ? Perhaps you may want to add a dump of
its size under -dbg ...


The cache sizes are completely arbitrary: from the code
/*
 * Limit (strwidth|breakat)_cache_ size to 512kB of string data.
 * Limit qtextlayout_cache_ size to 500 elements (we do not know the
 * size of the QTextLayout objects anyway).
 * Note that all these numbers are arbitrary.
 */

For the strwidth and breakat caches, what we cache is either the length 
of the string, or the cursor position at which the string should be 
broken (given the margin). We limit to 512k of string data, which is 
pretty conservative.


For the other cache which contains a full QTextLayout object, I have 
limited the size to 500 elements, because I have no idea of the size of 
the thing. If somebody can tell what is the price to pay for these 
things, we may decide to increase the cache size.


Could you try to run the massif tool of valgrind with the patch attached?

JMarc

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

On 09/12/2016 11:34, Jean-Marc Lasgouttes wrote:

Note though
that with my patch we directly draw the cached QTextLayout object.


your patch turns LyX from a snail to a lightning fast editor :-)!

It's a long time I don't have memory of being able to go through a doc at such 
a speed, just keeping the cursor down key pressed.
How big can this new cache grow ? Perhaps you may want to add a dump of its size 
under -dbg ...

Thanks,

T.

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 08/12/2016 à 23:18, Tommaso Cucinotta a écrit :

likely with this sequel [1], so the innermost LyX code seems:

lyx::frontend::GuiFontMetrics::breakAt,lyx::Row::Element::breakAt,
lyx::Row::shortenIfNeeded,lyx::TextMetrics::breakRow,lyx::TextMetrics::redoParagraph,


breakAt is the main method for which caching is added in my patch.


Now, I'm just moving the cursor and sometimes selecting with Shift down,
so do we actually need to redoParagraph() ?


This is an excellent question :) The problem is that this update code is 
very cery fragile. I have tried in the past to streamline 
BufferView::processUpdateFlags and BufferView::draw without much 
success. There are always cases where metrics are lost for some reason.


You can read development/PAINTING_ANALYSIS for some of my thoughts on 
the subject.
One thing I have been working towards is separating computation of 
metrics and inset positions, from drawing itself. This would have the 
obvious advantage to allow painting to happen at redraw events (or 
whatever they are called), like any normal application does.


However, I realize now that my assumption that painting is more 
expensive than metrics is not correct, especially now that we use 
QTextLayout to do the glyph combination for us. It might be that we 
should be at a lower level and leverage harfbuzz (or whatever thing I do 
not know) to this line breaking and cursor positioning job. I do not 
know what is the extra amount of work that is done by QTextLayout and 
that we do not really need. Note though that with my patch we directly 
draw the cached QTextLayout object.


And of course, the second work is to reduce the amount of metrics 
computation and of painting that happens. This is a pretty difficult 
job, especially since the code is peppered with random buffer.changed() 
calls (which force a redraw and/or metrics computation) which have been 
added for some forgotten reason, probably that it was easier than just 
thinking about what we were doing.


JMarc

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

In case it might help, this seems a recurrent stack trace during the slowness

writev,??,??,xcb_writev,_XSend,XRenderAddGlyphs,QFontEngineX11FT::uploadGlyphToServer(QFontEngineFT::QGlyphSet*,,
QFontEngineFT::loadGlyph(QFontEngineFT::QGlyphSet*,,QFontEngineFT::recalcAdvances(QGlyphLayout*,,??,??,??,??,
QTextEngine::shapeTextWithHarfbuzz(int),QTextEngine::shapeText(int),QTextEngine::shape(int)

likely with this sequel [1], so the innermost LyX code seems:

lyx::frontend::GuiFontMetrics::breakAt,lyx::Row::Element::breakAt,
lyx::Row::shortenIfNeeded,lyx::TextMetrics::breakRow,lyx::TextMetrics::redoParagraph,

Now, I'm just moving the cursor and sometimes selecting with Shift down, so do 
we actually need to redoParagraph() ?

Thanks,

T.

[1]

writev,??,??,xcb_writev,_XSend,XRenderAddGlyphs,QFontEngineX11FT::uploadGlyphToServer(QFontEngineFT::QGlyphSet*,,
QFontEngineFT::loadGlyph(QFontEngineFT::QGlyphSet*,,QFontEngineFT::recalcAdvances(QGlyphLayout*,,??,??,??,??,
QTextEngine::shapeTextWithHarfbuzz(int),QTextEngine::shapeText(int),QTextEngine::shape(int),
QTextLine::layout_helper(int),lyx::frontend::GuiFontMetrics::breakAt,lyx::Row::Element::breakAt,
lyx::Row::shortenIfNeeded,lyx::TextMetrics::breakRow,lyx::TextMetrics::redoParagraph,
lyx::BufferView::updateMetrics,lyx::BufferView::processUpdateFlags,lyx::BufferView::mouseEventDispatch,
lyx::frontend::GuiWorkArea::Private::dispatch,lyx::frontend::GuiWorkArea::mousePressEvent,QWidget::event(QEvent*),
QFrame::event(QEvent*),QCoreApplicationPrivate::sendThroughObjectEventFilters(QObject*,,
QApplicationPrivate::notify_helper(QObject*,,QApplication::notify(QObject*,,lyx::frontend::GuiApplication::notify,
QCoreApplication::notifyInternal(QObject*,,QApplicationPrivate::sendMouseEvent(QWidget*,,??,
QApplication::x11ProcessEvent(_XEvent*),??,g_main_context_dispatch,??,g_main_context_iteration,
QEventDispatcherGlib::processEvents(QFlags),??,
QEventLoop::processEvents(QFlags),
QEventLoop::exec(QFlags),QCoreApplication::exec(),
lyx::frontend::GuiApplication::exec,lyx::LyX::exec,main

On 08/12/2016 12:36, Tommaso Cucinotta wrote:

On 08/12/2016 11:09, Tommaso Cucinotta wrote:

I'm now trying oprofile/operf, just to compare output.


that's quite similar

CPU_CLK_UNHALT...|
  samples|  %|
--
  1873080 100.000 lyx
CPU_CLK_UNHALT...|
  samples|  %|
--
  1636151 87.3508 libfreetype.so.6.12.3
70092  3.7421 libQtGui.so.4.8.7
64930  3.4665 libc-2.24.so
52064  2.7796 libQtCore.so.4.8.7
27387  1.4621 kallsyms
 7808  0.4169 libXrender.so.1.3.0
 4479  0.2391 libX11.so.6.3.0
 4366  0.2331 libstdc++.so.6.0.22
 2532  0.1352 ld-2.24.so
 1226  0.0655 lyx
  866  0.0462 libxcb.so.1.1.0
  651  0.0348 libpthread-2.24.so
  275  0.0147 libglib-2.0.so.0.5000.0
   40  0.0021 libqgif.so
   30  0.0016 libdbus-1.so.3.14.7
   29  0.0015 libgdk-x11-2.0.so.0.2400.30


T.

Re: LyX 2.2 slowness

[Guillaume Munch]

Le 08/12/2016 à 16:07, Jean-Marc Lasgouttes a écrit :


Also, it would be nice to know what are the callers of freettype that
consume the most time.


As a reminder here is where I got last time with kcachegrind:
https://www.mail-archive.com/lyx-devel@lists.lyx.org/msg194401.html

The caching in Qt5 seems much lower-level than what you have access to,
so it is not impossible that you can no longer get good performance with
Qt4. Let us hope that your patch works.

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 08/12/2016 à 16:01, Tommaso Cucinotta a écrit :

On 08/12/2016 14:39, Jean-Marc Lasgouttes wrote:

On my ubuntu 12.40 station I get:
Xft.dpi:96
Xft.antialias:1
Xft.hinting:1
Xft.hintstyle:hintslight
Xft.rgba:none
Xft.lcdfilter:none



mine:


Did you try the patch I posted, BTW?

Also, it would be nice to know what are the callers of freettype that 
consume the most time.


An option to see this is the new sysprof. I am still trying to make it 
work, though :)


JMarc

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

On 08/12/2016 14:39, Jean-Marc Lasgouttes wrote:

On my ubuntu 12.40 station I get:
Xft.dpi:96
Xft.antialias:1
Xft.hinting:1
Xft.hintstyle:hintslight
Xft.rgba:none
Xft.lcdfilter:none



mine:

tommaso@tommylap:~/lyx-trunk-ws/lyx$ xrdb -query |grep Xft
Xft.antialias:  1
Xft.dpi:96
Xft.hinting:1
Xft.hintstyle:  hintslight
Xft.rgba:   rgb

T.

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 08/12/2016 à 14:29, Jean-Marc Lasgouttes a écrit :

Le 08/12/2016 à 12:36, Tommaso Cucinotta a écrit :

On 08/12/2016 11:09, Tommaso Cucinotta wrote:

I'm now trying oprofile/operf, just to compare output.


that's quite similar


How do we know what are the freetype settings in effect (anitalisaing,
hinting, ...)?


OK, it is
  xrdb -query |grep Xft

On my ubuntu 12.40 station I get:
Xft.dpi:96
Xft.antialias:  1
Xft.hinting:1
Xft.hintstyle:  hintslight
Xft.rgba:   none
Xft.lcdfilter:  none

This means that I have slgight grayscale antialiasing.

Note that AFAIK, subpixel aliasing does not work with LyX, since we 
paint on an intermediate Pixmap and not directly on screen.


JMarc

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 08/12/2016 à 12:36, Tommaso Cucinotta a écrit :

On 08/12/2016 11:09, Tommaso Cucinotta wrote:

I'm now trying oprofile/operf, just to compare output.


that's quite similar


How do we know what are the freetype settings in effect (anitalisaing, 
hinting, ...)?


JMarc

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

On 08/12/2016 11:09, Tommaso Cucinotta wrote:

I'm now trying oprofile/operf, just to compare output.


that's quite similar

CPU_CLK_UNHALT...|
  samples|  %|
--
  1873080 100.000 lyx
CPU_CLK_UNHALT...|
  samples|  %|
--
  1636151 87.3508 libfreetype.so.6.12.3
70092  3.7421 libQtGui.so.4.8.7
64930  3.4665 libc-2.24.so
52064  2.7796 libQtCore.so.4.8.7
27387  1.4621 kallsyms
 7808  0.4169 libXrender.so.1.3.0
 4479  0.2391 libX11.so.6.3.0
 4366  0.2331 libstdc++.so.6.0.22
 2532  0.1352 ld-2.24.so
 1226  0.0655 lyx
  866  0.0462 libxcb.so.1.1.0
  651  0.0348 libpthread-2.24.so
  275  0.0147 libglib-2.0.so.0.5000.0
   40  0.0021 libqgif.so
   30  0.0016 libdbus-1.so.3.14.7
   29  0.0015 libgdk-x11-2.0.so.0.2400.30


T.

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

On 08/12/2016 04:49, Richard Heck wrote:

I could do a valgrind thing of the same sort if you tell me what command
to run.


it's quite straightforward:

  valgrind --tool=callgrind /usr/bin/lyx
  # play with lyx, especially open a doc with a full page of text, move cursor 
on it, select parts with shift+{r,l,u,d}, etc..., then quit
  # you get a callgrind.out. file, open it with
  kcachegrind ./callgrind.out.

I'm now trying oprofile/operf, just to compare output.

T.

Re: LyX 2.2 slowness

[Richard Heck]

On 12/07/2016 06:46 PM, Tommaso Cucinotta wrote:
> On 08/12/2016 00:36, Richard Heck wrote:
>> On 12/07/2016 04:44 PM, Tommaso Cucinotta wrote:
>>> More info at: http://retis.sssup.it/~tommaso/callgrind.out.17492.gz
>>
>> Is it possible this has something to do with what screen fonts are being
>> used?
>
> The recompiled & Qt 4.8.7-linked version, with -g enabled (master):
>
>   http://retis.sssup.it/~tommaso/callgrind.out.10301.gz
>
> slowness seems there, changed screen font to a different one, slowness
> still there, e.g., when keeping a key pressed, but especially when
> selecting with shift+cursor ...

I wonder if it has something to do with the way fonts are rendered in
Ubuntu, as opposed to Fedora. It's weird it happens in the former but
not the latter, with the same Qt version. I've just checked again with
master, and there's nothing here.

I could do a valgrind thing of the same sort if you tell me what command
to run.

Richard

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

On 08/12/2016 00:36, Richard Heck wrote:

On 12/07/2016 04:44 PM, Tommaso Cucinotta wrote:

More info at: http://retis.sssup.it/~tommaso/callgrind.out.17492.gz


Is it possible this has something to do with what screen fonts are being
used?


The recompiled & Qt 4.8.7-linked version, with -g enabled (master):

  http://retis.sssup.it/~tommaso/callgrind.out.10301.gz

slowness seems there, changed screen font to a different one, slowness still 
there, e.g., when keeping a key pressed, but especially when selecting with 
shift+cursor ...

T.

Re: LyX 2.2 slowness

[Richard Heck]

On 12/07/2016 04:44 PM, Tommaso Cucinotta wrote:
> On 06/12/2016 23:12, Tommaso Cucinotta wrote:
>> I'll try to profile later, unless others made progress on this already.
>
> valgrind --tool=callgrind reports, without recompiling with -g, the
> following:
>
> libfreetype.so.6.12.377.14%
> libQtGui.so.4.8.78.10%
> ...
>
> ~46 million calls to smth in libfreetype.so, in smth like ~5 mins of
> running LyX.
>
> More info at: http://retis.sssup.it/~tommaso/callgrind.out.17492.gz

Is it possible this has something to do with what screen fonts are being
used?

Richard

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

On 07/12/2016 18:58, Richard Heck wrote:

- why is the slowness visible on Ubuntu 16.10 (Qt 4.8.7) but not
Ubuntu 12.04 (Qt 4.8.1)?


I'm having the issue on Ubuntu 16.10 with Qt 4.8.7.

tommaso@tommylap:~/lyx-trunk-ws/lyx$ dpkg -l | grep libqt4 | grep -v 386
ii  libqt4-dbus:amd64   
4:4.8.7+dfsg-7ubuntu1   amd64Qt 4 D-Bus module
ii  libqt4-declarative:amd64
4:4.8.7+dfsg-7ubuntu1   amd64Qt 4 Declarative module
ii  libqt4-designer:amd64   
4:4.8.7+dfsg-7ubuntu1   amd64Qt 4 designer module
ii  libqt4-dev  
4:4.8.7+dfsg-7ubuntu1   amd64Qt 4 development files
...

T.

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

On 06/12/2016 23:12, Tommaso Cucinotta wrote:

I'll try to profile later, unless others made progress on this already.


valgrind --tool=callgrind reports, without recompiling with -g, the following:

libfreetype.so.6.12.3   77.14%
libQtGui.so.4.8.7   8.10%
...


~46 million calls to smth in libfreetype.so, in smth like ~5 mins of running 
LyX.

More info at: http://retis.sssup.it/~tommaso/callgrind.out.17492.gz

Does it help?

T.

Re: LyX 2.2 slowness

[Richard Heck]

On 12/07/2016 06:15 AM, Jean-Marc Lasgouttes wrote:
>
> - why is the slowness visible on Ubuntu 16.10 (Qt 4.8.7) but not
> Ubuntu 12.04 (Qt 4.8.1)?

And more weirdly: I have never seen this kind of problem on Fedora with
Qt 4.8.7.

Richard

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 07/12/2016 à 12:15, Jean-Marc Lasgouttes a écrit :

I'll post a new version to try soon.


Here is a patch to play with. It is not perfect, but I would be 
interested to know whether it improves performance with Qt4.


JMarc

From 186d439af023942a6171a6a8ce5bafaa2c6715e0 Mon Sep 17 00:00:00 2001
From: Jean-Marc Lasgouttes 
Date: Tue, 5 Jul 2016 14:06:22 +0200
Subject: [PATCH] Add caching for the QTextLayout objects we use

The QTextLayout handling is terribly slow on Qt 4.8.7, but some
caching has been added in Qt5 that makes it much faster. For some
reason, it is not that slow with Qt 4.8.1.

This commit introduces some caching, which should probably only be
active for Qt 4.

Improve the caching of QTextLayout objects used for pos2x and x2pos
and use them for drawing too. We originally used a trivial caching
of the last QTextLayout, but now they are kept in a QCache.
Movoerover, caching is enabled for these QTextLayout object (not sure
what this does).

This patch also adds some caching in the breakAt method, the only user of
QTextLayout which did not have some kind of caching already.

For some weird reasons related to Argument-dependent lookup, the
qHash(docstring) function has to be defined in std namespace, since
lyx::docstring is actually std::basic_string.
---
 src/frontends/qt4/GuiFontMetrics.cpp |  124 ++
 src/frontends/qt4/GuiFontMetrics.h   |   27 
 src/frontends/qt4/GuiPainter.cpp |   13 +++-
 src/support/convert.cpp  |7 ++
 4 files changed, 114 insertions(+), 57 deletions(-)

diff --git a/src/frontends/qt4/GuiFontMetrics.cpp b/src/frontends/qt4/GuiFontMetrics.cpp
index d3c89f1..a974f49 100644
--- a/src/frontends/qt4/GuiFontMetrics.cpp
+++ b/src/frontends/qt4/GuiFontMetrics.cpp
@@ -21,11 +21,30 @@
 
 #include "insets/Inset.h"
 
+#include "support/convert.h"
 #include "support/lassert.h"
 
+#include 
+
 using namespace std;
 using namespace lyx::support;
 
+namespace std {
+
+/*
+ * Argument-dependent lookup implies that this function shall be
+ * declared in the namespace of its argument. But this is std
+ * namespace, since lyx::docstring is just std::basic_string.
+ */
+uint qHash(lyx::docstring const & s)
+{
+	return qHash(QByteArray(reinterpret_cast(s.data()),
+	s.size() * sizeof(lyx::docstring::value_type)));
+}
+
+}
+
+
 namespace lyx {
 namespace frontend {
 
@@ -51,11 +70,15 @@ inline QChar const ucs4_to_qchar(char_type const ucs4)
 } // anon namespace
 
 
-// Limit strwidth_cache_ size to 512kB of string data
+/*
+ * Limit (strwidth|breakat)_cache_ size to 512kB of string data.
+ * Limit qtextlayout_cache_ size to 500 elements (we do not know the
+ * size of the QTextLayout objects anyway).
+ * Note that all these numbers are arbitrary.
+ */
 GuiFontMetrics::GuiFontMetrics(QFont const & font)
 	: font_(font), metrics_(font, 0),
-	  strwidth_cache_(1 << 19),
-	  tl_cache_rtl_(false), tl_cache_wordspacing_(-1.0)
+	  strwidth_cache_(1 << 19), breakat_cache_(1 << 19),  qtextlayout_cache_(500)
 {
 }
 
@@ -140,10 +163,7 @@ int GuiFontMetrics::rbearing(char_type c) const
 
 int GuiFontMetrics::width(docstring const & s) const
 {
-	QByteArray qba =
-		QByteArray(reinterpret_cast(s.data()),
-		   s.size() * sizeof(docstring::value_type));
-	int * pw = strwidth_cache_[qba];
+	int * pw = strwidth_cache_[s];
 	if (pw)
 		return *pw;
 	// For some reason QMetrics::width returns a wrong value with Qt5
@@ -155,8 +175,7 @@ int GuiFontMetrics::width(docstring const & s) const
 	QTextLine line = tl.createLine();
 	tl.endLayout();
 	int w = int(line.naturalTextWidth());
-
-	strwidth_cache_.insert(qba, new int(w), qba.size());
+	strwidth_cache_.insert(s, new int(w), s.size() * sizeof(char_type));
 	return w;
 }
 
@@ -179,26 +198,27 @@ int GuiFontMetrics::signedWidth(docstring const & s) const
 }
 
 
-QTextLayout const &
-GuiFontMetrics::getTextLayout(docstring const & s, QFont font,
-  bool const rtl, double const wordspacing) const
+QTextLayout const *
+GuiFontMetrics::getTextLayout(docstring const & s, bool const rtl,
+  double const wordspacing) const
 {
-	if (s != tl_cache_s_ || font != tl_cache_font_ || rtl != tl_cache_rtl_
-	|| wordspacing != tl_cache_wordspacing_) {
-		tl_cache_.setText(toqstr(s));
-		font.setWordSpacing(wordspacing);
-		tl_cache_.setFont(font);
+	docstring const s_cache = s + (rtl ? "r" : "l") + convert(wordspacing);
+	QTextLayout * ptl = qtextlayout_cache_[s_cache];
+	if (!ptl) {
+		ptl = new QTextLayout();
+		ptl->setCacheEnabled(true);
+		ptl->setText(toqstr(s));
+		QFont copy = font_;
+		copy.setWordSpacing(wordspacing);
+		ptl->setFont(copy);
 		// Note that both setFlags and the enums are undocumented
-		tl_cache_.setFlags(rtl ? Qt::TextForceRightToLeft : Qt::TextForceLeftToRight);
-		tl_cache_.beginLayout();
-		tl_cache_.createLine();
-		tl_cache_.endLayout();
-		tl_cache_s_ = s;
-		tl_cache_font_ = 

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 07/12/2016 à 11:32, Tommaso Cucinotta a écrit :

On 06/12/2016 23:48, Scott Kostyshak wrote:

On Tue, Dec 06, 2016 at 11:12:56PM +0100, Tommaso Cucinotta wrote:

Hi there,

I haven't seen this, and I use LyX 2.2 on Ubuntu daily. Which Qt version
are you using?


tommaso@tommylap:~$ ldd `which lyx` | grep -i qt
libQtGui.so.4 => /usr/lib/x86_64-linux-gnu/libQtGui.so.4
(0x7fb5a7544000)
libQtCore.so.4 => /usr/lib/x86_64-linux-gnu/libQtCore.so.4
(0x7fb5a7052000)

Also, forgot to mention that in the recompiled from master I don't see
the slowness,
but that's recompiled with Qt5.


There is a known slowness problem in 2.2 and master that is due to our 
strong reliance on QTextLayout. In Qt4 this is expensive. In Qt5 the 
situaiton is better because there is some QTextLayout caching in place.


I have a patch to implement some caching that really improves the 
situation in Qt4. However, there are several unresolved things:

- what is the memory overhead?
- should we cache with Qt5 too (I guess not)
- why is the slowness visible on Ubuntu 16.10 (Qt 4.8.7) but not Ubuntu 
12.04 (Qt 4.8.1)?

- there are still a few quirks that I did not take time to chase

And of course, the patch has to be adapted to the recent code changes.

I'll post a new version to try soon.

JMarc

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

On 06/12/2016 23:48, Scott Kostyshak wrote:

On Tue, Dec 06, 2016 at 11:12:56PM +0100, Tommaso Cucinotta wrote:

Hi there,

I haven't seen this, and I use LyX 2.2 on Ubuntu daily. Which Qt version
are you using?


tommaso@tommylap:~$ ldd `which lyx` | grep -i qt
libQtGui.so.4 => /usr/lib/x86_64-linux-gnu/libQtGui.so.4 
(0x7fb5a7544000)
libQtCore.so.4 => /usr/lib/x86_64-linux-gnu/libQtCore.so.4 
(0x7fb5a7052000)

Also, forgot to mention that in the recompiled from master I don't see the 
slowness,
but that's recompiled with Qt5.


The only thing I've noticed regarding slowness is that
starting up LyX with Qt 4 is snappier than with Qt 5.


I have actually a shutdown slowness problem as well, which affects also master, 
in that,
when closing LyX (e.g., Alt+F4), the program stays there for ~2 secs before 
actually
closing ... quite annoying while developing for the frequent start/quit cycles.


I'll try to profile later, unless others made progress on this already.


This would be good I think. Also, are you able to reproduce the slowness
with e.g. LyX 2.1 compiled with the same version of Qt? If not, then
maybe a git bisect would be worth it.


Thanks for the tips, I'll try a few of these alternative compilations when I 
have some time.

T.

Re: LyX 2.2 slowness

[Scott Kostyshak]

On Tue, Dec 06, 2016 at 11:12:56PM +0100, Tommaso Cucinotta wrote:
> Hi there,
> 
> I just used LyX 2.2 (from Ubuntu) today to take some notes, and noticed also
> this terrible slowness on a very small document, without any pane open, just
> showing up while typing up to a ~3-4 lines paragraph without breaks, with the
> slowness also just for moving cursor, to the point that I had to point and
> click, in order to be faster than the usual Ctrl+.

I haven't seen this, and I use LyX 2.2 on Ubuntu daily. Which Qt version
are you using? The only thing I've noticed regarding slowness is that
starting up LyX with Qt 4 is snappier than with Qt 5.

> I'll try to profile later, unless others made progress on this already.

This would be good I think. Also, are you able to reproduce the slowness
with e.g. LyX 2.1 compiled with the same version of Qt? If not, then
maybe a git bisect would be worth it.

Scott


signature.asc
Description: PGP signature

Re: LyX 2.2 slowness

[Tommaso Cucinotta]

Hi there,

I just used LyX 2.2 (from Ubuntu) today to take some notes, and noticed also
this terrible slowness on a very small document, without any pane open, just
showing up while typing up to a ~3-4 lines paragraph without breaks, with the
slowness also just for moving cursor, to the point that I had to point and
click, in order to be faster than the usual Ctrl+.

I'll try to profile later, unless others made progress on this already.

Thanks,

T.

On 29/08/2016 12:07, Jean-Marc Lasgouttes wrote:

Le 29/08/2016 à 12:04, racoon a écrit :

LyX 2.2 is very slow compared to LyX 2.1.4 (on Windows). (I have only a
meager dual core processor though.)

For example, if I open a long document like the Users Guide and make a
text selection bigger or smaller 2.2 stutters a lot (up to not reacting)
while 2.1.4 works fine. I am not sure this is just the CPU load because
there is "only" a 10 percent difference (50 vs. 40 %).


What Qt version is in use? I guess it is Qt5.x.

Is it only when the document is long?

Do you have any way to run a profiler?

JMarc

Re: LyX 2.2 slowness

[racoon]

On 29.08.2016 12:04, racoon wrote:

LyX 2.2 is very slow compared to LyX 2.1.4 (on Windows). (I have only a
meager dual core processor though.)

For example, if I open a long document like the Users Guide and make a
text selection bigger or smaller 2.2 stutters a lot (up to not reacting)
while 2.1.4 works fine. I am not sure this is just the CPU load because
there is "only" a 10 percent difference (50 vs. 40 %).


Sorry, my mistake. I had the complete source preview open somewhere. I 
guess that explains the slowness...


Daniel

Re: LyX 2.2 slowness

[Jean-Marc Lasgouttes]

Le 29/08/2016 à 12:04, racoon a écrit :

LyX 2.2 is very slow compared to LyX 2.1.4 (on Windows). (I have only a
meager dual core processor though.)

For example, if I open a long document like the Users Guide and make a
text selection bigger or smaller 2.2 stutters a lot (up to not reacting)
while 2.1.4 works fine. I am not sure this is just the CPU load because
there is "only" a 10 percent difference (50 vs. 40 %).


What Qt version is in use? I guess it is Qt5.x.

Is it only when the document is long?

Do you have any way to run a profiler?

JMarc