Re: [Geany-devel] editing big files can be too slow with tag reparsing - HTML
On Tue, Sep 27, 2011 at 14:17, Nick Treleaven wrote: > On 26/09/2011 23:00, Colomban Wendling wrote: >> >> Le 26/09/2011 13:48, Nick Treleaven a écrit : >>> >>> I'm reduced to using Windows. I guess the reason is the regex code that >>> is really old which is used for Windows builds. >> >> Maybe yeah, but I find it a little weird you see *so much* slowness... >> >> For comparison, on my machine [1] parsing and updating the list takes >> less than 60ms for geany.html. > > Just to be sure, are you seeing the 'built-in regex' debug message: > > 12:42:07: (null) INFO : GTK 2.22.0, GLib 2.26.0, built-in regex > >>> It might help to update >>> it from CTags (we had a patch from Jiri Techet but it didn't look like >>> it would build on Windows, i.e. wasn't from CTags). >> >> Maybe we could use GRegex now we have a GLib recent enough? >> >> Anyway tagamanger/regex.c looks like a few glibc source file >> concatenated with a 0xff separator, I'd then guess any POSIX RE >> implementation (e.g. today's glibc's one) would do, wouldn't it? > > That's probably what Jiri did, but I noticed CTags had some modifications to > build on Windows. Perhaps glibc would work, but I think taking it from ctags > is safer. Basically I took the (then) latest versions of the regex files plus the makefile from ctags. The patch still applies by the way. I've just diffed the result of my patch with the ctags regex and there aren't big differences. What I've forgotten include into my patch is a conditional include under windows in regex_internal.h so if you want to use my patch, substitute this file with the ctags one. I haven't tested it under Windows myself so I don't know what happens. Cheers, Jiri ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 28 September 2011 22:01, Nick Treleaven wrote: > On 28/09/2011 12:56, Lex Trotman wrote: >> >> Actually that leads to the question, why is it reparsing an unchanged >> file after the first time anyway? > > It isn't, AFAICT. I was editing doc/geany.html (python not installed yet). So you was docutils :) no wonder it was annoying when it was slow. I know Colomban hasn't done the docs yet, we need to be careful to explain this well, there is no other automatic preference overrideing feature in Geany. Needs to explain that tags will stay there but won't update until save (they will still update on save right?) so the user can figure out how to realign tags and file after, for example, they delete a whole chapter of geany.html and make most of their tags useless. The statusbar message needs to be logged to the status window as well, status bar messages can be overwritten at any time. Cheers Lex ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 28/09/2011 12:56, Lex Trotman wrote: Actually that leads to the question, why is it reparsing an unchanged file after the first time anyway? It isn't, AFAICT. I was editing doc/geany.html (python not installed yet). ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 28 September 2011 21:12, Nick Treleaven wrote: > On 28/09/2011 12:03, Nick Treleaven wrote: >> >> @Nick, there are lots of large generated files a programmer might want >> to check, especially XMLs and logs, saying there is an xxxGb maximum >> is wrong, it depends on the machine the user has. Forcing them to >> have to go use another editor or to not have highlighting is bad > > Sorry, I missed the bit where you said no highlighting is bad. But in that > case, auto-disable works fine. Actually I should make it clear that I am not talking about editing huge files, but displaying them and showing all the navigational assistance that we can provide. So I guess so long as the tags generated by the first run that took too long and triggered the disable are going to be still there, then disabling it is ok. Actually that leads to the question, why is it reparsing an unchanged file after the first time anyway? In any case, there is no xxx Gb maximum, I > just meant that we shouldn't have a document menu option that is only for > disabling reparsing when it can be done automatically. > > Also I'm not sure whether Scintilla handles very big files well. I don't > think we need to be concerned about that. Like molasses :) if you try to edit it that is. Cheers Lex > ___ > Geany-devel mailing list > Geany-devel@uvena.de > https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel > ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 28/09/2011 12:03, Nick Treleaven wrote: @Nick, there are lots of large generated files a programmer might want to check, especially XMLs and logs, saying there is an xxxGb maximum is wrong, it depends on the machine the user has. Forcing them to have to go use another editor or to not have highlighting is bad Sorry, I missed the bit where you said no highlighting is bad. But in that case, auto-disable works fine. In any case, there is no xxx Gb maximum, I just meant that we shouldn't have a document menu option that is only for disabling reparsing when it can be done automatically. Also I'm not sure whether Scintilla handles very big files well. I don't think we need to be concerned about that. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 28/09/2011 00:45, Lex Trotman wrote: Nick, Thomas, [...] 0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :) @Nick, there are lots of large generated files a programmer might want to check, especially XMLs and logs, saying there is an xxxGb maximum is wrong, it depends on the machine the user has. Forcing them to have to go use another editor or to not have highlighting is bad. You edited out my reply which addressed that: On 27/09/2011 17:17, Nick Treleaven wrote: > 1. Very big files > x MB - I don't think we need to support that case > specially, Geany is a programmer's editor. The user can just open as > filetype none if they only want basic editing. In the open dialog, choose filetype None and you can easily open big files, plus it will be faster as syntax highlighting won't be enabled. @Thomas, and since the problem is with a specific large file, Colomban is right it is per document, since you only need to turn off the one that is too large. @Nick, therefore not per filetype either For large files an auto-disable reparsing feature would work fine. 0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK. This is more what I meant. I'm not so sure an extra dialog is needed, but if people want one that's ok. Would a "don't ask me again" checkbox be a good idea (if it doesn't already). @Thomas, While I still think automatic changing of settings is a last resort, and as Nick said what performance number is acceptable? If we do add it, the user must be notified, but a "don't tell me again" is ok (for the session) Perhaps I was wrong with 0.2s, maybe 0.5s (I didn't check Colomban's code to see what he used). @Nick no one reads the status bar, sadly A dialog is too intrusive. Consider opening 10 slow files and having to click the dialog for each one. Status bar is fine - as you sometimes say programmers don't need to be treated like newbies. Not having tag reparsing enabled will mean a few invalid bug reports at worst. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Nick, Thomas, [...] >> 0001-Per-document-real-time-symbols-setting.patch: >> Adds a per-document setting for real-time updates and a menu item in the >> Document menu. There is still a FIXME in it, feel free to give ideas :) > @Nick, there are lots of large generated files a programmer might want to check, especially XMLs and logs, saying there is an xxxGb maximum is wrong, it depends on the machine the user has. Forcing them to have to go use another editor or to not have highlighting is bad. @Thomas, and since the problem is with a specific large file, Colomban is right it is per document, since you only need to turn off the one that is too large. @Nick, therefore not per filetype either >> >> 0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: >> This one adds the check for the updating duration and asks the user. It >> is WIP, and I'm not really convinced by the dialog, as you can read in a >> TODO. Apart that, it seems to work pretty OK. >> > > This is more what I meant. I'm not so sure an extra dialog is needed, but if > people want one that's ok. Would a "don't ask me again" checkbox be a good > idea (if it doesn't already). @Thomas, While I still think automatic changing of settings is a last resort, and as Nick said what performance number is acceptable? If we do add it, the user must be notified, but a "don't tell me again" is ok (for the session) @Nick no one reads the status bar, sadly Cheers Lex ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Am 26.09.2011 23:38, schrieb Colomban Wendling: Hum, right, automagic might not be good, asking the user is always better. I've got 2 WIP patches, though they might not be appropriate to commit before release since they add strings... anyway, here they are: 0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :) I think that one isn't needed. I don't think anyone wants to set it off per document, except if one parses slowly in which case we can turn it of automatically (with notification). When I said per-document setting I meant internally, not one actually accessible in the menus. 0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK. This is more what I meant. I'm not so sure an extra dialog is needed, but if people want one that's ok. Would a "don't ask me again" checkbox be a good idea (if it doesn't already). Best regards. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 26/09/2011 22:38, Colomban Wendling wrote: Le 24/09/2011 13:32, Lex Trotman a écrit : [...] How about automagically disabling auto-reparsing per-file if it takes too long (1s?) at runtime? Without needing to change the setting that is. I disagree, it is a bad idea to turn off a setting that changes the behavior. The user is likely to be confused by the unexpected change in behavior (new symbols no longer in autocompletes, new types not being highlighted etc). Having it automagically turn off is bad. At most, pop up a dialog suggesting turning it off and noting where, but only ever once per session. Hum, right, automagic might not be good, asking the user is always better. I've got 2 WIP patches, though they might not be appropriate to commit before release since they add strings... anyway, here they are: 0001-Per-document-real-time-symbols-setting.patch: Adds a per-document setting for real-time updates and a menu item in the Document menu. There is still a FIXME in it, feel free to give ideas :) Not sure about the use case: 1. Very big files > x MB - I don't think we need to support that case specially, Geany is a programmer's editor. The user can just open as filetype none if they only want basic editing. 2. Slow tag parsers - a per-filetype setting might be more practical than per-document? I see it might be useful but I'm not convinced it's worth it. 0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: This one adds the check for the updating duration and asks the user. It is WIP, and I'm not really convinced by the dialog, as you can read in a TODO. Apart that, it seems to work pretty OK. I think this is overkill - who would want editing to have > ~0.2s delays? Auto-disable should be fine with a status bar message. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 26/09/2011 22:38, Colomban Wendling wrote: So, I'm soliciting your impressions, opinions, remarks, etc. Also, is this whole thing important enough to break string freeze less than a week before release? (read: to have some strings untranslated) Well, it's not just untranslated strings - it's best not to introduce new features close to a release in case we introduce bugs. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing - HTML
On 26/09/2011 23:00, Colomban Wendling wrote: Le 26/09/2011 13:48, Nick Treleaven a écrit : I'm reduced to using Windows. I guess the reason is the regex code that is really old which is used for Windows builds. Maybe yeah, but I find it a little weird you see *so much* slowness... For comparison, on my machine [1] parsing and updating the list takes less than 60ms for geany.html. Just to be sure, are you seeing the 'built-in regex' debug message: 12:42:07: (null) INFO : GTK 2.22.0, GLib 2.26.0, built-in regex It might help to update it from CTags (we had a patch from Jiri Techet but it didn't look like it would build on Windows, i.e. wasn't from CTags). Maybe we could use GRegex now we have a GLib recent enough? Anyway tagamanger/regex.c looks like a few glibc source file concatenated with a 0xff separator, I'd then guess any POSIX RE implementation (e.g. today's glibc's one) would do, wouldn't it? That's probably what Jiri did, but I noticed CTags had some modifications to build on Windows. Perhaps glibc would work, but I think taking it from ctags is safer. I don't know if GRegex would work, CTags parsers may use non-POSIX features - are \w \s part of POSIX? OK... with a fresh configdir? Just tried that, same problem. Task manager showed I still had about 230MB memory free whilst having the problem - the freeze caused 100% CPU as expected. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing - HTML
On 27/09/2011 01:53, Colomban Wendling wrote: Le 27/09/2011 01:44, Lex Trotman a écrit : [...] Pentium 4, 1.9 GHz, 512 MB ram. I'm no windows expert, but isn't 512mb considered kinda marginal, 1gb better. Whats your general windows performance like? I doubt that, even on Windows, it would so drastically reduce the performances. Was your RAM completely full so it needed to swap? Windows performance is fine, I run Firefox with tons of tabs easily. I have Windows XP BTW, so 512MB is not unusually low. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
[...]
> There is no prefs, but if autocompletion_update_freq is changed to or
> from 0 (e.g. disabled or enabled), then we need to update the setting
> for all documents where the user haven't done a choice. Otherwise the
> setting would only change for newly opened documents, which is no good IMHO.
>
Oops forgot about the global setting. But it shouldn't override a per
document setting chosen by the user. I think your below ensures it
doesn't?
> BTW, I just see I actually add a user_choice flag in 0002, so the info
> is here:
>
> foreach_document(i)
> {
> if (! documents[i]->priv->tag_list_update.user_choice)
> documents[i]->priv->tag_list_update.user_choice =
> editor_prefs.autocompletion_update_freq > 0;
> }
>
>
>>> 0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch:
>>> This one adds the check for the updating duration and asks the user. It
>>> is WIP, and I'm not really convinced by the dialog, as you can read in a
>>> TODO. Apart that, it seems to work pretty OK.
>>>
>>
>> Havn't had time to look at this, but as I said its a last resort.
>
[...]
>>
>> I'm not Nick, but I would suggest leaving it until after release,
>> thats not far away, and then as soon as it is committed Nick will have
>> built a new version :)
>
> Agreed, apart that if it is a real problem, Nick won't be the only one
> suffering of it. But Matthew tried on a Windows XP and didn't see any
> issue, so it at least doesn't affect everybody... /me still wonders
> what's happening.
I suspect we would have heard if it was a common problem, thats why I
think its ok to wait till after release. (I'm trying to save you work
:)
Cheers
Lex
___
Geany-devel mailing list
Geany-devel@uvena.de
https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Le 27/09/2011 02:03, Lex Trotman a écrit :
> Hey Colomban,
>
> [...]
>> 0001-Per-document-real-time-symbols-setting.patch:
>> Adds a per-document setting for real-time updates and a menu item in the
>> Document menu. There is still a FIXME in it, feel free to give ideas :)
>>
>
> This looks like a good idea.
>
> For the fixme, It shouldn't apply per document settings here, they are
> not in the prefs so they won't have changed. Just leave it out.
There is no prefs, but if autocompletion_update_freq is changed to or
from 0 (e.g. disabled or enabled), then we need to update the setting
for all documents where the user haven't done a choice. Otherwise the
setting would only change for newly opened documents, which is no good IMHO.
BTW, I just see I actually add a user_choice flag in 0002, so the info
is here:
foreach_document(i)
{
if (! documents[i]->priv->tag_list_update.user_choice)
documents[i]->priv->tag_list_update.user_choice =
editor_prefs.autocompletion_update_freq > 0;
}
>> 0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch:
>> This one adds the check for the updating duration and asks the user. It
>> is WIP, and I'm not really convinced by the dialog, as you can read in a
>> TODO. Apart that, it seems to work pretty OK.
>>
>
> Havn't had time to look at this, but as I said its a last resort.
Yeah, but if the user don't make a choice by accident just because she
was typing something else, I think it's better :)
>> Both misses docs, up to come when they are ready.
>>
>>
>> So, I'm soliciting your impressions, opinions, remarks, etc. Also, is
>> this whole thing important enough to break string freeze less than a
>> week before release? (read: to have some strings untranslated)
>>
>
> I'm not Nick, but I would suggest leaving it until after release,
> thats not far away, and then as soon as it is committed Nick will have
> built a new version :)
Agreed, apart that if it is a real problem, Nick won't be the only one
suffering of it. But Matthew tried on a Windows XP and didn't see any
issue, so it at least doesn't affect everybody... /me still wonders
what's happening.
Cheers,
Colomban
___
Geany-devel mailing list
Geany-devel@uvena.de
https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing - HTML
Le 27/09/2011 01:44, Lex Trotman a écrit : > [...] > >> Pentium 4, 1.9 GHz, 512 MB ram. >> > > I'm no windows expert, but isn't 512mb considered kinda marginal, 1gb > better. Whats your general windows performance like? I doubt that, even on Windows, it would so drastically reduce the performances. Was your RAM completely full so it needed to swap? Cheers, Colomban ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Hey Colomban, [...] > 0001-Per-document-real-time-symbols-setting.patch: > Adds a per-document setting for real-time updates and a menu item in the > Document menu. There is still a FIXME in it, feel free to give ideas :) > This looks like a good idea. For the fixme, It shouldn't apply per document settings here, they are not in the prefs so they won't have changed. Just leave it out. > 0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch: > This one adds the check for the updating duration and asks the user. It > is WIP, and I'm not really convinced by the dialog, as you can read in a > TODO. Apart that, it seems to work pretty OK. > Havn't had time to look at this, but as I said its a last resort. > Both misses docs, up to come when they are ready. > > > So, I'm soliciting your impressions, opinions, remarks, etc. Also, is > this whole thing important enough to break string freeze less than a > week before release? (read: to have some strings untranslated) > I'm not Nick, but I would suggest leaving it until after release, thats not far away, and then as soon as it is committed Nick will have built a new version :) Cheers Lex ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing - HTML
[...] > Pentium 4, 1.9 GHz, 512 MB ram. > I'm no windows expert, but isn't 512mb considered kinda marginal, 1gb better. Whats your general windows performance like? Cheers Lex ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing - HTML
Le 26/09/2011 13:48, Nick Treleaven a écrit : > On 24/09/2011 11:59, Colomban Wendling wrote: >> Le 22/09/2011 18:30, Nick Treleaven a écrit : >>> Hi, >>> Whilst opening doc/geany.html I found it takes>5s on my machine to >>> load. It's a big document though, still perhaps the HTML tag parser >>> performance could be improved. >> >> I think 5s is weird in the first place. I know geany.html is 6.5k-lines >> file, and that it may have say, 150 tags, but 5s seems way too much IMHO. >> >> Maybe we should try to find out why this is so slow on your machine, >> there might show a Geany bug, performance problem or something else. > > I'm reduced to using Windows. I guess the reason is the regex code that > is really old which is used for Windows builds. Maybe yeah, but I find it a little weird you see *so much* slowness... For comparison, on my machine [1] parsing and updating the list takes less than 60ms for geany.html. > It might help to update > it from CTags (we had a patch from Jiri Techet but it didn't look like > it would build on Windows, i.e. wasn't from CTags). Maybe we could use GRegex now we have a GLib recent enough? Anyway tagamanger/regex.c looks like a few glibc source file concatenated with a 0xff separator, I'd then guess any POSIX RE implementation (e.g. today's glibc's one) would do, wouldn't it? >> * What kind of machine did you run (basically, CPU's MHz) > > Pentium 4, 1.9 GHz, 512 MB ram. Not any kind of "slow" machine... Could anybody else try the same on another Windows machine, just to see if it is Windows-related or something? (Enrico, Matthew?) >> * Had you many HTML files open, with many tags each? >> * Had you many non-HTML files open? > > Just that one file. OK... with a fresh configdir? Regards, Colomban [1] OK, it is a little more powerful (i3 2.93GHz), but still ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Le 24/09/2011 13:32, Lex Trotman a écrit :
> [...]
>>> How about automagically disabling auto-reparsing per-file if it takes
>>> too long (1s?) at runtime? Without needing to change the setting that is.
>>
>> I like the idea.
>>
>
> Hi Colomban, Thomas,
Hey Lex, everyone,
> I disagree, it is a bad idea to turn off a setting that changes the
> behavior. The user is likely to be confused by the unexpected change
> in behavior (new symbols no longer in autocompletes, new types not
> being highlighted etc). Having it automagically turn off is bad. At
> most, pop up a dialog suggesting turning it off and noting where, but
> only ever once per session.
Hum, right, automagic might not be good, asking the user is always better.
I've got 2 WIP patches, though they might not be appropriate to commit
before release since they add strings... anyway, here they are:
0001-Per-document-real-time-symbols-setting.patch:
Adds a per-document setting for real-time updates and a menu item in the
Document menu. There is still a FIXME in it, feel free to give ideas :)
0002-Tell-the-user-if-real-time-reparsing-is-slow-and-let.patch:
This one adds the check for the updating duration and asks the user. It
is WIP, and I'm not really convinced by the dialog, as you can read in a
TODO. Apart that, it seems to work pretty OK.
Both misses docs, up to come when they are ready.
So, I'm soliciting your impressions, opinions, remarks, etc. Also, is
this whole thing important enough to break string freeze less than a
week before release? (read: to have some strings untranslated)
Cheers,
Colomban
PS:
> Although I have a great idea for an animated paper clip in the bottom
> right corner that can suggest it every couple of seconds ... :)
Hehe, I like the idea... :D
> Colomban is right in trying to evaluate where Nicks problem is first,
> thats far more likely to be productive, this sort of thing is the last
> option not the first.
BTW, after Nick's answer, it looks very weird to me, like if regexes was
eeaalyyy slow on Windows (or
everywhere?)
>From c080dec21d2bf18a525064922c9c88aeece9e1a9 Mon Sep 17 00:00:00 2001
From: Colomban Wendling
Date: Mon, 26 Sep 2011 23:25:11 +0200
Subject: [PATCH 1/2] Per-document real-time symbols setting
---
geany.glade | 10 ++
src/callbacks.c | 11 +++
src/callbacks.h |4
src/document.c| 25 +++--
src/document.h|2 ++
src/documentprivate.h |7 +--
src/editor.c |4
src/interface.c |9 +
src/ui_utils.c|4
9 files changed, 68 insertions(+), 8 deletions(-)
diff --git a/geany.glade b/geany.glade
index ba18b6f..aed5723 100644
--- a/geany.glade
+++ b/geany.glade
@@ -1407,6 +1407,16 @@
+
+ True
+ Real-time s_ymbols
+ True
+ False
+
+
+
+
+
True
In_dent Type
diff --git a/src/callbacks.c b/src/callbacks.c
index 18b98be..65af4eb 100644
--- a/src/callbacks.c
+++ b/src/callbacks.c
@@ -2127,3 +2127,14 @@ void on_detect_width_from_file_activate(GtkMenuItem *menuitem, gpointer user_dat
ui_document_show_hide(doc);
}
}
+
+
+void on_menu_real_time_symbols_activate(GtkCheckMenuItem *menuitem, gpointer user_data)
+{
+ GeanyDocument *doc = document_get_current();
+
+ if (doc != NULL)
+ {
+ document_set_auto_update_symbols(doc, gtk_check_menu_item_get_active(menuitem));
+ }
+}
diff --git a/src/callbacks.h b/src/callbacks.h
index 108b80e..d719c6d 100644
--- a/src/callbacks.h
+++ b/src/callbacks.h
@@ -670,3 +670,7 @@ on_detect_type_from_file_activate (GtkMenuItem *menuitem,
void
on_detect_width_from_file_activate (GtkMenuItem *menuitem,
gpointer user_data);
+
+void
+on_menu_real_time_symbols_activate (GtkCheckMenuItem *menuitem,
+gpointer user_data);
diff --git a/src/document.c b/src/document.c
index d185c0f..e610112 100644
--- a/src/document.c
+++ b/src/document.c
@@ -389,7 +389,8 @@ static void init_doc_struct(GeanyDocument *new_doc)
priv->undo_actions = NULL;
priv->redo_actions = NULL;
priv->line_count = 0;
- priv->tag_list_update_source = 0;
+ priv->tag_list_update.enabled = editor_prefs.autocompletion_update_freq > 0;
+ priv->tag_list_update.source = 0;
#ifndef USE_GIO_FILEMON
priv->last_check = time(NULL);
#endif
@@ -591,6 +592,17 @@ static GeanyDocument *document_create(const gchar *utf8_filename)
}
+/* set whether real-time tag parsing is enabled for a document */
+void document_set_auto_update_symbols(GeanyDocument *doc, gboolean enable)
+{
+ if (enable != doc->priv->tag_list_update.enabled)
+ {
+ doc->priv->tag_list_update.enabled = enable;
+ ui_document_show_hide(doc);
+ }
+}
+
+
/**
* Closes the given document.
*
@@ -2324,19 +2336,20 @@ static gboolean on_document_update_tag_list_idle
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Le 24/09/2011 12:39, Lex Trotman a écrit : > On 24 September 2011 20:33, Colomban Wendling > wrote: >> Le 23/09/2011 03:09, Lex Trotman a écrit : >>> On 23 September 2011 10:34, Matthew Brush wrote: On 09/22/2011 09:30 AM, Nick Treleaven wrote: > > Hi, > Whilst opening doc/geany.html I found it takes >5s on my machine to > load. It's a big document though, still perhaps the HTML tag parser > performance could be improved. > Well under 1 second to load geany.html here, and editing it is quite smooth as well. >>> >>> Ditto, but of course it depends on the machine. >> >> Same here, I don't see any kind of lag with that file. But yeah, >> depends a lot on the machine's power. >> Myself, I've found the biggest slowdown is when having lots of tags files, say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) loaded, Geany basically becomes unusable. >>> >>> IIUC the tags files are not re-loaded so the problem here suggests >>> symbol lookup problems rather than parsing problems. >> >> Actually, IIRC Tagmanager updates an array holding all workspace tags >> every time an update is done, so having many tags file or many open >> files of a same filetype *may* (?) reduce the performances. >> No guarantees on that though, there are too many dark corners in >> Tagmanager (:-') > > Are you saying that it re-reads all open tags files every time it > re-parses a source file? That would certainly make it slow. > > And if it is just an array, unless the array is indexed or sorted then > searching it would be slow for computing the auto-completions. 1) tm_workspace keeps a sorted array of all non-global tags, that is rebuilt on each reparse [1]. This needs to rebuild and sort an array holding all tags (I suspect that this may be "slow" with many tags), but don't need reparsing anything. 2) As Matthew said in another mail, tm_tags_find() uses bsearch() [2] so it's not that slow upon search. Note however that scope completion is a complete other topic -- BTW it's still broken... :/ Cheers, Colomban [1] tm_workspace.c:tm_workspace_update() [2] tm_tag.c:tm_tags_find() ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing - HTML
On 24/09/2011 11:59, Colomban Wendling wrote: Le 22/09/2011 18:30, Nick Treleaven a écrit : Hi, Whilst opening doc/geany.html I found it takes>5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved. I think 5s is weird in the first place. I know geany.html is 6.5k-lines file, and that it may have say, 150 tags, but 5s seems way too much IMHO. Maybe we should try to find out why this is so slow on your machine, there might show a Geany bug, performance problem or something else. I'm reduced to using Windows. I guess the reason is the regex code that is really old which is used for Windows builds. It might help to update it from CTags (we had a patch from Jiri Techet but it didn't look like it would build on Windows, i.e. wasn't from CTags). * What kind of machine did you run (basically, CPU's MHz) Pentium 4, 1.9 GHz, 512 MB ram. * Had you many HTML files open, with many tags each? * Had you many non-HTML files open? Just that one file. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 09/24/2011 05:49 AM, Lex Trotman wrote: Also Matthew's perceived slowness with lots of tags loaded needs checking, is it reloading them all the time (as Colomban seemed to suggest) or does the symbol table used by tagmanager have O(1), O(log), O(N) or worse performance. I'm too afraid to go back into the TagManager code, but IIRC it uses a sorted GPtrArray. I think for each new tag added it qsort()s the array and also removes duplicates by looping over the whole array. Lookups are done with bsearch() on the array I think. Not sure if that matters or is what you asked :) Cheers, Matthew Brush ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
[..] > I too think there should be a notification. But I think turning it of > per-file (without touching the actual setting) is also good and the user > will appreciate that. Per file is a good idea. > > Note that it would be turned off in the case symbol generation takes a long > time, so it wouldn't go unnoticed if it's deactivated. > I'm always surprised what users can overlook, but if we notify them, it doesn't matter how unobservant they are :) >> >> Colomban is right in trying to evaluate where Nicks problem is first, >> thats far more likely to be productive, this sort of thing is the last >> option not the first. >> > > I agree. Also Matthew's perceived slowness with lots of tags loaded needs checking, is it reloading them all the time (as Colomban seemed to suggest) or does the symbol table used by tagmanager have O(1), O(log), O(N) or worse performance. Cheers Lex ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Am 24.09.2011 13:32, schrieb Lex Trotman: Hi Colomban, Thomas, I disagree, it is a bad idea to turn off a setting that changes the behavior. The user is likely to be confused by the unexpected change in behavior (new symbols no longer in autocompletes, new types not being highlighted etc). Having it automagically turn off is bad. At most, pop up a dialog suggesting turning it off and noting where, but only ever once per session. I too think there should be a notification. But I think turning it of per-file (without touching the actual setting) is also good and the user will appreciate that. Note that it would be turned off in the case symbol generation takes a long time, so it wouldn't go unnoticed if it's deactivated. Colomban is right in trying to evaluate where Nicks problem is first, thats far more likely to be productive, this sort of thing is the last option not the first. I agree. Best regards. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
[...] >> How about automagically disabling auto-reparsing per-file if it takes >> too long (1s?) at runtime? Without needing to change the setting that is. > > I like the idea. > Hi Colomban, Thomas, I disagree, it is a bad idea to turn off a setting that changes the behavior. The user is likely to be confused by the unexpected change in behavior (new symbols no longer in autocompletes, new types not being highlighted etc). Having it automagically turn off is bad. At most, pop up a dialog suggesting turning it off and noting where, but only ever once per session. Although I have a great idea for an animated paper clip in the bottom right corner that can suggest it every couple of seconds ... :) Colomban is right in trying to evaluate where Nicks problem is first, thats far more likely to be productive, this sort of thing is the last option not the first. Cheers Lex ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Le 22/09/2011 18:30, Nick Treleaven a écrit : > Hi, > Whilst opening doc/geany.html I found it takes >5s on my machine to > load. It's a big document though, still perhaps the HTML tag parser > performance could be improved. I think 5s is weird in the first place. I know geany.html is 6.5k-lines file, and that it may have say, 150 tags, but 5s seems way too much IMHO. Maybe we should try to find out why this is so slow on your machine, there might show a Geany bug, performance problem or something else. * What kind of machine did you run (basically, CPU's MHz) * Had you many HTML files open, with many tags each? * Had you many non-HTML files open? ...and any information that might be useful (:-') (BTW, I've seen HTML tag parser is regex-based, maybe that's particularly slow?) > But now we have tag reparsing, editing geany.html becomes painful. I > know we can disable reparsing, but what else could we do to improve the > situation? It's not urgent to solve this problem, unless we want to make > reparsing off by default. I like Thomas' idea, see other mail. Regards, Colomban ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Le 23/09/2011 19:48, Thomas Martitz a écrit :
> Am 22.09.2011 18:30, schrieb Nick Treleaven:
>> Hi,
>> Whilst opening doc/geany.html I found it takes >5s on my machine to
>> load. It's a big document though, still perhaps the HTML tag parser
>> performance could be improved.
>>
>> But now we have tag reparsing, editing geany.html becomes painful. I
>> know we can disable reparsing, but what else could we do to improve
>> the situation? It's not urgent to solve this problem, unless we want
>> to make reparsing off by default.
>
>
> How about automagically disabling auto-reparsing per-file if it takes
> too long (1s?) at runtime? Without needing to change the setting that is.
I like the idea.
Having a per-document setting (enabled/disabled) that would be
automagically turned off if a configured performance impact is reached
would probably be a correct compromise.
Something like:
(arbitrary values)
max-reparse-duration: 500ms
max-reparse-duration-exceeding: 3
if (reparse_duration >= max_reparse_duration) {
/* give heavier weight to failures */
doc->n_reparse_duration_exceeding += 2;
if (doc->n_reparse_duration_exceeding >=
(max_reparse_duration_exceeding * 2)) {
doc->real_time_reparse = FALSE;
}
} else {
/* give lighter weight to success */
doc->n_reparse_duration_exceeding -= 1;
}
max-reparse-duration-exceeding is useful not to suffer of
false-positives like a reparsing happening when the machine is under
heavy load or any other temporary problem. And it is always possible to
set it to <= 1 to disable that heuristic.
Opinions, ideas?
Regards,
Colomban
___
Geany-devel mailing list
Geany-devel@uvena.de
https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 24 September 2011 20:33, Colomban Wendling wrote: > Le 23/09/2011 03:09, Lex Trotman a écrit : >> On 23 September 2011 10:34, Matthew Brush wrote: >>> On 09/22/2011 09:30 AM, Nick Treleaven wrote: Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved. >>> >>> Well under 1 second to load geany.html here, and editing it is quite smooth >>> as well. >> >> Ditto, but of course it depends on the machine. > > Same here, I don't see any kind of lag with that file. But yeah, > depends a lot on the machine's power. > >>> Myself, I've found the biggest slowdown is when having lots of tags files, >>> say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) >>> loaded, Geany basically becomes unusable. >> >> IIUC the tags files are not re-loaded so the problem here suggests >> symbol lookup problems rather than parsing problems. > > Actually, IIRC Tagmanager updates an array holding all workspace tags > every time an update is done, so having many tags file or many open > files of a same filetype *may* (?) reduce the performances. > No guarantees on that though, there are too many dark corners in > Tagmanager (:-') Are you saying that it re-reads all open tags files every time it re-parses a source file? That would certainly make it slow. And if it is just an array, unless the array is indexed or sorted then searching it would be slow for computing the auto-completions. Cheers Lex > > > Regards, > Colomban > ___ > Geany-devel mailing list > Geany-devel@uvena.de > https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel > ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Le 23/09/2011 03:09, Lex Trotman a écrit : > On 23 September 2011 10:34, Matthew Brush wrote: >> On 09/22/2011 09:30 AM, Nick Treleaven wrote: >>> >>> Hi, >>> Whilst opening doc/geany.html I found it takes >5s on my machine to >>> load. It's a big document though, still perhaps the HTML tag parser >>> performance could be improved. >>> >> >> Well under 1 second to load geany.html here, and editing it is quite smooth >> as well. > > Ditto, but of course it depends on the machine. Same here, I don't see any kind of lag with that file. But yeah, depends a lot on the machine's power. >> Myself, I've found the biggest slowdown is when having lots of tags files, >> say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) >> loaded, Geany basically becomes unusable. > > IIUC the tags files are not re-loaded so the problem here suggests > symbol lookup problems rather than parsing problems. Actually, IIRC Tagmanager updates an array holding all workspace tags every time an update is done, so having many tags file or many open files of a same filetype *may* (?) reduce the performances. No guarantees on that though, there are too many dark corners in Tagmanager (:-') Regards, Colomban ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
Am 22.09.2011 18:30, schrieb Nick Treleaven: Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved. But now we have tag reparsing, editing geany.html becomes painful. I know we can disable reparsing, but what else could we do to improve the situation? It's not urgent to solve this problem, unless we want to make reparsing off by default. How about automagically disabling auto-reparsing per-file if it takes too long (1s?) at runtime? Without needing to change the setting that is. Best regards. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 23 September 2011 10:34, Matthew Brush wrote: > On 09/22/2011 09:30 AM, Nick Treleaven wrote: >> >> Hi, >> Whilst opening doc/geany.html I found it takes >5s on my machine to >> load. It's a big document though, still perhaps the HTML tag parser >> performance could be improved. >> > > Well under 1 second to load geany.html here, and editing it is quite smooth > as well. Ditto, but of course it depends on the machine. > > Myself, I've found the biggest slowdown is when having lots of tags files, > say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) > loaded, Geany basically becomes unusable. IIUC the tags files are not re-loaded so the problem here suggests symbol lookup problems rather than parsing problems. I didn't spend much time > troubleshooting, but I suspect turning down the symbol update frequency > might help. If it is a parsing problem sure. > >> But now we have tag reparsing, editing geany.html becomes painful. I >> know we can disable reparsing, but what else could we do to improve the >> situation? It's not urgent to solve this problem, unless we want to make >> reparsing off by default. Of course if we turn it off by default a lot of people won't turn it on and won't know about the feature. IMHO its better to have it on by default and field a few ML/IRC questions about it being slow for big files. Cheers Lex > > IMO it would be best not to disabling automatic updating of the symbols, > it's a really good feature. > > Cheers, > Matthew Brush > ___ > Geany-devel mailing list > Geany-devel@uvena.de > https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel > ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] editing big files can be too slow with tag reparsing
On 09/22/2011 09:30 AM, Nick Treleaven wrote: Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved. Well under 1 second to load geany.html here, and editing it is quite smooth as well. Myself, I've found the biggest slowdown is when having lots of tags files, say more than 10 maybe. If I have all my Vala tags (the ones on the Wiki) loaded, Geany basically becomes unusable. I didn't spend much time troubleshooting, but I suspect turning down the symbol update frequency might help. But now we have tag reparsing, editing geany.html becomes painful. I know we can disable reparsing, but what else could we do to improve the situation? It's not urgent to solve this problem, unless we want to make reparsing off by default. IMO it would be best not to disabling automatic updating of the symbols, it's a really good feature. Cheers, Matthew Brush ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
[Geany-devel] editing big files can be too slow with tag reparsing
Hi, Whilst opening doc/geany.html I found it takes >5s on my machine to load. It's a big document though, still perhaps the HTML tag parser performance could be improved. But now we have tag reparsing, editing geany.html becomes painful. I know we can disable reparsing, but what else could we do to improve the situation? It's not urgent to solve this problem, unless we want to make reparsing off by default. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
