Re: [Geany-devel] tagmanager changes
On 27/04/2012 06:30, Lex Trotman wrote:
[...]
I don't understand why tagmanager has to be replaced, why not just replace
the parts you want to improve? Rewriting it is likely to lead to a new set
of bugs and be hard to review and merge changes from master.
One of the problems with tagmanager is its complexity, leading to
considerable wariness on the part of many of us about changing it
since we don't understand what we might break.
I don't see this myself, I see some complicating issues which could be
resolved (and I'm willing to work on them), but generally a sound design
for what it provides and for extra things we may want to add.
Actually documenting the overall structure of tagmanager and how it is
supposed to work would be a good thing, whats a workspace? what is it
meant to represent, how are scopes represented? etc.
Isn't it clear from the data structures? Look at TMWorkspace. Scopes and
other tag metadata is the same as CTags. Obviously if we had at-a-glance
overall documentation that would be good.
One confusing thing is that a TMTag can be used for an actual tag or for
a file. Probably that could be cleaned up.
- a multi-cache one that, as its name suggests, maintains multiple
caches (sorted tags arrays). it uses a little more memory and is
slower on insertion since it maintains several sorted lists, but a
search on an already existing cache is very fast.
Won't this be slow for adding many tags at once? How is this design better
than tagmanager?
Perhaps Colomban could confirm, but my first thought is that this is
for nested scopes.
I expect the design is better in some respects (and to be fair I didn't
look for better things), but finding a tag based on its name is
something we are always going to want to be fast. Even for scope
completion, you still need to lookup a tag structure from a name string.
So I think we will always want a sorted array of tags per document that
we can bsearch (or something equally fast).
Also, I've probably sounded quite harsh on Colomban's design, but I'm
commenting on what I think is important. I am genuinely interested in
why his design decisions are better. It's a lot to take in all at once,
so probably needs some explanation. Sorry if I didn't make that clear.
How does tagmanager handle nested scopes, or how would it need to be
modified to do so, considering the example (in C)
{ struct a o; struct a p;
o./* struct a members */
{ struct b o;
o./* struct b members */
p./* struct a members */
}
o./* struct a members */
p./* struct a members */
{ struct c o;
o./* struct c members */
p./* struct a members */
}
o./* struct a members */
p./* struct a members */
}
How much needs to be changed in tagmanager so that the right
autocompletes can be provided at each comment? (assuming c.c is
taught to parse local variables of course)
I don't know, but we still need fast tag lookup based on name. If O(n)
scope lookup is too slow, we will need additional data structures
arranged differently, but whatever we have should have something like
O(log n) lookup for names as this is by far the most common operation.
* this backend abstraction might be really overkill, and maybe we
could do better without it?;
I don't see why having two is better. The memory overhead for a pointer
array is not much vs. the size of the tag structures. Fast searching is
important.
It is but is it flexible enough to be expanded to nested scopes.
see above.
* tags (and most types) are reference-counted (so they aren't
necessarily duplicated, e.g. in the multicache backend);
I don't really understand src/symbols.c since the real-time parsing change,
so don't really understand why this is needed.
Blame C++ and overloaded names I think.
I looked at the thread about that, and from what I could tell, the
problem was for reparsing unsaved files. Wasn't the order OK for files
that have just been saved? (Also I don't follow what that has to do with
reference counting).
I don't really see what the problem understanding it is. I thought scope
completion was just tm_workspace_find_scoped and related functions, not some
tagmanager-wide problem.
I think the fact that this isn't clear is the problem :)
If I'm following you correctly, I think you're saying the design needs
to change, which I accept may be true. What I was saying was that
understanding the existing code for scope completion is not really that
complex.
Regards,
Nick
___
Geany-devel mailing list
Geany-devel@uvena.de
https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] tagmanager changes
Hi Nick,
I think maybe you just didn't realize how much everyone doesn't
understand TagManager because we always bitch about it on IRC in
passing. Actually, you might be the only person who *really* understands
it :)
I'll just rant a little bit about some problems with TM, as I see them
(and as bitched about on IRC), and maybe that can spin off some
discussions on ways we could improve it:
- Not invented here; none of us wrote it and not in Geany's coding
style, file system layout and naming convention, etc. I personally see
it as an upstream project like Scintilla, even though the upstream
project is dead (at least the TM part).
- Seems to be overly complex for what it needs to do (this might not be
true, but it's how it seems at a glance).
- Contains a *whole other fork* of CTags; for me this is the worst part.
It's far too difficult to take upstream improvements on files like c.c,
for example.
- Isn't threaded, blocks the UI for several seconds while parsing many
tags files before Geany can start, and even worse for the project
plugins that parse all the project files on opening. This makes Geany
appear really slow and in some cases *too* slow (ie. several minutes or
more, if there's enough files to parse).
- Isn't re-entrant or thread-safe, uses lots of global state, I guess
this is mostly due to CTags but also I think TM itself has some same
issues. This means it's really hard to get tag parsing out of the main
thread.
- Upstream project doesn't use or support TM anymore, just us. AFAIK
they are using a simpler scheme[1] involving forking out to a CTags
binary and using a (seemingly) more logical database (sqlite) for
storing and accessing tags.
- Doesn't complete local variables, scope completion doesn't seem to
work properly either.
- Doesn't support CTags format files for some reason (though I added
this previously in my fork, so it's certainly do-able).
Of course I don't mean to make it sound like TM is garbage, looking at
the code shows it's quite well engineered/optimized, and I'm confident
that it has lots of good qualities, even if I don't understand them.
Anyways, I'll end ranting here and hope it might give some ideas about
the problems some of us see with TM, and we could work towards fixing,
if we aren't to replace TM altogether.
Cheers,
Matthew Brush
[1] http://git.gnome.org/browse/anjuta/tree/plugins/symbol-db
On 12-04-29 05:07 AM, Nick Treleaven wrote:
On 27/04/2012 06:30, Lex Trotman wrote:
[...]
I don't understand why tagmanager has to be replaced, why not just
replace
the parts you want to improve? Rewriting it is likely to lead to a
new set
of bugs and be hard to review and merge changes from master.
One of the problems with tagmanager is its complexity, leading to
considerable wariness on the part of many of us about changing it
since we don't understand what we might break.
I don't see this myself, I see some complicating issues which could be
resolved (and I'm willing to work on them), but generally a sound design
for what it provides and for extra things we may want to add.
Actually documenting the overall structure of tagmanager and how it is
supposed to work would be a good thing, whats a workspace? what is it
meant to represent, how are scopes represented? etc.
Isn't it clear from the data structures? Look at TMWorkspace. Scopes and
other tag metadata is the same as CTags. Obviously if we had at-a-glance
overall documentation that would be good.
One confusing thing is that a TMTag can be used for an actual tag or for
a file. Probably that could be cleaned up.
- a multi-cache one that, as its name suggests, maintains multiple
caches (sorted tags arrays). it uses a little more memory and is
slower on insertion since it maintains several sorted lists, but a
search on an already existing cache is very fast.
Won't this be slow for adding many tags at once? How is this design
better
than tagmanager?
Perhaps Colomban could confirm, but my first thought is that this is
for nested scopes.
I expect the design is better in some respects (and to be fair I didn't
look for better things), but finding a tag based on its name is
something we are always going to want to be fast. Even for scope
completion, you still need to lookup a tag structure from a name string.
So I think we will always want a sorted array of tags per document that
we can bsearch (or something equally fast).
Also, I've probably sounded quite harsh on Colomban's design, but I'm
commenting on what I think is important. I am genuinely interested in
why his design decisions are better. It's a lot to take in all at once,
so probably needs some explanation. Sorry if I didn't make that clear.
How does tagmanager handle nested scopes, or how would it need to be
modified to do so, considering the example (in C)
{ struct a o; struct a p;
o./* struct a members */
{ struct b o;
o./* struct b members */
p./* struct a members */
}
o./* struct a members */
p./* struct
Re: [Geany-devel] tagmanager changes
Hi, Le 26/04/2012 17:02, Nick Treleaven a écrit : On 24/04/2012 22:31, Colomban Wendling wrote: Le 17/04/2012 18:20, Nick Treleaven a écrit : Sorry for the long delays -- and also small activity -- recently. I have/had a lot of non-Geany stuff to do and stuff, the whole story, you know. No problem. I finally committed it and pushed it so you can see it, comment, blame, flame more: see https://github.com/b4n/geany/tree/wip%2Fctagsmanager A few points, as they comes to my mind: * it support asynchronous parsing (though not concurrent parsing); What's the difference? Also, what does it buy us? What I meant when saying it's asynchronous but not concurrent is that it supports launching the parsers in a separate thread, but it cannot launch several parsers at once. This is because CTags parsers aren't thread-safe (have a lot of global states and no locks). What asynchronous parsing gives us is quite simple: no blocking. This means that a slow parser (like e.g. the HTML one on Windows before you changed the regex library) wouldn't lock Geany. Same for project plugins that want to parse thousands of files: this would still take a long time, but Geany would be still usable in the meantime. * it is under the new ctagsmanager/ directory; * it uses the same tag parsers tagmanager used, in ctagsmanager/ctags; * all types have different names than the tagmanager ones, though currently the API is almost an 1:1 mapping -- and that's maybe a huge mistake?; I don't understand why tagmanager has to be replaced, why not just replace the parts you want to improve? Rewriting it is likely to lead to a new set of bugs and be hard to review and merge changes from master. I think Lex and Matthew probably summarize it quite correctly: it's not that TM is bad or has to be replaced; it is that (I'm) unable to understand it enough to fix anything in it, like scope completion. Maybe it's only me, but I tried hard :) And the only reason why maybe TagManager would need replacement/large changes is asynchronous parsing. I'm not sure how hard it would be to get it with TM -- but again, maybe it's only that I don't understand it enough. * there is 2 backends as of today: - a simple one that is simple and that doesn't waste (too much) memory, but that searches in O(n); Linear searching seems like a bad idea when we currently have O(log n). Removing x tags is O(n * x) by the look of it. I agree it's not interesting regarding performances, and this backend isn't meant for it. I needed an implementation for early testing, so I wrote it simple. And it showed useful for testing later on too, since because of it's simplicity it isn't bugged -- just slow :) BTW, how does TagManager do fast searches? I always though it did a sorting with new attributes each time they changed, so in such situations it's even worse than O(n), isn't it? - a multi-cache one that, as its name suggests, maintains multiple caches (sorted tags arrays). it uses a little more memory and is slower on insertion since it maintains several sorted lists, but a search on an already existing cache is very fast. Won't this be slow for adding many tags at once? How is this design better than tagmanager? Well, adding a tag would require a bsearch() on each cache, yes. However, adding tags is mostly done in a separate thread (if async parsing it used), so it can be less of a problem. And a search is simply a bsearch() (O(n log n), right?) given the cache already exists. If the cache doesn't exist, it has to be created so yeah on the first search it's slow. Given my global tags merge change (see below), I think tagmanager needs some changes to avoid sorting the workspace tags array each time tags are updated, but this is certainly doable. In practice I haven't yet tested anything big enough to see any difference in performances between these two backends, and that's something that probably isn't worth bothering about for now. * this backend abstraction might be really overkill, and maybe we could do better without it?; I don't see why having two is better. The memory overhead for a pointer array is not much vs. the size of the tag structures. Fast searching is important. Multiple backends isn't really useful probably. As said above, the first was mostly a testing one, and I wanted to be able to keep both during development for better testing. At one point I also though maybe there could be some backends optimized for particular situations, like fast search, fast insertion, low memory consumption, etc., but I don't see much use for this anymore. * tags (and most types) are reference-counted (so they aren't necessarily duplicated, e.g. in the multicache backend); I don't really understand src/symbols.c since the real-time parsing change, so don't really understand why this is needed. It's not strictly needed, but it makes some memory management
Re: [Geany-devel] tagmanager changes
Le 26/04/2012 18:53, Nick Treleaven a écrit : On 26/04/2012 16:02, Nick Treleaven wrote: On 24/04/2012 22:31, Colomban Wendling wrote: * it uses the same tag parsers tagmanager used, in ctagsmanager/ctags; BTW this is a good idea to clearly separate CTags from tagmanager. If this change can be applied separately, perhaps it could be merged into master. It should be quite easy -- though it won't still be a vanilla CTags of course, my own isn't either (yet?). For avoiding resorting of workspace tags when only reparsing one source object, we can remove the source object's old tags merge the new tags. This is all O(n) for the workspace array. I haven't looked into implementing this yet because now it's clear you're working on a competing change. Another option is to remove the workspace array altogether and just have Geany collate tags from each (sorted) source object when needed. Not sure the implications of this yet but it would simplify tagmanager. Well, tagmanager needs to know all tags to perform e.g. scope completion beyond file's boundaries. And for search too, it would need us to pass it everything, or to perform a search on each document's tags and then merge the result. Doesn't sound sensible at a first glance, but maybe I'm missing something. ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] tagmanager changes
Le 27/04/2012 07:30, Lex Trotman a écrit :
[...]
- a multi-cache one that, as its name suggests, maintains multiple
caches (sorted tags arrays). it uses a little more memory and is
slower on insertion since it maintains several sorted lists, but a
search on an already existing cache is very fast.
Won't this be slow for adding many tags at once? How is this design better
than tagmanager?
Perhaps Colomban could confirm, but my first thought is that this is
for nested scopes.
Not at all, sorry :)
It's only for performance, so search on different criteria can remain
fast (just a bsearch()).
How does tagmanager handle nested scopes, or how would it need to be
modified to do so, considering the example (in C)
{ struct a o; struct a p;
o./* struct a members */
{ struct b o;
o./* struct b members */
p./* struct a members */
}
o./* struct a members */
p./* struct a members */
{ struct c o;
o./* struct c members */
p./* struct a members */
}
o./* struct a members */
p./* struct a members */
}
How much needs to be changed in tagmanager so that the right
autocompletes can be provided at each comment? (assuming c.c is
taught to parse local variables of course)
And of course the same question for Colomban.
To really support such thing would require the parsers to provide a true
tree, not a flat thing. However, my scope completion algorithm takes
into account a few things to try to find The Right Thing™:
1) the file in which the tag appears (e.g. tries to resolve things
locally first);
2) the distance in lines between the tags (e.g. the tag that comes
just before the one to complete has precedence over another one).
but this wouldn't be enough to get correct results with your example; it
would really need a tree.
Cheers,
Colomban
___
Geany-devel mailing list
Geany-devel@uvena.de
https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
Re: [Geany-devel] tagmanager changes
Le 29/04/2012 14:07, Nick Treleaven a écrit : On 27/04/2012 06:30, Lex Trotman wrote: [...] I don't understand why tagmanager has to be replaced, why not just replace the parts you want to improve? Rewriting it is likely to lead to a new set of bugs and be hard to review and merge changes from master. One of the problems with tagmanager is its complexity, leading to considerable wariness on the part of many of us about changing it since we don't understand what we might break. I don't see this myself, I see some complicating issues which could be resolved (and I'm willing to work on them), but generally a sound design for what it provides and for extra things we may want to add. Actually documenting the overall structure of tagmanager and how it is supposed to work would be a good thing, whats a workspace? what is it meant to represent, how are scopes represented? etc. Isn't it clear from the data structures? Look at TMWorkspace. Scopes and other tag metadata is the same as CTags. Obviously if we had at-a-glance overall documentation that would be good. What I personally blame TM for is not the data structure or overall design, but the code. I can't get my head around the implementation. Every time I try to get it, I get a headache and generally no fix :( But maybe I'm just plain stupid, or maybe I just miss one or two key things of how it's written to get it. One confusing thing is that a TMTag can be used for an actual tag or for a file. Probably that could be cleaned up. Agreed, that's something I think that should be quite easy to fix and that would make the thing easier to understand at first (though it's not one of the things that makes me not understand TM ;)). BTW, do we actually use any file tags? - a multi-cache one that, as its name suggests, maintains multiple caches (sorted tags arrays). it uses a little more memory and is slower on insertion since it maintains several sorted lists, but a search on an already existing cache is very fast. Won't this be slow for adding many tags at once? How is this design better than tagmanager? Perhaps Colomban could confirm, but my first thought is that this is for nested scopes. I expect the design is better in some respects (and to be fair I didn't look for better things), but finding a tag based on its name is something we are always going to want to be fast. Even for scope completion, you still need to lookup a tag structure from a name string. So I think we will always want a sorted array of tags per document that we can bsearch (or something equally fast). Also, I've probably sounded quite harsh on Colomban's design, but I'm commenting on what I think is important. I am genuinely interested in why his design decisions are better. It's a lot to take in all at once, so probably needs some explanation. Sorry if I didn't make that clear. Don't worry about sounding harsh or something, I totally agree that changing for changing is not a good idea. To be honest, there were really two reasons why I tried to write a new tagmanager on the first place: 1) because I wasn't able to fix the current one; 2) to support asynchronous parsing. And actually I haven't added much great design, it actually works quite the same as does TM currently -- per-file tags, a workspace holding a merge of all tags. [...] * tags (and most types) are reference-counted (so they aren't necessarily duplicated, e.g. in the multicache backend); I don't really understand src/symbols.c since the real-time parsing change, so don't really understand why this is needed. Blame C++ and overloaded names I think. I looked at the thread about that, and from what I could tell, the problem was for reparsing unsaved files. Wasn't the order OK for files that have just been saved? (Also I don't follow what that has to do with reference counting). We don't parse unsaved files at all because TM can't do that. And that's once another thing that should be fixed. However I don't get the point you're talking about, maybe my answer is OT then. Cheers, Colomban ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
[Geany-devel] plugin_add_toolbar_item()
Hi, According to it's description, $subject Inserts a toolbar item before the Quit button, or after the previous plugin toolbar item. A separator is added on the first call to this function [...]. In reality, they are added like this: b c d e f | a Quit. Looking at $subject source, it's obvious that the 2nd and subsequent items are inserted before the separator created with the 1st item. There are two ways to fix this: Keep item counter, and insert the 2nd+ items at separator_position + counter + 1; we must count the removals as well. Always insert before Quit, which will reduce plugin_add_toolbar_item by about 1/3. Practically the two are identical, but the latter is much simpler. If you agree to it, I'll write a patch. Currently $subject is used by 4 plugins; they all add a single item. -- E-gards: Jimmy ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
[Geany-devel] plugin_add_toolbar_item() was my mistake
Hi, Actually, plugin_add_toolbar_item() works properly. Now, if there isn't enough space, the first item(s?) are placed in the drop-down toolbar menu, instead of the last. But that's probably a gtk+ thing. -- E-gards: Jimmy ___ Geany-devel mailing list Geany-devel@uvena.de https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
[Geany-devel] gtk_separator_tool_item_new() patch
Hi again, and excuse me for stuffing the list.
Actually there is 1/2 error. The plugin toolbar items are inserted
improperly, but added to plugin_items list in the right order. So
using Customize Toolbar and adding/removing items or otherwise
changing them fixes the order.
Patch attached. Not in git format, sorry.
--
E-gards: Jimmy
--- ./src/pluginutils.c.orig 2011-10-15 19:28:57.0 +0300
+++ ./src/pluginutils.c 2012-04-29 21:17:36.685006711 +0300
@@ -47,7 +47,7 @@
void plugin_add_toolbar_item(GeanyPlugin *plugin, GtkToolItem *item)
{
GtkToolbar *toolbar = GTK_TOOLBAR(main_widgets.toolbar);
- gint pos;
+ gint pos = toolbar_get_insert_position();
GeanyAutoSeparator *autosep;
g_return_if_fail(plugin);
@@ -55,26 +55,15 @@
if (!autosep-widget)
{
- GtkToolItem *sep;
+ GtkToolItem *sep = gtk_separator_tool_item_new();
- pos = toolbar_get_insert_position();
-
- sep = gtk_separator_tool_item_new();
- gtk_toolbar_insert(toolbar, sep, pos);
+ gtk_toolbar_insert(toolbar, sep, pos++);
autosep-widget = GTK_WIDGET(sep);
-
- gtk_toolbar_insert(toolbar, item, pos + 1);
-
toolbar_item_ref(sep);
- toolbar_item_ref(item);
- }
- else
- {
- pos = gtk_toolbar_get_item_index(toolbar, GTK_TOOL_ITEM(autosep-widget));
- g_return_if_fail(pos = 0);
- gtk_toolbar_insert(toolbar, item, pos);
- toolbar_item_ref(item);
}
+
+ gtk_toolbar_insert(toolbar, item, pos);
+ toolbar_item_ref(item);
/* hide the separator widget if there are no toolbar items showing for the plugin */
ui_auto_separator_add_ref(autosep, GTK_WIDGET(item));
}
___
Geany-devel mailing list
Geany-devel@uvena.de
https://lists.uvena.de/cgi-bin/mailman/listinfo/geany-devel
