Re: [Geany-Devel] [FT-plugins] Proposed Design

[Matthew Brush]

On 2016-08-30 06:29 AM, Colomban Wendling wrote:

Le 29/08/2016 à 03:09, Matthew Brush a écrit :

On 2016-08-28 05:47 PM, Matthew Brush wrote:

[...]

To give an idea, the registration function called by plugins might look
something like this:

gboolean ftplugin_register_provider(GeanyPlugin*,
GeanyFiletypeID, GeanyFiletypeFeature, GCallback, gpointer);

[...]


I forgot to mention, it may turn out that in order to provide a feature,
there may be a need for multiple callbacks (ex. activate, deactivate,
init_styles, prepare_list, whatever). If this ends up being the case, we
would need to either pass a table of callbacks here or perhaps a GObject
implementing a particular interface or whatever.


Then probably better make the provide a structure like

GeanyAutoCompleteProvider {
GeanyFiletypeID filetype;
GCallback feature1;
...
}


As long as it's uniform across all features (ie. all or none are using 
structs whether or not they have multiple features), that's pretty much 
what I had in mind, and is basically how I represented the internal 
tracking structures in my experimental code I was playing with.


API-wise I find it neater to keep the parameterization in the parameters 
list and keep any structures like that as simple vtables, but the 
difference is rather minute (probably whatever would work better for 
auto-binding generators to read).




In theory I have nothing against GInterface, but I'm not quite sure it's
a good idea to start riddling Geany with GObject API where we already
have like 2 non-GObject-ish API styles (plugin registration, Proxy
registration, signals…)

Not saying GInterface is a presona non-grata, but that IMO it would have
to prove very useful over the other solutions to be used.  Especially as
people are likely to implement these filetype features in C or C++ more
than Python, JavaScript or Ruby, so it should be comfortable in C and
C++, and you know you don't like GObject boilerplate :)



I personally believe interfaces/abstract-bases to be the most 
appropriate technical solution (at least from OOP-languages) at our 
disposal, but I chose to stay away from that mostly to avoid the 
social/political/emotional ramifications of it on the discussion. 
Besides, re-implementing a small portion of it isn't that big a deal and 
doesn't feel so unnatural for C.


Cheers,
Matthew Brush
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] [FT-plugins] Proposed Design

[Matthew Brush]

On 2016-08-30 06:24 AM, Colomban Wendling wrote:

Le 29/08/2016 à 02:47, Matthew Brush a écrit :

[…]

To give an idea, the registration function called by plugins might look
something like this:

gboolean ftplugin_register_provider(GeanyPlugin*,
GeanyFiletypeID, GeanyFiletypeFeature, GCallback, gpointer);


Maybe we could simply use GSignals?  Something like


[...]


Just a though, and seeing some of the questions raised further down
might suggest it's not flexible enough, but using the GSignal mechanism
we already have at hand seemed like at least a good idea to consider.



It's a good thought, and I did consider it, but I believe for it to 
work, we would have to store an object of a different GObject type for 
each combination of Filetype and Feature, so that each object could have 
their distinct signals installed in their classes and be connected to. 
Otherwise we'd have to provide some kind of DSL embedded inside the 
signal-name like property notifications (ex. `c::autocomplete`) or 
something, and have Filetype*Feature signals for the GeanyObject.


Cheers,
Matthew Brush
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] [FT-plugins] Proposed Design

[Colomban Wendling]

Le 29/08/2016 à 03:09, Matthew Brush a écrit :
> On 2016-08-28 05:47 PM, Matthew Brush wrote:
>> [...]
>>
>> To give an idea, the registration function called by plugins might look
>> something like this:
>>
>> gboolean ftplugin_register_provider(GeanyPlugin*,
>> GeanyFiletypeID, GeanyFiletypeFeature, GCallback, gpointer);
>>
>> [...]
> 
> I forgot to mention, it may turn out that in order to provide a feature,
> there may be a need for multiple callbacks (ex. activate, deactivate,
> init_styles, prepare_list, whatever). If this ends up being the case, we
> would need to either pass a table of callbacks here or perhaps a GObject
> implementing a particular interface or whatever.

Then probably better make the provide a structure like

GeanyAutoCompleteProvider {
GeanyFiletypeID filetype;
GCallback feature1;
...
}

In theory I have nothing against GInterface, but I'm not quite sure it's
a good idea to start riddling Geany with GObject API where we already
have like 2 non-GObject-ish API styles (plugin registration, Proxy
registration, signals…)

Not saying GInterface is a presona non-grata, but that IMO it would have
to prove very useful over the other solutions to be used.  Especially as
people are likely to implement these filetype features in C or C++ more
than Python, JavaScript or Ruby, so it should be comfortable in C and
C++, and you know you don't like GObject boilerplate :)
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] [FT-plugins] Proposed Design

[Matthew Brush]

On 2016-08-28 08:17 PM, Lex Trotman wrote:

On 29 August 2016 at 12:30, Matthew Brush  wrote:

[...]

That's the idea with boolean results, a plugin could actually provide the
feature but return `FALSE` and Geany would call the next provider to perform
the feature as well. It's kind of like a poor-man's inheritance.


Actually, after more thought it doesn't make sense to me.

If a more specific provider wants to build on a more general one, it
wants the general one to run before it, so it has to be ordered after
the general provider, which has to return FALSE to allow the more
specific one to run.  But if the more specific provider isn't loaded
the general provider has to return TRUE to stop the geany built-in
functionality from running.  This means the general provider has to
know about the specific providers and if they are loaded, which is the
wrong way around.

Also a more specific provider that does it all itself and doesn't want
the general one to muck stuff up has to be ordered before the general
one to return TRUE so the general one doesn't run.  So providers have
to be able to re-order themselves or to specify an order.

I think if a provider depends on another it should be ordered first
and directly call the other, then it can return TRUE so the other
isn't called twice.



It's a good point. I think at first just using the activation and manual 
user-customized order will likely suffice. I expect the common case to 
be activating a single plugin (ex. my CDK plugin) to cover a set of 
features/filetypes and no other plugins conflicting or dependent. In the 
future it will probably be useful to make this more automatic by adding 
some kind of inter-plugin dependency manager and GUIs whatnot.


Cheers,
Matthew Brush
___
Devel mailing list
Devel@lists.geany.org
https://lists.geany.org/cgi-bin/mailman/listinfo/devel

Re: [Geany-Devel] [FT-plugins] Proposed Design

[Lex Trotman]

On 29 August 2016 at 12:30, Matthew Brush  wrote:
> On 2016-08-28 06:59 PM, Lex Trotman wrote:
>>
>> On 29 August 2016 at 10:47, Matthew Brush  wrote:
>>>
>>> [...]
>>>
>>> "Registration" - the act (ie. function call) of an ft-plugin declaring
>>> its
>>> interest in providing a feature for a filetype. An ft-plugin can register
>>> to
>>> provide one or more features for one or more filetypes.
>>
>>
>> I think the registration data should go in the plugin data file that
>> Thomas added, not require plugins to be loaded to register, otherwise
>> all language plugins will be loaded at all times just so they can call
>> the function to be registered.
>>
>
> That's a thought. I think the plugin data file you refer to may only be for
> the libpeas proxy plugin which is not part of Geany (yet).
>
> IIUC, at least when the plugin manager is opened, Geany loads all of the
> plugin DLLs anyway. Plugins could register their providers at
> `geany_load_module()`-time before they're actually initialized. Needs more
> consideration.

Ok, as noted below, may need to be a separate mechanism anyway.

[...]
>>>
>>> The order of the list of plugins registered to provide a feature for a
>>> given
>>> feature/filetype pair would determine the priority given when Geany asks
>>> the
>>> provider to perform its function. The callback functions could return a
>>> boolean telling Geany whether the provider performed its function or
>>> whether
>>> it should try the next provider in the list, similar to many GTK+
>>> callbacks.
>>> If no provider performs its function, or there are no providers
>>> registered
>>> for a given feature/filetype, then Geany would take its existing code
>>> path
>>> to provide the feature itself.
>>
>>
>> This makes sense, but so does Colombans desire for plugins to be able
>> to improve on the work of other plugins so you don't have to have all
>> plugins re-implement the basic functionality.  Not immediately sure
>> how to reconcile these two options, unless Colombans desire is
>> provided by having the dependent plugin have to call the basic plugin
>> function directly, so its not Geany's problem.
>>
>
> That's the idea with boolean results, a plugin could actually provide the
> feature but return `FALSE` and Geany would call the next provider to perform
> the feature as well. It's kind of like a poor-man's inheritance.

Actually, after more thought it doesn't make sense to me.

If a more specific provider wants to build on a more general one, it
wants the general one to run before it, so it has to be ordered after
the general provider, which has to return FALSE to allow the more
specific one to run.  But if the more specific provider isn't loaded
the general provider has to return TRUE to stop the geany built-in
functionality from running.  This means the general provider has to
know about the specific providers and if they are loaded, which is the
wrong way around.

Also a more specific provider that does it all itself and doesn't want
the general one to muck stuff up has to be ordered before the general
one to return TRUE so the general one doesn't run.  So providers have
to be able to re-order themselves or to specify an order.

I think if a provider depends on another it should be ordered first
and directly call the other, then it can return TRUE so the other
isn't called twice.

And Geany should make its fallback functionality available to call, so
a provider that wants to just make a small adjustment doesn't have to
re-implement the default (right Colomban).  Or the Geany default can
be removed and added to core plugins :)

[...]
>>>
>>> To enable Geany to use the providers, in the existing code just before
>>> it's
>>> about to provide the feature itself as it does now, we could insert a
>>> check/call to try the ft-plugin providers.
>>
>>
>> This may require some unwinding of the spaghetti to clearly identify
>> the beginning of a "feature" and ensure it happens in one place.  Some
>> places there is stuff split between different callbacks depending on
>> how the feature was triggered.
>>
>
> Most likely yeah. In some cases though, it might be as simple as sticking an
> "if" statement at the beginning of a single function with an early return.

Yes, thats where we want to get to, but I suspect that its not allways
the reality now.  Untangling spaghetti is a GOOD thing, its just more
work :)

[...]
>>
>> What is missing is how the FT plugins get loaded in the first place so
>> they can register, if the user can control that from the PM GUI, or if
>> these plugins are controlled from your GUI only (my preference).  If
>> the FT plugins are loaded via a different path and don't appear in the
>> PM that may solve the order issue as well.
>>
>
> At least initially, I think when the plugins are activated they could
> register their providers. The user would just check the plugin in the plugin
> manager dialog as usual. In the future it could be at
> 

Re: [Geany-Devel] [FT-plugins] Proposed Design

[Matthew Brush]

On 2016-08-28 06:59 PM, Lex Trotman wrote:

On 29 August 2016 at 10:47, Matthew Brush  wrote:

[...]

"Registration" - the act (ie. function call) of an ft-plugin declaring its
interest in providing a feature for a filetype. An ft-plugin can register to
provide one or more features for one or more filetypes.


I think the registration data should go in the plugin data file that
Thomas added, not require plugins to be loaded to register, otherwise
all language plugins will be loaded at all times just so they can call
the function to be registered.



That's a thought. I think the plugin data file you refer to may only be 
for the libpeas proxy plugin which is not part of Geany (yet).


IIUC, at least when the plugin manager is opened, Geany loads all of the 
plugin DLLs anyway. Plugins could register their providers at 
`geany_load_module()`-time before they're actually initialized. Needs 
more consideration.






In this design, a new module (c/h file) would be added to manage the
filetype plugins and which features they provide for which filetypes. A
mapping (ex. GHashTable) could be used to map from Filetype to a list (ex.
GQueue) of data describing the needed information for a registered provider.
The list would be ordered in the same order as registration and could also
be re-ordered by the user using a GUI (more on this below).


Would need to ensure plugin loading will re-load in the
original/specified order, IIRC it currently re-loads in alpha order.



Yeah, possibly storing this info in separate ftplugin-specific keys in 
the config file.




The order of the list of plugins registered to provide a feature for a given
feature/filetype pair would determine the priority given when Geany asks the
provider to perform its function. The callback functions could return a
boolean telling Geany whether the provider performed its function or whether
it should try the next provider in the list, similar to many GTK+ callbacks.
If no provider performs its function, or there are no providers registered
for a given feature/filetype, then Geany would take its existing code path
to provide the feature itself.


This makes sense, but so does Colombans desire for plugins to be able
to improve on the work of other plugins so you don't have to have all
plugins re-implement the basic functionality.  Not immediately sure
how to reconcile these two options, unless Colombans desire is
provided by having the dependent plugin have to call the basic plugin
function directly, so its not Geany's problem.



That's the idea with boolean results, a plugin could actually provide 
the feature but return `FALSE` and Geany would call the next provider to 
perform the feature as well. It's kind of like a poor-man's inheritance.






When a plugin registers its intent to provide a feature (or perhaps after it
has registered all the features it wishes to provide), Geany could check
whether there is already another plugin providing this feature. Geany could
ask the user if they would like to resolve the conflict, and if they would,
then it could show a management dialog (see attachment for mockup), allowing
the user to control the priority of the plugin's provider for a given
feature/filetype by moving them up or down in a list, and possibly being
able to completely disable a provider entirely (via a checkbox in the list
or something).


Not sure about this dialog happening automatically, think it would
need a "Shut TF up" option, and also be quiet at re-load.  But
certainly providing the advanced user control of the providers is
fine.



Yeah, I thought it might be annoying too. Will need experimentation.





To enable Geany to use the providers, in the existing code just before it's
about to provide the feature itself as it does now, we could insert a
check/call to try the ft-plugin providers.


This may require some unwinding of the spaghetti to clearly identify
the beginning of a "feature" and ensure it happens in one place.  Some
places there is stuff split between different callbacks depending on
how the feature was triggered.



Most likely yeah. In some cases though, it might be as simple as 
sticking an "if" statement at the beginning of a single function with an 
early return.




 If nobody performed the feature,

then it would continue to the existing code path. This should limit the
number of changes needed to Geany. Some features would necessarily require
more changes, for example syntax highlighting would require Geany to switch
from the Scintilla lexer to the container lexer and back as plugins
start/stop providing the feature. It will require care for features that are
activated often to ensure minimal performance degradation when looking up
and calling into the provider, as this would happen in the main code paths
(unless someone has a better way).


It is always a requirement for plugins to not hog or block the main
thread, they can do that by other threads or separate processes, but
its 

[Geany-Devel] [FT-plugins] Proposed Design

[Matthew Brush]

Hi All,

After experimenting with a few different ideas and approaches, I think I 
have come up with a way to implement the core of "filetype plugins"[0] 
without too much effort or code.




Terminology:

"Filetype plugin" - or "ft-plugin"; at this point is just normal plugin 
that would call special API functions and implement any number of 
features for any number of filetypes.


"Feature" - one of the features an ft-plugin would want to override, for 
example syntax highlighting or auto-completion.


"Provider" - an ft-plugin that provides one or more features for one or 
more filetypes. For example if it can provide calltips, it could be 
referred to as a "Calltip Provider". Each ft-plugin can have a number of 
"providers" for the various features and filetypes.


"Registration" - the act (ie. function call) of an ft-plugin declaring 
its interest in providing a feature for a filetype. An ft-plugin can 
register to provide one or more features for one or more filetypes.




In this design, a new module (c/h file) would be added to manage the 
filetype plugins and which features they provide for which filetypes. A 
mapping (ex. GHashTable) could be used to map from Filetype to a list 
(ex. GQueue) of data describing the needed information for a registered 
provider. The list would be ordered in the same order as registration 
and could also be re-ordered by the user using a GUI (more on this below).


The order of the list of plugins registered to provide a feature for a 
given feature/filetype pair would determine the priority given when 
Geany asks the provider to perform its function. The callback functions 
could return a boolean telling Geany whether the provider performed its 
function or whether it should try the next provider in the list, similar 
to many GTK+ callbacks. If no provider performs its function, or there 
are no providers registered for a given feature/filetype, then Geany 
would take its existing code path to provide the feature itself.


When a plugin is unloaded, the mapping and lists of providers would be 
updated to remove any providers registered by that plugin so Geany 
doesn't call into an unloaded plugin. The next provider registered (if 
any) would have the first chance to now provide the feature.


When a plugin registers its intent to provide a feature (or perhaps 
after it has registered all the features it wishes to provide), Geany 
could check whether there is already another plugin providing this 
feature. Geany could ask the user if they would like to resolve the 
conflict, and if they would, then it could show a management dialog (see 
attachment for mockup), allowing the user to control the priority of the 
plugin's provider for a given feature/filetype by moving them up or down 
in a list, and possibly being able to completely disable a provider 
entirely (via a checkbox in the list or something).


To give an idea, the registration function called by plugins might look 
something like this:


gboolean ftplugin_register_provider(GeanyPlugin*,
GeanyFiletypeID, GeanyFiletypeFeature, GCallback, gpointer);

Or perhaps it could use varargs similar to many GTK+/GObject functions, 
terminated by a sentinel, to register many filetype feature providers in 
one call, making it easier to implement a less annoying conflict 
handling scheme - not nagging after each registration call.


The callback function would have an actual signature suitable for 
implementing the specific feature. For example, the callback for a 
syntax highlighting provider might be something like this:


gboolean (*) (GeanyPlugin*, GeanyDocument*,
guint start_pos, guint end_pos, gpointer user_data);

The document would be the document that needs highlighting, and allow 
the plugin to access Scintilla and its buffer. The start/end positions 
would indicate where to highlight. This is in-line with Scintilla's 
'style-needed' notification used to implement container lexers. I don't 
want to get bogged down on the actual specific signatures at this point, 
I just wanted to give an example.


To enable Geany to use the providers, in the existing code just before 
it's about to provide the feature itself as it does now, we could insert 
a check/call to try the ft-plugin providers. If nobody performed the 
feature, then it would continue to the existing code path. This should 
limit the number of changes needed to Geany. Some features would 
necessarily require more changes, for example syntax highlighting would 
require Geany to switch from the Scintilla lexer to the container lexer 
and back as plugins start/stop providing the feature. It will require 
care for features that are activated often to ensure minimal performance 
degradation when looking up and calling into the provider, as this would 
happen in the main code paths (unless someone has a better way).


Hopefully I have described enough details of my proposed design to allow 
everyone to understand what I mean.