Hello all,

Here are some thoughts and questions about plugins in Go 1.8 (which I use 
on Linux/x86-64).

*Plugins in other languages*

In C and C++ code (on Linux), plugins are practically not exactly the same 
as e.g. shared objects (a good reference would be Drepper's *How To Write 
Shared Libraries* <https://www.akkadia.org/drepper/dsohowto.pdf> paper). In 
practice, a plugin is a (dlopen-ed) shared object *with references to 
symbols in the main program*. For example, a GCC plugin 
<https://gcc.gnu.org/onlinedocs/gccint/Plugins.html> (this is an example 
that both Ian Taylor & me know well) will call functions from GCC itself 
(such as register_callback or many others, e.g. gimple_block etc etc...). 
Likewise, a GEDIT plugin <https://wiki.gnome.org/Apps/Gedit/ShippedPlugins> 
(for example the wordcompletion 
one) will call GEDIT or GTK functions like gedit_debug or 
gtk_text_view_get_buffer etc...). Notice that to make that possible the 
main program (cc1plus for GCC
or gedit) has to be linked with the -rdynamic flag (to make the symbols of 
the main program visible from the plugin). Even if that is in theory 
principle, it is 

uncommon to dlopen a shared object which is not a plugin, designed and coded to 
call *symbols* from the *main program* (a program might in theory dlopen some 
plain shared library  like /usr/lib/x86_64-linux-gnu/libgdbm.so.3 but in 
practice this is never done; program loading plugins are dlopen-ing some 
shared object -the plugin- *specifically designed* for them, and *calling 
functions* from the main program, and the main program uses dlsym to find some 
symbols such as plugin_init for GCC obeying some signature convention). I 
deliberately ignore plugins loading other plugins (but I know about RTLD_GLOBAL 
flag to dlopen <http://man7.org/linux/man-pages/man3/dlopen.3.html>)
and I know about constructor function attributes 
<https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.htm> in plugins.

In JAVA, there are class loaders 
<https://en.wikipedia.org/wiki/Java_Classloader> with a sophisticated protocol 
for loading them.

In Ocaml, you have the dynlink 
<http://caml.inria.fr/pub/docs/manual-ocaml/libdynlink.html> library. The 
Dynlink.loadfile function 
<http://caml.inria.fr/pub/docs/manual-ocaml/libref/Dynlink.html> is similar to 
dlopen (and will run the plugin initialization code).
There is no equivalent of dlsym: the plugin is supposed to e.g.  register 
function values at initialization time, perhaps by passing them to 
some function of the main program. Of course the plugin can call functions from 
the main program.

*Plugins and packages in Go*

The *package* concept is a core concept of Go since every source file belongs 
to some package (with main being a special case) and often imports several 
other ones.
Practically speaking, a Go plugin is likely to call functions from some package 
defined by the main program and having itself (the plugin) some packages. So the
 tiny example in plugin documentation <https://tip.golang.org/pkg/plugin/> is a 
bit too naive (even if it is calling fmt.Printf). A *realistic example* would 
be a program having not only some main function 
in its main package, but also defining some purple package having some Foo 
public function called as purple.Foo from main and having a Bar public function
(with purple.Bar called from the plugin). The plugin would have not only some 
public F function in its main package (which should call purple.Bar from the 
plugin-loading program) but also some plugin specific yellow package with a 
public Y function called (as yellow.Y) from F. I hope that such a realistic 
example will be 
given in the documentation of future Go 1.9.

Unfortunately, I know no public mechanism to say, within the source code of 
a plugin, that some imported package is (and has to be) provided by the 
main loading program.
I would suggest to re-use and extend the import comment 
<https://golang.org/cmd/go/#hdr-Import_path_checking> convention (with the 
path "*" being a notation to say *import that package from the main loading 
program*). For example, our plugin.go source code might have import 
"purple" // import "*" on a single line, and then the purple package would 
be externally linked (not incorporated in the plugin; exactly like in some 
GCC plugin calling gimple_block does not add all the existing GIMPLE 
processing code into the plugin).

Currently, a plugin which uses some package (even a standard one like fmt; 
but I am thinking of a program-supplied one like purple)  is linking it 
*twice* (once in the main program, and once in the plugin). This is 
*inefficient* (compile time of plugins -even a tiny one- is huge, and their 
shared object size is much too big) and *error prone* (it is not defined 
what happens if the version of that package is different, and there might 
be -in weird cases- infelicities with multiple initialization of the same 

*A use case*In my monimelt <https://github.com/bstarynk/monimelt> program 
(which is now buildable with go tool and don't use gb anymore, since gb 
does not support plugins yet) commit 6bb8b56160749 
the test-plugins/makename.go 
is a simplistic plugin which just calls a few functions from the objvalmo & 
payloadmo packages defined in the main program. But its compilation 
time with build-plugin-monimelt.sh 
script is ridiculously big, since all the packages of the main program (and 
even external ones like go-sqlite) get linked both in the main program and 
the plugin. To run that test, do ./monimelt -run-plugin 
test-plugins/makename.go which will compile test-plugins/makename.go into 
makename.so using build-plugin-monimelt.sh script and open that plugin. The 
whole point of monimelt will be to generate Go source code (of plugins) at 
runtime and compile and load them (I followed the same idea in GCC MELT 

BTW, I am interested in reading some white paper about the precise 
semantics of packages, and how the Go compiler works internally.

Basile Starynkevitch <http://starynkevitch.net/Basile/> (France)

You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to