Re: Plug-ins on Windows
Manuel López-Ibáñez wrote: > So I would prefer that GCC developers do not say "We want to provide > an API" when what is actually meant is "We will be happy if people > contribute patches towards providing an API". OK, I agree with that. As far as I know, nobody right now is actively working on improving the plug-in API in the way that I suggested. So, yes, I would be happy if someone would like to work on that and to contribute patches. I will volunteer to help review patches that move in this direction. -- Mark Mitchell CodeSourcery m...@codesourcery.com (650) 331-3385 x713
Re: Plug-ins on Windows
On 6 July 2010 23:05, Grigori Fursin wrote: > I don't disagree with your comments too, Manuel. > > I spent some years developing plugin framework for pass selection and > reordering, and later we managed to get minimal hooks to mainline GCC based on > our needs. Of course, I personally would like to see a coherent and stable API > for most of the parts of GCC, but as you said it will not happen itself. > Still I believe > that through gradual API-zation of different parts of GCC, there will be an > eventual > convergence to the global and stable API ... Yes, but I would like our language to explicitly encourage external forces to make this happen. Even despite (perceived or real) resistance from "the powers that decide" in GCC, progress happens when enough people push for it (in the right way and with the right force, just pushing will get you nowhere). So I would prefer that GCC developers do not say "We want to provide an API" when what is actually meant is "We will be happy if people contribute patches towards providing an API". The first gives the impression that people wishing for such API just have to sit and wait. When nothing happens, they will feel frustrated and betrayed. The latter invites those same people to scratch their itch by working together with the GCC community. "Together" means progressively and interactively, dumping a complete API design+huge patch will certainly fail. Cheers, Manuel.
RE: Plug-ins on Windows
I don't disagree with your comments too, Manuel. I spent some years developing plugin framework for pass selection and reordering, and later we managed to get minimal hooks to mainline GCC based on our needs. Of course, I personally would like to see a coherent and stable API for most of the parts of GCC, but as you said it will not happen itself. Still I believe that through gradual API-zation of different parts of GCC, there will be an eventual convergence to the global and stable API ... Cheers, Grigori -Original Message- From: ctuning-discussi...@googlegroups.com [mailto:ctuning-discussi...@googlegroups.com] On Behalf Of Manuel Lopez-Ibanez Sent: Tuesday, July 06, 2010 6:42 PM To: Grigori Fursin Cc: ctuning-discussi...@googlegroups.com; Joern Rennecke; David Brown; gcc@gcc.gnu.org Subject: Re: Plug-ins on Windows On 6 July 2010 17:54, Grigori Fursin wrote: >>I view the current plug-in mechanism as a prototype. I think that we >>should be working toward a much more robust mechanism, similar to >>plug-ins for Eclipse, Firefox, MySQL, or other popular software stacks. >>I certainly see no reason that plug-ins cannot work on any system that >>has something roughly equivalent to dlopen (which Windows and OS X >>certainly do). There's lots of prior art here. >> >>The key change we need to make is that instead of the current >>"unstructured" approach where we essentially expose the entirety of the >>GCC internals, we provide a stable, documented API that exposes a >>portion of the internals. (It's fine with me if there's an expert mode >>in which you can do anything you want, but it should be easy to stay >>within the stable API if you want to do that.) I would start with an >>API for "observing" (rather than modifying) the internal representation, >> and add modification later. "We need to make"? Are there any plans (or even interest) on designing such an API in the future? My understanding is that we hope an API will slowly arise from the requirements of external tools. So plugins' authors should not wait for such an API but instead start proposing it, ideally, with patches. >>With observation alone, you could start to build interesting >>static-checking tools, including, for example, domain-specific tools >>that could check requirements of the Linux kernel. This would be a >>powerful and exciting feature to have in GCC. > > Agree with this vision, but it will take some time I guess ;) ... I don't think it requires time. It requires people asking for it and working on it. Current GCC developers do not need this and they will not work on this (as far as I know), so no one should be waiting for such tools to magically appear. However, now that the plugins framework is in-place, anyone is welcome to not only develop such tools but to propose changes that make developing such tools easier. The plugins framework is a statement of the willingness of the GCC community, despite the initial opposition from the FSF, to open the project to external contributions and usages. GCC internals have been (and are still being) renovated, and there is ongoing work to make GCC more modular. However, if there is no one proposing the particular changes required by external usages, those changes won't be made. This is also a warning against working around and second-guessing GCC. Please, don't. Instead contribute the changes (API, code, and documentation) required to make GCC more open and easier to use. I know that this is the hardest route, but it is the only one that makes us progress. Cheers, Manuel. -- You received this message because you are subscribed to the Google Groups "ctuning-discussions" group. To post to this group, send email to ctuning-discussi...@googlegroups.com. To unsubscribe from this group, send email to ctuning-discussions+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/ctuning-discussions?hl=en.
Re: Plug-ins on Windows
On 6 July 2010 17:54, Grigori Fursin wrote: >>I view the current plug-in mechanism as a prototype. I think that we >>should be working toward a much more robust mechanism, similar to >>plug-ins for Eclipse, Firefox, MySQL, or other popular software stacks. >>I certainly see no reason that plug-ins cannot work on any system that >>has something roughly equivalent to dlopen (which Windows and OS X >>certainly do). There's lots of prior art here. >> >>The key change we need to make is that instead of the current >>"unstructured" approach where we essentially expose the entirety of the >>GCC internals, we provide a stable, documented API that exposes a >>portion of the internals. (It's fine with me if there's an expert mode >>in which you can do anything you want, but it should be easy to stay >>within the stable API if you want to do that.) I would start with an >>API for "observing" (rather than modifying) the internal representation, >> and add modification later. "We need to make"? Are there any plans (or even interest) on designing such an API in the future? My understanding is that we hope an API will slowly arise from the requirements of external tools. So plugins' authors should not wait for such an API but instead start proposing it, ideally, with patches. >>With observation alone, you could start to build interesting >>static-checking tools, including, for example, domain-specific tools >>that could check requirements of the Linux kernel. This would be a >>powerful and exciting feature to have in GCC. > > Agree with this vision, but it will take some time I guess ;) ... I don't think it requires time. It requires people asking for it and working on it. Current GCC developers do not need this and they will not work on this (as far as I know), so no one should be waiting for such tools to magically appear. However, now that the plugins framework is in-place, anyone is welcome to not only develop such tools but to propose changes that make developing such tools easier. The plugins framework is a statement of the willingness of the GCC community, despite the initial opposition from the FSF, to open the project to external contributions and usages. GCC internals have been (and are still being) renovated, and there is ongoing work to make GCC more modular. However, if there is no one proposing the particular changes required by external usages, those changes won't be made. This is also a warning against working around and second-guessing GCC. Please, don't. Instead contribute the changes (API, code, and documentation) required to make GCC more open and easier to use. I know that this is the hardest route, but it is the only one that makes us progress. Cheers, Manuel.
RE: Plug-ins on Windows
>I view the current plug-in mechanism as a prototype. I think that we >should be working toward a much more robust mechanism, similar to >plug-ins for Eclipse, Firefox, MySQL, or other popular software stacks. >I certainly see no reason that plug-ins cannot work on any system that >has something roughly equivalent to dlopen (which Windows and OS X >certainly do). There's lots of prior art here. > >The key change we need to make is that instead of the current >"unstructured" approach where we essentially expose the entirety of the >GCC internals, we provide a stable, documented API that exposes a >portion of the internals. (It's fine with me if there's an expert mode >in which you can do anything you want, but it should be easy to stay >within the stable API if you want to do that.) I would start with an >API for "observing" (rather than modifying) the internal representation, > and add modification later. > >With observation alone, you could start to build interesting >static-checking tools, including, for example, domain-specific tools >that could check requirements of the Linux kernel. This would be a >powerful and exciting feature to have in GCC. Agree with this vision, but it will take some time I guess ;) ... Grigori
Re: Plug-ins on Windows
Joern Rennecke wrote: > The main utility of plugins is that they make developing, testing and > deploying modifications to gcc easier. This may be true now, but I certainly hope it will not be true in future. I view the current plug-in mechanism as a prototype. I think that we should be working toward a much more robust mechanism, similar to plug-ins for Eclipse, Firefox, MySQL, or other popular software stacks. I certainly see no reason that plug-ins cannot work on any system that has something roughly equivalent to dlopen (which Windows and OS X certainly do). There's lots of prior art here. The key change we need to make is that instead of the current "unstructured" approach where we essentially expose the entirety of the GCC internals, we provide a stable, documented API that exposes a portion of the internals. (It's fine with me if there's an expert mode in which you can do anything you want, but it should be easy to stay within the stable API if you want to do that.) I would start with an API for "observing" (rather than modifying) the internal representation, and add modification later. With observation alone, you could start to build interesting static-checking tools, including, for example, domain-specific tools that could check requirements of the Linux kernel. This would be a powerful and exciting feature to have in GCC. -- Mark Mitchell CodeSourcery m...@codesourcery.com (650) 331-3385 x713
Re: Plug-ins on Windows
> The attached patch still has some rough edges, like missing main gcc > ChangeLog, missing documentation, no dso building implemented (should > eventually work both in-tree to be installed in the plugin dir and > out-of-tree with a previous built compiler). > Well, I definitely owe you some beer. The gcc I am using isn't trunk but the majority of the changes applied cleanly. I had to back port some changes from trunk to my version and do a little extra hack or two to get it to work on Windows but it works like a charm! On Windows, when you configure gcc it doesn't set ENABLE_PLUGIN because dynamically linked plugins are (rightfully so) unsupported and LINK_PLUGIN_NAMES doesn't get set by configure. Therefore I had to add LINK_PLUGIN_NAMES define to the compile options for opts.o in the gcc/Makefile.in. The exact same options your patch added for plugin.o, after that and fixing a missing return type and semi colon in plugin.c everything works like a charm. You definitely saved me quite a bit of time thanks again! I think if this patch was polished up it could be at least a short term solution for plugin support on Windows. Plus it's a cool feature on other platforms as well. Cheers, Kyle
Re: Plug-ins on Windows
Quoting Kyle Girard :
A generic linked-in plugin ability would definitely solve my
plugin-on-windows problem. From what I've been reading on this list it
looks like I'm going to have to do some sort of similar hack to gcc to
get my plugin working on windows at least in the short term.
I hacked GNU hello 1.3 a little to make it work as a linked-in GCC plugin,
thus bringing much needed mail reader funtionality to GCC. Well, at least
on unix-like systems with mailboxes it does.
So during configuration, I added:
--enable-link-plugins=all
alternatively, it should be possible to specify a list, although I
haven't tested this.
When you've built your compiler, you can access the GNU hello functionality
via -fplugin / -fplugin-arg:
[amyl...@laria gcc]$ ./xgcc -B. -fplugin=hello -fplugin-arg-hello-h -x
c /dev/null -c
This is GNU Hello, THE greeting printing program.
Usage: -fplugin=hello
{ -fplugin-arg-hello([h[elp]] | [t[raditional]] | [v[ersion]] | [m[ail]]) }
h[elp]Print a summary of the options
t[raditional] Use traditional greeting format
v[ersion] Print plugin version information
m[ail]Print your mail
[amyl...@laria gcc]$ ./xgcc -B. -fplugin=hello
-fplugin-arg-hello-sailor -x c /dev/null -c
Nothing happens here.
The attached patch still has some rough edges, like missing main gcc
ChangeLog, missing documentation, no dso building implemented (should
eventually work both in-tree to be installed in the plugin dir and
out-of-tree with a previous built compiler).
I though it'd make sense to ask first if the patch so far goes in the
right direction.
link-plugin-diff.bz2
Description: BZip2 compressed data
Re: Plug-ins on Windows
> The main utility of plugins is that they make developing, testing and > deploying modifications to gcc easier. I don't think that MS windows users > will miss too much if they can't dynamically load the plugins, as long > as their sysadmin can provide them with a linked-in version - the sysadmin > might actually be happier about the greater control, as plugins are > potential virus/trojan vectors. > It makes sense to write a plugin in such a way that it can alternatively > be used as a GCC patch - in the sense that you add the code alongside > the main gcc code and it gets linked in. > Maybe we should create an interface for linked-in plugin code, i.e. > arrays of plugin names and their plugin_init function so that they can > be activated with a matching -fplugin=built-in:NAME (or propose a different > syntax if you think of a better one) and get passed any plugin options > like a dynamically loaded one would. > A generic linked-in plugin ability would definitely solve my plugin-on-windows problem. From what I've been reading on this list it looks like I'm going to have to do some sort of similar hack to gcc to get my plugin working on windows at least in the short term. Is there so little demand for dynamically loaded plugin on windows platform that a generic linked-in plugin framework could be how plugins are supported on windows? Just drop the plugin source and a config of some sort in a directory of the main gcc build and it's pulled in auto-magically? Kyle
Re: Plug-ins on Windows
Joern Rennecke wrote: [gcc plugins rely on ELF features and therefore don't work on MS windows] Quoting David Brown : In reality, if embedded developers are unhappy (for whatever reason) with running gcc on Windows, they will move to proprietary compilers under windows - not gcc on Linux. Of course, this is not a problem at the moment - gcc plugins are currently only for particular special purposes and specially interested developers and users, and so requiring Linux is not a big issue. But as gcc plugins become more mainstream, it will become a limitation. Should the famously cross-platform, cross-host, cross-target gcc now be limited to specific hosts? The main utility of plugins is that they make developing, testing and deploying modifications to gcc easier. I don't think that MS windows users will miss too much if they can't dynamically load the plugins, as long as their sysadmin can provide them with a linked-in version - the sysadmin might actually be happier about the greater control, as plugins are potential virus/trojan vectors. I think I am viewing this situation from a different angle from others on this list. You guys are (I presume) mostly gcc developers - I am mostly a user, who is interested in the development of gcc and would like to get involved if I can ever find the time. I am particularly interested in the "smaller" targets, such as for the AVR - I work as a professional embedded developer and use several gcc host-target combinations for my job. I am therefore trying to offer a slightly different viewpoint here, in the hope that it will be of help or interest to others. The fact is, windows sysadmins do not, as a rule, build gcc for others. In most cases, there are two sorts of people who compile gcc to run on windows. There are people who build gcc, and package it along with libraries, debuggers, and other tools in an easy-to-install package for others to use. Sometimes this is done for fun or interest, sometimes it is done for money. Then there are the few specially interested users who build the tools for themselves. For most windows users, the concept of compiling your compiler is alien. They don't have the required tools (such as cygwin or mingw/msys, or a Linux machine for cross-compiling), they are not used to the command line, and they are not used to the idea of the compiler being separate from the linker, library, and related development tools. Another issue is that for embedded developers, it is very important to have the consistent toolchains, and sometimes to have them certified or qualified in some way. In this business, often makes sense to pay someone like CodeSourcery for a build that they have tested thoroughly, rather than compiling your own build with the latest and greatest version that hasn't yet had the same level of testing. So having said all that, it's fairly clear that almost all development work for gcc is done on Linux (or other *nix-like platforms), and that's where the development benefits of plugins are by far the most relevant. But there are plenty of potential uses for plugins beyond development. There is a lot of scope for plugins that are not tied tightly to particular compiler versions or targets, and could be decoupled from the main compiler development. One I mentioned in an earlier post would be a "ccache" plugin. Others include static analysis plugins, or perhaps even code style standards checkers. I think it would be a good thing for everyone if plugins like that could be developed independently of gcc and distributed as compiled shared library files (as well as the source, of course) on both Linux and Windows. It makes sense to write a plugin in such a way that it can alternatively be used as a GCC patch - in the sense that you add the code alongside the main gcc code and it gets linked in. Maybe we should create an interface for linked-in plugin code, i.e. arrays of plugin names and their plugin_init function so that they can be activated with a matching -fplugin=built-in:NAME (or propose a different syntax if you think of a better one) and get passed any plugin options like a dynamically loaded one would. The plugin_init function name would of course have to be mangled or put into a namespace to avoid clashes with other plugins.
Re: Plug-ins on Windows
On 06/30/2010 04:21 PM, Jan Hubicka wrote: >> Long term we could arrange for libbackend.a to become libbackend.dll and >> have that library be used for plugins. The existing practice of linking >> back into the main executable is more or less an efficiency hack that >> happens to work with ELF. > > It also makes WHOPR with -fwhole-program possible on GCC. If we will want to > have dynamically linkable backend library, we would need to clean up our > interfaces quite a lot so frontend does not link into backend other way than > by > langhooks. (or make other crosslinking explicit via externally_visible) Not > that would be a bad thing. I made absolutely no measurements yet if linking > frotned into backend improves performance in any sensible way. For i386-elf (and probably several other important targets) host, linking the front end and backend together means that libbackend does not need to be compiled -fPIC which would definitely affect performance. For elf host, I think we would have to have a much stronger reason to split the compiler into a shared library than plugins. We could perhaps enable it during development to help make sure that Windows continues to build, but I would hope there would be enough testers there to let us know quickly if something broke. r~
Re: Plug-ins on Windows
[gcc plugins rely on ELF features and therefore don't work on MS windows] Quoting David Brown : In reality, if embedded developers are unhappy (for whatever reason) with running gcc on Windows, they will move to proprietary compilers under windows - not gcc on Linux. Of course, this is not a problem at the moment - gcc plugins are currently only for particular special purposes and specially interested developers and users, and so requiring Linux is not a big issue. But as gcc plugins become more mainstream, it will become a limitation. Should the famously cross-platform, cross-host, cross-target gcc now be limited to specific hosts? The main utility of plugins is that they make developing, testing and deploying modifications to gcc easier. I don't think that MS windows users will miss too much if they can't dynamically load the plugins, as long as their sysadmin can provide them with a linked-in version - the sysadmin might actually be happier about the greater control, as plugins are potential virus/trojan vectors. It makes sense to write a plugin in such a way that it can alternatively be used as a GCC patch - in the sense that you add the code alongside the main gcc code and it gets linked in. Maybe we should create an interface for linked-in plugin code, i.e. arrays of plugin names and their plugin_init function so that they can be activated with a matching -fplugin=built-in:NAME (or propose a different syntax if you think of a better one) and get passed any plugin options like a dynamically loaded one would. The plugin_init function name would of course have to be mangled or put into a namespace to avoid clashes with other plugins.
Re: Plug-ins on Windows
On Thu, Jul 1, 2010 at 3:23 PM, Joern Rennecke wrote: > Quoting Richard Guenther : > >> Re-compiling the same plugin sources for different gcc versions is >> not supported. Of course you might be lucky for minor version >> changes such as 4.5.3 to 4.5.4. > > I think that's putting it a bit too strong. If the maintainer of a plugin > cares about the plugin working for a new gcc version that comes out, it > is up to that plugin maintainer to determine if any relevant interfaces > have changed so that the plugin source needs changing to continue support, > or if the plugin simply won't be portable to that gcc version. > > Once it has been determined that no relevant interface has changed for a > plugin, re-compiling it for the new gcc version should work fine. > (Well, strictly speaking, it would work or not work regardless of > people having found out about this before, but that's not a safe mode > to operate.) > > It's pretty much the same as for out-of-tree patches. They may or may not > work in a new version. There's nothing wrong with applying an old patch > to a newer gcc version, if you know what you are doing. I was making this strong statement to warn people that all bugreports like "plugin $foo stopped to compile on the 4.5 branch after rev. XYZ" will be closed as invalid. Thus, there is no ABI/API compatibility guarantee for plugins at all. I expect that most of the time you will be lucky on a release branch though (but it's still not supported, in the sense of the first sentence). Richard.
Re: Plug-ins on Windows
Quoting Richard Guenther : Re-compiling the same plugin sources for different gcc versions is not supported. Of course you might be lucky for minor version changes such as 4.5.3 to 4.5.4. I think that's putting it a bit too strong. If the maintainer of a plugin cares about the plugin working for a new gcc version that comes out, it is up to that plugin maintainer to determine if any relevant interfaces have changed so that the plugin source needs changing to continue support, or if the plugin simply won't be portable to that gcc version. Once it has been determined that no relevant interface has changed for a plugin, re-compiling it for the new gcc version should work fine. (Well, strictly speaking, it would work or not work regardless of people having found out about this before, but that's not a safe mode to operate.) It's pretty much the same as for out-of-tree patches. They may or may not work in a new version. There's nothing wrong with applying an old patch to a newer gcc version, if you know what you are doing.
Re: Plug-ins on Windows
On Thu, Jul 1, 2010 at 2:48 PM, David Brown wrote: > I was perhaps over-generalising - obviously anything that depends on target > specifics will be dependent on the target. And I'd also expect some things > to change in the plugin interface between major gcc versions - while it > would be nice to have binary compatibility for plugins between different > major versions of the compiler, I fully understand it if some things break. > The main requirement is that it must be possible to re-compile the plugin > for different gcc versions, but it would be very useful if re-compilation > were not necessary, if such support is not too costly. Re-compiling the same plugin sources for different gcc versions is not supported. Of course you might be lucky for minor version changes such as 4.5.3 to 4.5.4. Richard.
Re: Plug-ins on Windows
On 01/07/2010 12:14, Joern Rennecke wrote: Quoting David Brown : But it strikes me that a system where the main programs and the plugins are directly linking to each other is going to make it hard to separate the development of the two sides, and hard to distribute compiled plugins that will work with separately compiled main binaries. And surely it is this sort of separation and modularisation that is the whole purpose of the plugins? For the "average" gcc user running Linux and with a single version of gcc producing native code, it doesn't matter much - they've either compiled their plugins at the same time as their gcc binary, or (more likely) their Linux distribution maintainers compiled them at the same time. No, both binary plugins and their sources are generally much smaller than the GCC distribution. And a plugin is built faster from source. Also, it is often more straightforward to combine different plugins than to take different patchsets, merge them into gcc sources, and bootstrap the compiler. But gcc is far more flexible than that. I use gcc for perhaps six different targets on a regular basis, with a variety of versions of gcc for some of the these targets, with different hosts (Windows or Linux), and with different target OSes (mostly bare metal elf, sometimes Linux, and some old bare metal coff). To my mind, a plugin that works for gcc-mips-linux-4.5 should work without recompilation for gcc-arm-elf-4.6, assuming the same host. It depends on the plugin. If to uses target-specific macros or types, (e.g.: FIRST_PSEUDO_REGISTER, CUMULATIVE_ARGS) or invokes functions specific to a target (could be autoconfigured), then I wouldn't expect this to work. Also, the name mangling has to match. I was perhaps over-generalising - obviously anything that depends on target specifics will be dependent on the target. And I'd also expect some things to change in the plugin interface between major gcc versions - while it would be nice to have binary compatibility for plugins between different major versions of the compiler, I fully understand it if some things break. The main requirement is that it must be possible to re-compile the plugin for different gcc versions, but it would be very useful if re-compilation were not necessary, if such support is not too costly. Still, recompiling the plugin for two different gcc builds tend to be much more lightweight than to rebuild & install two gcc configurations. True enough. And it should be possible to build the plugin binary for Linux, Window s (native/mingw, not just cygwin) and other major gcc hosts (Mac, BSDs, etc.). Like it or not, a great deal of cross-compilation embedded development is done using gcc on windows, and it's a shame if something as potentially powerful as gcc plugins is not available to those users. Well, it's their choice (or their employers/sysadmins), we can't force them to switch to a more powerful development platform. As you said, there are a number of major gcc host platforms available, and several elf-based ones run on the same hardware as MS windows. No, it is often /not/ the choice of their employers or IT folk. I am the sys-admin for my company, and I also have a lot of influence over the chips we use and the tools we use for them. But like it or not, a great deal of tools available are windows-only, and a great many of our customers and users are windows-only. I fully agree that Linux is a more powerful platform for /programming/, but for many gcc users there is more to their daily work (or play) than writing and compiling code. Many of the tools I use - Firefox, Thunderbird, OpenOffice - are cross-platform. But for others, such as my electronics CAD software, it's windows-only. For many of the bigger microcontrollers, there are development tools available that run on Linux - very often gcc, perhaps packages with libraries, debuggers, IDEs, etc, (such as those CodeSourcery makes). But most non-gcc tools are Windows only, and for most manufacturers, the only tools they support are on Windows. For example, when programming Atmel AVR processors the great majority of developers use Atmel's AVR Studio debugger, simulator, programmer, and IDE. So most people who use avr-gcc do so under Windows. In reality, if embedded developers are unhappy (for whatever reason) with running gcc on Windows, they will move to proprietary compilers under windows - not gcc on Linux. Of course, this is not a problem at the moment - gcc plugins are currently only for particular special purposes and specially interested developers and users, and so requiring Linux is not a big issue. But as gcc plugins become more mainstream, it will become a limitation. Should the famously cross-platform, cross-host, cross-target gcc now be limited to specific hosts?
Re: Plug-ins on Windows
On 1 July 2010 12:14, Joern Rennecke wrote: > >> And it should be possible to >> build the plugin binary for Linux, Window s (native/mingw, not just >> cygwin) and other major gcc hosts (Mac, BSDs, etc.). Like it or not, a >> great deal of cross-compilation embedded development is done using gcc >> on windows, and it's a shame if something as potentially powerful as >> gcc plugins is not available to those users. > > Well, it's their choice (or their employers/sysadmins), we can't force them > to switch to a more powerful development platform. As you said, there are > a number of major gcc host platforms available, and several elf-based ones > run on the same hardware as MS windows. Keeping in mind that there is nothing stopping anyone from designing and implementing a radically different approach to plugins that overcomes the limitations of the current one. If such new plugin architecture were compelling enough, it will be adopted by GCC. We are definitely not afraid of replacing stuff with better stuff. It just has to actually be better. Cheers, Manuel.
Re: Plug-ins on Windows
Quoting David Brown : But it strikes me that a system where the main programs and the plugins are directly linking to each other is going to make it hard to separate the development of the two sides, and hard to distribute compiled plugins that will work with separately compiled main binaries. And surely it is this sort of separation and modularisation that is the whole purpose of the plugins? For the "average" gcc user running Linux and with a single version of gcc producing native code, it doesn't matter much - they've either compiled their plugins at the same time as their gcc binary, or (more likely) their Linux distribution maintainers compiled them at the same time. No, both binary plugins and their sources are generally much smaller than the GCC distribution. And a plugin is built faster from source. Also, it is often more straightforward to combine different plugins than to take different patchsets, merge them into gcc sources, and bootstrap the compiler. But gcc is far more flexible than that. I use gcc for perhaps six different targets on a regular basis, with a variety of versions of gcc for some of the these targets, with different hosts (Windows or Linux), and with different target OSes (mostly bare metal elf, sometimes Linux, and some old bare metal coff). To my mind, a plugin that works for gcc-mips-linux-4.5 should work without recompilation for gcc-arm-elf-4.6, assuming the same host. It depends on the plugin. If to uses target-specific macros or types, (e.g.: FIRST_PSEUDO_REGISTER, CUMULATIVE_ARGS) or invokes functions specific to a target (could be autoconfigured), then I wouldn't expect this to work. Also, the name mangling has to match. Still, recompiling the plugin for two different gcc builds tend to be much more lightweight than to rebuild & install two gcc configurations. And it should be possible to build the plugin binary for Linux, Window s (native/mingw, not just cygwin) and other major gcc hosts (Mac, BSDs, etc.). Like it or not, a great deal of cross-compilation embedded development is done using gcc on windows, and it's a shame if something as potentially powerful as gcc plugins is not available to those users. Well, it's their choice (or their employers/sysadmins), we can't force them to switch to a more powerful development platform. As you said, there are a number of major gcc host platforms available, and several elf-based ones run on the same hardware as MS windows.
Re: Plug-ins on Windows
On 01/07/2010 00:46, Dave Korn wrote: On 30/06/2010 21:38, Kyle Girard wrote: Hello, I am playing around with a plug-in for gcc but recently ran into the road block that plug-ins aren't supported on Windows. Are there any plans to add support for the windows platform in the future? If not, what are the issues with supporting Windows and how much effort would it be to add support? I'm assuming that since it's not done already there has to be some difficulty somewhere... The main problem is that plugins rely on the property of the ELF format that you can leave undefined references in a shared library and have them filled in by symbols exported from the main exe at runtime; they use this to call the various functions exported from GCC, like register_callback, and to access global variables and so on. Isn't that an unusual way to handle plugins and callbacks? It also sounds to me like it would make a lot of co-dependencies between the main gcc parts and the plugins, and make it hard to modularise. I would have thought a better way for gcc to export functions (or data) to the plugin is by calling some sort of "register" function in the plugin, and passing references over. In other words, gcc should call a "register_callback" function in the plugin, not vice versa. I haven't implemented a plugin system on either Windows or Linux, nor looked much at gcc plugins, so I could be way off the mark here. But it strikes me that a system where the main programs and the plugins are directly linking to each other is going to make it hard to separate the development of the two sides, and hard to distribute compiled plugins that will work with separately compiled main binaries. And surely it is this sort of separation and modularisation that is the whole purpose of the plugins? For the "average" gcc user running Linux and with a single version of gcc producing native code, it doesn't matter much - they've either compiled their plugins at the same time as their gcc binary, or (more likely) their Linux distribution maintainers compiled them at the same time. But gcc is far more flexible than that. I use gcc for perhaps six different targets on a regular basis, with a variety of versions of gcc for some of the these targets, with different hosts (Windows or Linux), and with different target OSes (mostly bare metal elf, sometimes Linux, and some old bare metal coff). To my mind, a plugin that works for gcc-mips-linux-4.5 should work without recompilation for gcc-arm-elf-4.6, assuming the same host. And it should be possible to build the plugin binary for Linux, Windows (native/mingw, not just cygwin) and other major gcc hosts (Mac, BSDs, etc.). Like it or not, a great deal of cross-compilation embedded development is done using gcc on windows, and it's a shame if something as potentially powerful as gcc plugins is not available to those users. Although we could build plugins as Windows DLLs and have GCC load them at runtime, if those DLLs needed to refer to anything in the main GCC executable, it would have to be specifically linked to import it - and imports on Windows have to explicitly specify the name of the DLL (or executable) they are imported from. That means that the plugin would need to explicitly refer to cc1.exe or cc1plus.exe, etc; we'd need to build separate versions of the plugin for each of the different GCC language compilers. (Long term, we might be able to extend the toolchain and libltdl to co-operate to do this kind of deferred runtime linking for us, but that's not imminent.) Would it be a lot faster/easier to create a custom gcc that has my plug-in compiled in directly for the windows platform? Yes, much! At least for the foreseeable future. Sorry. cheers, DaveK
Re: Plug-ins on Windows
> On 06/30/2010 03:46 PM, Dave Korn wrote: > > Although we could build plugins as Windows DLLs and have GCC load them at > > runtime, if those DLLs needed to refer to anything in the main GCC > > executable, > > it would have to be specifically linked to import it - and imports on > > Windows > > have to explicitly specify the name of the DLL (or executable) they are > > imported from. That means that the plugin would need to explicitly refer to > > cc1.exe or cc1plus.exe, etc; we'd need to build separate versions of the > > plugin for each of the different GCC language compilers. > > > > (Long term, we might be able to extend the toolchain and libltdl to > > co-operate to do this kind of deferred runtime linking for us, but that's > > not > > imminent.) > > Long term we could arrange for libbackend.a to become libbackend.dll and > have that library be used for plugins. The existing practice of linking > back into the main executable is more or less an efficiency hack that > happens to work with ELF. It also makes WHOPR with -fwhole-program possible on GCC. If we will want to have dynamically linkable backend library, we would need to clean up our interfaces quite a lot so frontend does not link into backend other way than by langhooks. (or make other crosslinking explicit via externally_visible) Not that would be a bad thing. I made absolutely no measurements yet if linking frotned into backend improves performance in any sensible way. Honza > > > r~
Re: Plug-ins on Windows
On 06/30/2010 03:46 PM, Dave Korn wrote: > Although we could build plugins as Windows DLLs and have GCC load them at > runtime, if those DLLs needed to refer to anything in the main GCC executable, > it would have to be specifically linked to import it - and imports on Windows > have to explicitly specify the name of the DLL (or executable) they are > imported from. That means that the plugin would need to explicitly refer to > cc1.exe or cc1plus.exe, etc; we'd need to build separate versions of the > plugin for each of the different GCC language compilers. > > (Long term, we might be able to extend the toolchain and libltdl to > co-operate to do this kind of deferred runtime linking for us, but that's not > imminent.) Long term we could arrange for libbackend.a to become libbackend.dll and have that library be used for plugins. The existing practice of linking back into the main executable is more or less an efficiency hack that happens to work with ELF. r~
Re: Plug-ins on Windows
On 30/06/2010 21:38, Kyle Girard wrote: > Hello, > > I am playing around with a plug-in for gcc but recently ran into the > road block that plug-ins aren't supported on Windows. Are there any > plans to add support for the windows platform in the future? If not, > what are the issues with supporting Windows and how much effort would it > be to add support? I'm assuming that since it's not done already there > has to be some difficulty somewhere... The main problem is that plugins rely on the property of the ELF format that you can leave undefined references in a shared library and have them filled in by symbols exported from the main exe at runtime; they use this to call the various functions exported from GCC, like register_callback, and to access global variables and so on. Although we could build plugins as Windows DLLs and have GCC load them at runtime, if those DLLs needed to refer to anything in the main GCC executable, it would have to be specifically linked to import it - and imports on Windows have to explicitly specify the name of the DLL (or executable) they are imported from. That means that the plugin would need to explicitly refer to cc1.exe or cc1plus.exe, etc; we'd need to build separate versions of the plugin for each of the different GCC language compilers. (Long term, we might be able to extend the toolchain and libltdl to co-operate to do this kind of deferred runtime linking for us, but that's not imminent.) > Would it be a lot faster/easier to create a custom gcc that > has my plug-in compiled in directly for the windows platform? Yes, much! At least for the foreseeable future. Sorry. cheers, DaveK
Re: Plug-ins on Windows
Quoting Kyle Girard : Would it be a lot faster/easier to create a custom gcc that has my plug-in compiled in directly for the windows platform? Depends on how many plugins you use. The distgcc page says it's reported to work on cygwin. So you could use a cygwin distgcc to send the compile job to some other machine, or a virtual machine on the same machine, which runs a Unix or GNU system with elf object format.
Re: Plug-ins on Windows
On Wed, 2010-06-30 at 16:38 -0400, Kyle Girard wrote: > Hello, > > I am playing around with a plug-in for gcc but recently ran into the > road block that plug-ins aren't supported on Windows. Are there any > plans to add support for the windows platform in the future? If not, > what are the issues with supporting Windows and how much effort would it > be to add support? I'm assuming that since it's not done already there > has to be some difficulty somewhere... Disclaimer: I know nothing about Windows & I never used it (except twice a year to change a mandatory password). I absolutely know nothing about Windows programming in the WIN32 API. Back to GCC plugins. If I remember correctly, we did slightly consider using libtldl, a wrapper providing dlopen variant http://www.gnu.org/software/libtool/manual/html_node/Libltdl-interface.html but we later rejected that idea to avoid adding yet another library dependency to GCC. Apparently libltdl is supposed to work on Windows. Perhaps you could try to use it in your own variant of GCC. However, I would imagine that making the GCC trunk accept it is much more difficult (& personally I don't care that much anymore). Perhaps also they are some technical issues that makes using libltdl impractical. Maybe your concern should be to make sure that the "-rdynamic" functionality of Linux linker can be emulated on Windows. You really want most public symbols of GCC to be visible from your Windows GCC plugin. I heard it could be hard (i.e. boring) to achieve (perhaps requiring a dllexport on all public symbols of GCC). I'll be interested to know if you succeeded in making some GCC plugin work under Windows. In case you are very brave, you might even as an exercice port the MELT plugin http://gcc.gnu.org/wiki/MELT to Windows! But I am not sure it is worth the effort. Cheers.
