Re: [HACKERS] Compiling extensions on Windows
Craig Ringer wrote: Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is Do a source build of PG, compile your ext in-tree in contrib/, and hope the result is binary compatible with release PostgreSQL builds for Windows. Certainly that's how I've been doing it to date. How about everyone else here? Does anyone actually build and distribute extensions out of tree at all? I build binaries for oracle_fdw for EDB's binaries by hand, i.e. I look at what PGXS does on Linux and try to do something comparable with cl. It's rather painful, and something like PGXS for MSVC would be very welcome. Yours, Laurenz Albe -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
On 01/12/2014 12:00 AM, Tom Lane wrote: Craig Ringer cr...@2ndquadrant.com writes: We don't set __declspec(dllexport) on extension functions automatically when building stand-alone on Windows. So it's necessary to explicitly specify PGDLLEXPORT for each function. I'm not sure I believe this. I don't see any PGDLLEXPORT symbols in any of the standard contrib modules; how is it that they work? We force all symbols to be exported using the project file generation tools in the Pg build system, so PGDLLEXPORT is not required there. That doesn't help stand alone builds. We don't have anything like PGXS for Windows, so the only real option for building and distributing reliably compatible extensions is your own Visual Studio project. Just build in contrib/ in a source tree is not an acceptable answer, because extensions need binary compatibility with public installer distributions like the one EDB maintains. Unless you're sure your build is set up just the same way I'm not at all convinced you're going to get that, though you may get something that works *most* of the time. This hasn't historically been a project keen on mostly solutions though. So if it's really necessary to change anything here, I'd rather see us take the approach of hiding it in PG_FUNCTION_INFO_V1. What happens if we do that and there's also a manually-written prototype? I've run out of weekend and have to go full-speed on upd. sb. views, BDR, and event triggers tomorrow, so I don't think I'll have a chance to test it immediately. I'll check after the CF deadline. That's certainly one option, anyway, and one that'll solve the immediate issue with extensions, and would be the most practical short term solution if it works. Otherwise we can just add DLLEXPORT macros on public functions in contrib/ and the docs. They're harmless on other platforms - and in fact can be quite handy. Those windows droppings can actually be useful: They annotate sources with a useful piece of information we don't currently use on *nix, but probably should. They specify a shared library or executable's public interface. Imagine that the macro was called PGAPI or PG_PUBLIC_API, not PGDLLEXPORT. That's what it actually *means*, really. On *nix we treat the executable and shared lib APIs as the union of the exported symbols of all compilation units in the object. That's rather crude, as it means that something very internal to Pg must be public if it's to be visible across more than one compilation unit, and that if we wish to hide it we have to squash things together into bigger compilation units than we might otherwise want. That's a pain for backward compat, as theoretically all of PostgreSQL's exported symbols are public API. We don't really offer a good way to tell what's public and what's not anyway. Instead, we can and (IMO) should be marking our public API explicitly. If it's supposed to be usable from a contrib / extension / fdw / etc, mark it PGDLLEXPORT. Or rename that PG_PUBLIC_API and label it that. Whatever. On Windows, these expand to __declspec(dllexport) and cause symbol export. On gcc, we switch to building with -fvisiblity=hidden and have these macros expand to __attribute__ ((dllexport)) . Access to non-exported symbols becomes a link error. On ancient GCCs and on other toolchains we continue to export all symbols by default, as now. This'd reduce symbol table sizes a bit - though it's really only a big win for C++ code with monster symbol tables. More importantly, it'd make it easier to tell what's supposed to be reasonably stable across versions as public API, and what's totally internal. (It might be that we have enough exts delving deep in to Pg's guts already that this is impractical. It'd be fun to see.) Related references: http://gcc.gnu.org/wiki/Visibility http://people.redhat.com/drepper/dsohowto.pdf -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
On 01/12/2014 04:54 PM, Craig Ringer wrote: On 01/12/2014 12:00 AM, Tom Lane wrote: So if it's really necessary to change anything here, I'd rather see us take the approach of hiding it in PG_FUNCTION_INFO_V1. What happens if we do that and there's also a manually-written prototype? That's certainly one option, anyway, and one that'll solve the immediate issue with extensions, and would be the most practical short term solution if it works. ... which it kind-of does. Turned out to be trivial to test. If the prototype with PGDLLEXPORT appears *first*, then all is well. If the prototype with PGDLLEXPORT appears AFTER a user-provided prototype it fails with: 1DemoExtension.c(16): error C2375: 'add_one' : redefinition; different linkage 1 DemoExtension.c(14) : see declaration of 'add_one' Two copies of the prototype, both with PGDLLEXPORT, work fine. So really the question is: Do we care? The current usage of extensions built outside the Pg tree on Windows is likely to be nearly zero, and is already fiddly. I'm not too fussed if we make people fix up their prototypes. I think we can just emit a prototype for the function from PG_FUNCTION_INFO_V1 . The only things it's going to break is the odd bit of weirdly-built, not really supported extensions on Windows. Are there any platforms that object to prototype redefinition? If not, we can just emit a prototype on all platforms, not just Windows. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
Craig Ringer cr...@2ndquadrant.com writes: Turned out to be trivial to test. If the prototype with PGDLLEXPORT appears *first*, then all is well. If the prototype with PGDLLEXPORT appears AFTER a user-provided prototype it fails with: That's sort of what I thought would happen. It's problematic because putting the extern before the PG_FUNCTION_INFO_V1 is standard practice, especially if you have the extern in a header file. I think we can just emit a prototype for the function from PG_FUNCTION_INFO_V1. Doesn't sound like it; we'll still be forced to modify or remove manually-written externs in basically all contrib and third-party code, if we want it to build on Windows. Which, as I said before, does not seem to me to be a workable solution path at all. It would take years to track down all the third-party code and get it fixed, if the authors don't themselves build/test on Windows. And I continue to maintain that it's not acceptable for the Windows port to require this anyway. If any other platform came to us and said hey guys, you need to plaster this non-ANSI-C decoration on every exported function, what response do you think they'd get? One last point is that automatically export every external symbol is exactly what happens for these modules on Unix-ish platforms. If it doesn't work the same way on Windows, we're just opening ourselves up to subtle portability issues. This needs to be fixed in the Windows build system, not the source code. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
On 01/13/2014 02:04 AM, Tom Lane wrote: Craig Ringer cr...@2ndquadrant.com writes: I think we can just emit a prototype for the function from PG_FUNCTION_INFO_V1. Doesn't sound like it. On second thought, agreed. The externs tending to appear in headers kills that. In that case - after the rush for the CF close, expect a patch from me against post-9.4 that adds PGDLLEXPORT to the documentation examples, and another that adds them for contribs (to help people using them as examples). Nothing else needed, and it doesn't have to affect the *nix build process or server/ code in any way. I'll also add a note in the docs explaining what's wrong if you get an error about an obviously-present function being missing in your extension on Windows when it works on other platforms. And I continue to maintain that it's not acceptable for the Windows port to require this anyway. If any other platform came to us and said hey guys, you need to plaster this non-ANSI-C decoration on every exported function, what response do you think they'd get? One last point is that automatically export every external symbol is exactly what happens for these modules on Unix-ish platforms. If it doesn't work the same way on Windows, we're just opening ourselves up to subtle portability issues. The portability issues are easily dealt with by using gcc's symbol visibility features on *nix, which will produce slightly smaller binaries with faster linkage too. I'll demo that once I've got the current work off my plate. Platforms w/o gcc visibility don't need to care, nothing changes, they just don't get the benefits. Everyone develops on platforms that have gcc and visibility features so they'll spot any issues introduced. As described earlier, doing this helps differentiate internal stuff from public API if we choose to, as well. If we don't want to attempt to treat anything as internal, non-public-use, non-stable API, then there's no point using visibility - everything not static is public API and should be exported to the world. That's how things are now. In that case I'd stick with just improving the docs to cover PGDLLEXPORT, and maybe the contribs. I do think we should think about actually hiding private internal API post-9.4, though. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/07/2014 12:41 AM, Joe Conway wrote: Yes, this pretty much exactly describes how I build PL/R for Windows. I had to match my build system SDK with the one EDB uses to get a compatible binary. It would be nice if we had something equivalent to PGXS on Windows, or maybe even a community build system where authors could get Windows binaries built. I'd be interested in hearing if you have any luck doing this with a standlone MSVC build: .dll build type, no manifest, compile as C code, appropriate include and lib paths, define WIN32, link to postgres.lib, add PGDLLEXPORT to all declarations of functions with a PG_FUNCTION_INFO_V1, build. - -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJS01XNAAoJELBXNkqjr+S2S4QH/icetcqZDmsHFHjZnm3hY+LP M0+rU8+E3on5V/JmWR3bBESmlYXRYM53LmgwapwCfQ45rsrMSiKACyxtl/XkWsEh 38NTsagjlZtsyZIiUoe9d0szSNQerS86ZDBwXAvJnBSNLQy1AnDQ5tzsFbdeZph4 veL6MnoKNIacfbkEBoCjM0KyYdjnYnt4nRlmbGKQNg/h4Y9KqgJsFFpk0r8dfz+v KNPPyOmdHIcMyCgJS9hIdAdzc+CPPjYBZC3oVVQAuIOsqccmTykPr4Nh3MVcSfCy JJTRZhzgU6TdRZIi4adY8Ys39O+TJM2T5Wr1xJ6+Eapnd6L1AiY/08jbZIrsFVc= =5/Y6 -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/12/2014 06:56 PM, Craig Ringer wrote: On 01/07/2014 12:41 AM, Joe Conway wrote: Yes, this pretty much exactly describes how I build PL/R for Windows. I had to match my build system SDK with the one EDB uses to get a compatible binary. It would be nice if we had something equivalent to PGXS on Windows, or maybe even a community build system where authors could get Windows binaries built. I'd be interested in hearing if you have any luck doing this with a standlone MSVC build: .dll build type, no manifest, compile as C code, appropriate include and lib paths, define WIN32, link to postgres.lib, add PGDLLEXPORT to all declarations of functions with a PG_FUNCTION_INFO_V1, build. Yes, that's more-or-less how I do it. I checked with EDB to be sure I had the same SDK (looks like last time I did it it was SDK 7.1), then I: 1) download postgres source 2) copy plr source into contrib directory 3) modify src/tools/msvc/* to make plr look like any other contrib 4) build As long as the SDK is matched, the resulting plr.dll works fine with the EDB one-click installer version of postgres (or at least so far). Joe - -- Joe Conway credativ LLC: http://www.credativ.us Linux, PostgreSQL, and general Open Source Training, Service, Consulting, 24x7 Support -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJS01m8AAoJEDfy90M199hlkPEP/A1iCDJeRchfBM4OQB73k0VN +4o9aVxo3n4jjoOiKI3zctpP6bHQI9/wyI9OYAKnRr3PutttrEyDn1MO3wBffTVN k84rXyKKROgZOnGpQrE33QG82J1TRHmxZRyeQirQplekWEsB2RpiGu+jZJDjUsLn svmgPZOka5tc+KqC4OJ3qXprv5OW4h6xMfZmUljn6jdzsuEubE6p4X/XqqGUJmVP t6caIJ3yKvIcedQ7QZmxWEJXYGGoKlr6qv2Vo99SfqK9lGX6A7DFwAG3DaLmWqa/ rULapnfnET5jUtfncu3zkz6Kok848BsoaSfFZpHyyzvU2RlfAAPWCv0ZnNrSYTsl s7blVBzanBoX63gy4MwMOcGdHmjPyYjeJ78m53GE28cvv62JEGubISkWoCbmOPYu +UmxaCPR1l03n/QAv3Qo0fCEtU09OXzYci8NDEKqbdkWt9K/M4bNlv207J0CywiX pEwQGVciaR5ylKYFZpKpCpgf6OElBVFaUk8uE41swgxM3McXWoPFuKocnhTWd0u6 9Fyb7W8W2kAJeOkiNWcpsX2DNHKMkZ8siTfMufyqHDxVfaJLie/0oTZejnGJf4Lh vOBpqxdowN6xVhBR796vtzxjLg50vDj8NzOKvn6evtJMnWJkUiRBsY8JrR6g3fII lb+3NTRa5x0ptApbsUKJ =wFy+ -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/13/2014 11:13 AM, Joe Conway wrote: Yes, that's more-or-less how I do it. I checked with EDB to be sure I had the same SDK (looks like last time I did it it was SDK 7.1), then I: 1) download postgres source 2) copy plr source into contrib directory What I mean is that you should not need a full Pg build tree to compile extensions. Just as we use PGXS on *nix, so it is possible to just use Visual Studio to build the extension on Windows as a VS project like any other. As long as the SDK is matched, the resulting plr.dll works fine with the EDB one-click installer version of postgres (or at least so far). The SDK shouldn't even need to match. Windows software is expected to cope with mismatched, mix-and-match C runtimes. Icky, but occasionally useful. So long as you don't pass a FILE* across DLL boundaries, or free() memory that was malloc()'d in another DLL, mixing runtimes should work fine. I'd be interested in knowing what problems you encountered when the runtime didn't match. Maybe it's specific to how we do build within a Pg source tree? I was able to use three different SDK builds against EDB's latest 9.3 packages - SDK 7.1 (x86 and x64), toolchain v90 (x86), and toolchain v100 (x86) - to compile and test a moderately trivial extension. - -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJS01vxAAoJELBXNkqjr+S2zcoIAI75MetuO+osAlSOMdMDAgKh 7rqKu8VPyF2bjFVAiYdZOS9Yvlg3yZDzEG2JMbUSJOz3KtGIirZkIub8hgI4q4/W 6PBg5UZAiwK30AXILTqGBXio/Z+olbCPOKvcVv05OF4WExYLMek5Hc2SKy3UIudj HHvC10LkSeVvJvNj+rK7SUjQIpwa4a2+cc0gx87z7kd8ElVwJ3D/c1Eb3DM8mdsg KqIlmWkGolwnk1fb/JoabkO9XVvRjPgpj/aR9Ak6mUH7QTXGMqafdpPeCV9BhGRK d6MDDT3ncoEzHZd7GsKeqVWBFz9eZ7hLXiQR6rZ7bidvNVc4V7NnM2dr50uhoec= =pgpf -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/12/2014 07:22 PM, Craig Ringer wrote: On 01/13/2014 11:13 AM, Joe Conway wrote: What I mean is that you should not need a full Pg build tree to compile extensions. Just as we use PGXS on *nix, so it is possible to just use Visual Studio to build the extension on Windows as a VS project like any other. I have never gotten that to work. As long as the SDK is matched, the resulting plr.dll works fine with the EDB one-click installer version of postgres (or at least so far). The SDK shouldn't even need to match. Windows software is expected to cope with mismatched, mix-and-match C runtimes. Icky, but occasionally useful. Unfortunately my experience tells me otherwise. Mismatched SDK = plr.dll that does not work correctly with EDB builds of postgres. So long as you don't pass a FILE* across DLL boundaries, or free() memory that was malloc()'d in another DLL, mixing runtimes should work fine. I'd be interested in knowing what problems you encountered when the runtime didn't match. plr would not load. I'm not good enough with MSVC and Windows debuggers to figure out why, and don't have the time or interest to become so, I just know matching the SDK to the postgres build resulted in it owrking for me and no more complaints of this type from the field. It would work fine with the postgres build I made, but not with the EDB build which of course is what virtually everyone running pg on Windows uses. Maybe it's specific to how we do build within a Pg source tree? I was able to use three different SDK builds against EDB's latest 9.3 packages - SDK 7.1 (x86 and x64), toolchain v90 (x86), and toolchain v100 (x86) - to compile and test a moderately trivial extension. Trivial is probably the key thing. With PL/R it also has to link against the R.dll which is built with yet another toolchain on Windows. Joe - -- Joe Conway credativ LLC: http://www.credativ.us Linux, PostgreSQL, and general Open Source Training, Service, Consulting, 24x7 Support -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJS018UAAoJEDfy90M199hl/mAP/2OhXbvrlKSzP6fWi8g9Tez0 PnWaOFXXPIeOi5lJ/o5rC7HrQdiCCBljSBAffq0PKl8SVn2cDwmA5E2n4T3JfZQF PYGZiYuqoiZ6i+svW7x6XocDnIhJgMJvTvys7ToApjmkD3VEgj7RO8MHQyYmVDsh A9WIPpyb96mTmzlTLnHZDkfL7MgEof4kTHHC2jOa6i3wMq+zATc6lBTXOcrrGzS8 qd/iIap0kNdwKgLEX/jXip0YOB8SMfxOeHVLV+790JUwnWmBJnbn3XDqFEmj39kK dGEP8vaxjPppyEmMvkGZd5Hxw6WIGFASTjyn6kXH1VfVqsLNZYO+rwTXnSVtyqH0 aFCKLT7awMBjFSh3plFQcqxeeqElZZaCRNVO5xooQ28xoUoUl/wVYqI0yoF9hKKm NlJ8jJGB6aEImFlQ7QUg2eZRfMpyYc9J06uaX1+/L3g71O4O2Xzgc6gPVWvYCIQP BvcNBtUlxA38H5wjiMSlyyz4Si95cIIbqDfUliKZ1Ab0W24en0vnvISxk6v/2GKc vE9X7GRFUjmUJNoIvkRu0hnzp5S955sO0X6Q6pDmgM2esGRADMTntY0Bcxp8R2qg qiZkVs1vfuewLmzz4LqixItW9BhMHK3zdGcv07xntNt+EAaz8g3cU5tBZP8CP6y8 JTa/fyFth7hL+ttkH5hc =9+nY -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/13/2014 11:35 AM, Joe Conway wrote: On 01/12/2014 07:22 PM, Craig Ringer wrote: On 01/13/2014 11:13 AM, Joe Conway wrote: What I mean is that you should not need a full Pg build tree to compile extensions. Just as we use PGXS on *nix, so it is possible to just use Visual Studio to build the extension on Windows as a VS project like any other. I have never gotten that to work. As long as the SDK is matched, the resulting plr.dll works fine with the EDB one-click installer version of postgres (or at least so far). The SDK shouldn't even need to match. Windows software is expected to cope with mismatched, mix-and-match C runtimes. Icky, but occasionally useful. Unfortunately my experience tells me otherwise. Mismatched SDK = plr.dll that does not work correctly with EDB builds of postgres. Something for me to play with once this CF is over, then. - -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.15 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJS01/mAAoJELBXNkqjr+S2ZQcH/0rHjCW+6pvg8727dJHSmYfM fY18VBlgYzjfzgPaFLYgp4OqT7VHl2h1d7AapVo8wRblVD4z1hEFtW6/j56Mx1wm UTw7rWhs7utA9G+gzWcHJz9VDxED0ROFH+IwurSM85P57ztPKaRhvp6YT3fibhYb kqw51FrEfvlnhCMi3Art3DGmHtzDRLwGTI03YOr/GRfWsccHPwpRLkpQkyuMsyOX UBhPenz0OhtkVxEfFSmyVWmu6NzlOQyxgFgl8zW7R9pq4fTBgOfo198RDkGKnCno 9KYq3H8VqLxHgpyR2KP3netrqDvDBzk0xVmgoaJbQyT6HyuQDWH6lIHE+5RTyyM= =nreR -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
On 01/06/2014 07:44 PM, Sandeep Thakkar wrote: Okay. BTW, I just checked that Windows 32bit installer ships the libintl.h. Did you try if it works for you? Or it requires more headers? Somehow, Windows 64bit installer installer missed it. I'll fix the build script. That appears to be the only header missing for simple compilation. I did some testing to verify that it works. http://blog.2ndquadrant.com/compiling-postgresql-extensions-visual-studio-windows/ -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
On 01/06/2014 07:44 PM, Sandeep Thakkar wrote: Okay. BTW, I just checked that Windows 32bit installer ships the libintl.h. Did you try if it works for you? Or it requires more headers? Somehow, Windows 64bit installer installer missed it. I'll fix the build script. Actually, on second thoughts there were two other issues. One I reported separately (double-inclusion of pg_config_os.h due to _WIN32 vs WIN32). The other is worth looking at here. We don't set __declspec(dllexport) on extension functions automatically when building stand-alone on Windows. So it's necessary to explicitly specify PGDLLEXPORT for each function. We seem to work around this in the Perl build toolchain by forcing export of everything not explicitly static (which is, btw, a pretty crappy thing we should revisit in favour of using PGDLLEXPORT and -fvisibility=hidden on Linux). Instead we should perhaps be adding this automatically via a prototype generated by PG_FUNCTION_INFO_V1, or adding PGDLLEXPORT to all our example documentation. I think the latter is preferable because if we start generating a prototype for the base function in PG_FUNCTION_INFO when we didn't before it could break existing code. Comments? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
Craig Ringer cr...@2ndquadrant.com writes: We don't set __declspec(dllexport) on extension functions automatically when building stand-alone on Windows. So it's necessary to explicitly specify PGDLLEXPORT for each function. I'm not sure I believe this. I don't see any PGDLLEXPORT symbols in any of the standard contrib modules; how is it that they work? Instead we should perhaps be adding this automatically via a prototype generated by PG_FUNCTION_INFO_V1, or adding PGDLLEXPORT to all our example documentation. I think the latter is preferable because if we start generating a prototype for the base function in PG_FUNCTION_INFO when we didn't before it could break existing code. Comments? One of the things I've always found particularly vile about Microsoft is the way that they seem to think it's fine to make people sprinkle Windows-only droppings throughout code that's supposed to be portable. I'm not in favor of asking people to write out PGDLLEXPORT manually on every function unless it's *absolutely* necessary, and the available evidence suggests to me that it isn't. So if it's really necessary to change anything here, I'd rather see us take the approach of hiding it in PG_FUNCTION_INFO_V1. What happens if we do that and there's also a manually-written prototype? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
On Sat, Jan 11, 2014 at 5:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: Craig Ringer cr...@2ndquadrant.com writes: We don't set __declspec(dllexport) on extension functions automatically when building stand-alone on Windows. So it's necessary to explicitly specify PGDLLEXPORT for each function. I'm not sure I believe this. I don't see any PGDLLEXPORT symbols in any of the standard contrib modules; how is it that they work? They are built through our perl toolkit, which enables exporting of *all* symbols, regardless of flags in the code. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Re: [HACKERS] Compiling extensions on Windows
Magnus Hagander mag...@hagander.net writes: On Sat, Jan 11, 2014 at 5:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure I believe this. I don't see any PGDLLEXPORT symbols in any of the standard contrib modules; how is it that they work? They are built through our perl toolkit, which enables exporting of *all* symbols, regardless of flags in the code. That seems like a perfectly reasonable solution, given the way we use loadable modules. Excess symbols in the module shouldn't really do any harm. Can't we just document the flags to use for this, if you're building in some other way? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
On Sat, Jan 11, 2014 at 7:05 PM, Tom Lane t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net writes: On Sat, Jan 11, 2014 at 5:00 PM, Tom Lane t...@sss.pgh.pa.us wrote: I'm not sure I believe this. I don't see any PGDLLEXPORT symbols in any of the standard contrib modules; how is it that they work? They are built through our perl toolkit, which enables exporting of *all* symbols, regardless of flags in the code. That seems like a perfectly reasonable solution, given the way we use loadable modules. Excess symbols in the module shouldn't really do any harm. Can't we just document the flags to use for this, if you're building in some other way? It's not a build flag, and that's the main problem. It's the src/tools/msvc/ gendef.pl script that builds the export list. And what Craig is after here is being able to build extensions using standard tools without needing our full build infrastructure. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/
Re: [HACKERS] Compiling extensions on Windows
On 01/11/2014 01:55 PM, Magnus Hagander wrote: On Sat, Jan 11, 2014 at 7:05 PM, Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.us wrote: Magnus Hagander mag...@hagander.net mailto:mag...@hagander.net writes: On Sat, Jan 11, 2014 at 5:00 PM, Tom Lane t...@sss.pgh.pa.us mailto:t...@sss.pgh.pa.us wrote: I'm not sure I believe this. I don't see any PGDLLEXPORT symbols in any of the standard contrib modules; how is it that they work? They are built through our perl toolkit, which enables exporting of *all* symbols, regardless of flags in the code. That seems like a perfectly reasonable solution, given the way we use loadable modules. Excess symbols in the module shouldn't really do any harm. Can't we just document the flags to use for this, if you're building in some other way? It's not a build flag, and that's the main problem. It's the src/tools/msvc/gendef.pl http://gendef.pl script that builds the export list. And what Craig is after here is being able to build extensions using standard tools without needing our full build infrastructure. What I'd like is something that would use or mimic our msvc build tools but for extensions. (And no, I don't have time to build it.) cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
On Mon, Jan 6, 2014 at 3:32 AM, Craig Ringer cr...@2ndquadrant.com wrote: Hi all Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is Do a source build of PG, compile your ext in-tree in contrib/, and hope the result is binary compatible with release PostgreSQL builds for Windows. Certainly that's how I've been doing it to date. How about everyone else here? Does anyone actually build and distribute extensions out of tree at all? I'm interested in making the Windows installer distributions a bit more extension dev friendly. In particular, I'd really like to see EDB's Windows installers include the libintl.h for the included libintl, since its omission, combined with Pg being built with ENABLE_NLS, tends to break things horribly. Users can always undefine ENABLE_NLS, but it's an unnecessary roadblock. Sandeep, can you work on fixing this please? Thanks. Are there any objections from -hackers to including 3rd party headers for libs we expose in our public headers in the binary distribution? Other than bundling 3rd party headers, any ideas/suggestions for how we might make ext building saner on Windows? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
Sure. I'll make the changes so that the next available Windows installers include lbintl.h in $Installdir/include. How about the changes with respect to NLS? On Mon, Jan 6, 2014 at 2:44 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Jan 6, 2014 at 3:32 AM, Craig Ringer cr...@2ndquadrant.com wrote: Hi all Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is Do a source build of PG, compile your ext in-tree in contrib/, and hope the result is binary compatible with release PostgreSQL builds for Windows. Certainly that's how I've been doing it to date. How about everyone else here? Does anyone actually build and distribute extensions out of tree at all? I'm interested in making the Windows installer distributions a bit more extension dev friendly. In particular, I'd really like to see EDB's Windows installers include the libintl.h for the included libintl, since its omission, combined with Pg being built with ENABLE_NLS, tends to break things horribly. Users can always undefine ENABLE_NLS, but it's an unnecessary roadblock. Sandeep, can you work on fixing this please? Thanks. Are there any objections from -hackers to including 3rd party headers for libs we expose in our public headers in the binary distribution? Other than bundling 3rd party headers, any ideas/suggestions for how we might make ext building saner on Windows? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sandeep Thakkar
Re: [HACKERS] Compiling extensions on Windows
On Mon, Jan 6, 2014 at 10:57 AM, Sandeep Thakkar sandeep.thak...@enterprisedb.com wrote: Sure. I'll make the changes so that the next available Windows installers include lbintl.h in $Installdir/include. How about the changes with respect to NLS? No, there's nothing to change there. Craig was suggesting that users could disable NLS in their extension to avoid issues with the missing header, but that's not a good solution. On Mon, Jan 6, 2014 at 2:44 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Jan 6, 2014 at 3:32 AM, Craig Ringer cr...@2ndquadrant.com wrote: Hi all Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is Do a source build of PG, compile your ext in-tree in contrib/, and hope the result is binary compatible with release PostgreSQL builds for Windows. Certainly that's how I've been doing it to date. How about everyone else here? Does anyone actually build and distribute extensions out of tree at all? I'm interested in making the Windows installer distributions a bit more extension dev friendly. In particular, I'd really like to see EDB's Windows installers include the libintl.h for the included libintl, since its omission, combined with Pg being built with ENABLE_NLS, tends to break things horribly. Users can always undefine ENABLE_NLS, but it's an unnecessary roadblock. Sandeep, can you work on fixing this please? Thanks. Are there any objections from -hackers to including 3rd party headers for libs we expose in our public headers in the binary distribution? Other than bundling 3rd party headers, any ideas/suggestions for how we might make ext building saner on Windows? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sandeep Thakkar -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
If libintl.h and any headers it in turn includes are bundled, there is no longer an issue with NLS. That was just a workaround for building exts when Pg's headers tried to refer to nonexistent headers when NLS was enabled. On 6 Jan 2014 18:57, Sandeep Thakkar sandeep.thak...@enterprisedb.com wrote:Sure. Ill make the changes so that the next available Windows installers include lbintl.h in $Installdir/include. How about the changes with respect to NLS? On Mon, Jan 6, 2014 at 2:44 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Jan 6, 2014 at 3:32 AM, Craig Ringer cr...@2ndquadrant.com wrote: Hi all Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is Do a source build of PG, compile your ext in-tree in contrib/, and hope the result is binary compatible with release PostgreSQL builds for Windows. Certainly thats how Ive been doing it to date. How about everyone else here? Does anyone actually build and distribute extensions out of tree at all? Im interested in making the Windows installer distributions a bit more extension dev friendly. In particular, Id really like to see EDBs Windows installers include the libintl.h for the included libintl, since its omission, combined with Pg being built with ENABLE_NLS, tends to break things horribly. Users can always undefine ENABLE_NLS, but its an unnecessary roadblock. Sandeep, can you work on fixing this please? Thanks. Are there any objections from -hackers to including 3rd party headers for libs we expose in our public headers in the binary distribution? Other than bundling 3rd party headers, any ideas/suggestions for how we might make ext building saner on Windows? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sandeep Thakkar
Re: [HACKERS] Compiling extensions on Windows
Okay. BTW, I just checked that Windows 32bit installer ships the libintl.h. Did you try if it works for you? Or it requires more headers? Somehow, Windows 64bit installer installer missed it. I'll fix the build script. On Mon, Jan 6, 2014 at 4:55 PM, Craig Ringer cr...@2ndquadrant.com wrote: If libintl.h and any headers it in turn includes are bundled, there is no longer an issue with NLS. That was just a workaround for building exts when Pg's headers tried to refer to nonexistent headers when NLS was enabled. On 6 Jan 2014 18:57, Sandeep Thakkar sandeep.thak...@enterprisedb.com wrote: Sure. I'll make the changes so that the next available Windows installers include lbintl.h in $Installdir/include. How about the changes with respect to NLS? On Mon, Jan 6, 2014 at 2:44 PM, Dave Page dp...@pgadmin.org wrote: On Mon, Jan 6, 2014 at 3:32 AM, Craig Ringer cr...@2ndquadrant.com wrote: Hi all Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is Do a source build of PG, compile your ext in-tree in contrib/, and hope the result is binary compatible with release PostgreSQL builds for Windows. Certainly that's how I've been doing it to date. How about everyone else here? Does anyone actually build and distribute extensions out of tree at all? I'm interested in making the Windows installer distributions a bit more extension dev friendly. In particular, I'd really like to see EDB's Windows installers include the libintl.h for the included libintl, since its omission, combined with Pg being built with ENABLE_NLS, tends to break things horribly. Users can always undefine ENABLE_NLS, but it's an unnecessary roadblock. Sandeep, can you work on fixing this please? Thanks. Are there any objections from -hackers to including 3rd party headers for libs we expose in our public headers in the binary distribution? Other than bundling 3rd party headers, any ideas/suggestions for how we might make ext building saner on Windows? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sandeep Thakkar -- Sandeep Thakkar
Re: [HACKERS] Compiling extensions on Windows
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 01/05/2014 07:32 PM, Craig Ringer wrote: Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is Do a source build of PG, compile your ext in-tree in contrib/, and hope the result is binary compatible with release PostgreSQL builds for Windows. Certainly that's how I've been doing it to date. Yes, this pretty much exactly describes how I build PL/R for Windows. I had to match my build system SDK with the one EDB uses to get a compatible binary. It would be nice if we had something equivalent to PGXS on Windows, or maybe even a community build system where authors could get Windows binaries built. Joe - -- Joe Conway credativ LLC: http://www.credativ.us Linux, PostgreSQL, and general Open Source Training, Service, Consulting, 24x7 Support -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.14 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSytzOAAoJEDfy90M199hlgdUP/3wHL7Akq4ONkYlFXq8d3LGJ h5LNAWvILfJVOI3dmpHHzw8FCmWSsrsnDKuN/GUvJvhioPL0QsWgU1YGLwKNb7Mt Q3famk8hJCe8PikrL5KvItk1jQtxem+M4wGcKZZoM2cb4soeRDxM5FtTKzZMHGi0 9GA3Tx/TpgGfIP35Xg9ckL/LejyOZIndrRHuJHREGZlIP27AW0SEscZMDq25Q5yy jPcWki7hAIABohwkRPkFWVZArhSrCe1dA1Yy0Gad5cB/JdVB4xAjkmwLSa2suBod nQ65G/8Hz88GIRxY1FlzPn+06IDnSdnZmrhxfZLn8Vl/mnMoW9h6pKmBNyWTQMoP 25Ex5/tYIQ6iYyUO3Ic/B/23OMVHu9bWXeyk1hKEpqCFR/1BctzafQI/vA4dRs0u KRN3hua9GYnX+guw+d9mIkujPAeXphbjaMlgY6ckkENmiAg1HXfzv8+tKsQT4Pwx IqcSNzIsnzTRSag1IKklwUW6DuTSGyFHyXsbWRA+kkxL2/ucHL7f2mCmbZ3Qg8WW zthp6dN+9dLC/iH92qiS/nkFFxkkikyBpG2wb+Xcc0Ko1u26xp3e7ZFnCUZQ0Bse DTiOIywW89ICk7pokI8vMEwJIN5d42dZ01GL6XdLT9iPTnGXSuCQsE2GSMBxUuHs KbY+hsZrrvWH0QaVkrq5 =V4H5 -END PGP SIGNATURE- -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Compiling extensions on Windows
Hi all Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is Do a source build of PG, compile your ext in-tree in contrib/, and hope the result is binary compatible with release PostgreSQL builds for Windows. Certainly that's how I've been doing it to date. How about everyone else here? Does anyone actually build and distribute extensions out of tree at all? I'm interested in making the Windows installer distributions a bit more extension dev friendly. In particular, I'd really like to see EDB's Windows installers include the libintl.h for the included libintl, since its omission, combined with Pg being built with ENABLE_NLS, tends to break things horribly. Users can always undefine ENABLE_NLS, but it's an unnecessary roadblock. Are there any objections from -hackers to including 3rd party headers for libs we expose in our public headers in the binary distribution? Other than bundling 3rd party headers, any ideas/suggestions for how we might make ext building saner on Windows? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Compiling extensions on Windows
On 01/05/2014 10:32 PM, Craig Ringer wrote: Hi all Out of personal interest (in pain and suffering) I was recently looking into how to compile extensions out-of-tree on Windows using Visual Studio (i.e. no PGXS). It looks like the conventional answer to this is Do a source build of PG, compile your ext in-tree in contrib/, and hope the result is binary compatible with release PostgreSQL builds for Windows. Certainly that's how I've been doing it to date. How about everyone else here? Does anyone actually build and distribute extensions out of tree at all? I'm interested in making the Windows installer distributions a bit more extension dev friendly. In particular, I'd really like to see EDB's Windows installers include the libintl.h for the included libintl, since its omission, combined with Pg being built with ENABLE_NLS, tends to break things horribly. Users can always undefine ENABLE_NLS, but it's an unnecessary roadblock. Are there any objections from -hackers to including 3rd party headers for libs we expose in our public headers in the binary distribution? Other than bundling 3rd party headers, any ideas/suggestions for how we might make ext building saner on Windows? If you're bundling a DLL then I don't see why the corresponding header file shouldn't be included also. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
