Re: [HACKERS] narwhal and PGDLLIMPORT
On Wed, Oct 22, 2014 at 12:12:36AM -0400, Noah Misch wrote: On Mon, Oct 20, 2014 at 01:03:31AM -0400, Noah Misch wrote: I reproduced narwhal's problem using its toolchain on another 32-bit Windows Server 2003 system. The crash happens at the SHGetFolderPath() call in pqGetHomeDirectory(). A program can acquire that function via shfolder.dll or via shell32.dll; we've used the former method since commit 889f038, for better compatibility[1] with Windows NT 4.0. On this system, shfolder.dll's version loads and unloads shell32.dll. In PostgreSQL built using this older compiler, shfolder.dll:SHGetFolderPath() unloads libpq in addition to unloading shell32! That started with commit 846e91e. I don't expect to understand the mechanism behind it, but I recommend we switch back to linking libpq with shell32.dll. The MSVC build already does that in all supported branches, and it feels right for the MinGW build to follow suit in 9.4+. Windows versions that lack the symbol in shell32.dll are now ancient history. That allowed narwhal to proceed a bit further than before, but it crashes later in the dblink test suite. Will test again... Calls to ldap_init() exhibit the same problem shfolder.dll:SHGetFolderPath() exhibited: it loads and unloads some DLLs, and it manages to unload libpq in passing. There's nothing comparable to the above workaround this time, but I found a more fundamental fix. Each DLL header records the DLL's presumed name, viewable with objdump -x libpq.dll | grep ^Name. Per the GNU ld documentation, one can override it in a .def file: The optional LIBRARY name command indicates the internal name of the output DLL. If `name' does not include a suffix, the default library suffix, `.DLL' is appended. libpqdll.def uses LIBRARY LIBPQ, which yields an internal name of LIBPQ.dll under MinGW-w64 i686-4.9.1-release-win32-dwarf-rt_v3-rev1. Our MSVC build process also gives LIBPQ.dll. Under narwhal's older toolchain, libpq.dll gets a name of just LIBPQ. The attached patch brings the product of narwhal's toolchain in line with the others. I don't know by what magic wldap32.dll:ldap_init() and shfolder.dll:SHGetFolderPath() care about finding .dll in the names of loaded libraries, but it does fix the bug. This erases the impetus for my recent commit 53566fc. I'm inclined to keep that commit in the name of consistency with the MSVC build, but one could argue for reverting its 9.4 counterpart. I don't feel strongly either way, so I expect to let 2f51f42 stand. FYI, the problem is reproducible in a 32-bit PostgreSQL build running on 32-bit or 64-bit Windows Server 2003. It does not occur on 64-bit Windows Server 2008, even after marking postgres.exe to run in the compatibility mode for Windows Server 2003. (I did not try 32-bit Windows Server 2008.) commit 912846f (HEAD, master) Author: Noah Misch n...@leadboat.com AuthorDate: Sun Oct 26 18:02:55 2014 -0400 Commit: Noah Misch n...@leadboat.com CommitDate: Sun Oct 26 18:17:34 2014 -0400 MinGW: Include .dll extension in .def file LIBRARY commands. Newer toolchains append the extension implicitly if missing, but buildfarm member narwhal (gcc 3.4.2, ld 2.15.91 20040904) does not. This affects most core libraries having an exports.txt file, namely libpq and the ECPG support libraries. On Windows Server 2003, Windows API functions that load and unload DLLs internally will mistakenly unload a libpq whose DLL header reports LIBPQ instead of LIBPQ.dll. When, subsequently, control would return to libpq, the backend crashes. Back-patch to 9.4, like commit 846e91e0223cf9f2821c3ad4dfbb929cb027. Before that commit, we used a different linking technique that yielded libpq.dll in the DLL header. Commit 53566fc0940cf557416b13252df57350a4511ce4 worked around this by eliminating a call to a function that loads and unloads DLLs internally. That commit is no longer necessary for correctness, but its improving consistency with the MSVC build remains valid. diff --git a/src/Makefile.shlib b/src/Makefile.shlib index 012dd12..e3a06ef 100644 --- a/src/Makefile.shlib +++ b/src/Makefile.shlib @@ -423,13 +423,13 @@ UC_NAME = $(shell echo $(NAME) | tr 'abcdefghijklmnopqrstuvwxyz' 'ABCDEFGHIJKLMN lib$(NAME)dll.def: $(SHLIB_EXPORTS) echo '; DEF file for MS VC++' $@ - echo 'LIBRARY LIB$(UC_NAME)' $@ + echo 'LIBRARY LIB$(UC_NAME).dll' $@ echo 'EXPORTS' $@ sed -e '/^#/d' -e 's/^\(.*[ ]\)\([0-9][0-9]*\)/\1@ \2/' $ $@ lib$(NAME)ddll.def: $(SHLIB_EXPORTS) echo '; DEF file for MS VC++' $@ - echo 'LIBRARY LIB$(UC_NAME)D' $@ + echo 'LIBRARY LIB$(UC_NAME)D.dll' $@ echo 'EXPORTS' $@ sed -e '/^#/d' -e 's/^\(.*[ ]\)\([0-9][0-9]*\)/\1@ \2/' $ $@ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription:
Re: [HACKERS] narwhal and PGDLLIMPORT
Noah Misch n...@leadboat.com writes: Calls to ldap_init() exhibit the same problem shfolder.dll:SHGetFolderPath() exhibited: it loads and unloads some DLLs, and it manages to unload libpq in passing. There's nothing comparable to the above workaround this time, but I found a more fundamental fix. ... libpqdll.def uses LIBRARY LIBPQ, which yields an internal name of LIBPQ.dll under MinGW-w64 i686-4.9.1-release-win32-dwarf-rt_v3-rev1. Our MSVC build process also gives LIBPQ.dll. Under narwhal's older toolchain, libpq.dll gets a name of just LIBPQ. The attached patch brings the product of narwhal's toolchain in line with the others. I don't know by what magic wldap32.dll:ldap_init() and shfolder.dll:SHGetFolderPath() care about finding .dll in the names of loaded libraries, but it does fix the bug. Nice detective work! This erases the impetus for my recent commit 53566fc. I'm inclined to keep that commit in the name of consistency with the MSVC build, but one could argue for reverting its 9.4 counterpart. I don't feel strongly either way, so I expect to let 2f51f42 stand. Yeah, I lean the same way. The fewer differences in the results of our assorted Windows build processes, the better. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On Mon, Oct 20, 2014 at 01:03:31AM -0400, Noah Misch wrote: I reproduced narwhal's problem using its toolchain on another 32-bit Windows Server 2003 system. The crash happens at the SHGetFolderPath() call in pqGetHomeDirectory(). A program can acquire that function via shfolder.dll or via shell32.dll; we've used the former method since commit 889f038, for better compatibility[1] with Windows NT 4.0. On this system, shfolder.dll's version loads and unloads shell32.dll. In PostgreSQL built using this older compiler, shfolder.dll:SHGetFolderPath() unloads libpq in addition to unloading shell32! That started with commit 846e91e. I don't expect to understand the mechanism behind it, but I recommend we switch back to linking libpq with shell32.dll. The MSVC build already does that in all supported branches, and it feels right for the MinGW build to follow suit in 9.4+. Windows versions that lack the symbol in shell32.dll are now ancient history. That allowed narwhal to proceed a bit further than before, but it crashes later in the dblink test suite. Will test again... -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Noah Misch n...@leadboat.com writes: I reproduced narwhal's problem using its toolchain on another 32-bit Windows Server 2003 system. The crash happens at the SHGetFolderPath() call in pqGetHomeDirectory(). A program can acquire that function via shfolder.dll or via shell32.dll; we've used the former method since commit 889f038, for better compatibility[1] with Windows NT 4.0. On this system, shfolder.dll's version loads and unloads shell32.dll. In PostgreSQL built using this older compiler, shfolder.dll:SHGetFolderPath() unloads libpq in addition to unloading shell32! That started with commit 846e91e. Thanks for doing the detective work on this! I don't expect to understand the mechanism behind it, but I recommend we switch back to linking libpq with shell32.dll. The MSVC build already does that in all supported branches, and it feels right for the MinGW build to follow suit in 9.4+. Windows versions that lack the symbol in shell32.dll are now ancient history. This is basically reverting 889f038, right? It looks to me like we made that change only to support NT4, which was obsolete even in 2005, so no objection from me. Magnus might have a different idea though. I happened to try the same contrib/dblink test suite on PostgreSQL built with modern MinGW-w64 (i686-4.9.1-release-win32-dwarf-rt_v3-rev1). That, too, gave a crash-like symptom starting with commit 846e91e. Ick. Passing -static-libgcc to the link restores the libgcc situation as it stood before commit 846e91e. The main beneficiary of shared libgcc is C++/Java exception handling, so PostgreSQL doesn't care. No doubt there's some deeper bug in libgcc or in PostgreSQL; loading a module that links with shared libgcc should not disrupt exit(). I'm content with this workaround. Works for me too, until such time as somebody feels like digging deeper. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On Mon, Oct 20, 2014 at 11:06:54AM -0400, Tom Lane wrote: Noah Misch n...@leadboat.com writes: I don't expect to understand the mechanism behind it, but I recommend we switch back to linking libpq with shell32.dll. The MSVC build already does that in all supported branches, and it feels right for the MinGW build to follow suit in 9.4+. Windows versions that lack the symbol in shell32.dll are now ancient history. This is basically reverting 889f038, right? Mostly so. Keeping the SHGetSpecialFolderPath() - SHGetFolderPath() part, but reverting the shell32.dll - shfolder.dll part. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-10-20 01:03:31 -0400, Noah Misch wrote: On Wed, Oct 15, 2014 at 12:53:03AM -0400, Noah Misch wrote: On Tue, Oct 14, 2014 at 07:07:17PM -0400, Tom Lane wrote: Dave Page dp...@pgadmin.org writes: On Tue, Oct 14, 2014 at 11:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think we're hoping that somebody will step up and investigate how narwhal's problem might be fixed. I have planned to look at reproducing narwhal's problem once the dust settles on orangutan, but I wouldn't mind if narwhal went away instead. No argument here. I would kind of like to have more than zero understanding of *why* it's failing, just in case there's more to it than oh, probably a bug in this old toolchain. But finding that out might well take significant time, and in the end not tell us anything very useful. Agreed on all those points. I reproduced narwhal's problem using its toolchain on another 32-bit Windows Server 2003 system. The crash happens at the SHGetFolderPath() call in pqGetHomeDirectory(). A program can acquire that function via shfolder.dll or via shell32.dll; we've used the former method since commit 889f038, for better compatibility[1] with Windows NT 4.0. On this system, shfolder.dll's version loads and unloads shell32.dll. In PostgreSQL built using this older compiler, shfolder.dll:SHGetFolderPath() unloads libpq in addition to unloading shell32! That started with commit 846e91e. I don't expect to understand the mechanism behind it, but I recommend we switch back to linking libpq with shell32.dll. The MSVC build already does that in all supported branches, and it feels right for the MinGW build to follow suit in 9.4+. Windows versions that lack the symbol in shell32.dll are now ancient history. Ick. Nice detective work of a ugly situation. I happened to try the same contrib/dblink test suite on PostgreSQL built with modern MinGW-w64 (i686-4.9.1-release-win32-dwarf-rt_v3-rev1). That, too, gave a crash-like symptom starting with commit 846e91e. Specifically, a backend that LOADed any module linked to libpq (libpqwalreceiver, dblink, postgres_fdw) would suffer this after calling exit(0): === 3056 2014-10-20 00:40:15.163 GMT LOG: disconnection: session time: 0:00:00.515 user=cyg_server database=template1 host=127.0.0.1 port=3936 This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. 9300 2014-10-20 00:40:15.163 GMT LOG: server process (PID 3056) exited with exit code 3 === The mechanism turned out to be disjoint from the mechanism behind the ancient-compiler crash. Based on the functions called from exit(), my best guess is that exit() encountered recursion and used something like an abort() to escape. Hm. (I can send the gdb transcript if anyone is curious to see the gory details.) That would be interesting. The proximate cause was commit 846e91e allowing modules to use shared libgcc. A 32-bit libpq acquires 64-bit integer division from libgcc. Passing -static-libgcc to the link restores the libgcc situation as it stood before commit 846e91e. The main beneficiary of shared libgcc is C++/Java exception handling, so PostgreSQL doesn't care. No doubt there's some deeper bug in libgcc or in PostgreSQL; loading a module that links with shared libgcc should not disrupt exit(). I'm content with this workaround. I'm unconvinced by this reasoning. Popular postgres extensions like postgis do use C++. It's imo not hard to imagine situations where switching to a statically linked libgcc statically could cause problems. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On Mon, Oct 20, 2014 at 10:24:47PM +0200, Andres Freund wrote:
On 2014-10-20 01:03:31 -0400, Noah Misch wrote:
On Wed, Oct 15, 2014 at 12:53:03AM -0400, Noah Misch wrote:
I happened to try the same contrib/dblink test suite on PostgreSQL built
with
modern MinGW-w64 (i686-4.9.1-release-win32-dwarf-rt_v3-rev1). That, too,
gave
a crash-like symptom starting with commit 846e91e. Specifically, a backend
that LOADed any module linked to libpq (libpqwalreceiver, dblink,
postgres_fdw) would suffer this after calling exit(0):
===
3056 2014-10-20 00:40:15.163 GMT LOG: disconnection: session time:
0:00:00.515 user=cyg_server database=template1 host=127.0.0.1 port=3936
This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application's support team for more information.
9300 2014-10-20 00:40:15.163 GMT LOG: server process (PID 3056) exited
with exit code 3
===
The mechanism turned out to be disjoint from the mechanism behind the
ancient-compiler crash. Based on the functions called from exit(), my best
guess is that exit() encountered recursion and used something like an
abort()
to escape.
Hm.
(I can send the gdb transcript if anyone is curious to see the
gory details.)
That would be interesting.
Attached. (rep 100 s calls a macro equivalent to issuing s 100 times.)
The proximate cause was commit 846e91e allowing modules to use
shared libgcc. A 32-bit libpq acquires 64-bit integer division from libgcc.
Passing -static-libgcc to the link restores the libgcc situation as it stood
before commit 846e91e. The main beneficiary of shared libgcc is C++/Java
exception handling, so PostgreSQL doesn't care. No doubt there's some
deeper
bug in libgcc or in PostgreSQL; loading a module that links with shared
libgcc
should not disrupt exit(). I'm content with this workaround.
I'm unconvinced by this reasoning. Popular postgres extensions like
postgis do use C++. It's imo not hard to imagine situations where
switching to a statically linked libgcc statically could cause problems.
True; I was wrong to say that PostgreSQL doesn't care. MinGW builds of every
released PostgreSQL version use static libgcc. That changed as an unintended
consequence of a patch designed to remove our reliance on dlltool. Given the
lack of complaints about our historic use of static libgcc, I think it's fair
to revert to 9.3's use thereof. Supporting shared libgcc would be better
still, should someone make that effort.
Breakpoint 1, 0x77bcaf46 in msvcrt!exit () from C:\WINDOWS\system32\msvcrt.dll
#0 0x77bcaf46 in msvcrt!exit () from C:\WINDOWS\system32\msvcrt.dll
#1 0x0065637b in proc_exit (code=code@entry=0) at ipc.c:143
#2 0x006798bf in PostgresMain (argc=1, argv=argv@entry=0x225830,
dbname=0x224498 template1, username=0x224460 cyg_server)
at postgres.c:4232
#3 0x0061b26e in BackendRun (port=0x205f520) at postmaster.c:4113
#4 SubPostmasterMain (argc=argc@entry=3, argv=argv@entry=0x222e88)
at postmaster.c:4617
#5 0x007af2a1 in main (argc=3, argv=0x222e88) at main.c:196
(gdb) s
Single stepping until exit from function msvcrt!exit,
which has no line number information.
0x77bcae7a in msvcrt!_initterm () from C:\WINDOWS\system32\msvcrt.dll
(gdb)
Single stepping until exit from function msvcrt!_initterm,
which has no line number information.
0x77bc84c4 in strerror () from C:\WINDOWS\system32\msvcrt.dll
(gdb)
Single stepping until exit from function strerror,
which has no line number information.
0x77bcae86 in msvcrt!_initterm () from C:\WINDOWS\system32\msvcrt.dll
(gdb)
Single stepping until exit from function msvcrt!_initterm,
which has no line number information.
atexit_callback () at ipc.c:283
warning: Source file is more recent than executable.
283 proc_exit_prepare(-1);
(gdb)
proc_exit_prepare (code=-1) at ipc.c:153
153 {
(gdb) fin
Run till exit from #0 proc_exit_prepare (code=-1) at ipc.c:153
0x77bcaed6 in msvcrt!_initterm () from C:\WINDOWS\system32\msvcrt.dll
(gdb) s
Single stepping until exit from function msvcrt!_initterm,
which has no line number information.
Breakpoint 1, 0x77bcaf61 in msvcrt!_exit ()
from C:\WINDOWS\system32\msvcrt.dll
(gdb) rep 100 s
Single stepping until exit from function msvcrt!_exit,
which has no line number information.
0x77bcae7a in msvcrt!_initterm () from C:\WINDOWS\system32\msvcrt.dll
Single stepping until exit from function msvcrt!_initterm,
which has no line number information.
0x77bc84c4 in strerror () from C:\WINDOWS\system32\msvcrt.dll
Single stepping until exit from function strerror,
which has no line number information.
0x77bcae86 in msvcrt!_initterm () from C:\WINDOWS\system32\msvcrt.dll
Single stepping until exit from function msvcrt!_initterm,
which has no line
Re: [HACKERS] narwhal and PGDLLIMPORT
On 10/15/2014 01:53 AM, Craig Ringer wrote: On 10/15/2014 12:53 PM, Noah Misch wrote: Windows Server 2003 isn't even EOL yet. I'd welcome a buildfarm member with that OS and a modern toolchain. It's possible to run multiple buildfarm animals on a single Windows instance, each with a different toolchain. Indeed, and even using the same buildroot. That's been built into the buildfarm for a long time. For example, nightjar and friarbird do this. There's the chance of interactions that can't occur if each SDK is used in isolation on its own machine, but it should be pretty minimal. Make sure each has a distinct base_port. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
It seems we left this in broken state. Do we need to do more here to fix narwhal, or do we want to retire narwhal now? Something else? Are we waiting on someone in particular to do something specific? -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Alvaro Herrera alvhe...@2ndquadrant.com writes: It seems we left this in broken state. Do we need to do more here to fix narwhal, or do we want to retire narwhal now? Something else? Are we waiting on someone in particular to do something specific? I think we're hoping that somebody will step up and investigate how narwhal's problem might be fixed. However, the machine's owner (Dave) doesn't appear to have the time/interest to do that. That means that our realistic choices are to retire narwhal or revert the linker changes that broke it. Since those linker changes were intended to help expose missing-PGDLLIMPORT bugs, I don't much care for the second alternative. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On Tue, Oct 14, 2014 at 11:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: It seems we left this in broken state. Do we need to do more here to fix narwhal, or do we want to retire narwhal now? Something else? Are we waiting on someone in particular to do something specific? I think we're hoping that somebody will step up and investigate how narwhal's problem might be fixed. However, the machine's owner (Dave) doesn't appear to have the time/interest to do that. That means that our realistic choices are to retire narwhal or revert the linker changes that broke it. Since those linker changes were intended to help expose missing-PGDLLIMPORT bugs, I don't much care for the second alternative. It's a time issue right now I'm afraid (always interested in fixing bugs). However, if fixing it comes down to upgrading the seriously old compiler and toolchain on that box (which frankly is so obsolete, I can't see why anyone would want to use anything like it these days), then I think the best option is to retire it, and replace it with Windows 2012R2 and a modern release of MinGW/Msys which is far more likely to be similar to what someone would want to use at present. Does anyone really think there's a good reason to keep maintaining such an obsolete animal? -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 10/14/2014 06:44 PM, Dave Page wrote: On Tue, Oct 14, 2014 at 11:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: It seems we left this in broken state. Do we need to do more here to fix narwhal, or do we want to retire narwhal now? Something else? Are we waiting on someone in particular to do something specific? I think we're hoping that somebody will step up and investigate how narwhal's problem might be fixed. However, the machine's owner (Dave) doesn't appear to have the time/interest to do that. That means that our realistic choices are to retire narwhal or revert the linker changes that broke it. Since those linker changes were intended to help expose missing-PGDLLIMPORT bugs, I don't much care for the second alternative. It's a time issue right now I'm afraid (always interested in fixing bugs). However, if fixing it comes down to upgrading the seriously old compiler and toolchain on that box (which frankly is so obsolete, I can't see why anyone would want to use anything like it these days), then I think the best option is to retire it, and replace it with Windows 2012R2 and a modern release of MinGW/Msys which is far more likely to be similar to what someone would want to use at present. Does anyone really think there's a good reason to keep maintaining such an obsolete animal? I do not. I upgraded from this ancient toolset quite a few years ago, and I'm actually thinking of retiring what I replaced it with. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Dave Page dp...@pgadmin.org writes: On Tue, Oct 14, 2014 at 11:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think we're hoping that somebody will step up and investigate how narwhal's problem might be fixed. However, the machine's owner (Dave) doesn't appear to have the time/interest to do that. It's a time issue right now I'm afraid (always interested in fixing bugs). However, if fixing it comes down to upgrading the seriously old compiler and toolchain on that box (which frankly is so obsolete, I can't see why anyone would want to use anything like it these days), then I think the best option is to retire it, and replace it with Windows 2012R2 and a modern release of MinGW/Msys which is far more likely to be similar to what someone would want to use at present. No argument here. I would kind of like to have more than zero understanding of *why* it's failing, just in case there's more to it than oh, probably a bug in this old toolchain. But finding that out might well take significant time, and in the end not tell us anything very useful. Does anyone really think there's a good reason to keep maintaining such an obsolete animal? Is it likely that anyone is still using Windows 2003 in the field? A possible compromise is to update the toolchain but stay on the same OS release, so that we still have testing that's relevant to people using older OS releases. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On Tue, Oct 14, 2014 at 07:07:17PM -0400, Tom Lane wrote: Dave Page dp...@pgadmin.org writes: On Tue, Oct 14, 2014 at 11:38 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think we're hoping that somebody will step up and investigate how narwhal's problem might be fixed. I have planned to look at reproducing narwhal's problem once the dust settles on orangutan, but I wouldn't mind if narwhal went away instead. However, if fixing it comes down to upgrading the seriously old compiler and toolchain on that box (which frankly is so obsolete, I can't see why anyone would want to use anything like it these days), then I think the best option is to retire it, and replace it with Windows 2012R2 and a modern release of MinGW/Msys which is far more likely to be similar to what someone would want to use at present. If you upgrade the toolchain, you really have a new animal. No argument here. I would kind of like to have more than zero understanding of *why* it's failing, just in case there's more to it than oh, probably a bug in this old toolchain. But finding that out might well take significant time, and in the end not tell us anything very useful. Agreed on all those points. Is it likely that anyone is still using Windows 2003 in the field? A possible compromise is to update the toolchain but stay on the same OS release, so that we still have testing that's relevant to people using older OS releases. Windows Server 2003 isn't even EOL yet. I'd welcome a buildfarm member with that OS and a modern toolchain. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 10/15/2014 12:53 PM, Noah Misch wrote: Windows Server 2003 isn't even EOL yet. I'd welcome a buildfarm member with that OS and a modern toolchain. It's possible to run multiple buildfarm animals on a single Windows instance, each with a different toolchain. There's the chance of interactions that can't occur if each SDK is used in isolation on its own machine, but it should be pretty minimal. I have a machine that currently does this with Jenkins. I'll look at bringing up buildfarm members on it. It's win2k8 though. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(Top post, on phone) The @number part is optional. It indicates an export ordinal. (You don't want to know, if you do, see MSDN). If you remove them or change them then binaries linked to the older version will fail to link to the newer; it breaks binary compat. The ordinals are part of the library ABI. On 20 Feb 2014 07:28, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Tom Lane escribió: Hiroshi Inoue in...@tpf.co.jp writes: Attached is a patch to remove dllwarp from pgevent/Makefile. Actually, it looks like this patch doesn't work at all: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacanadt=2014-02-20%2001%3A00%3A53 Did I fat-finger the commit somehow? I made a couple of cosmetic changes, or at least I thought they were cosmetic. The format of the file is completely unlike that of the other *dll.def files we use elsewhere. AFAICT each line is tabsymbolname@tabsome_number -- Álvaro Herrera http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/18 0:02), Dave Page wrote: On Mon, Feb 17, 2014 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Fri, Feb 14, 2014 at 5:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Not sure - it's certainly installed on the box. I've enabled it for now, and will see what happens. Sigh ... stop the presses. In 9.3, narwhal is *still* showing a PGDLLIMPORT-type failure that no other Windows critter is unhappy about: dlltool --export-all --output-def worker_spi.def worker_spi.o dllwrap -o worker_spi.dll --def worker_spi.def worker_spi.o -L../../src/port -L../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -L../../src/backend -lpostgres Info: resolving _MyBgworkerEntry by linking to __imp__MyBgworkerEntry (auto-import) fu01.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' fu02.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' nmth00.o(.idata$4+0x0): undefined reference to `_nm__MyBgworkerEntry' collect2: ld returned 1 exit status So we are back to square one AFAICS: we still have no idea why narwhal is pickier than everything else. (BTW, to save people the trouble of looking: MyBgworkerEntry is marked PGDLLIMPORT in HEAD but not 9.3.) Also, in HEAD narwhal is building things OK, but then seems to be dumping core in the dblink regression test, leaving one with not a very warm feeling about whether the contrib executables it's building are any good. Well, as we know, Narwhal is really quite old now. I think I built it seven+ years ago. There is a difference between narwhal and my pretty old machine (Windows Vista gcc 3.4.5). Linking perl58.lib or tcl84.lib works on my machine but neither works on narwhal. Therefore dlltool still remains in Makefiles of plperl, pltcl and plpython (linking pythonxx.lib doesn't work even on a newer machine). I tried another way which links the dll directly instead of msvc import library and got successful results on both old and newer machines except for plpython on an old machine which fails with a compilation error before linkage phase. I attached a patch which eliminates pexports and dlltool from makefile of plperl. The patch links mingw gcc import library if it exists, otherwise links the dll directly. When it works on narwhal, I can provide similar patches for pltcl and plpython. regards, Hiroshi Inoue diff --git a/src/pl/plperl/GNUmakefile b/src/pl/plperl/GNUmakefile index 1e800a3..55fe9b9 100644 --- a/src/pl/plperl/GNUmakefile +++ b/src/pl/plperl/GNUmakefile @@ -36,23 +36,17 @@ DATA = plperl.control plperl--1.0.sql plperl--unpackaged--1.0.sql \ PERLCHUNKS = plc_perlboot.pl plc_trusted.pl -# Perl on win32 ships with import libraries only for Microsoft Visual C++, -# which are not compatible with mingw gcc. Therefore we need to build a -# new import library to link with. +# When Perl on win32 doesn't ship with mingw import libraries, link to the DLL +# directly. ifeq ($(PORTNAME), win32) +perllibdir := $(subst -L,,$(filter -L%, $(perl_embed_ldflags))) perlwithver := $(subst -l,,$(filter -l%, $(perl_embed_ldflags))) -PERLDLL := $(dir $(subst ',,$(PERL)))$(perlwithver).dll -# we no longer want to include the original -l spec in SHLIB_LINK -override perl_embed_ldflags := - -OBJS += lib$(perlwithver).a - -lib$(perlwithver).a: $(perlwithver).def - dlltool --dllname $(perlwithver).dll --def $(perlwithver).def --output-lib lib$(perlwithver).a - -$(perlwithver).def: $(PERLDLL) - pexports $^ $@ +# Link to the perl DLL directly in the bin directory unless the mingw import +# library exists. +ifeq (, $(shell find $(perllibdir) -maxdepth 1 -name lib$(perlwithver).a)) +override perl_embed_ldflags := -L$(dir $(PERL))' -l$(perlwithver) +endif endif # win32 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Hiroshi Inoue in...@tpf.co.jp writes: (2014/02/12 3:03), Tom Lane wrote: Also, the only remaining usage of dllwrap is in src/bin/pgevent/Makefile. Do we need that either? Sorry for the late reply. Attached is a patch to remove dllwarp from pgevent/Makefile. Pushed, thanks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Hiroshi Inoue in...@tpf.co.jp writes: Attached is a patch to remove dllwarp from pgevent/Makefile. Actually, it looks like this patch doesn't work at all: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacanadt=2014-02-20%2001%3A00%3A53 Did I fat-finger the commit somehow? I made a couple of cosmetic changes, or at least I thought they were cosmetic. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Tom Lane escribió: Hiroshi Inoue in...@tpf.co.jp writes: Attached is a patch to remove dllwarp from pgevent/Makefile. Actually, it looks like this patch doesn't work at all: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacanadt=2014-02-20%2001%3A00%3A53 Did I fat-finger the commit somehow? I made a couple of cosmetic changes, or at least I thought they were cosmetic. The format of the file is completely unlike that of the other *dll.def files we use elsewhere. AFAICT each line is tabsymbolname@tabsome_number -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/20 10:32), Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: Attached is a patch to remove dllwarp from pgevent/Makefile. Actually, it looks like this patch doesn't work at all: Strangely enough it works here though I see double EXPORTS lines in libpgeventdll.def. http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jacanadt=2014-02-20%2001%3A00%3A53 Did I fat-finger the commit somehow? I made a couple of cosmetic changes, or at least I thought they were cosmetic. Seems EXPORTS line in exports.txt should be removed. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Hiroshi Inoue in...@tpf.co.jp writes: Seems EXPORTS line in exports.txt should be removed. Done. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/12 15:31), Inoue, Hiroshi wrote: (2014/02/12 3:03), Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: (2014/02/09 8:06), Andrew Dunstan wrote: Yeah. Incidentally, we didn't quite get rid of dlltool for Cygwin. We did get rid of dllwrap. But I agree this is worth trying for Mingw. I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. I cleaned this up a bit (the if-nesting in Makefile.shlib was making my head hurt, not to mention that it left a bunch of dead code) and committed it. Thanks. By my count, the only remaining usage of dlltool is in plpython's Makefile. Can we get rid of that? Maybe this is one of the few use cases of dlltool. Because python doesn't ship with its MINGW import library, the Makefile uses dlltool to generate an import library from the python DLL. Also, the only remaining usage of dllwrap is in src/bin/pgevent/Makefile. Do we need that either? Maybe this can be removed. I would make a patch later. Sorry for the late reply. Attached is a patch to remove dllwarp from pgevent/Makefile. regards, Hiroshi Inoue diff --git a/src/bin/pgevent/Makefile b/src/bin/pgevent/Makefile index 1d90276..0a7c16d 100644 --- a/src/bin/pgevent/Makefile +++ b/src/bin/pgevent/Makefile @@ -17,30 +17,21 @@ include $(top_builddir)/src/Makefile.global ifeq ($(PORTNAME), win32) OBJS=pgevent.o pgmsgevent.o -NAME=pgevent.dll +NAME=pgevent +SHLIB_LINK = +SHLIB_EXPORTS = exports.txt -all: $(NAME) +all: all-lib -install: all install-lib +include $(top_srcdir)/src/Makefile.shlib -pgevent.dll: pgevent.def $(OBJS) - $(DLLWRAP) --def $ -o $(NAME) $(OBJS) +install: all install-lib pgmsgevent.o: pgmsgevent.rc win32ver.rc $(WINDRES) $ -o $@ --include-dir=$(top_builddir)/src/include --include-dir=$(top_srcdir)/src/include --include-dir=$(srcdir) --include-dir=. -all-lib: $(NAME) - -install-lib: $(NAME) - $(INSTALL_STLIB) $ '$(DESTDIR)$(libdir)/$' - -uninstall-lib: - rm -f '$(DESTDIR)$(libdir)/$(NAME)' - -clean distclean: - rm -f $(OBJS) $(NAME) win32ver.rc -clean-lib: - rm -f $(NAME) +clean distclean: clean-lib + rm -f $(OBJS) win32ver.rc $(DLL_DEFFILE) endif diff --git a/src/bin/pgevent/exports.txt b/src/bin/pgevent/exports.txt new file mode 100644 index 000..70dcd25 --- /dev/null +++ b/src/bin/pgevent/exports.txt @@ -0,0 +1,5 @@ +; dlltool --output-def pgevent.def pgevent.o pgmsgevent.o +EXPORTS + DllUnregisterServer@0 ; + DllRegisterServer@0 ; + DllInstall ; -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On Fri, Feb 14, 2014 at 5:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: One thing I'm wondering about is that plperl is linking perlxx.lib not libperlxx.a. I made a patch following plpython and it also works here. Is it worth trying? I hadn't noticed that part of plpython's Makefile before. Man, that's an ugly technique :-(. Still, there's little about this platform that isn't ugly. Let's try it and see what happens. And what happens is this: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhaldt=2014-02-14%2017%3A00%3A02 namely, it gets through plperl now and then chokes with the same symptoms on pltcl. So I guess we need the same hack in pltcl. The fun never stops ... (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Not sure - it's certainly installed on the box. I've enabled it for now, and will see what happens. -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 02/16/2014 07:03 AM, Andres Freund wrote: On 2014-02-15 17:48:00 -0500, Tom Lane wrote: Marco Atzeri marco.atz...@gmail.com writes: 32 $ grep -rH in6addr_any * cygwin/in6.h:extern const struct in6_addr in6addr_any; cygwin/version.h: in6addr_any, in6addr_loopback. So how come there's a declspec on the getopt.h variables, but not this one? Well, those are real constants, so they probably are fully contained in the static link library, no need to dynamically resolve them. If they're externs that're then defined in a single module, that's not the case. Only if they're *not* declared extern, and thus added to each compilation unit then de-duplicated during linking, may they be safely referred to without a __declspec(dllimport) because each module gets its own copy. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Hi, I just wanted to mention that it should probably not be too hard to emulate the current windows behaviour in gcc/clang elf targets. Somebody (won't be me) could add a --emulate-windows-linkage configure flag or such. By mapping PGDLLIMPORT to__attribute__((section(...))) it should be relatively straightforward to put all exported variables into a special section and use a linker script to change the visibility of the default data, bss, rodata sections... /crazytalk Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Dave Page dp...@pgadmin.org writes: On Fri, Feb 14, 2014 at 5:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Not sure - it's certainly installed on the box. I've enabled it for now, and will see what happens. Sigh ... stop the presses. In 9.3, narwhal is *still* showing a PGDLLIMPORT-type failure that no other Windows critter is unhappy about: dlltool --export-all --output-def worker_spi.def worker_spi.o dllwrap -o worker_spi.dll --def worker_spi.def worker_spi.o -L../../src/port -L../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -L../../src/backend -lpostgres Info: resolving _MyBgworkerEntry by linking to __imp__MyBgworkerEntry (auto-import) fu01.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' fu02.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' nmth00.o(.idata$4+0x0): undefined reference to `_nm__MyBgworkerEntry' collect2: ld returned 1 exit status So we are back to square one AFAICS: we still have no idea why narwhal is pickier than everything else. (BTW, to save people the trouble of looking: MyBgworkerEntry is marked PGDLLIMPORT in HEAD but not 9.3.) Also, in HEAD narwhal is building things OK, but then seems to be dumping core in the dblink regression test, leaving one with not a very warm feeling about whether the contrib executables it's building are any good. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On Mon, Feb 17, 2014 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Dave Page dp...@pgadmin.org writes: On Fri, Feb 14, 2014 at 5:32 PM, Tom Lane t...@sss.pgh.pa.us wrote: (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Not sure - it's certainly installed on the box. I've enabled it for now, and will see what happens. Sigh ... stop the presses. In 9.3, narwhal is *still* showing a PGDLLIMPORT-type failure that no other Windows critter is unhappy about: dlltool --export-all --output-def worker_spi.def worker_spi.o dllwrap -o worker_spi.dll --def worker_spi.def worker_spi.o -L../../src/port -L../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -L../../src/backend -lpostgres Info: resolving _MyBgworkerEntry by linking to __imp__MyBgworkerEntry (auto-import) fu01.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' fu02.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' nmth00.o(.idata$4+0x0): undefined reference to `_nm__MyBgworkerEntry' collect2: ld returned 1 exit status So we are back to square one AFAICS: we still have no idea why narwhal is pickier than everything else. (BTW, to save people the trouble of looking: MyBgworkerEntry is marked PGDLLIMPORT in HEAD but not 9.3.) Also, in HEAD narwhal is building things OK, but then seems to be dumping core in the dblink regression test, leaving one with not a very warm feeling about whether the contrib executables it's building are any good. Well, as we know, Narwhal is really quite old now. I think I built it seven+ years ago. Is it really worth banging heads against walls to support something that noone in their right mind should be using for a build these days? -- Dave Page Blog: http://pgsnake.blogspot.com Twitter: @pgsnake EnterpriseDB UK: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-17 15:02:15 +, Dave Page wrote: On Mon, Feb 17, 2014 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: Not sure - it's certainly installed on the box. I've enabled it for now, and will see what happens. Sigh ... stop the presses. In 9.3, narwhal is *still* showing a PGDLLIMPORT-type failure that no other Windows critter is unhappy about: dlltool --export-all --output-def worker_spi.def worker_spi.o dllwrap -o worker_spi.dll --def worker_spi.def worker_spi.o -L../../src/port -L../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -L../../src/backend -lpostgres Info: resolving _MyBgworkerEntry by linking to __imp__MyBgworkerEntry (auto-import) fu01.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' fu02.o(.idata$3+0xc): undefined reference to `libpostgres_a_iname' nmth00.o(.idata$4+0x0): undefined reference to `_nm__MyBgworkerEntry' collect2: ld returned 1 exit status So we are back to square one AFAICS: we still have no idea why narwhal is pickier than everything else. (BTW, to save people the trouble of looking: MyBgworkerEntry is marked PGDLLIMPORT in HEAD but not 9.3.) Also, in HEAD narwhal is building things OK, but then seems to be dumping core in the dblink regression test, leaving one with not a very warm feeling about whether the contrib executables it's building are any good. Well, as we know, Narwhal is really quite old now. I think I built it seven+ years ago. Is it really worth banging heads against walls to support something that noone in their right mind should be using for a build these days? The problem is that lots of those issues are bugs that actually cause problems for msvc builds. If there were tests in worker_spi it'd quite possibly crash when run in 9.3. The problem is rather that the other animals are *not* erroring. Unless I am missing something. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-17 15:02:15 +, Dave Page wrote: On Mon, Feb 17, 2014 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote: In 9.3, narwhal is *still* showing a PGDLLIMPORT-type failure that no other Windows critter is unhappy about: Well, as we know, Narwhal is really quite old now. I think I built it seven+ years ago. Is it really worth banging heads against walls to support something that noone in their right mind should be using for a build these days? The problem is that lots of those issues are bugs that actually cause problems for msvc builds. If there were tests in worker_spi it'd quite possibly crash when run in 9.3. The problem is rather that the other animals are *not* erroring. Exactly. Although on second thought, the lack of complaints from other Windows animals can probably be blamed on the fact that we didn't back-port any of the recent hacking on the Windows build processes. Maybe we should think about doing so, now that the dust seems to have settled. We still need to know why narwhal is crashing on dblink though. I have a bad feeling that that may indicate still-unresolved linkage problems. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-17 10:21:12 -0500, Tom Lane wrote:
Andres Freund and...@2ndquadrant.com writes:
On 2014-02-17 15:02:15 +, Dave Page wrote:
On Mon, Feb 17, 2014 at 2:58 PM, Tom Lane t...@sss.pgh.pa.us wrote:
In 9.3, narwhal is *still* showing a PGDLLIMPORT-type failure that no
other Windows critter is unhappy about:
Well, as we know, Narwhal is really quite old now. I think I built it
seven+ years ago. Is it really worth banging heads against walls to
support something that noone in their right mind should be using for a
build these days?
The problem is that lots of those issues are bugs that actually cause
problems for msvc builds. If there were tests in worker_spi it'd quite
possibly crash when run in 9.3. The problem is rather that the other
animals are *not* erroring.
Exactly.
Although on second thought, the lack of complaints from other Windows
animals can probably be blamed on the fact that we didn't back-port
any of the recent hacking on the Windows build processes. Maybe we
should think about doing so, now that the dust seems to have settled.
Yea, at the very least the gendef.pl thing should be backported,
possibly the mingw --disable-auto-import thing as well. But it's
probably a gooid idea to wait till the branches are stamped?
We still need to know why narwhal is crashing on dblink though.
I have a bad feeling that that may indicate still-unresolved
linkage problems.
It's odd:
[53019d05.f58:2] LOG: server process (PID 2428) exited with exit code 128
[53019d05.f58:3] DETAIL: Failed process was running: SELECT *
FROM dblink('dbname=contrib_regression','SELECT * FROM foo') AS t(a
int, b text, c text[])
WHERE t.a 7;
[53019d05.f58:4] LOG: server process (PID 2428) exited with exit code 0
[53019d05.f58:5] LOG: terminating any other active server processes
[53019d06.e9c:2] WARNING: terminating connection because of crash of another
server process
[53019d06.e9c:3] DETAIL: The postmaster has commanded this server process to
roll back the curreGreetings,
Not sure if that's actually a segfault and not something else. Why is
the same death reported twice? With different exit codes?
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-17 10:21:12 -0500, Tom Lane wrote: Although on second thought, the lack of complaints from other Windows animals can probably be blamed on the fact that we didn't back-port any of the recent hacking on the Windows build processes. Maybe we should think about doing so, now that the dust seems to have settled. Yea, at the very least the gendef.pl thing should be backported, possibly the mingw --disable-auto-import thing as well. But it's probably a gooid idea to wait till the branches are stamped? Certainly --- I'm not touching that till the releases are out ;-). Anything that's broken now has been broken in past releases too, so it's not worth the risk. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Marco, Andrew: On 2014-02-15 22:11:37 +0100, Marco Atzeri wrote: ../../src/timezone/localtime.o ../../src/timezone/strftime.o ../../src/timezone/pgtz.o ../../src/port/libpgport_srv.a ../../src/common/libpgcommon_srv.a -lintl -lssl -lcrypto -lcrypt -lldap -o postgres libpq/auth.o:auth.c:(.text+0x1940): undefined reference to `in6addr_any' libpq/auth.o:auth.c:(.text+0x1954): undefined reference to `in6addr_any' libpq/auth.o:auth.c:(.text+0x196d): undefined reference to `in6addr_any' libpq/auth.o:auth.c:(.text+0x1979): undefined reference to `in6addr_any' /usr/lib/gcc/i686-pc-cygwin/4.8.2/../../../../i686-pc-cygwin/bin/ld: libpq/auth.o: bad reloc address 0xce4 in section `.rdata' collect2: error: ld returned 1 exit status Makefile:66: recipe for target 'postgres' failed make[2]: *** [postgres] Error 1 make[2]: Leaving directory '/pub/devel/postgresql/git/postgresql_build/src/backend' Could either of you try whether compiling with the attached hack fixes anything on cygwin? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services diff --git a/src/backend/Makefile b/src/backend/Makefile index 870a022..a991134 100644 --- a/src/backend/Makefile +++ b/src/backend/Makefile @@ -62,6 +62,8 @@ endif ifeq ($(PORTNAME), cygwin) +LIBS += -lwsock32 -lws2_32 + postgres: $(OBJS) $(CC) $(CFLAGS) $(LDFLAGS) $(LDFLAGS_EX) $(export_dynamic) -Wl,--stack,$(WIN32_STACK_RLIMIT) -Wl,--export-all-symbols -Wl,--out-implib=libpostgres.a $(call expand_subsys,$^) $(LIBS) -o $@ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Hiroshi Inoue in...@tpf.co.jp writes: (2014/02/15 2:32), Tom Lane wrote: And what happens is this: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhaldt=2014-02-14%2017%3A00%3A02 namely, it gets through plperl now and then chokes with the same symptoms on pltcl. So I guess we need the same hack in pltcl. The fun never stops ... Pltcl still fails. tclxx.dll lives in bin directory not in lib directory. The attached patch would fix the problem. Pushed, thanks. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 16/02/2014 15:43, Andres Freund wrote: Marco, Andrew: On 2014-02-15 22:11:37 +0100, Marco Atzeri wrote: ../../src/timezone/localtime.o ../../src/timezone/strftime.o ../../src/timezone/pgtz.o ../../src/port/libpgport_srv.a ../../src/common/libpgcommon_srv.a -lintl -lssl -lcrypto -lcrypt -lldap -o postgres libpq/auth.o:auth.c:(.text+0x1940): undefined reference to `in6addr_any' libpq/auth.o:auth.c:(.text+0x1954): undefined reference to `in6addr_any' libpq/auth.o:auth.c:(.text+0x196d): undefined reference to `in6addr_any' libpq/auth.o:auth.c:(.text+0x1979): undefined reference to `in6addr_any' /usr/lib/gcc/i686-pc-cygwin/4.8.2/../../../../i686-pc-cygwin/bin/ld: libpq/auth.o: bad reloc address 0xce4 in section `.rdata' collect2: error: ld returned 1 exit status Makefile:66: recipe for target 'postgres' failed make[2]: *** [postgres] Error 1 make[2]: Leaving directory '/pub/devel/postgresql/git/postgresql_build/src/backend' Could either of you try whether compiling with the attached hack fixes anything on cygwin? Greetings, Andres Freund on cygwin32 bit it works, but it stops later on --- sl -lcrypto -lz -lreadline -lcrypt -o psql.exe tab-complete.o:tab-complete.c:(.text+0xa98): undefined reference to `rl_line_buffer' tab-complete.o:tab-complete.c:(.text+0xa387): undefined reference to `rl_attempted_completion_function' tab-complete.o:tab-complete.c:(.text+0xa391): undefined reference to `rl_basic_word_break_characters' tab-complete.o:tab-complete.c:(.text+0xa3a4): undefined reference to `rl_readline_name' /usr/lib/gcc/i686-pc-cygwin/4.8.2/../../../../i686-pc-cygwin/bin/ld: tab-complete.o: bad reloc address 0x30ec in section `.rdata' /usr/lib/gcc/i686-pc-cygwin/4.8.2/../../../../i686-pc-cygwin/bin/ld: final link failed: Invalid operation collect2: error: ld returned 1 exit status Makefile:33: recipe for target 'psql' failed --- on cygwin 64bit, that I was not testing before, something is strange -- -lintl -lssl -lcrypto -lcrypt -lldap -lwsock32 -lws2_32 -o postgres postmaster/postmaster.o:postmaster.c:(.rdata$.refptr.environ[.refptr.environ]+0x0): undefined reference to `environ' collect2: error: ld returned 1 exit status Makefile:68: recipe for target 'postgres' failed make[2]: *** [postgres] Error 1 --- of course 9.3.2 builds fine on cygwin 64bit. Regards Marco -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Marco Atzeri marco.atz...@gmail.com writes: On 16/02/2014 15:43, Andres Freund wrote: Could either of you try whether compiling with the attached hack fixes anything on cygwin? on cygwin32 bit it works, but it stops later on --- sl -lcrypto -lz -lreadline -lcrypt -o psql.exe tab-complete.o:tab-complete.c:(.text+0xa98): undefined reference to `rl_line_buffer' on cygwin 64bit, that I was not testing before, something is strange -- -lintl -lssl -lcrypto -lcrypt -lldap -lwsock32 -lws2_32 -o postgres postmaster/postmaster.o:postmaster.c:(.rdata$.refptr.environ[.refptr.environ]+0x0): undefined reference to `environ' So what we currently know is that on cygwin, some of the core system include files have been declspec'd, but others haven't; and headers for third-party libraries like libxml and libreadline mostly haven't. I'm starting to get the feeling that we're going to have to admit defeat and not try to use --disable-auto-import on cygwin builds. That platform is evidently not capable of supporting it. We seem to be pretty nearly there on getting the MSVC and Mingw builds to reliably complain about missing PGDLLIMPORTs, so maybe it's good enough if those builds do it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-16 12:57:46 -0500, Tom Lane wrote: Marco Atzeri marco.atz...@gmail.com writes: On 16/02/2014 15:43, Andres Freund wrote: Could either of you try whether compiling with the attached hack fixes anything on cygwin? on cygwin32 bit it works, but it stops later on --- sl -lcrypto -lz -lreadline -lcrypt -o psql.exe tab-complete.o:tab-complete.c:(.text+0xa98): undefined reference to `rl_line_buffer' on cygwin 64bit, that I was not testing before, something is strange -- -lintl -lssl -lcrypto -lcrypt -lldap -lwsock32 -lws2_32 -o postgres postmaster/postmaster.o:postmaster.c:(.rdata$.refptr.environ[.refptr.environ]+0x0): undefined reference to `environ' That's in this case because it's our own extern, that itself would probably be fixable, but: So what we currently know is that on cygwin, some of the core system include files have been declspec'd, but others haven't; and headers for third-party libraries like libxml and libreadline mostly haven't. it's not going to work for the external libraries. I'm starting to get the feeling that we're going to have to admit defeat and not try to use --disable-auto-import on cygwin builds. That platform is evidently not capable of supporting it. Agreed. It's probably doable if somebody actually using cygwin themselves would invest a day or two and work on upstreaming the changes, but it looks painful to do indirectly. We seem to be pretty nearly there on getting the MSVC and Mingw builds to reliably complain about missing PGDLLIMPORTs, so maybe it's good enough if those builds do it. Is there anything missing on that end? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-16 12:57:46 -0500, Tom Lane wrote: I'm starting to get the feeling that we're going to have to admit defeat and not try to use --disable-auto-import on cygwin builds. That platform is evidently not capable of supporting it. Agreed. It's probably doable if somebody actually using cygwin themselves would invest a day or two and work on upstreaming the changes, but it looks painful to do indirectly. Yeah, and I doubt that anybody in the Cygwin project is going to see the point of maintaining such patches anyway. As Marco said, their idea is to provide a Unix-ish platform as best they can, not adopt all the worst features of Windows. I kinda wonder why they have declspec's on the getopt variables at all. We seem to be pretty nearly there on getting the MSVC and Mingw builds to reliably complain about missing PGDLLIMPORTs, so maybe it's good enough if those builds do it. Is there anything missing on that end? I think it's all fixed, but I want to wait for another buildfarm cycle just to be sure all the relevant critters are failing where we expect them to and not somewhere else (like pltcl). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-16 13:25:58 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-16 12:57:46 -0500, Tom Lane wrote: I'm starting to get the feeling that we're going to have to admit defeat and not try to use --disable-auto-import on cygwin builds. That platform is evidently not capable of supporting it. Agreed. It's probably doable if somebody actually using cygwin themselves would invest a day or two and work on upstreaming the changes, but it looks painful to do indirectly. Yeah, and I doubt that anybody in the Cygwin project is going to see the point of maintaining such patches anyway. As Marco said, their idea is to provide a Unix-ish platform as best they can, not adopt all the worst features of Windows. I kinda wonder why they have declspec's on the getopt variables at all. When searching for others having issues, I discovered several other projects also disabling auto-import, so it's not just us. And, afaics, it's mostly upstream bugs, e.g. libxml's header just misses an extern in one of the cygwin specific defines. I don't think we've actually found bugs in cygwin yet. We'd overridden the provided declspec for getopt et al, that's why it failed, and we're failing to link to several of the libraries we're using. I think the only reason the latter isn't breaking our neck is that they are indirectly included and the auto-import code is saving our neck. That said, even if the upstream projects were fixed, we probably wouldn't want to enable this unconditionally for a fair while. And I am surely not volunteering to do the necessary work, and I somehow think neither are you... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/15 11:42), Tom Lane wrote: Hiroshi Inoue in...@tpf.co.jp writes: (2014/02/15 2:32), Tom Lane wrote: (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Due to *initializer element is not constant* error which also can be see on my old machine. Hmm, isn't that one of the symptoms of inadequacies in --enable-auto-import? Wonder if the disable change fixed it. The error is a compilation error not a linkage one. gcc on narwhal or my old machine is too old unfortunately. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-12 14:11:10 -0500, Tom Lane wrote: Marco Atzeri marco.atz...@gmail.com writes: On 12/02/2014 19:19, Andres Freund wrote: On 2014-02-12 19:13:07 +0100, Marco Atzeri wrote: About PGDLLIMPORT , my build log is full of warning: ‘optarg’ redeclared without dllimport attribute: previous dllimport ignored That should be fixed then. I guess cygwin's getopt.h declares it that way? from /usr/include/getopt.h extern char __declspec(dllimport) *optarg; /* argument associated with option */ Hm. All of our files that use getopt also do extern char *optarg; extern intoptind, opterr; #ifdef HAVE_INT_OPTRESET extern intoptreset;/* might not be declared by system headers */ #endif At least zic.c doesn't... So, now that brolga is revived, this unsurprisingly causes breakage there after the --disable-auto-export change. ISTM the easiest way to deal with this is to just adorn all those with a PGDLLIMPORT? I don't immediately see how the HAVE_INT_OPTRESET stuff could be reused in any interesting way as it's really independent of optind/optarg whether it exists? It strikes me that we should just move all that to src/include/getopt_long.h and include that from the places that currently include getopt.h directly. It's pretty pointless to have all those externs, HAVE_GETOPT_H, HAVE_INT_OPTRESET in every file. It'd be prettier if it were named getopt.h, but well... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 14:11:10 -0500, Tom Lane wrote: Marco Atzeri marco.atz...@gmail.com writes: from /usr/include/getopt.h extern char __declspec(dllimport) *optarg; /* argument associated with option */ Hm. All of our files that use getopt also do extern char *optarg; So, now that brolga is revived, this unsurprisingly causes breakage there after the --disable-auto-export change. ISTM the easiest way to deal with this is to just adorn all those with a PGDLLIMPORT? Uh, no, because that's backwards: we'd need the __declspec(dllimport) in the backend code. The best thing probably is not to have the duplicate declarations on platforms that don't need 'em. Unfortunately, I seem to recall that the current coding was arrived at to forestall link problems on weird platforms that *had* these symbols declared and yet we needed externs anyway. We might have to do something as ugly as #ifndef CYGWIN. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-15 10:16:50 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 14:11:10 -0500, Tom Lane wrote: Marco Atzeri marco.atz...@gmail.com writes: from /usr/include/getopt.h extern char __declspec(dllimport) *optarg; /* argument associated with option */ Hm. All of our files that use getopt also do extern char *optarg; So, now that brolga is revived, this unsurprisingly causes breakage there after the --disable-auto-export change. ISTM the easiest way to deal with this is to just adorn all those with a PGDLLIMPORT? Uh, no, because that's backwards: we'd need the __declspec(dllimport) in the backend code. Yuck, it's even uglier than that. On normal windows it's not actually the platform's getopt() that ends up getting really used, but the one from port/... # mingw has adopted a GNU-centric interpretation of optind/optreset, # so always use our version on Windows. if test $PORTNAME = win32; then AC_LIBOBJ(getopt) AC_LIBOBJ(getopt_long) fi The best thing probably is not to have the duplicate declarations on platforms that don't need 'em. Unfortunately, I seem to recall that the current coding was arrived at to forestall link problems on weird platforms that *had* these symbols declared and yet we needed externs anyway. We might have to do something as ugly as #ifndef CYGWIN. Hm, according to a quick blame, they are there unconditionally since at least 2000 (c.f. a70e74b06 moving them around). So it very well might be that that reasoning isn't current anymore. One ugly thing to do is to fall back to the port implementation of getopt on cygwin as well... That'd still have the warning parade tho. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-15 10:16:50 -0500, Tom Lane wrote: The best thing probably is not to have the duplicate declarations on platforms that don't need 'em. Unfortunately, I seem to recall that the current coding was arrived at to forestall link problems on weird platforms that *had* these symbols declared and yet we needed externs anyway. We might have to do something as ugly as #ifndef CYGWIN. Hm, according to a quick blame, they are there unconditionally since at least 2000 (c.f. a70e74b06 moving them around). So it very well might be that that reasoning isn't current anymore. I don't have time right now to research it (have to go shovel snow), but I think that at least some of the issue was that we needed the externs when we force use of our src/port implementation. One ugly thing to do is to fall back to the port implementation of getopt on cygwin as well... That'd still have the warning parade tho. Yeah, that doesn't sound terribly satisfactory. Another idea would be to wrap the externs in #ifndef HAVE_GETOPT_H. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-15 10:59:17 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-15 10:16:50 -0500, Tom Lane wrote: The best thing probably is not to have the duplicate declarations on platforms that don't need 'em. Unfortunately, I seem to recall that the current coding was arrived at to forestall link problems on weird platforms that *had* these symbols declared and yet we needed externs anyway. We might have to do something as ugly as #ifndef CYGWIN. Hm, according to a quick blame, they are there unconditionally since at least 2000 (c.f. a70e74b06 moving them around). So it very well might be that that reasoning isn't current anymore. I don't have time right now to research it (have to go shovel snow), but I think that at least some of the issue was that we needed the externs when we force use of our src/port implementation. I think that'd be solvable easy enough if we'd just always included pg's getopt_long.h (or a new getopt.h) which properly deals with defining them when included. That'd centralize all the magic and it'd overall get rid of a ton of ifdefs and externs. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-15 10:59:17 -0500, Tom Lane wrote: I don't have time right now to research it (have to go shovel snow), but I think that at least some of the issue was that we needed the externs when we force use of our src/port implementation. I think that'd be solvable easy enough if we'd just always included pg's getopt_long.h (or a new getopt.h) which properly deals with defining them when included. That'd centralize all the magic and it'd overall get rid of a ton of ifdefs and externs. Yeah, there are enough copies of that stuff that centralizing them sounds like a great idea. Call it pg_getopt.h, perhaps? AFAICS, getopt_long.h just defines them unconditionally, which is as wrong as everyplace else. The reasonable alternatives seem to be (1) invent pg_getopt.h, which would #include getopt.h if available and then provide properly-ifdef'd externs for optarg and friends; getopt_long.h would #include pg_getopt.h. (2) Just do the above in getopt_long.h, and include it in the places that currently have externs for optarg and friends. Option (2) seems a bit odd in files that aren't actually using getopt_long(), but it avoids making a new header file. Preferences? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-15 12:16:58 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-15 10:59:17 -0500, Tom Lane wrote: I don't have time right now to research it (have to go shovel snow), but I think that at least some of the issue was that we needed the externs when we force use of our src/port implementation. I think that'd be solvable easy enough if we'd just always included pg's getopt_long.h (or a new getopt.h) which properly deals with defining them when included. That'd centralize all the magic and it'd overall get rid of a ton of ifdefs and externs. Yeah, there are enough copies of that stuff that centralizing them sounds like a great idea. Call it pg_getopt.h, perhaps? I'm just working on it. pg_getopt.h was exactly what I came up with. (1) invent pg_getopt.h, which would #include getopt.h if available and then provide properly-ifdef'd externs for optarg and friends; getopt_long.h would #include pg_getopt.h. That's what I've done. I'll post in a minute, just wanted to give a headsup so we don't duplicate work. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-15 18:21:56 +0100, Andres Freund wrote: On 2014-02-15 12:16:58 -0500, Tom Lane wrote: Yeah, there are enough copies of that stuff that centralizing them sounds like a great idea. Call it pg_getopt.h, perhaps? Patch attached. I am not sure whether HAVE_GETOPT is the best condition to use, since it's set by configure by a link based check, same goes for HAVE_INT_OPTERR. The other choices would be relying on HAVE_GETOPT_H or a new AC_CHECK_DECL. Thanks. I'll look this over and try to get it committed before brolga's next scheduled run. At least this'll ensure we only have one place to tweak if more tweaking is needed. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: Patch attached. Committed with some cosmetic adjustments --- thanks! I am not sure whether HAVE_GETOPT is the best condition to use, since it's set by configure by a link based check, same goes for HAVE_INT_OPTERR. The other choices would be relying on HAVE_GETOPT_H or a new AC_CHECK_DECL. I'm pretty sure HAVE_GETOPT is *not* what we want to use for the variable declarations. I've tried HAVE_GETOPT_H for the moment, but that may need more tweaking depending on what the buildfarm has to say. I haven't touched entab.c because it's not linking with pgport, so there seems little use in changing it. Agreed. We don't have very high standards for its portability anyway. I've also removed some #ifndef WIN32's that didn't seem to make much sense. They might've been needed at one time, but we'll see. (I also took out a few includes of unistd.h that were clearly now redundant, though I didn't try to be thorough about it.) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-15 14:35:02 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: Patch attached. Committed with some cosmetic adjustments --- thanks! I am not sure whether HAVE_GETOPT is the best condition to use, since it's set by configure by a link based check, same goes for HAVE_INT_OPTERR. The other choices would be relying on HAVE_GETOPT_H or a new AC_CHECK_DECL. I'm pretty sure HAVE_GETOPT is *not* what we want to use for the variable declarations. I've tried HAVE_GETOPT_H for the moment, but that may need more tweaking depending on what the buildfarm has to say. I only used it because it was what previously protected the getopt() declaration in port.h... But that should probably have been depending on !HAVE_GETOPT_H in the first place. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 12/02/2014 17:39, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? Hm, I don't see a big advantage in detecting the errors as It won't hugely increase the buildfarm coverage. But --auto-import seems to require marking the .text sections as writable, avoiding that seems to be a good idea if we don't need it anymore. Given the rather small number of Windows machines in the buildfarm, I'd really like them all to be on the same page about what's valid and what isn't. Having to wait ~24 hours to find out if they're all happy with something isn't my idea of a good time. In any case, as long as we have discrepancies between them, I'm not going to be entirely convinced that any of them are telling the full truth. I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. regards, tom lane disabling is not working on cygwin gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard zic.o ialloc.o scheck.o localtime.o -L../../src/port -L../../src/common -Wl,--allow-multiple-definition -Wl,--disable-auto-import -L/usr/local/lib -Wl,--as-needed -lpgcommon -lpgport -lintl -lssl -lcrypto -lz -lreadline -lcrypt -o zic.exe zic.o:zic.c:(.text.startup+0xc9): undefined reference to `optarg' zic.o:zic.c:(.text.startup+0x10d): undefined reference to `optarg' /usr/lib/gcc/i686-pc-cygwin/4.8.2/../../../../i686-pc-cygwin/bin/ld: zic.o: bad reloc address 0x10d in section `.text.startup' /usr/lib/gcc/i686-pc-cygwin/4.8.2/../../../../i686-pc-cygwin/bin/ld: final link failed: Invalid operation collect2: error: ld returned 1 exit status just removing -Wl,--disable-auto-import and everything works $ gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard zic.o ialloc.o scheck.o localtime.o -L../../src/port -L../../src/common -Wl,--allow-multiple-definition -L/usr/local/lib -Wl,--as-needed -lpgcommon -lpgport -lintl -lssl -lcrypto -lz -lreadline -lcrypt -o zic.exe Regards Marco -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-15 21:49:43 +0100, Marco Atzeri wrote: On 12/02/2014 17:39, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? Hm, I don't see a big advantage in detecting the errors as It won't hugely increase the buildfarm coverage. But --auto-import seems to require marking the .text sections as writable, avoiding that seems to be a good idea if we don't need it anymore. Given the rather small number of Windows machines in the buildfarm, I'd really like them all to be on the same page about what's valid and what isn't. Having to wait ~24 hours to find out if they're all happy with something isn't my idea of a good time. In any case, as long as we have discrepancies between them, I'm not going to be entirely convinced that any of them are telling the full truth. I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. regards, tom lane disabling is not working on cygwin gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard zic.o ialloc.o scheck.o localtime.o -L../../src/port -L../../src/common -Wl,--allow-multiple-definition -Wl,--disable-auto-import -L/usr/local/lib -Wl,--as-needed -lpgcommon -lpgport -lintl -lssl -lcrypto -lz -lreadline -lcrypt -o zic.exe zic.o:zic.c:(.text.startup+0xc9): undefined reference to `optarg' zic.o:zic.c:(.text.startup+0x10d): undefined reference to `optarg' /usr/lib/gcc/i686-pc-cygwin/4.8.2/../../../../i686-pc-cygwin/bin/ld: zic.o: bad reloc address 0x10d in section `.text.startup' /usr/lib/gcc/i686-pc-cygwin/4.8.2/../../../../i686-pc-cygwin/bin/ld: final link failed: Invalid operation collect2: error: ld returned 1 exit status just removing -Wl,--disable-auto-import and everything works $ gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard zic.o ialloc.o scheck.o localtime.o -L../../src/port -L../../src/common -Wl,--allow-multiple-definition -L/usr/local/lib -Wl,--as-needed -lpgcommon -lpgport -lintl -lssl -lcrypto -lz -lreadline -lcrypt -o zic.exe Please pull and retry, that already might fix it. The reason it's probably failing is the warnings about declspec you reported earlier. See 60ff2fdd9970ba29f5267317a5e7354d2658c1e5 Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 15/02/2014 21:54, Andres Freund wrote: Please pull and retry, that already might fix it. The reason it's probably failing is the warnings about declspec you reported earlier. See 60ff2fdd9970ba29f5267317a5e7354d2658c1e5 Greetings, Andres Freund that is fine. there is a different one, later on [cut] ../../src/timezone/localtime.o ../../src/timezone/strftime.o ../../src/timezone/pgtz.o ../../src/port/libpgport_srv.a ../../src/common/libpgcommon_srv.a -lintl -lssl -lcrypto -lcrypt -lldap -o postgres libpq/auth.o:auth.c:(.text+0x1940): undefined reference to `in6addr_any' libpq/auth.o:auth.c:(.text+0x1954): undefined reference to `in6addr_any' libpq/auth.o:auth.c:(.text+0x196d): undefined reference to `in6addr_any' libpq/auth.o:auth.c:(.text+0x1979): undefined reference to `in6addr_any' /usr/lib/gcc/i686-pc-cygwin/4.8.2/../../../../i686-pc-cygwin/bin/ld: libpq/auth.o: bad reloc address 0xce4 in section `.rdata' collect2: error: ld returned 1 exit status Makefile:66: recipe for target 'postgres' failed make[2]: *** [postgres] Error 1 make[2]: Leaving directory '/pub/devel/postgresql/git/postgresql_build/src/backend' Regards Marco -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-15 22:11:37 +0100, Marco Atzeri wrote: On 15/02/2014 21:54, Andres Freund wrote: Please pull and retry, that already might fix it. The reason it's probably failing is the warnings about declspec you reported earlier. See 60ff2fdd9970ba29f5267317a5e7354d2658c1e5 Greetings, Andres Freund that is fine. there is a different one, later on [cut] ../../src/timezone/localtime.o ../../src/timezone/strftime.o ../../src/timezone/pgtz.o ../../src/port/libpgport_srv.a ../../src/common/libpgcommon_srv.a -lintl -lssl -lcrypto -lcrypt -lldap -o postgres libpq/auth.o:auth.c:(.text+0x1940): undefined reference to `in6addr_any' Could you try additionally linking with -lwsock32? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-15 22:11:37 +0100, Marco Atzeri wrote: ../../src/timezone/localtime.o ../../src/timezone/strftime.o ../../src/timezone/pgtz.o ../../src/port/libpgport_srv.a ../../src/common/libpgcommon_srv.a -lintl -lssl -lcrypto -lcrypt -lldap -o postgres libpq/auth.o:auth.c:(.text+0x1940): undefined reference to `in6addr_any' Could you try additionally linking with -lwsock32? The interesting question here is why it used to work. There is no extern for in6addr_any in our code, so there must have been a declaration of that constant in some system header. Which one, and what linkage is it defining, and where was the linkage getting resolved before? I notice that brolga is showing both this failure and some libxml issues. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-15 17:26:30 -0500, Tom Lane wrote:
Andres Freund and...@2ndquadrant.com writes:
On 2014-02-15 22:11:37 +0100, Marco Atzeri wrote:
../../src/timezone/localtime.o ../../src/timezone/strftime.o
../../src/timezone/pgtz.o ../../src/port/libpgport_srv.a
../../src/common/libpgcommon_srv.a -lintl -lssl -lcrypto -lcrypt -lldap -o
postgres
libpq/auth.o:auth.c:(.text+0x1940): undefined reference to `in6addr_any'
Could you try additionally linking with -lwsock32?
The interesting question here is why it used to work. There is no
extern for in6addr_any in our code, so there must have been a
declaration of that constant in some system header. Which one, and
what linkage is it defining, and where was the linkage getting
resolved before?
mingwcompat.c has the following ugly as heck tidbit:
#ifndef WIN32_ONLY_COMPILER
/*
* MingW defines an extern to this struct, but the actual struct isn't present
* in any library. It's trivial enough that we can safely define it
* ourselves.
*/
const struct in6_addr in6addr_any = {{{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0}}};
I think when that was added the problem might just have been
misanalyzed, but due to the auto import magic this probably wasn't
noticed...
Greetings,
Andres Freund
--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Marco Atzeri marco.atz...@gmail.com writes: 32 $ grep -rH in6addr_any * cygwin/in6.h:extern const struct in6_addr in6addr_any; cygwin/version.h: in6addr_any, in6addr_loopback. So how come there's a declspec on the getopt.h variables, but not this one? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes:
On 2014-02-15 17:26:30 -0500, Tom Lane wrote:
The interesting question here is why it used to work. There is no
extern for in6addr_any in our code, so there must have been a
declaration of that constant in some system header. Which one, and
what linkage is it defining, and where was the linkage getting
resolved before?
mingwcompat.c has the following ugly as heck tidbit:
#ifndef WIN32_ONLY_COMPILER
/*
* MingW defines an extern to this struct, but the actual struct isn't present
* in any library. It's trivial enough that we can safely define it
* ourselves.
*/
const struct in6_addr in6addr_any = {{{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0}}};
Yeah, I noticed that. AFAICS, mingwcompat.c isn't built in Cygwin builds,
so we'd probably need to put the constant someplace else entirely if we
end up defining it ourselves.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 15/02/2014 23:37, Andres Freund wrote:
On 2014-02-15 17:26:30 -0500, Tom Lane wrote:
Andres Freund and...@2ndquadrant.com writes:
On 2014-02-15 22:11:37 +0100, Marco Atzeri wrote:
../../src/timezone/localtime.o ../../src/timezone/strftime.o
../../src/timezone/pgtz.o ../../src/port/libpgport_srv.a
../../src/common/libpgcommon_srv.a -lintl -lssl -lcrypto -lcrypt -lldap -o
postgres
libpq/auth.o:auth.c:(.text+0x1940): undefined reference to `in6addr_any'
Could you try additionally linking with -lwsock32?
The interesting question here is why it used to work. There is no
extern for in6addr_any in our code, so there must have been a
declaration of that constant in some system header. Which one, and
what linkage is it defining, and where was the linkage getting
resolved before?
mingwcompat.c has the following ugly as heck tidbit:
#ifndef WIN32_ONLY_COMPILER
/*
* MingW defines an extern to this struct, but the actual struct isn't present
* in any library. It's trivial enough that we can safely define it
* ourselves.
*/
const struct in6_addr in6addr_any = {{{0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0}}};
I think when that was added the problem might just have been
misanalyzed, but due to the auto import magic this probably wasn't
noticed...
Greetings,
Andres Freund
on cygwin header in /usr/include
32 $ grep -rH in6addr_any *
cygwin/in6.h:extern const struct in6_addr in6addr_any;
cygwin/version.h: in6addr_any, in6addr_loopback.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-15 17:48:00 -0500, Tom Lane wrote: Marco Atzeri marco.atz...@gmail.com writes: 32 $ grep -rH in6addr_any * cygwin/in6.h:extern const struct in6_addr in6addr_any; cygwin/version.h: in6addr_any, in6addr_loopback. So how come there's a declspec on the getopt.h variables, but not this one? Well, those are real constants, so they probably are fully contained in the static link library, no need to dynamically resolve them. Note that the frontend libpq indeed links to wsock32, just as Mkvcbuild.pm does for both frontend and backend... It might be that ipv6 never worked for mingw and cygwin, and in6addr_any just resolved to some magic thunk --auto-import created... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: It might be that ipv6 never worked for mingw and cygwin, and in6addr_any just resolved to some magic thunk --auto-import created... Yeah, it would not be surprising if this exercise is exposing bugs we didn't even know we had. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-15 18:26:46 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: It might be that ipv6 never worked for mingw and cygwin, and in6addr_any just resolved to some magic thunk --auto-import created... Yeah, it would not be surprising if this exercise is exposing bugs we didn't even know we had. Agreed, but I think we should start fixing the missing PGDLLIMPORTs soon, at least the ones needed in the backbranches. AFAICT there's live bugs with postgres_fdw there. And possibly it might need more than one full cycle to get right... Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/15 2:32), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: One thing I'm wondering about is that plperl is linking perlxx.lib not libperlxx.a. I made a patch following plpython and it also works here. Is it worth trying? I hadn't noticed that part of plpython's Makefile before. Man, that's an ugly technique :-(. Still, there's little about this platform that isn't ugly. Let's try it and see what happens. And what happens is this: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhaldt=2014-02-14%2017%3A00%3A02 namely, it gets through plperl now and then chokes with the same symptoms on pltcl. So I guess we need the same hack in pltcl. The fun never stops ... Pltcl still fails. tclxx.dll lives in bin directory not in lib directory. The attached patch would fix the problem. regards, Hiroshi Inoue diff --git a/src/pl/tcl/Makefile b/src/pl/tcl/Makefile index e0fb13e..2ab2a27 100644 --- a/src/pl/tcl/Makefile +++ b/src/pl/tcl/Makefile @@ -53,7 +53,7 @@ PSQLDIR = $(bindir) ifeq ($(PORTNAME), win32) tclwithver = $(subst -l,,$(filter -l%, $(TCL_LIB_SPEC))) -TCLDLL = $(subst -L,,$(filter -L%, $(TCL_LIB_SPEC)))/$(tclwithver).dll +TCLDLL = $(dir $(TCLSH))/$(tclwithver).dll OBJS += lib$(tclwithver).a -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/12 8:30), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. I cleaned this up a bit (the if-nesting in Makefile.shlib was making my head hurt, not to mention that it left a bunch of dead code) and committed it. Hm ... according to buildfarm member narwhal, this doesn't work so well for plperl: gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -g -Wno-comment -shared -o plperl.dll plperl.o SPI.o Util.o -L../../../src/port -L../../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -LC:/Perl/lib/CORE -lperl58 -L../../../src/backend -lpostgres -lpgcommon -lpgport -lintl -lxslt -lxml2 -lssleay32 -leay32 -lz -lm -lws2_32 -lshfolder -Wl,--export-all-symbols -Wl,--out-implib=libplperl.a Cannot export .idata$4: symbol not found Cannot export .idata$5: symbol not found Cannot export .idata$6: symbol not found Cannot export .text: symbol not found Cannot export perl58_NULL_THUNK_DATA: symbol not found Creating library file: libplperl.a collect2: ld returned 1 exit status make[3]: *** [plperl.dll] Error 1 Not very clear what's going on there; could this be a problem in narwhal's admittedly-ancient toolchain? The error doesn't occur here (un)fortunately. One thing I'm wondering about is that plperl is linking perlxx.lib not libperlxx.a. I made a patch following plpython and it also works here. Is it worth trying? regards, Hiroshi Inoue diff --git a/src/pl/plperl/GNUmakefile b/src/pl/plperl/GNUmakefile index e0e31ec..065c5d3 100644 --- a/src/pl/plperl/GNUmakefile +++ b/src/pl/plperl/GNUmakefile @@ -16,6 +16,8 @@ endif ifeq ($(shared_libperl),yes) ifeq ($(PORTNAME), win32) +generate_perl_implib = yes + override CPPFLAGS += -DPLPERL_HAVE_UID_GID # Perl on win32 contains /* within comment all over the header file, # so disable this warning. @@ -36,7 +38,21 @@ DATA = plperl.control plperl--1.0.sql plperl--unpackaged--1.0.sql \ PERLCHUNKS = plc_perlboot.pl plc_trusted.pl +# Perl on win32 ships with import libraries only for Microsoft Visual C++, +# which may not be handled with mingw gcc. Therefore we choose to build a +# new import library to link with. +ifeq ($(generate_perl_implib), yes) + perlwithver = $(subst -l,,$(filter -l%, $(perl_embed_ldflags))) + OBJS += lib$(perlwithver).a +lib$(perlwithver).a: $(perlwithver).def + dlltool --dllname $(perlwithver).dll --def $(perlwithver).def --output-lib lib$(perlwithver).a +$(perlwithver).def: + pexports $(dir $(subst ',,$(PERL)))$(perlwithver).dll $(perlwithver).def + +SHLIB_LINK = +else SHLIB_LINK = $(perl_embed_ldflags) +endif REGRESS_OPTS = --dbname=$(PL_TESTDB) --load-extension=plperl --load-extension=plperlu REGRESS = plperl plperl_lc plperl_trigger plperl_shared plperl_elog plperl_util plperl_init plperlu plperl_array @@ -105,6 +121,9 @@ submake: clean distclean maintainer-clean: clean-lib rm -f SPI.c Util.c $(OBJS) perlchunks.h plperl_opmask.h rm -rf $(pg_regress_clean_files) +ifeq (yes, $(generate_perl_implib)) + rm -rf $(perlwithver).def +endif else # can't build -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Hiroshi Inoue in...@tpf.co.jp writes: (2014/02/12 8:30), Tom Lane wrote: Not very clear what's going on there; could this be a problem in narwhal's admittedly-ancient toolchain? The error doesn't occur here (un)fortunately. One thing I'm wondering about is that plperl is linking perlxx.lib not libperlxx.a. I made a patch following plpython and it also works here. Is it worth trying? I hadn't noticed that part of plpython's Makefile before. Man, that's an ugly technique :-(. Still, there's little about this platform that isn't ugly. Let's try it and see what happens. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
I wrote: Hiroshi Inoue in...@tpf.co.jp writes: One thing I'm wondering about is that plperl is linking perlxx.lib not libperlxx.a. I made a patch following plpython and it also works here. Is it worth trying? I hadn't noticed that part of plpython's Makefile before. Man, that's an ugly technique :-(. Still, there's little about this platform that isn't ugly. Let's try it and see what happens. And what happens is this: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhaldt=2014-02-14%2017%3A00%3A02 namely, it gets through plperl now and then chokes with the same symptoms on pltcl. So I guess we need the same hack in pltcl. The fun never stops ... (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/15 2:32), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: One thing I'm wondering about is that plperl is linking perlxx.lib not libperlxx.a. I made a patch following plpython and it also works here. Is it worth trying? I hadn't noticed that part of plpython's Makefile before. Man, that's an ugly technique :-(. Still, there's little about this platform that isn't ugly. Let's try it and see what happens. And what happens is this: http://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhaldt=2014-02-14%2017%3A00%3A02 namely, it gets through plperl now and then chokes with the same symptoms on pltcl. So I guess we need the same hack in pltcl. The fun never stops ... There seems some risk to link msvc import library. Linking perlxx.lib or tcl.lib worls here but linking pythonxx.lib doesn't work even in newer machine. (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Due to *initializer element is not constant* error which also can be see on my old machine. http://www.postgresql.org/message-id/CA+OCxozr=0wkQDF7kfd2n-bJQOwdSUs0Myohg29pA_U5=2p...@mail.gmail.com regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Hiroshi Inoue in...@tpf.co.jp writes: (2014/02/15 2:32), Tom Lane wrote: (BTW, narwhal is evidently not trying to build plpython. I wonder why not?) Due to *initializer element is not constant* error which also can be see on my old machine. Hmm, isn't that one of the symptoms of inadequacies in --enable-auto-import? Wonder if the disable change fixed it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/13 9:51), Hiroshi Inoue wrote: (2014/02/12 12:28), Inoue, Hiroshi wrote: (2014/02/12 8:30), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. I cleaned this up a bit (the if-nesting in Makefile.shlib was making my head hurt, not to mention that it left a bunch of dead code) and committed it. Hm ... according to buildfarm member narwhal, this doesn't work so well for plperl: gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -g -Wno-comment -shared -o plperl.dll plperl.o SPI.o Util.o -L../../../src/port -L../../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -LC:/Perl/lib/CORE -lperl58 -L../../../src/backend -lpostgres -lpgcommon -lpgport -lintl -lxslt -lxml2 -lssleay32 -leay32 -lz -lm -lws2_32 -lshfolder -Wl,--export-all-symbols -Wl,--out-implib=libplperl.a Cannot export .idata$4: symbol not found Cannot export .idata$5: symbol not found Cannot export .idata$6: symbol not found Cannot export .text: symbol not found Cannot export perl58_NULL_THUNK_DATA: symbol not found Creating library file: libplperl.a collect2: ld returned 1 exit status make[3]: *** [plperl.dll] Error 1 Oops I forgot to inclule plperl, tcl or python, sorry. I would retry build operations with them. Unfortunately it would take pretty long time because build operations are pretty (or veeery in an old machine) slow. Not very clear what's going on there; could this be a problem in narwhal's admittedly-ancient toolchain? As for build, plperl and pltcl are OK on both Windows7+gcc4.6.1 machine and Windows Vista+gcc3.4.5 machine. plpython is OK on gcc4.6.1 machine but causes a *initializer element is not constant* error on gcc3.4.5 machine. I've not run regression test yet. Rebuild with --disable-auto-import causes errors in contrib on both machines. Errors occur in pg_buffercache, pg_stat_statements, postgres_fdw and test_shm_mq. --enable-auto-import cures all of them. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Hiroshi Inoue in...@tpf.co.jp writes: Rebuild with --disable-auto-import causes errors in contrib on both machines. Errors occur in pg_buffercache, pg_stat_statements, postgres_fdw and test_shm_mq. Yeah, that's the idea: we want to get the same failures as on MSVC. I'm going to put in PGDLLIMPORT macros to fix these cases, but I'm waiting for verification that Cygwin also sees the problem before making it go away. brolga unfortunately seems to have been AWOL for the past day and a half. Andrew, could you give it a kick? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Craig Ringer cr...@2ndquadrant.com writes: Great, that's what I was hoping to see - proper errors where we've omitted things, not silent miscompilation. And, just to make sure that ain't nobody happy ... brolga, our one operational Cygwin animal, is still building without complaints. We really need to understand what is going on here and why some toolchains don't seem to feel that lack of PGDLLIMPORT is a problem. Nosing around, I happened to notice this in src/template/cygwin: # --enable-auto-import gets rid of a diagnostics linker message LDFLAGS=-Wl,--allow-multiple-definition -Wl,--enable-auto-import Anybody know *exactly* what --enable-auto-import does? The name is, um, suggestive. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-12 10:58:20 -0500, Tom Lane wrote: Craig Ringer cr...@2ndquadrant.com writes: Great, that's what I was hoping to see - proper errors where we've omitted things, not silent miscompilation. And, just to make sure that ain't nobody happy ... brolga, our one operational Cygwin animal, is still building without complaints. Hm, isn't that pretty much expected? Cygwin's infrastructure tries to be unixoid, so it's not surprising that the toolchain doesn't require such strange things? Otherwise even larger amounts of unix software wouldn't run, right? I am pretty sure halfway recent versions of mingw can be made to behave similarly, but I don't really see the advantage in doing so, to the contrary even. # --enable-auto-import gets rid of a diagnostics linker message LDFLAGS=-Wl,--allow-multiple-definition -Wl,--enable-auto-import Anybody know *exactly* what --enable-auto-import does? The name is, um, suggestive. My ld(1)'s manpage has three screen's worth of details... Most of it seems to be on http://www.freebsd.org/cgi/man.cgi?query=ldsektion=1 It's essentially elf like shared library linking in pe-coff through dirty tricks. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 10:58:20 -0500, Tom Lane wrote: Anybody know *exactly* what --enable-auto-import does? The name is, um, suggestive. My ld(1)'s manpage has three screen's worth of details... Most of it seems to be on http://www.freebsd.org/cgi/man.cgi?query=ldsektion=1 It's essentially elf like shared library linking in pe-coff through dirty tricks. Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 10:58:20 -0500, Tom Lane wrote: Anybody know *exactly* what --enable-auto-import does? The name is, um, suggestive. My ld(1)'s manpage has three screen's worth of details... Most of it seems to be on http://www.freebsd.org/cgi/man.cgi?query=ldsektion=1 It's essentially elf like shared library linking in pe-coff through dirty tricks. Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? Hm, I don't see a big advantage in detecting the errors as It won't hugely increase the buildfarm coverage. But --auto-import seems to require marking the .text sections as writable, avoiding that seems to be a good idea if we don't need it anymore. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? Hm, I don't see a big advantage in detecting the errors as It won't hugely increase the buildfarm coverage. But --auto-import seems to require marking the .text sections as writable, avoiding that seems to be a good idea if we don't need it anymore. Given the rather small number of Windows machines in the buildfarm, I'd really like them all to be on the same page about what's valid and what isn't. Having to wait ~24 hours to find out if they're all happy with something isn't my idea of a good time. In any case, as long as we have discrepancies between them, I'm not going to be entirely convinced that any of them are telling the full truth. I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-12 11:39:43 -0500, Tom Lane wrote: I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. The manpage also has a --disable-auto-import, but doesn't document wheter enabling or disabling is the default... So maybe try to be explicit? Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On Wed, Feb 12, 2014 at 11:39:43AM -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? Hm, I don't see a big advantage in detecting the errors as It won't hugely increase the buildfarm coverage. But --auto-import seems to require marking the .text sections as writable, avoiding that seems to be a good idea if we don't need it anymore. Given the rather small number of Windows machines in the buildfarm, I'd really like them all to be on the same page about what's valid and what isn't. Having to wait ~24 hours to find out if they're all happy with something isn't my idea of a good time. In any case, as long as we have discrepancies between them, I'm not going to be entirely convinced that any of them are telling the full truth. I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. Can document the issue in case it comes up again, please? -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com + Everyone has their own god. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-12 11:50:41 -0500, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:39:43 -0500, Tom Lane wrote: I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. The manpage also has a --disable-auto-import, but doesn't document wheter enabling or disabling is the default... So maybe try to be explicit? Yeah, I was just wondering about that myself. Presumably --enable is not the default, or it would not have gotten added. However, I notice that it was added a long time ago (2004) ... is it possible the default changed since then? Some release notes I just found seem to suggest it is http://sourceware.org/binutils/docs-2.17/ld/WIN32.html : This feature is enabled with the `--enable-auto-import' command-line option, although it is enabled by default on cygwin/mingw. The `--enable-auto-import' option itself now serves mainly to suppress any warnings that are ordinarily emitted when linked objects trigger the feature's use. 2.17 is from 2007, so it's been enabled by default at least since then. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: Some release notes I just found seem to suggest it is http://sourceware.org/binutils/docs-2.17/ld/WIN32.html : This feature is enabled with the `--enable-auto-import' command-line option, although it is enabled by default on cygwin/mingw. Curious ... narwhal is mingw but evidently doesn't have that switch turned on. OTOH we also know it's an old version. I'm inclined to change template/win32 also. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:39:43 -0500, Tom Lane wrote: I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. The manpage also has a --disable-auto-import, but doesn't document wheter enabling or disabling is the default... So maybe try to be explicit? Yeah, I was just wondering about that myself. Presumably --enable is not the default, or it would not have gotten added. However, I notice that it was added a long time ago (2004) ... is it possible the default changed since then? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 12/02/2014 17:26, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 10:58:20 -0500, Tom Lane wrote: Anybody know *exactly* what --enable-auto-import does? The name is, um, suggestive. My ld(1)'s manpage has three screen's worth of details... Most of it seems to be on http://www.freebsd.org/cgi/man.cgi?query=ldsektion=1 It's essentially elf like shared library linking in pe-coff through dirty tricks. Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? regards, tom lane If I am not wrong --enable-auto-import is already the default on cygwin build chain ( binutils = 2.19.51 ) so it should make no difference on latest cygwin. Not sure for you 1.7.7 build enviroment. About PGDLLIMPORT , my build log is full of warning: ‘optarg’ redeclared without dllimport attribute: previous dllimport ignored I suspect that removing will also make no difference. Marco PS: we aim unix-like builds not windows one -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-12 19:13:07 +0100, Marco Atzeri wrote: On 12/02/2014 17:26, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? If I am not wrong --enable-auto-import is already the default on cygwin build chain ( binutils = 2.19.51 ) so it should make no difference on latest cygwin. Not sure for you 1.7.7 build enviroment. We're *disabling* not *enabling* it. About PGDLLIMPORT , my build log is full of warning: ‘optarg’ redeclared without dllimport attribute: previous dllimport ignored That should be fixed then. I guess cygwin's getopt.h declares it that way? I suspect that removing will also make no difference. The committed patch explicitly disables the functionality. PS: we aim unix-like builds not windows one Well, there are a significant number of caveats around the auto import functionality, so there seems little benefit in using it if all the declspec's have to be there anyway. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 12/02/2014 19:19, Andres Freund wrote: On 2014-02-12 19:13:07 +0100, Marco Atzeri wrote: On 12/02/2014 17:26, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? If I am not wrong --enable-auto-import is already the default on cygwin build chain ( binutils = 2.19.51 ) so it should make no difference on latest cygwin. Not sure for you 1.7.7 build enviroment. We're *disabling* not *enabling* it. remove is not disable if enable is already the default inside binutils and gcc. Or I am missing something ? About PGDLLIMPORT , my build log is full of warning: ‘optarg’ redeclared without dllimport attribute: previous dllimport ignored That should be fixed then. I guess cygwin's getopt.h declares it that way? from /usr/include/getopt.h #ifndef __INSIDE_CYGWIN__ extern int __declspec(dllimport) opterr;/* if error message should be printed */ extern int __declspec(dllimport) optind;/* index into parent argv vector */ extern int __declspec(dllimport) optopt;/* character checked for validity */ extern int __declspec(dllimport) optreset; /* reset getopt */ extern char __declspec(dllimport) *optarg; /* argument associated with option */ #endif I suspect that removing will also make no difference. The committed patch explicitly disables the functionality. PS: we aim unix-like builds not windows one Well, there are a significant number of caveats around the auto import functionality, so there seems little benefit in using it if all the declspec's have to be there anyway. I think that I am not currently using anymore the declspec in the build. But I could be wrong, as the the postgresql build way is the most complicated between all the ones I am dealing with. Greetings, Andres Freund Cheers Marco -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Marco Atzeri marco.atz...@gmail.com writes: On 12/02/2014 19:19, Andres Freund wrote: On 2014-02-12 19:13:07 +0100, Marco Atzeri wrote: About PGDLLIMPORT , my build log is full of warning: âoptargâ redeclared without dllimport attribute: previous dllimport ignored That should be fixed then. I guess cygwin's getopt.h declares it that way? from /usr/include/getopt.h extern char __declspec(dllimport) *optarg; /* argument associated with option */ Hm. All of our files that use getopt also do extern char *optarg; extern intoptind, opterr; #ifdef HAVE_INT_OPTRESET extern intoptreset;/* might not be declared by system headers */ #endif regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2/11/14, 7:04 PM, Craig Ringer wrote: I don't see any use for that with plperl, but it might be a valid thing to be doing for (e.g.) hstore.dll. Though you can't really link to it from another module anyway, you have to go through the fmgr to get access to its symbols at rutime, so we can probably just skip generation of import libraries for contribs and PLs. There are cases where one module needs symbols from another directly. Would that be affected by this? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On February 12, 2014 10:23:21 PM CET, Peter Eisentraut pete...@gmx.net wrote: On 2/11/14, 7:04 PM, Craig Ringer wrote: I don't see any use for that with plperl, but it might be a valid thing to be doing for (e.g.) hstore.dll. Though you can't really link to it from another module anyway, you have to go through the fmgr to get access to its symbols at rutime, so we can probably just skip generation of import libraries for contribs and PLs. There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither from the POV of loading several .so's, nor from a symbol visibility. Afaics we'd need a working definition of PGDLLIMPORT which inverts the declspecs. I think Tom just removed the remnants of that. Andres -- Please excuse brevity and formatting - I am writing this on my mobile phone. Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Andres Freund and...@2ndquadrant.com writes: On February 12, 2014 10:23:21 PM CET, Peter Eisentraut pete...@gmx.net wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither from the POV of loading several .so's, nor from a symbol visibility. Afaics we'd need a working definition of PGDLLIMPORT which inverts the declspecs. I think Tom just removed the remnants of that. No, I've not touched the PGDLLIMPORT macros. I was hoping to, but it looks like we're not getting there :-( regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2/12/14, 4:30 PM, Andres Freund wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither from the POV of loading several .so's, nor from a symbol visibility. Afaics we'd need a working definition of PGDLLIMPORT which inverts the declspecs. I think Tom just removed the remnants of that. It works reasonably well on other platforms. Of course, we can barely build extension modules on Windows, so maybe this is a bit much to ask. But as long as we're dealing only with functions, not variables, it should work without any dllimport dances, right? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Peter Eisentraut pete...@gmx.net writes: There are cases where one module needs symbols from another directly. Would that be affected by this? It works reasonably well on other platforms. Of course, we can barely build extension modules on Windows, so maybe this is a bit much to ask. But as long as we're dealing only with functions, not variables, it should work without any dllimport dances, right? AFAIK we've not changed anything that would affect that. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On February 12, 2014 10:35:06 PM CET, Tom Lane t...@sss.pgh.pa.us wrote: Andres Freund and...@2ndquadrant.com writes: On February 12, 2014 10:23:21 PM CET, Peter Eisentraut pete...@gmx.net wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither from the POV of loading several .so's, nor from a symbol visibility. Afaics we'd need a working definition of PGDLLIMPORT which inverts the declspecs. I think Tom just removed the remnants of that. No, I've not touched the PGDLLIMPORT macros. I was hoping to, but it looks like we're not getting there :-( Right, that was just the test patch... Then the macros we're using in fmgr.h for the magic macros (even if not strictly needed) should work for Peter's case. Andres -- Please excuse brevity and formatting - I am writing this on my mobile phone. Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 02/12/2014 02:31 PM, Inoue, Hiroshi wrote: Maybe this is one of the few use cases of dlltool. Because python doesn't ship with its MINGW import library, the Makefile uses dlltool to generate an import library from the python DLL. Wow. Has anyone been in touch with the Python package maintainers to see if they can fix that and bundle the .lib ? -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 02/13/2014 12:39 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On 2014-02-12 11:26:56 -0500, Tom Lane wrote: Hm. So if we're giving up on the idea of ever getting rid of PGDLLIMPORT, we ought to actually remove that, so that the Cygwin build works more like the other Windows builds? Hm, I don't see a big advantage in detecting the errors as It won't hugely increase the buildfarm coverage. But --auto-import seems to require marking the .text sections as writable, avoiding that seems to be a good idea if we don't need it anymore. Given the rather small number of Windows machines in the buildfarm, I'd really like them all to be on the same page about what's valid and what isn't. Having to wait ~24 hours to find out if they're all happy with something isn't my idea of a good time. In any case, as long as we have discrepancies between them, I'm not going to be entirely convinced that any of them are telling the full truth. I'm going to go remove that switch and see if brolga starts failing. If it does, I'll be satisfied that we understand the issues here. I wouldn't assume that _anything_ Cygwin does makes sense, or is consistent with anything else. Its attempts to produce UNIX-like behaviour on Windows cause some truly bizarre behaviour at times. It is not totally unfair to compare the level of weirdness of Cygwin to that of WINE, though they operate in completely different ways. I wouldn't suggest making any conclusions about the _other_ platforms based on Cygwin. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 02/13/2014 05:23 AM, Peter Eisentraut wrote: On 2/11/14, 7:04 PM, Craig Ringer wrote: I don't see any use for that with plperl, but it might be a valid thing to be doing for (e.g.) hstore.dll. Though you can't really link to it from another module anyway, you have to go through the fmgr to get access to its symbols at rutime, so we can probably just skip generation of import libraries for contribs and PLs. There are cases where one module needs symbols from another directly. Would that be affected by this? Yes, in that you cannot link directly to another DLL on Windows (without hoop jumping and lots of pain), you link to the import library. So if we don't generate an import library then (eg) MyExtension cannot link to hstore.dll . It can still look up function exports via the fmgr. As concluded upthread, it's easier to just generate import libraries for everything since we need it for the client library, so nothing's going to change anyway. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Craig Ringer cr...@2ndquadrant.com writes: On 02/12/2014 02:31 PM, Inoue, Hiroshi wrote: Maybe this is one of the few use cases of dlltool. Because python doesn't ship with its MINGW import library, the Makefile uses dlltool to generate an import library from the python DLL. Wow. Has anyone been in touch with the Python package maintainers to see if they can fix that and bundle the .lib ? Indeed somebody should pester them about that, but it's not clear how much it'd help us even if they fixed it tomorrow. We're going to have to be able to build with existing Python distributions for a long time to come. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 02/13/2014 05:35 AM, Tom Lane wrote: Andres Freund and...@2ndquadrant.com writes: On February 12, 2014 10:23:21 PM CET, Peter Eisentraut pete...@gmx.net wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither from the POV of loading several .so's, nor from a symbol visibility. Afaics we'd need a working definition of PGDLLIMPORT which inverts the declspecs. I think Tom just removed the remnants of that. No, I've not touched the PGDLLIMPORT macros. I was hoping to, but it looks like we're not getting there :-( In theory we could now remove the __declspec(dllexport) case for BUILDING_DLL, as it should now be redundant with the fixed .DEF generator. Should. Unfortunately it looks like there's not going to be any getting around having something that can turn into __declspec(dllimport) in the headers for compiling things that link to postgres.exe, though. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 02/13/2014 05:35 AM, Peter Eisentraut wrote: On 2/12/14, 4:30 PM, Andres Freund wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither from the POV of loading several .so's, nor from a symbol visibility. Afaics we'd need a working definition of PGDLLIMPORT which inverts the declspecs. I think Tom just removed the remnants of that. It works reasonably well on other platforms. Of course, we can barely build extension modules on Windows, so maybe this is a bit much to ask. But as long as we're dealing only with functions, not variables, it should work without any dllimport dances, right? Don't think so. If you don't have __declspec(dllexport) or a .DEF file marking something as exported, it's not part of the DLL interface at all, it's like you compiled it with gcc using -fvisibility=hidden and didn't give the symbol __attribute__((visibility (default)) . If you _do_ have the symbol exported from the DLL, using __declspec(dllimport) or a generated .DEF file that exposes all externs, you can link to the symbol. However, from the reading I've done recently, I'm pretty sure that if you fail to declare __declspec(dllimport) on the importing side, you actually land up statically linking to a thunk function that in turn calls the real function in the DLL. So it works, but at a performance cost. So you should do the dance. Sorry. It gets worse, too. Say you want hstore to export a couple of symbols. Those symbols must be __declspec(dllexport) while everything else in headers must be __declspec(dllimport). This means you can't just use PGDLLIMPORT. You must define a HSTOREDLLIMPORT that's __declspec(dllexport) when compiling hstore and otherwise __declspec(dllimport). Then set a preprocessor macro like -DCOMPILING_HSTORE to trigger it. Even though we're generating .DEF files, I don't think we can avoid that, because we at minimum need to have the hstore exported symbols *not* annotated __declspec(dllimport) when compiling hstore, but have them so annotated when importing the header into anything else. (Come to think of it, how're we dealing with this in libpq? I wonder if postgres is linking to libpq using thunk functions?) -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
Craig Ringer cr...@2ndquadrant.com writes: However, from the reading I've done recently, I'm pretty sure that if you fail to declare __declspec(dllimport) on the importing side, you actually land up statically linking to a thunk function that in turn calls the real function in the DLL. So it works, but at a performance cost. Meh. I'm not really willing to go through the pushups that would be needed to deal with that, even if it can be shown that the performance cost is noticeable. We've already kluged our source code for Windows far beyond what anybody would tolerate for any other platform, and I find that to be quite enough bending over for Redmond. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 2014-02-13 07:58:09 +0800, Craig Ringer wrote: On 02/13/2014 05:35 AM, Peter Eisentraut wrote: On 2/12/14, 4:30 PM, Andres Freund wrote: There are cases where one module needs symbols from another directly. Would that be affected by this? I don't think we have real infrastructure for that yet. Neither from the POV of loading several .so's, nor from a symbol visibility. Afaics we'd need a working definition of PGDLLIMPORT which inverts the declspecs. I think Tom just removed the remnants of that. It works reasonably well on other platforms. Of course, we can barely build extension modules on Windows, so maybe this is a bit much to ask. But as long as we're dealing only with functions, not variables, it should work without any dllimport dances, right? Don't think so. If you don't have __declspec(dllexport) or a .DEF file marking something as exported, it's not part of the DLL interface at all, it's like you compiled it with gcc using -fvisibility=hidden and didn't give the symbol __attribute__((visibility (default)) . If you _do_ have the symbol exported from the DLL, using __declspec(dllimport) or a generated .DEF file that exposes all externs, you can link to the symbol. However, from the reading I've done recently, I'm pretty sure that if you fail to declare __declspec(dllimport) on the importing side, you actually land up statically linking to a thunk function that in turn calls the real function in the DLL. So it works, but at a performance cost. So you should do the dance. Sorry. I don't think the thunk function will have such a high overhead in this day and age. And it's what we essentially already do for all functions called *from* extensions, no? It gets worse, too. Say you want hstore to export a couple of symbols. Those symbols must be __declspec(dllexport) while everything else in headers must be __declspec(dllimport). This means you can't just use PGDLLIMPORT. You must define a HSTOREDLLIMPORT that's __declspec(dllexport) when compiling hstore and otherwise __declspec(dllimport). Then set a preprocessor macro like -DCOMPILING_HSTORE to trigger it. We actually have a a macro that should do that, namely PGDLLEXPORT. I am not sure though, why it's not dependent on dependent on BUILDING_DLL (which is absolutely horribly misnamed, being essentially inverted). Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
On 02/13/2014 08:26 AM, Andres Freund wrote: It gets worse, too. Say you want hstore to export a couple of symbols. Those symbols must be __declspec(dllexport) while everything else in headers must be __declspec(dllimport). This means you can't just use PGDLLIMPORT. You must define a HSTOREDLLIMPORT that's __declspec(dllexport) when compiling hstore and otherwise __declspec(dllimport). Then set a preprocessor macro like -DCOMPILING_HSTORE to trigger it. We actually have a a macro that should do that, namely PGDLLEXPORT. I am not sure though, why it's not dependent on dependent on BUILDING_DLL (which is absolutely horribly misnamed, being essentially inverted). Yes, it does that *for building postgres*. What I'm saying is that you need another one for hstore if you wish to be able to export symbols from hstore and import them into other libs. You can't use PGDLLEXPORT because BUILDING_DLL will be unset, so it'll expand to __declspec(dllimport) while compiling hstore. Which is wrong if you want to be exporting symbols; even if you use a .DEF file, it must expand blank, and if you don't use a .DEF it must expand to __declspec(dllexport) ... but ONLY for those symbols exported by hstore its self. So each lib that wants to export symbols needs its own equivalent of PGDLLIMPORT, and a flag set just when compiling that lib. This is the normal way it's done on Windows builds, not stuff I'm looking into and saying how _I_ think we should do it. AFAIK this is just how it's done on Windows, horrible as that is. I'll see if I can find a couple of relevant links. BTW, BUILDING_DLL is so-named because the macro definition will be taken from examples that talk about compiling a DLL, that's all. -- Craig Ringer http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] narwhal and PGDLLIMPORT
(2014/02/12 12:28), Inoue, Hiroshi wrote: (2014/02/12 8:30), Tom Lane wrote: I wrote: Hiroshi Inoue in...@tpf.co.jp writes: I tried MINGW port with the attached change and successfully built src and contrib and all pararell regression tests were OK. I cleaned this up a bit (the if-nesting in Makefile.shlib was making my head hurt, not to mention that it left a bunch of dead code) and committed it. Hm ... according to buildfarm member narwhal, this doesn't work so well for plperl: gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -g -Wno-comment -shared -o plperl.dll plperl.o SPI.o Util.o -L../../../src/port -L../../../src/common -Wl,--allow-multiple-definition -L/mingw/lib -Wl,--as-needed -LC:/Perl/lib/CORE -lperl58 -L../../../src/backend -lpostgres -lpgcommon -lpgport -lintl -lxslt -lxml2 -lssleay32 -leay32 -lz -lm -lws2_32 -lshfolder -Wl,--export-all-symbols -Wl,--out-implib=libplperl.a Cannot export .idata$4: symbol not found Cannot export .idata$5: symbol not found Cannot export .idata$6: symbol not found Cannot export .text: symbol not found Cannot export perl58_NULL_THUNK_DATA: symbol not found Creating library file: libplperl.a collect2: ld returned 1 exit status make[3]: *** [plperl.dll] Error 1 Oops I forgot to inclule plperl, tcl or python, sorry. I would retry build operations with them. Unfortunately it would take pretty long time because build operations are pretty (or veeery in an old machine) slow. Not very clear what's going on there; could this be a problem in narwhal's admittedly-ancient toolchain? As for build, plperl and pltcl are OK on both Windows7+gcc4.6.1 machine and Windows Vista+gcc3.4.5 machine. plpython is OK on gcc4.6.1 machine but causes a *initializer element is not constant* error on gcc3.4.5 machine. I've not run regression test yet. regards, Hiroshi Inoue -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
