Re: pl/perl extension fails on Windows

[Noah Misch]

On Sun, Jan 21, 2018 at 11:34:07AM +0100, Christian Ullrich wrote:
> * Christian Ullrich wrote:
> >* Noah Misch wrote:
> >>On Wed, Nov 29, 2017 at 11:45:35PM -0500, Tom Lane wrote:
> >>>Oh, OK.  In that case, we need to get some representatives of these
> >>>more modern builds into the buildfarm while we're at it.
> >>
> >>Yep.  Among machines already in the buildfarm, the one running member
> >>woodlouse is the best candidate for this.  Its owner could install
> >>http://strawberryperl.com/download/5.26.1.1/strawberry-perl-5.26.1.1-32bit.msi
> >>and setup another animal on the same machine that builds 32-bit and enables
> >>Perl.  Christian, are you interested in doing this?
> >
> >Ready to go, waiting for animal assignment. For now, I can confirm that it 
> >works, that is, the buildfarm --test run is successful.
> 
> Up and running now, name is whelk, first report on REL9_6_STABLE.
> 
> Sorry it took me another ten days to complete the configuration.

This is great.  Thanks.

Buildfarm metadata reports whelk using "Microsoft Visual C++ 2010", but the
run logs show it's using MSVC 2013, like woodlouse does.  Would you update
that buildfarm metadata (with update_personality.pl)?

Re: pl/perl extension fails on Windows

[Christian Ullrich]

* Christian Ullrich wrote:


* Noah Misch wrote:


On Wed, Nov 29, 2017 at 11:45:35PM -0500, Tom Lane wrote:

Oh, OK.  In that case, we need to get some representatives of these
more modern builds into the buildfarm while we're at it.


Yep.  Among machines already in the buildfarm, the one running member
woodlouse is the best candidate for this.  Its owner could install
http://strawberryperl.com/download/5.26.1.1/strawberry-perl-5.26.1.1-32bit.msi
and setup another animal on the same machine that builds 32-bit and enables
Perl.  Christian, are you interested in doing this?


Ready to go, waiting for animal assignment. For now, I can confirm that it 
works, that is, the buildfarm --test run is successful.


Up and running now, name is whelk, first report on REL9_6_STABLE.

Sorry it took me another ten days to complete the configuration.

--
Christian

Re: pl/perl extension fails on Windows

[Andrew Dunstan]

On 01/11/2018 12:08 AM, Noah Misch wrote:
> On Sun, Dec 10, 2017 at 11:46:08AM -0800, Noah Misch wrote:
>> On Sun, Dec 10, 2017 at 12:36:13PM +, Christian Ullrich wrote:
>>> * Noah Misch wrote:
 On Wed, Nov 29, 2017 at 11:45:35PM -0500, Tom Lane wrote:
> Oh, OK.  In that case, we need to get some representatives of these
> more modern builds into the buildfarm while we're at it.
 Yep.  Among machines already in the buildfarm, the one running member
 woodlouse is the best candidate for this.  Its owner could install
 http://strawberryperl.com/download/5.26.1.1/strawberry-perl-5.26.1.1-32bit.msi
 and setup another animal on the same machine that builds 32-bit and enables
 Perl.  Christian, are you interested in doing this?
>>> Ready to go, waiting for animal assignment. For now, I can confirm that it 
>>> works, that is, the buildfarm --test run is successful.
>> Thanks!
> Did the animal assignment come through?  I don't see such an animal reporting.


Looks like it's still in the queue. Will approve now.

cheers

andrew

-- 
Andrew Dunstanhttps://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

RE: pl/perl extension fails on Windows

[Christian Ullrich]

* From: Noah Misch [mailto:n...@leadboat.com]

> > > Ready to go, waiting for animal assignment. For now, I can
> confirm that it works, that is, the buildfarm --test run is
> successful.

> Did the animal assignment come through?  I don't see such an animal
> reporting.

No, not yet. Sorry, I lost track of it, or I would have mentioned it again 
earlier.

-- 
Christian

Re: pl/perl extension fails on Windows

[Noah Misch]

On Sun, Dec 10, 2017 at 11:46:08AM -0800, Noah Misch wrote:
> On Sun, Dec 10, 2017 at 12:36:13PM +, Christian Ullrich wrote:
> > * Noah Misch wrote:
> > > On Wed, Nov 29, 2017 at 11:45:35PM -0500, Tom Lane wrote:
> > >> Oh, OK.  In that case, we need to get some representatives of these
> > >> more modern builds into the buildfarm while we're at it.
> > > 
> > > Yep.  Among machines already in the buildfarm, the one running member
> > > woodlouse is the best candidate for this.  Its owner could install
> > > http://strawberryperl.com/download/5.26.1.1/strawberry-perl-5.26.1.1-32bit.msi
> > > and setup another animal on the same machine that builds 32-bit and 
> > > enables
> > > Perl.  Christian, are you interested in doing this?
> > 
> > Ready to go, waiting for animal assignment. For now, I can confirm that it 
> > works, that is, the buildfarm --test run is successful.
> 
> Thanks!

Did the animal assignment come through?  I don't see such an animal reporting.

Re: pl/perl extension fails on Windows

[Noah Misch]

On Sun, Dec 10, 2017 at 12:36:13PM +, Christian Ullrich wrote:
> * Noah Misch wrote:
> > On Wed, Nov 29, 2017 at 11:45:35PM -0500, Tom Lane wrote:
> >> Oh, OK.  In that case, we need to get some representatives of these
> >> more modern builds into the buildfarm while we're at it.
> > 
> > Yep.  Among machines already in the buildfarm, the one running member
> > woodlouse is the best candidate for this.  Its owner could install
> > http://strawberryperl.com/download/5.26.1.1/strawberry-perl-5.26.1.1-32bit.msi
> > and setup another animal on the same machine that builds 32-bit and enables
> > Perl.  Christian, are you interested in doing this?
> 
> Ready to go, waiting for animal assignment. For now, I can confirm that it 
> works, that is, the buildfarm --test run is successful.

Thanks!

> Although I have to admit, I fail to see the need for Windows x86 builds, too. 
> Who in their right mind would want them today?

I can't see installing a 32-bit Windows postmaster in 2018.  The 32-bit libpq
might be useful.  Download statistics for the binary distributions would be
informative.  On the other hand, removing 32-bit Windows support would
eliminate three veteran buildfarm animals, and maintenance was cheap in the
few years before this thread.  I don't think today is the day to remove
support, but it's coming one of these years.

Re: pl/perl extension fails on Windows

[Noah Misch]

On Wed, Nov 29, 2017 at 11:45:35PM -0500, Tom Lane wrote:
> Noah Misch  writes:
> > On Wed, Nov 29, 2017 at 11:34:56PM -0500, Tom Lane wrote:
> >> I don't really have an opinion about the relative merits of these changes,
> >> but why do anything?  The existing solution has the buildfarm happy, and
> >> we've not heard any field complaints that I saw.  I'm not sure we should
> >> spend more time on supporting obsolete toolchain combinations that aren't
> >> represented in the buildfarm.
> 
> > It's the other way around.  The buildfarm's 32-bit MSVC animals each use an
> > obsolete Perl.  PostgreSQL is incompatible with today's 32-bit ActivePerl 
> > and
> > Strawberry Perl.
> 
> Oh, OK.  In that case, we need to get some representatives of these
> more modern builds into the buildfarm while we're at it.

Yep.  Among machines already in the buildfarm, the one running member
woodlouse is the best candidate for this.  Its owner could install
http://strawberryperl.com/download/5.26.1.1/strawberry-perl-5.26.1.1-32bit.msi
and setup another animal on the same machine that builds 32-bit and enables
Perl.  Christian, are you interested in doing this?

Re: pl/perl extension fails on Windows

[Noah Misch]

On Wed, Nov 29, 2017 at 08:14:41PM -0800, Noah Misch wrote:
> 1. If $Config{gccversion} is nonempty, add _USE_32BIT_TIME_T.  This will do
>the wrong thing if MinGW changes its default to match modern MSVC.  It will
>do the wrong thing for a Perl built with "gcc -D__MINGW_USE_VC2005_COMPAT".
> 
> 2. When configuring the build, determine whether to add _USE_32BIT_TIME_T by
>running a test program built with and without that symbol.  Perhaps have
>the test program store and retrieve a PL_modglobal value.  (PL_modglobal
>maps to a PerlInterpreter field that follows the fields sensitive to
>_USE_32BIT_TIME_T, and perlapi documents it since 5.8.0 or earlier.)  This
>is more principled than (1), but it will be more code and may have weird
>interactions with rare Perl build options.
> 
> I am inclined toward (2) if it takes no more than roughly a hundred lines of
> code, else (1).  Opinions?  I regret investing in 32-bit Windows.  If there's
> any OS where a 32-bit PostgreSQL server makes sense today, it's not Windows.

Here's an implementation of (2).  This is more intricate than I hoped.  One
could argue for (1), but I estimate (2) wins by a nose.  I successfully tested
http://strawberryperl.com/download/5.14.4.1/strawberry-perl-5.14.4.1-32bit.msi
(Perl 5.14.4; MinGW-built; must have -D_USE_32BIT_TIME_T) and
http://get.enterprisedb.com/languagepacks/edb-languagepack-10-3-windows.exe
(Perl 5.24.0; MSVC-built; must not have -D_USE_32BIT_TIME_T).  I also tried
http://strawberryperl.com/download/5.8.9/strawberry-perl-5.8.9.5.msi, which
experienced a StackHash_0a9e crash during PERL_SYS_INIT3(), with or without
-D_USE_32BIT_TIME_T.  I expect that breakage is orthogonal.  I didn't have
ready access to obsolete MSVC-built Perl, so it will be interesting to see how
the buildfarm likes this.
MSVC: Test whether 32-bit Perl needs -D_USE_32BIT_TIME_T.

Commits 5a5c2feca3fd858e70ea348822595547e6fa6c15 and
b5178c5d08ca59e30f9d9428fa6fdb2741794e65 introduced support for modern
MSVC-built, 32-bit Perl, but they broke use of MinGW-built, 32-bit Perl
distributions like Strawberry Perl and modern ActivePerl.  Perl has no
robust means to report whether it expects a -D_USE_32BIT_TIME_T ABI, so
test this.  Back-patch to 9.3 (all supported versions).

The chief alternative was a heuristic of adding -D_USE_32BIT_TIME_T when
$Config{gccversion} is nonempty.  That banks on every gcc-built Perl
using the same ABI.  gcc could change its default ABI the way MSVC once
did, and one could build Perl with gcc and the non-default ABI.

The GNU make build system could benefit from a similar test, without
which it does not support MSVC-built Perl.  For now, just add a comment.
Most users taking the special step of building Perl with MSVC probably
build PostgreSQL with MSVC.

Discussion: https://postgr.es/m/20171130041441.ga3161...@rfd.leadboat.com


diff --git a/config/perl.m4 b/config/perl.m4
index 76b1a92..caefb07 100644
--- a/config/perl.m4
+++ b/config/perl.m4
@@ -48,19 +48,23 @@ AC_DEFUN([PGAC_CHECK_PERL_CONFIGS],
 
 # PGAC_CHECK_PERL_EMBED_CCFLAGS
 # -
-# We selectively extract stuff from $Config{ccflags}.  We don't really need
-# anything except -D switches, and other sorts of compiler switches can
-# actively break things if Perl was compiled with a different compiler.
-# Moreover, although Perl likes to put stuff like -D_LARGEFILE_SOURCE and
-# -D_FILE_OFFSET_BITS=64 here, it would be fatal to try to compile PL/Perl
-# to a different libc ABI than core Postgres uses.  The available information
-# says that all the symbols that affect Perl's own ABI begin with letters,
-# so it should be sufficient to adopt -D switches for symbols not beginning
-# with underscore.  An exception is that we need to let through
-# -D_USE_32BIT_TIME_T if it's present.  (We probably could restrict that to
-# only get through on Windows, but for the moment we let it through always.)
-# For debugging purposes, let's have the configure output report the raw
-# ccflags value as well as the set of flags we chose to adopt.
+# We selectively extract stuff from $Config{ccflags}.  For debugging purposes,
+# let's have the configure output report the raw ccflags value as well as the
+# set of flags we chose to adopt.  We don't really need anything except -D
+# switches, and other sorts of compiler switches can actively break things if
+# Perl was compiled with a different compiler.  Moreover, although Perl likes
+# to put stuff like -D_LARGEFILE_SOURCE and -D_FILE_OFFSET_BITS=64 here, it
+# would be fatal to try to compile PL/Perl to a different libc ABI than core
+# Postgres uses.  The available information says that most symbols that affect
+# Perl's own ABI begin with letters, so it's almost sufficient to adopt -D
+# switches for symbols not beginning with underscore.  Some exceptions are the
+# Windows-specific -D_USE_32BIT_TIME_T and -D__MINGW_USE_VC2005_COMPAT; see
+# Mkvcbuild.pm for details.  We absorb 

Re: pl/perl extension fails on Windows

[Michael Paquier]

On Thu, Nov 30, 2017 at 1:34 PM, Tom Lane  wrote:
> Noah Misch  writes:
>> On Thu, Aug 17, 2017 at 12:15:58PM -0400, Tom Lane wrote:
>>> ... it's now looking to me like we should do the above with X = 5.13.4.
>>> That won't be a perfect solution, but it's about the best we can
>>> readily do.  Realistically, nobody out in the wider world is likely
>>> to care about building current PG releases against such old Perl
>>> versions on Windows; if we satisfy our older buildfarm critters,
>>> it's enough for me.
>
>> MinGW default behavior matches "cl -D_USE_32BIT_TIME_T", and MSVC >= 2005
>> default behavior matches "gcc -D__MINGW_USE_VC2005_COMPAT"[1].  MinGW-built
>> Perl[2] does not mention _USE_32BIT_TIME_T in $Config{ccflags}, so we
>> typically must add _USE_32BIT_TIME_T when using MSVC to build 32-bit against
>> MinGW-built Perl.  I'm considering two ways to achieve this:
>
> I don't really have an opinion about the relative merits of these changes,
> but why do anything?  The existing solution has the buildfarm happy, and
> we've not heard any field complaints that I saw.  I'm not sure we should
> spend more time on supporting obsolete toolchain combinations that aren't
> represented in the buildfarm.

I agree with this position. If people are looking for getting better
coverage about weird component combinations, I'd like to think that
they should provide an animal so as support is live and not
investigated afterwards. Remember for example the recent thread about
overlayfs 
(https://www.postgresql.org/message-id/20171107135454.lbelbbvfgadlj...@home.ouaza.com).
On top of that this thread deals with rather old components with 32b
stuff on Windows..
-- 
Michael

Re: pl/perl extension fails on Windows

[Noah Misch]

On Wed, Nov 29, 2017 at 11:34:56PM -0500, Tom Lane wrote:
> Noah Misch  writes:
> > On Thu, Aug 17, 2017 at 12:15:58PM -0400, Tom Lane wrote:
> >> ... it's now looking to me like we should do the above with X = 5.13.4.
> >> That won't be a perfect solution, but it's about the best we can
> >> readily do.  Realistically, nobody out in the wider world is likely
> >> to care about building current PG releases against such old Perl
> >> versions on Windows; if we satisfy our older buildfarm critters,
> >> it's enough for me.
> 
> > MinGW default behavior matches "cl -D_USE_32BIT_TIME_T", and MSVC >= 2005
> > default behavior matches "gcc -D__MINGW_USE_VC2005_COMPAT"[1].  MinGW-built
> > Perl[2] does not mention _USE_32BIT_TIME_T in $Config{ccflags}, so we
> > typically must add _USE_32BIT_TIME_T when using MSVC to build 32-bit against
> > MinGW-built Perl.  I'm considering two ways to achieve this:
> 
> I don't really have an opinion about the relative merits of these changes,
> but why do anything?  The existing solution has the buildfarm happy, and
> we've not heard any field complaints that I saw.  I'm not sure we should
> spend more time on supporting obsolete toolchain combinations that aren't
> represented in the buildfarm.

It's the other way around.  The buildfarm's 32-bit MSVC animals each use an
obsolete Perl.  PostgreSQL is incompatible with today's 32-bit ActivePerl and
Strawberry Perl.