Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andres Freund]

On 2017-10-10 18:10:15 -0400, Tom Lane wrote:
> Andres Freund  writes:
> > Here's a fix. Not quite sure whether we really need the
> > HAVE_DECL_STRNLEN test, added it for symmetry.
> 
> LGTM.

Thanks for checking.


> I think the DECL test is a good idea; the system definition
> might be a macro or otherwise weird, in which case we'd cause problems
> if we write an extern definition anyway.

I was thinking about protecting it with HAVE_STRNLEN rather than not
protecting it at all. But this works, so whatever ;)

Greetings,

Andres Freund


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Tom Lane]

Andres Freund  writes:
> Here's a fix. Not quite sure whether we really need the
> HAVE_DECL_STRNLEN test, added it for symmetry.

LGTM.  I think the DECL test is a good idea; the system definition
might be a macro or otherwise weird, in which case we'd cause problems
if we write an extern definition anyway.

regards, tom lane


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andres Freund]

On 2017-10-10 17:34:17 -0400, Andrew Dunstan wrote:
> This test is governed by the test at line 946 of configure.in. You need
> to move it somewhere else.

Yea, sorry for the noise :(.

Greetings,

Andres Freund


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andres Freund]

On 2017-10-10 13:53:15 -0700, Andres Freund wrote:
> On 2017-10-10 16:51:39 -0400, Tom Lane wrote:
> > Andres Freund  writes:
> > > As far as I can tell it's still somehow using a configure from before
> > > the last commits:
> > 
> > No, it's pilot error.  The AC_CHECK_FUNCS call you added strnlen to
> > is only executed if
> > AS_IF([test "$enable_thread_safety" = yes -a "$PORTNAME" != "win32"],
> 
> Dear god, I'm daft. Dear god, m4 is a horrible lanuagage.

Here's a fix. Not quite sure whether we really need the
HAVE_DECL_STRNLEN test, added it for symmetry.

Afaict windows "always" had strnlen, so no need to meddle with the
windows build.

Will eat lunch and push, even if there's some futher adjustments, I'd
like to get Andrew's animals green again.

Greetings,

Andres Freund
>From fffd651e83ccbd6191a76be6ec7c6b1b27888fde Mon Sep 17 00:00:00 2001
From: Andres Freund 
Date: Tue, 10 Oct 2017 14:42:16 -0700
Subject: [PATCH] Rewrite strnlen replacement implementation from 8a241792f96.

The previous placement of the fallback implementation in libpgcommon
was problematic, because libpqport functions need strnlen
functionality.

Move replacement into libpgport. Provide strnlen() under its posix
name, instead of pg_strnlen(). Fix stupid configure bug, executing the
test only when compiled with threading support.

Author: Andres Freund
Discussion: https://postgr.es/m/e1e1gr2-0005fb...@gemulon.postgresql.org
---
 configure | 25 -
 configure.in  |  6 +++---
 src/backend/utils/mmgr/mcxt.c |  3 +--
 src/common/string.c   | 20 
 src/include/common/string.h   | 15 ---
 src/include/pg_config.h.in|  4 
 src/include/pg_config.h.win32 | 10 +++---
 src/include/port.h|  4 
 src/port/snprintf.c   |  4 +---
 src/port/strnlen.c| 33 +
 10 files changed, 77 insertions(+), 47 deletions(-)
 create mode 100644 src/port/strnlen.c

diff --git a/configure b/configure
index a1283c05005..b0582657bf4 100755
--- a/configure
+++ b/configure
@@ -8777,7 +8777,7 @@ fi
 
 
 
-for ac_func in strerror_r getpwuid_r gethostbyname_r strnlen
+for ac_func in strerror_r getpwuid_r gethostbyname_r
 do :
   as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
 ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
@@ -13161,6 +13161,16 @@ fi
 cat >>confdefs.h <<_ACEOF
 #define HAVE_DECL_STRLCPY $ac_have_decl
 _ACEOF
+ac_fn_c_check_decl "$LINENO" "strnlenfrak" "ac_cv_have_decl_strnlen" "$ac_includes_default"
+if test "x$ac_cv_have_decl_strnlen" = xyes; then :
+  ac_have_decl=1
+else
+  ac_have_decl=0
+fi
+
+cat >>confdefs.h <<_ACEOF
+#define HAVE_DECL_STRNLEN $ac_have_decl
+_ACEOF
 
 # This is probably only present on macOS, but may as well check always
 ac_fn_c_check_decl "$LINENO" "F_FULLFSYNC" "ac_cv_have_decl_F_FULLFSYNC" "#include 
@@ -13528,6 +13538,19 @@ esac
 
 fi
 
+ac_fn_c_check_func "$LINENO" "strnlenfrak" "ac_cv_func_strnlen"
+if test "x$ac_cv_func_strnlen" = xyes; then :
+  $as_echo "#define HAVE_STRNLEN 1" >>confdefs.h
+
+else
+  case " $LIBOBJS " in
+  *" strnlen.$ac_objext "* ) ;;
+  *) LIBOBJS="$LIBOBJS strnlen.$ac_objext"
+ ;;
+esac
+
+fi
+
 
 
 case $host_os in
diff --git a/configure.in b/configure.in
index e1381b4ead6..4548db0dd3c 100644
--- a/configure.in
+++ b/configure.in
@@ -961,7 +961,7 @@ LIBS="$LIBS $PTHREAD_LIBS"
 AC_CHECK_HEADER(pthread.h, [], [AC_MSG_ERROR([
 pthread.h not found;  use --disable-thread-safety to disable thread safety])])
 
-AC_CHECK_FUNCS([strerror_r getpwuid_r gethostbyname_r strnlen])
+AC_CHECK_FUNCS([strerror_r getpwuid_r gethostbyname_r])
 
 # Do test here with the proper thread flags
 PGAC_FUNC_STRERROR_R_INT
@@ -1422,7 +1422,7 @@ AC_CHECK_DECLS(posix_fadvise, [], [], [#include ])
 fi
 
 AC_CHECK_DECLS(fdatasync, [], [], [#include ])
-AC_CHECK_DECLS([strlcat, strlcpy])
+AC_CHECK_DECLS([strlcat, strlcpy, strnlen])
 # This is probably only present on macOS, but may as well check always
 AC_CHECK_DECLS(F_FULLFSYNC, [], [], [#include ])
 
@@ -1514,7 +1514,7 @@ else
   AC_CHECK_FUNCS([fpclass fp_class fp_class_d class], [break])
 fi
 
-AC_REPLACE_FUNCS([crypt fls getopt getrusage inet_aton mkdtemp random rint srandom strerror strlcat strlcpy])
+AC_REPLACE_FUNCS([crypt fls getopt getrusage inet_aton mkdtemp random rint srandom strerror strlcat strlcpy strnlen])
 
 case $host_os in
 
diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c
index 64e0408d5af..c5c311fad39 100644
--- a/src/backend/utils/mmgr/mcxt.c
+++ b/src/backend/utils/mmgr/mcxt.c
@@ -21,7 +21,6 @@
 
 #include "postgres.h"
 
-#include "common/string.h"
 #include "miscadmin.h"
 #include "utils/memdebug.h"
 #include "utils/memutils.h"
@@ -1089,7 +1088,7 @@ pnstrdup(const char *in, Size len)
 {
 	char	   *out;
 
-	len = pg_strnlen(in, len);
+	len = strnlen(in, len);
 
 	out = palloc(len + 1);
 	memcpy(out, in, 

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andrew Dunstan]

On 10/10/2017 03:48 PM, Andres Freund wrote:
> On 2017-10-09 23:33:36 -0400, Andrew Dunstan wrote:
>>
>> On 10/09/2017 07:15 PM, Andres Freund wrote:
>>> Hi Andrew,
>>>
>>> On 2017-10-09 22:22:04 +, Andres Freund wrote:
 Add pg_strnlen() a portable implementation of strlen.

 As the OS version is likely going to be more optimized, fall back to
 it if available, as detected by configure.
>>> I'm a bit confused, frogmouth
>>> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmouth=2017-10-09%2022%3A30%3A41
>>> shows that it compiled the new code, but the configure output doesn't
>>> show it ran through the new configure test.  Additionally, without the
>>> the config define, this should result in the replacement being
>>> used. Which doesn't seem to be the case either.
>>>
>>> Kinda sounds like this used some halfway outdated build or such?
>>>
>>
>>
>> frogmouth is using some code not yet released that makes the config
>> cache persistent. I just identified and fixed a stupid bug in the code
>> that obsoletes the cache, and I have removed frogmouth's cache file and
>> set it running again, so we'll see if that fixes things.
> As far as I can tell it's still somehow using a configure from before
> the last commits:
> https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=frogmouth=2017-10-10%2018%3A35%3A06=configure
>
> Note that it's not running the new test.
>
> (there's definitely fixes to be made to where strnlen's replacement is
> located, but regardless, this needs to be fixed too)
>



This test is governed by the test at line 946 of configure.in. You need
to move it somewhere else.

cheers

andrew

-- 
Andrew Dunstanhttps://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services




-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andres Freund]

On 2017-10-10 16:51:39 -0400, Tom Lane wrote:
> Andres Freund  writes:
> > As far as I can tell it's still somehow using a configure from before
> > the last commits:
> 
> No, it's pilot error.  The AC_CHECK_FUNCS call you added strnlen to
> is only executed if
> AS_IF([test "$enable_thread_safety" = yes -a "$PORTNAME" != "win32"],

Dear god, I'm daft. Dear god, m4 is a horrible lanuagage.

Thanks.


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Tom Lane]

Andres Freund  writes:
> As far as I can tell it's still somehow using a configure from before
> the last commits:

No, it's pilot error.  The AC_CHECK_FUNCS call you added strnlen to
is only executed if
AS_IF([test "$enable_thread_safety" = yes -a "$PORTNAME" != "win32"],

A better place would be configure.in:1403, probably.

regards, tom lane


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andres Freund]

On 2017-10-10 16:37:04 -0400, Tom Lane wrote:
> Andres Freund  writes:
> > (there's definitely fixes to be made to where strnlen's replacement is
> > located, but regardless, this needs to be fixed too)
> 
> Given that strnlen is standardized by POSIX, and has been for nigh a
> decade, I think it'd be all right for us to treat it as a straight
> port replacement function, a la strlcpy() for instance.  That is,
> forget the pg_ prefix and just create a strnlen() function if the
> platform has not got it.

Yea, I'm ok with that. But can't do that before the configure issue on
these boxes is fixed, otherwise we'll likely get symbol conflicts...

Greetings,

Andres Freund


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Tom Lane]

Andres Freund  writes:
> (there's definitely fixes to be made to where strnlen's replacement is
> located, but regardless, this needs to be fixed too)

Given that strnlen is standardized by POSIX, and has been for nigh a
decade, I think it'd be all right for us to treat it as a straight
port replacement function, a la strlcpy() for instance.  That is,
forget the pg_ prefix and just create a strnlen() function if the
platform has not got it.

regards, tom lane


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andres Freund]

On 2017-10-09 23:33:36 -0400, Andrew Dunstan wrote:
> 
> 
> On 10/09/2017 07:15 PM, Andres Freund wrote:
> > Hi Andrew,
> >
> > On 2017-10-09 22:22:04 +, Andres Freund wrote:
> >> Add pg_strnlen() a portable implementation of strlen.
> >>
> >> As the OS version is likely going to be more optimized, fall back to
> >> it if available, as detected by configure.
> > I'm a bit confused, frogmouth
> > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmouth=2017-10-09%2022%3A30%3A41
> > shows that it compiled the new code, but the configure output doesn't
> > show it ran through the new configure test.  Additionally, without the
> > the config define, this should result in the replacement being
> > used. Which doesn't seem to be the case either.
> >
> > Kinda sounds like this used some halfway outdated build or such?
> >
> 
> 
> 
> frogmouth is using some code not yet released that makes the config
> cache persistent. I just identified and fixed a stupid bug in the code
> that obsoletes the cache, and I have removed frogmouth's cache file and
> set it running again, so we'll see if that fixes things.

As far as I can tell it's still somehow using a configure from before
the last commits:
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=frogmouth=2017-10-10%2018%3A35%3A06=configure

Note that it's not running the new test.

(there's definitely fixes to be made to where strnlen's replacement is
located, but regardless, this needs to be fixed too)

- Andres


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Tom Lane]

Andres Freund  writes:
> I think the current split between common and port isn't particularly
> meaningful. But as long as we have it, this probably belongs more in
> port than in common.

Well, port is supposed to be for stuff that we expect to find in libc
on common platforms, but is missing some places.  strnlen seems to
fall in that category well enough.

regards, tom lane


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andres Freund]

On 2017-10-10 00:25:52 -0400, Tom Lane wrote:
> Andrew Dunstan  writes:
> > frogmouth is using some code not yet released that makes the config
> > cache persistent. I just identified and fixed a stupid bug in the code
> > that obsoletes the cache, and I have removed frogmouth's cache file and
> > set it running again, so we'll see if that fixes things.

Interesting. Although an outdated cache file doesn't explain why
configure didn't even do the relevant check, no?  This kinda sounds more
like configure as a whole is outdated, rather than it's cache file.


> I doubt it.

It'll quite possibly fix it, but probably not for good reasons.
Presumably after a proper configure run the fallback won't be used
anymore.


> I think the problem with this patch is that Andres has
> made libpgport dependent on libpgcommon, which is backwards, or at
> least circular.  The module layering is supposed to go the other way.

Yes, quite possibly. At least one platform without strnlen [1], as well
as my local machine when intentionally marking strnlen as not available,
ran successfully. But that's likely just a difference in how/when
symbols are resolved.

I think the current split between common and port isn't particularly
meaningful. But as long as we have it, this probably belongs more in
port than in common.

Greetings,

Andres Freund

[1] 
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=gharial=2017-10-10%2000%3A31%3A38=config


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Tom Lane]

Andrew Dunstan  writes:
> frogmouth is using some code not yet released that makes the config
> cache persistent. I just identified and fixed a stupid bug in the code
> that obsoletes the cache, and I have removed frogmouth's cache file and
> set it running again, so we'll see if that fixes things.

I doubt it.  I think the problem with this patch is that Andres has
made libpgport dependent on libpgcommon, which is backwards, or at
least circular.  The module layering is supposed to go the other way.

regards, tom lane


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andrew Dunstan]

On 10/09/2017 07:15 PM, Andres Freund wrote:
> Hi Andrew,
>
> On 2017-10-09 22:22:04 +, Andres Freund wrote:
>> Add pg_strnlen() a portable implementation of strlen.
>>
>> As the OS version is likely going to be more optimized, fall back to
>> it if available, as detected by configure.
> I'm a bit confused, frogmouth
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmouth=2017-10-09%2022%3A30%3A41
> shows that it compiled the new code, but the configure output doesn't
> show it ran through the new configure test.  Additionally, without the
> the config define, this should result in the replacement being
> used. Which doesn't seem to be the case either.
>
> Kinda sounds like this used some halfway outdated build or such?
>



frogmouth is using some code not yet released that makes the config
cache persistent. I just identified and fixed a stupid bug in the code
that obsoletes the cache, and I have removed frogmouth's cache file and
set it running again, so we'll see if that fixes things.

cheers

andrew




-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andres Freund]

Hi Andrew,

On 2017-10-09 22:22:04 +, Andres Freund wrote:
> Add pg_strnlen() a portable implementation of strlen.
> 
> As the OS version is likely going to be more optimized, fall back to
> it if available, as detected by configure.

I'm a bit confused, frogmouth
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmouth=2017-10-09%2022%3A30%3A41
shows that it compiled the new code, but the configure output doesn't
show it ran through the new configure test.  Additionally, without the
the config define, this should result in the replacement being
used. Which doesn't seem to be the case either.

Kinda sounds like this used some halfway outdated build or such?

Greetings,

Andres Freund


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers

[COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.

[Andres Freund]

Add pg_strnlen() a portable implementation of strlen.

As the OS version is likely going to be more optimized, fall back to
it if available, as detected by configure.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/8a241792f968ed5be6cf4d41e32c0d264f6c0c65

Modified Files
--
configure |  2 +-
configure.in  |  2 +-
src/common/string.c   | 20 
src/include/common/string.h   | 15 +++
src/include/pg_config.h.in|  3 +++
src/include/pg_config.h.win32 |  3 +++
src/port/snprintf.c   | 12 ++--
7 files changed, 45 insertions(+), 12 deletions(-)


-- 
Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-committers