Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
On 2017-10-10 18:10:15 -0400, Tom Lane wrote: > Andres Freundwrites: > > Here's a fix. Not quite sure whether we really need the > > HAVE_DECL_STRNLEN test, added it for symmetry. > > LGTM. Thanks for checking. > I think the DECL test is a good idea; the system definition > might be a macro or otherwise weird, in which case we'd cause problems > if we write an extern definition anyway. I was thinking about protecting it with HAVE_STRNLEN rather than not protecting it at all. But this works, so whatever ;) Greetings, Andres Freund -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
Andres Freundwrites: > Here's a fix. Not quite sure whether we really need the > HAVE_DECL_STRNLEN test, added it for symmetry. LGTM. I think the DECL test is a good idea; the system definition might be a macro or otherwise weird, in which case we'd cause problems if we write an extern definition anyway. regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
On 2017-10-10 17:34:17 -0400, Andrew Dunstan wrote: > This test is governed by the test at line 946 of configure.in. You need > to move it somewhere else. Yea, sorry for the noise :(. Greetings, Andres Freund -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
On 2017-10-10 13:53:15 -0700, Andres Freund wrote: > On 2017-10-10 16:51:39 -0400, Tom Lane wrote: > > Andres Freundwrites: > > > As far as I can tell it's still somehow using a configure from before > > > the last commits: > > > > No, it's pilot error. The AC_CHECK_FUNCS call you added strnlen to > > is only executed if > > AS_IF([test "$enable_thread_safety" = yes -a "$PORTNAME" != "win32"], > > Dear god, I'm daft. Dear god, m4 is a horrible lanuagage. Here's a fix. Not quite sure whether we really need the HAVE_DECL_STRNLEN test, added it for symmetry. Afaict windows "always" had strnlen, so no need to meddle with the windows build. Will eat lunch and push, even if there's some futher adjustments, I'd like to get Andrew's animals green again. Greetings, Andres Freund >From fffd651e83ccbd6191a76be6ec7c6b1b27888fde Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Tue, 10 Oct 2017 14:42:16 -0700 Subject: [PATCH] Rewrite strnlen replacement implementation from 8a241792f96. The previous placement of the fallback implementation in libpgcommon was problematic, because libpqport functions need strnlen functionality. Move replacement into libpgport. Provide strnlen() under its posix name, instead of pg_strnlen(). Fix stupid configure bug, executing the test only when compiled with threading support. Author: Andres Freund Discussion: https://postgr.es/m/e1e1gr2-0005fb...@gemulon.postgresql.org --- configure | 25 - configure.in | 6 +++--- src/backend/utils/mmgr/mcxt.c | 3 +-- src/common/string.c | 20 src/include/common/string.h | 15 --- src/include/pg_config.h.in| 4 src/include/pg_config.h.win32 | 10 +++--- src/include/port.h| 4 src/port/snprintf.c | 4 +--- src/port/strnlen.c| 33 + 10 files changed, 77 insertions(+), 47 deletions(-) create mode 100644 src/port/strnlen.c diff --git a/configure b/configure index a1283c05005..b0582657bf4 100755 --- a/configure +++ b/configure @@ -8777,7 +8777,7 @@ fi -for ac_func in strerror_r getpwuid_r gethostbyname_r strnlen +for ac_func in strerror_r getpwuid_r gethostbyname_r do : as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh` ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var" @@ -13161,6 +13161,16 @@ fi cat >>confdefs.h <<_ACEOF #define HAVE_DECL_STRLCPY $ac_have_decl _ACEOF +ac_fn_c_check_decl "$LINENO" "strnlenfrak" "ac_cv_have_decl_strnlen" "$ac_includes_default" +if test "x$ac_cv_have_decl_strnlen" = xyes; then : + ac_have_decl=1 +else + ac_have_decl=0 +fi + +cat >>confdefs.h <<_ACEOF +#define HAVE_DECL_STRNLEN $ac_have_decl +_ACEOF # This is probably only present on macOS, but may as well check always ac_fn_c_check_decl "$LINENO" "F_FULLFSYNC" "ac_cv_have_decl_F_FULLFSYNC" "#include @@ -13528,6 +13538,19 @@ esac fi +ac_fn_c_check_func "$LINENO" "strnlenfrak" "ac_cv_func_strnlen" +if test "x$ac_cv_func_strnlen" = xyes; then : + $as_echo "#define HAVE_STRNLEN 1" >>confdefs.h + +else + case " $LIBOBJS " in + *" strnlen.$ac_objext "* ) ;; + *) LIBOBJS="$LIBOBJS strnlen.$ac_objext" + ;; +esac + +fi + case $host_os in diff --git a/configure.in b/configure.in index e1381b4ead6..4548db0dd3c 100644 --- a/configure.in +++ b/configure.in @@ -961,7 +961,7 @@ LIBS="$LIBS $PTHREAD_LIBS" AC_CHECK_HEADER(pthread.h, [], [AC_MSG_ERROR([ pthread.h not found; use --disable-thread-safety to disable thread safety])]) -AC_CHECK_FUNCS([strerror_r getpwuid_r gethostbyname_r strnlen]) +AC_CHECK_FUNCS([strerror_r getpwuid_r gethostbyname_r]) # Do test here with the proper thread flags PGAC_FUNC_STRERROR_R_INT @@ -1422,7 +1422,7 @@ AC_CHECK_DECLS(posix_fadvise, [], [], [#include ]) fi AC_CHECK_DECLS(fdatasync, [], [], [#include ]) -AC_CHECK_DECLS([strlcat, strlcpy]) +AC_CHECK_DECLS([strlcat, strlcpy, strnlen]) # This is probably only present on macOS, but may as well check always AC_CHECK_DECLS(F_FULLFSYNC, [], [], [#include ]) @@ -1514,7 +1514,7 @@ else AC_CHECK_FUNCS([fpclass fp_class fp_class_d class], [break]) fi -AC_REPLACE_FUNCS([crypt fls getopt getrusage inet_aton mkdtemp random rint srandom strerror strlcat strlcpy]) +AC_REPLACE_FUNCS([crypt fls getopt getrusage inet_aton mkdtemp random rint srandom strerror strlcat strlcpy strnlen]) case $host_os in diff --git a/src/backend/utils/mmgr/mcxt.c b/src/backend/utils/mmgr/mcxt.c index 64e0408d5af..c5c311fad39 100644 --- a/src/backend/utils/mmgr/mcxt.c +++ b/src/backend/utils/mmgr/mcxt.c @@ -21,7 +21,6 @@ #include "postgres.h" -#include "common/string.h" #include "miscadmin.h" #include "utils/memdebug.h" #include "utils/memutils.h" @@ -1089,7 +1088,7 @@ pnstrdup(const char *in, Size len) { char *out; - len = pg_strnlen(in, len); + len = strnlen(in, len); out = palloc(len + 1); memcpy(out, in,
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
On 10/10/2017 03:48 PM, Andres Freund wrote: > On 2017-10-09 23:33:36 -0400, Andrew Dunstan wrote: >> >> On 10/09/2017 07:15 PM, Andres Freund wrote: >>> Hi Andrew, >>> >>> On 2017-10-09 22:22:04 +, Andres Freund wrote: Add pg_strnlen() a portable implementation of strlen. As the OS version is likely going to be more optimized, fall back to it if available, as detected by configure. >>> I'm a bit confused, frogmouth >>> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmouth=2017-10-09%2022%3A30%3A41 >>> shows that it compiled the new code, but the configure output doesn't >>> show it ran through the new configure test. Additionally, without the >>> the config define, this should result in the replacement being >>> used. Which doesn't seem to be the case either. >>> >>> Kinda sounds like this used some halfway outdated build or such? >>> >> >> >> frogmouth is using some code not yet released that makes the config >> cache persistent. I just identified and fixed a stupid bug in the code >> that obsoletes the cache, and I have removed frogmouth's cache file and >> set it running again, so we'll see if that fixes things. > As far as I can tell it's still somehow using a configure from before > the last commits: > https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=frogmouth=2017-10-10%2018%3A35%3A06=configure > > Note that it's not running the new test. > > (there's definitely fixes to be made to where strnlen's replacement is > located, but regardless, this needs to be fixed too) > This test is governed by the test at line 946 of configure.in. You need to move it somewhere else. cheers andrew -- Andrew Dunstanhttps://www.2ndQuadrant.com PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
On 2017-10-10 16:51:39 -0400, Tom Lane wrote: > Andres Freundwrites: > > As far as I can tell it's still somehow using a configure from before > > the last commits: > > No, it's pilot error. The AC_CHECK_FUNCS call you added strnlen to > is only executed if > AS_IF([test "$enable_thread_safety" = yes -a "$PORTNAME" != "win32"], Dear god, I'm daft. Dear god, m4 is a horrible lanuagage. Thanks. -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
Andres Freundwrites: > As far as I can tell it's still somehow using a configure from before > the last commits: No, it's pilot error. The AC_CHECK_FUNCS call you added strnlen to is only executed if AS_IF([test "$enable_thread_safety" = yes -a "$PORTNAME" != "win32"], A better place would be configure.in:1403, probably. regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
On 2017-10-10 16:37:04 -0400, Tom Lane wrote: > Andres Freundwrites: > > (there's definitely fixes to be made to where strnlen's replacement is > > located, but regardless, this needs to be fixed too) > > Given that strnlen is standardized by POSIX, and has been for nigh a > decade, I think it'd be all right for us to treat it as a straight > port replacement function, a la strlcpy() for instance. That is, > forget the pg_ prefix and just create a strnlen() function if the > platform has not got it. Yea, I'm ok with that. But can't do that before the configure issue on these boxes is fixed, otherwise we'll likely get symbol conflicts... Greetings, Andres Freund -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
Andres Freundwrites: > (there's definitely fixes to be made to where strnlen's replacement is > located, but regardless, this needs to be fixed too) Given that strnlen is standardized by POSIX, and has been for nigh a decade, I think it'd be all right for us to treat it as a straight port replacement function, a la strlcpy() for instance. That is, forget the pg_ prefix and just create a strnlen() function if the platform has not got it. regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
On 2017-10-09 23:33:36 -0400, Andrew Dunstan wrote: > > > On 10/09/2017 07:15 PM, Andres Freund wrote: > > Hi Andrew, > > > > On 2017-10-09 22:22:04 +, Andres Freund wrote: > >> Add pg_strnlen() a portable implementation of strlen. > >> > >> As the OS version is likely going to be more optimized, fall back to > >> it if available, as detected by configure. > > I'm a bit confused, frogmouth > > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmouth=2017-10-09%2022%3A30%3A41 > > shows that it compiled the new code, but the configure output doesn't > > show it ran through the new configure test. Additionally, without the > > the config define, this should result in the replacement being > > used. Which doesn't seem to be the case either. > > > > Kinda sounds like this used some halfway outdated build or such? > > > > > > frogmouth is using some code not yet released that makes the config > cache persistent. I just identified and fixed a stupid bug in the code > that obsoletes the cache, and I have removed frogmouth's cache file and > set it running again, so we'll see if that fixes things. As far as I can tell it's still somehow using a configure from before the last commits: https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=frogmouth=2017-10-10%2018%3A35%3A06=configure Note that it's not running the new test. (there's definitely fixes to be made to where strnlen's replacement is located, but regardless, this needs to be fixed too) - Andres -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
Andres Freundwrites: > I think the current split between common and port isn't particularly > meaningful. But as long as we have it, this probably belongs more in > port than in common. Well, port is supposed to be for stuff that we expect to find in libc on common platforms, but is missing some places. strnlen seems to fall in that category well enough. regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
On 2017-10-10 00:25:52 -0400, Tom Lane wrote: > Andrew Dunstanwrites: > > frogmouth is using some code not yet released that makes the config > > cache persistent. I just identified and fixed a stupid bug in the code > > that obsoletes the cache, and I have removed frogmouth's cache file and > > set it running again, so we'll see if that fixes things. Interesting. Although an outdated cache file doesn't explain why configure didn't even do the relevant check, no? This kinda sounds more like configure as a whole is outdated, rather than it's cache file. > I doubt it. It'll quite possibly fix it, but probably not for good reasons. Presumably after a proper configure run the fallback won't be used anymore. > I think the problem with this patch is that Andres has > made libpgport dependent on libpgcommon, which is backwards, or at > least circular. The module layering is supposed to go the other way. Yes, quite possibly. At least one platform without strnlen [1], as well as my local machine when intentionally marking strnlen as not available, ran successfully. But that's likely just a difference in how/when symbols are resolved. I think the current split between common and port isn't particularly meaningful. But as long as we have it, this probably belongs more in port than in common. Greetings, Andres Freund [1] https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=gharial=2017-10-10%2000%3A31%3A38=config -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
Andrew Dunstanwrites: > frogmouth is using some code not yet released that makes the config > cache persistent. I just identified and fixed a stupid bug in the code > that obsoletes the cache, and I have removed frogmouth's cache file and > set it running again, so we'll see if that fixes things. I doubt it. I think the problem with this patch is that Andres has made libpgport dependent on libpgcommon, which is backwards, or at least circular. The module layering is supposed to go the other way. regards, tom lane -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
On 10/09/2017 07:15 PM, Andres Freund wrote: > Hi Andrew, > > On 2017-10-09 22:22:04 +, Andres Freund wrote: >> Add pg_strnlen() a portable implementation of strlen. >> >> As the OS version is likely going to be more optimized, fall back to >> it if available, as detected by configure. > I'm a bit confused, frogmouth > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmouth=2017-10-09%2022%3A30%3A41 > shows that it compiled the new code, but the configure output doesn't > show it ran through the new configure test. Additionally, without the > the config define, this should result in the replacement being > used. Which doesn't seem to be the case either. > > Kinda sounds like this used some halfway outdated build or such? > frogmouth is using some code not yet released that makes the config cache persistent. I just identified and fixed a stupid bug in the code that obsoletes the cache, and I have removed frogmouth's cache file and set it running again, so we'll see if that fixes things. cheers andrew -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers
Re: [COMMITTERS] pgsql: Add pg_strnlen() a portable implementation of strlen.
Hi Andrew, On 2017-10-09 22:22:04 +, Andres Freund wrote: > Add pg_strnlen() a portable implementation of strlen. > > As the OS version is likely going to be more optimized, fall back to > it if available, as detected by configure. I'm a bit confused, frogmouth https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmouth=2017-10-09%2022%3A30%3A41 shows that it compiled the new code, but the configure output doesn't show it ran through the new configure test. Additionally, without the the config define, this should result in the replacement being used. Which doesn't seem to be the case either. Kinda sounds like this used some halfway outdated build or such? Greetings, Andres Freund -- Sent via pgsql-committers mailing list (pgsql-committers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-committers