Use mbsnrtowcs() in char2wchar() to avoid temporary string copies

[David Geier]

Hi hackers,

While playing around with optimizing trigram detection and splitting in
generate_trgm_only(), I found that char2wchar() creates a string copy
via pstrdup() because mbstowcs() requires the input string to be
null-terminated. Switching to mbsnrtowcs() saves 2-3% of the total
CREATE INDEX statement.

Seems like an easy win, or is there a good reason not to use
mbsnrtowcs() on platforms where it is supported?

The patch needs a bit more work to remove mbstowcs_l(), including the
build system part, and possibly doing the build system part for
mbsnrtowcs().


Beyond that, why do a lot of (all?) string conversion functions not
write anything into the output buffer if it turned out to be too small?
That requires creating a temp buffer that the result is written into to
then copy it into the output buffer if its big enough, e.g.
strlower_libc_mb() does the following:

max_size = curr_char * pg_database_encoding_max_length();
result = palloc(max_size + 1);
result_size = wchar2char(result, workspace, max_size + 1, loc);

if (destsize >= result_size + 1)
{
        memcpy(dest, result, result_size);
        dest[result_size] = '\0';
}

The problem with that is that even if the output buffer is big enough,
the function always needs to palloc() + memcpy() which would be great to
avoid as most places can simple allocate num_characters *
pg_database_encoding_max_length() many characters upfront and be done
with it.

On top of that we could think about allowing to call the conversion
functions iteratively. The call site could realloc as needed and then
call the conversion function only on the not-yet-converted data instead
of converting everything again.

--
David Geier
From 869fc004ca6816d26bab288a16b55205e42c9d4f Mon Sep 17 00:00:00 2001
From: David Geier <geidav...@gmail.com>
Date: Wed, 7 Oct 2026 16:59:25 +0200
Subject: [PATCH v1] Avoid string copy in char2wchar()

---
 src/backend/utils/adt/pg_locale_libc.c | 22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/src/backend/utils/adt/pg_locale_libc.c b/src/backend/utils/adt/pg_locale_libc.c
index 02c419aa62a..e568e1088ee 100644
--- a/src/backend/utils/adt/pg_locale_libc.c
+++ b/src/backend/utils/adt/pg_locale_libc.c
@@ -13,6 +13,7 @@
 
 #include <limits.h>
 #include <wctype.h>
+#include <wchar.h>
 
 #include "access/htup_details.h"
 #include "catalog/pg_database.h"
@@ -1332,21 +1333,28 @@ char2wchar(wchar_t *to, size_t tolen, const char *from, size_t fromlen,
 	else
 #endif							/* WIN32 */
 	{
-		/* mbstowcs requires ending '\0' */
-		char	   *str = pnstrdup(from, fromlen);
+		const char *src_ptr = from;
+		mbstate_t st = {0};
 
 		if (loc == (locale_t) 0)
 		{
-			/* Use mbstowcs directly for the default locale */
-			result = mbstowcs(to, str, tolen);
+			/* Use mbsnrtowcs directly for the default locale */
+			result = mbsnrtowcs(to, &src_ptr, fromlen, tolen - 1, &st);
 		}
 		else
 		{
-			/* Use mbstowcs_l for nondefault locales */
-			result = mbstowcs_l(to, str, tolen, loc);
+			/* For nondefault locales, we need to use mbsnrtowcs via uselocale */
+			locale_t	save_locale = uselocale(loc);
+			result = mbsnrtowcs(to, &src_ptr, fromlen, tolen - 1, &st);
+			uselocale(save_locale);
 		}
 
-		pfree(str);
+		if (result != (size_t) -1)
+		{
+			Assert(result < tolen);
+			/* Append trailing null wchar (mbsnrtowcs() does not) */
+			to[result] = 0;
+		}
 	}
 
 	if (result == -1)
-- 
2.53.0