On Wed, Sep 23, 2026 at 1:19 PM Sehrope Sarkuni <[email protected]> wrote: > > On Wed, Sep 23, 2026 at 1:09 PM Nathan Bossart <[email protected]> > wrote: > > > > On Wed, Sep 23, 2026 at 09:41:36AM -0500, Nathan Bossart wrote: > > > On Wed, Sep 23, 2026 at 10:10:09AM -0400, Sehrope Sarkuni wrote: > > >> lpad() and rpad() pad one character at a time, calling > > >> pg_mblen_range() and memcpy() once per padding char. When the padding > > >> string is a single byte, e.g., lpad(x, n, '0') or rpad(x, n, ' '), > > >> the padding is that byte repeated, so the attached patch fills it with > > >> one memset(). > > > > > > I wonder if we could expand these gains by using SIMD whenever the vector > > > length is divisible by the padding string length. My hunch is that's > > > where > > > a lot of the memset() gains come from. > > > > Actually, I think we can expand this to any padding string length by > > copying the padding string once, and then copying from the beginning of the > > padding to the end repeatedly so that we write double the padding each > > time. This is a bit like what commit c60e520 added for pglz_decompress(). > > I've attached some proof-of-concept grade patches. This doesn't quite > > match the performance of your 1-byte fast-path, but it's pretty close and > > applies to many more cases. > > Ah! just saw this after I hit send.
I like your split with the refactor into the separate function first better. It makes the second piece purely the perf improvement. And the copy-as-you-count for the first piece is nifty. Attached is v4 (retroactively referring to my v2, as v3). I stepped through and renamed it a bit. The rest of it is your v2 with some updated tests. Passes tests, CI, and the numbers match my v3. Regards, -- Sehrope Sarkuni Founder & CEO | JackDB, Inc. | https://www.jackdb.com/
From d8a3b9507845c28c2e458cd57637524e1eacfc95 Mon Sep 17 00:00:00 2001 From: Nathan Bossart <[email protected]> Date: Wed, 23 Sep 2026 11:47:02 -0500 Subject: [PATCH v4 1/2] Factor the padding loop out of lpad() and rpad(). lpad() and rpad() each carry an identical copy of the loop that writes the padding characters. This commit moves it into a helper function that both call. No functional change. This is preparatory work for a follow-up commit that will teach the helper to write long padding with a few large memcpy() calls instead of one per character. --- src/backend/utils/adt/oracle_compat.c | 60 ++++++++++++--------------- 1 file changed, 27 insertions(+), 33 deletions(-) diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c index 7422a454397..e5238e44813 100644 --- a/src/backend/utils/adt/oracle_compat.c +++ b/src/backend/utils/adt/oracle_compat.c @@ -143,6 +143,31 @@ casefold(PG_FUNCTION_ARGS) } +/* + * Append m characters of the padding string pad (padlen bytes) to dst, + * cycling through pad as needed, and return a pointer past the last byte + * written. + */ +static char * +append_padding(char *dst, const char *pad, int padlen, int m) +{ + const char *p = pad; + const char *pend = pad + padlen; + + while (m--) + { + int mlen = pg_mblen_range(p, pend); + + memcpy(dst, p, mlen); + dst += mlen; + p += mlen; + if (p == pend) /* wrap around at end of pad */ + p = pad; + } + + return dst; +} + /******************************************************************** * * lpad @@ -167,10 +192,7 @@ lpad(PG_FUNCTION_ARGS) text *string2 = PG_GETARG_TEXT_PP(2); text *ret; char *ptr1, - *ptr2, - *ptr2start, *ptr_ret; - const char *ptr2end; int m, s1len, s2len; @@ -209,20 +231,7 @@ lpad(PG_FUNCTION_ARGS) m = len - s1len; - ptr2 = ptr2start = VARDATA_ANY(string2); - ptr2end = ptr2 + s2len; - ptr_ret = VARDATA(ret); - - while (m--) - { - int mlen = pg_mblen_range(ptr2, ptr2end); - - memcpy(ptr_ret, ptr2, mlen); - ptr_ret += mlen; - ptr2 += mlen; - if (ptr2 == ptr2end) /* wrap around at end of s2 */ - ptr2 = ptr2start; - } + ptr_ret = append_padding(VARDATA(ret), VARDATA_ANY(string2), s2len, m); ptr1 = VARDATA_ANY(string1); @@ -265,10 +274,7 @@ rpad(PG_FUNCTION_ARGS) text *string2 = PG_GETARG_TEXT_PP(2); text *ret; char *ptr1, - *ptr2, - *ptr2start, *ptr_ret; - const char *ptr2end; int m, s1len, s2len; @@ -320,19 +326,7 @@ rpad(PG_FUNCTION_ARGS) ptr1 += mlen; } - ptr2 = ptr2start = VARDATA_ANY(string2); - ptr2end = ptr2 + s2len; - - while (m--) - { - int mlen = pg_mblen_range(ptr2, ptr2end); - - memcpy(ptr_ret, ptr2, mlen); - ptr_ret += mlen; - ptr2 += mlen; - if (ptr2 == ptr2end) /* wrap around at end of s2 */ - ptr2 = ptr2start; - } + ptr_ret = append_padding(ptr_ret, VARDATA_ANY(string2), s2len, m); SET_VARSIZE(ret, ptr_ret - (char *) ret); -- 2.17.1
From a5007ff32afe3bd608a88d0f64a79bdb327e8448 Mon Sep 17 00:00:00 2001 From: Sehrope Sarkuni <[email protected]> Date: Wed, 23 Sep 2026 21:41:57 +0000 Subject: [PATCH v4 2/2] Optimize padding in lpad() and rpad(). lpad() and rpad() write the padding one character at a time, calling pg_mblen_range() and memcpy() for each one, which gets slow once the padding runs to many kilobytes. Since the padding is just the pad string repeated, this commit copies the pad string once, as before, and then produces the rest by copying what has already been written onto the end of itself, doubling the length each time. That takes a handful of memcpy() calls for a pad string of any length, and it is dramatically faster for long padding. Note that the pad string is still only checked for a truncated multibyte character as far as it is actually used, so a bad tail is accepted when no padding is needed, as before. --- src/backend/utils/adt/oracle_compat.c | 47 ++++++++++++++++++++++++-- src/test/regress/expected/encoding.out | 37 ++++++++++++++++++++ src/test/regress/expected/strings.out | 26 ++++++++++++++ src/test/regress/sql/encoding.sql | 10 ++++++ src/test/regress/sql/strings.sql | 7 ++++ 5 files changed, 124 insertions(+), 3 deletions(-) diff --git a/src/backend/utils/adt/oracle_compat.c b/src/backend/utils/adt/oracle_compat.c index e5238e44813..f457e01a2ae 100644 --- a/src/backend/utils/adt/oracle_compat.c +++ b/src/backend/utils/adt/oracle_compat.c @@ -147,22 +147,63 @@ casefold(PG_FUNCTION_ARGS) * Append m characters of the padding string pad (padlen bytes) to dst, * cycling through pad as needed, and return a pointer past the last byte * written. + * + * The pad string is validated with pg_mblen_range() only as far as it is + * used, so an incomplete multibyte character at its end is an error only + * if the padding reaches it. */ static char * append_padding(char *dst, const char *pad, int padlen, int m) { const char *p = pad; const char *pend = pad + padlen; + char *start = dst; + int nchars = 0; + int nbytes; + int written; - while (m--) + /* copy pad once, one character at a time, or until m runs out */ + while (m > 0 && p < pend) { int mlen = pg_mblen_range(p, pend); memcpy(dst, p, mlen); dst += mlen; p += mlen; - if (p == pend) /* wrap around at end of pad */ - p = pad; + m--; + nchars++; + } + + if (m == 0) + return dst; + + /* + * The rest of the padding is pad repeated, so work out how many bytes + * that is: whole copies of pad, plus the first m % nchars characters of + * one more. + */ + nbytes = (m / nchars) * padlen; + p = pad; + for (m %= nchars; m > 0; m--) + p += pg_mblen_unbounded(p); + nbytes += p - pad; + + /* + * Produce those bytes by copying what has already been written onto the + * end, doubling the length each time, so the work is done by a few large + * memcpy() calls rather than one per character. The last chunk, if + * shorter, is a prefix of the padding written so far and therefore of + * pad, which is the partial final repetition. + */ + written = dst - start; + while (nbytes > 0) + { + int chunk = Min(written, nbytes); + + memcpy(dst, start, chunk); + dst += chunk; + nbytes -= chunk; + written += chunk; } return dst; diff --git a/src/test/regress/expected/encoding.out b/src/test/regress/expected/encoding.out index 0bb72a1df6f..9fc871215b0 100644 --- a/src/test/regress/expected/encoding.out +++ b/src/test/regress/expected/encoding.out @@ -60,6 +60,43 @@ SELECT reverse(good) FROM regress_encoding; éfac (1 row) +-- multibyte pad strings: whole repetitions, a partial final repetition, and +-- fewer than one repetition +SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding; + lpad | rpad +---------+--------- + ééécafé | caféééé +(1 row) + +SELECT lpad(good, 12, 'éab'), rpad(good, 12, 'éab') FROM regress_encoding; + lpad | rpad +--------------+-------------- + éabéabéacafé | cafééabéabéa +(1 row) + +SELECT lpad(good, 5, 'éab'), rpad(good, 5, 'éab') FROM regress_encoding; + lpad | rpad +-------+------- + écafé | caféé +(1 row) + +-- a lone lead byte in the pad string is an error if the padding reaches it +SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding; +ERROR: invalid byte sequence for encoding "UTF8": 0xc3 +SELECT rpad(good, 7, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding; +ERROR: invalid byte sequence for encoding "UTF8": 0xc3 +SELECT lpad(good, 5, 'ab' || test_bytea_to_text('\xc3')), rpad(good, 6, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding; + lpad | rpad +-------+-------- + acafé | caféab +(1 row) + +SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding; + lpad | rpad +------+------ + café | café +(1 row) + -- invalid short mb character = error SELECT length(truncated) FROM regress_encoding; ERROR: invalid byte sequence for encoding "UTF8": 0xc3 diff --git a/src/test/regress/expected/strings.out b/src/test/regress/expected/strings.out index fa29abfd829..6e064e3807f 100644 --- a/src/test/regress/expected/strings.out +++ b/src/test/regress/expected/strings.out @@ -3441,6 +3441,32 @@ SELECT rpad('hi', 5, ''); hi (1 row) +-- whole repetitions of the pad string, a partial final repetition, and +-- fewer than one repetition +SELECT lpad('hi', 8, 'abc'), rpad('hi', 8, 'abc'); + lpad | rpad +----------+---------- + abcabchi | hiabcabc +(1 row) + +SELECT lpad('hi', 9, 'abc'), rpad('hi', 9, 'abc'); + lpad | rpad +-----------+----------- + abcabcahi | hiabcabca +(1 row) + +SELECT lpad('hi', 12, 'ab'), rpad('hi', 12, 'ab'); + lpad | rpad +--------------+-------------- + abababababhi | hiababababab +(1 row) + +SELECT lpad('hi', 3, 'abc'), rpad('hi', 3, 'abc'); + lpad | rpad +------+------ + ahi | hia +(1 row) + SELECT ltrim('zzzytrim', 'xyz'); ltrim ------- diff --git a/src/test/regress/sql/encoding.sql b/src/test/regress/sql/encoding.sql index 26caa93a5d5..3ea6e54e52d 100644 --- a/src/test/regress/sql/encoding.sql +++ b/src/test/regress/sql/encoding.sql @@ -37,6 +37,16 @@ SELECT substring(good, 3, 1) FROM regress_encoding; SELECT substring(good, 4, 1) FROM regress_encoding; SELECT regexp_replace(good, '^caf(.)$', '\1') FROM regress_encoding; SELECT reverse(good) FROM regress_encoding; +-- multibyte pad strings: whole repetitions, a partial final repetition, and +-- fewer than one repetition +SELECT lpad(good, 7, 'é'), rpad(good, 7, 'é') FROM regress_encoding; +SELECT lpad(good, 12, 'éab'), rpad(good, 12, 'éab') FROM regress_encoding; +SELECT lpad(good, 5, 'éab'), rpad(good, 5, 'éab') FROM regress_encoding; +-- a lone lead byte in the pad string is an error if the padding reaches it +SELECT lpad(good, 7, test_bytea_to_text('\xc3')) FROM regress_encoding; +SELECT rpad(good, 7, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding; +SELECT lpad(good, 5, 'ab' || test_bytea_to_text('\xc3')), rpad(good, 6, 'ab' || test_bytea_to_text('\xc3')) FROM regress_encoding; +SELECT lpad(good, 4, test_bytea_to_text('\xc3')), rpad(good, 4, test_bytea_to_text('\xc3')) FROM regress_encoding; -- invalid short mb character = error SELECT length(truncated) FROM regress_encoding; diff --git a/src/test/regress/sql/strings.sql b/src/test/regress/sql/strings.sql index 7d9c7275a02..04651a0a46f 100644 --- a/src/test/regress/sql/strings.sql +++ b/src/test/regress/sql/strings.sql @@ -1165,6 +1165,13 @@ SELECT rpad('hi', -5, 'xy'); SELECT rpad('hello', 2); SELECT rpad('hi', 5, ''); +-- whole repetitions of the pad string, a partial final repetition, and +-- fewer than one repetition +SELECT lpad('hi', 8, 'abc'), rpad('hi', 8, 'abc'); +SELECT lpad('hi', 9, 'abc'), rpad('hi', 9, 'abc'); +SELECT lpad('hi', 12, 'ab'), rpad('hi', 12, 'ab'); +SELECT lpad('hi', 3, 'abc'), rpad('hi', 3, 'abc'); + SELECT ltrim('zzzytrim', 'xyz'); SELECT translate('', '14', 'ax'); -- 2.17.1
