Zsolt Parragi <[email protected]> writes:
> Commit 23d9ad77181 added additional length checks for tsvector, but it
> is inconsistent at one place: most checks use toklen > MAXSTRLEN as
> the error condition, but array_to_tsvector uses >=. This causes a
> strange corner-case issue: we can convert a 2047 length lexeme to an
> array, as tsvector_to_array allows it, but we can't convert it back
> with array_to_tsvector as it errors out.
> 0001 is a fix for this specific case.
> After looking into this further, I realized that the length handling
> is also inconsistent at other places, even before the mentioned
> commit.
Yeah. I noticed this inconsistency while preparing 23d9ad771,
but felt that fixing it was out of scope for a security patch.
There's a similar problem with inconsistent enforcement of
MAXSTRPOS. Attached is what I had staged to propose post-release.
regards, tom lane
From f4b36a5fb83dd23500cd956e400342689b99d39b Mon Sep 17 00:00:00 2001
From: Tom Lane <[email protected]>
Date: Wed, 12 Aug 2026 15:02:17 -0400
Subject: [PATCH v1] Consistently enforce tsvector/tsquery maximum lengths.
Some places rejected individual tokens longer than MAXSTRLEN, while
others rejected ones longer than MAXSTRLEN-1. The data structure is
perfectly capable of handling MAXSTRLEN, so there's nothing wrong
with using the looser bound. Moreover, as things stand there is a
dump/reload hazard: some code paths permit construction of a tsvector
or tsquery that would later be rejected by tsvectorin or tsqueryin.
So standardize on using MAXSTRLEN.
Identical remarks apply to MAXSTRPOS (the total data length),
so fix that too.
---
src/backend/tsearch/ts_parse.c | 4 ++--
src/backend/utils/adt/tsquery.c | 6 +++---
src/backend/utils/adt/tsvector.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
4 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/src/backend/tsearch/ts_parse.c b/src/backend/tsearch/ts_parse.c
index cb69e9899c5..93d46f1d642 100644
--- a/src/backend/tsearch/ts_parse.c
+++ b/src/backend/tsearch/ts_parse.c
@@ -379,7 +379,7 @@ parsetext(Oid cfgId, ParsedText *prs, char *buf, int buflen)
PointerGetDatum(&lemm),
PointerGetDatum(&lenlemm)));
- if (type > 0 && lenlemm >= MAXSTRLEN)
+ if (type > 0 && lenlemm > MAXSTRLEN)
{
#ifdef IGNORE_LONGLEXEME
ereport(NOTICE,
@@ -582,7 +582,7 @@ hlparsetext(Oid cfgId, HeadlineParsedText *prs, TSQuery query, char *buf, int bu
PointerGetDatum(&lemm),
PointerGetDatum(&lenlemm)));
- if (type > 0 && lenlemm >= MAXSTRLEN)
+ if (type > 0 && lenlemm > MAXSTRLEN)
{
#ifdef IGNORE_LONGLEXEME
ereport(NOTICE,
diff --git a/src/backend/utils/adt/tsquery.c b/src/backend/utils/adt/tsquery.c
index e411a21f07e..3247d5db082 100644
--- a/src/backend/utils/adt/tsquery.c
+++ b/src/backend/utils/adt/tsquery.c
@@ -548,12 +548,12 @@ pushValue_internal(TSQueryParserState state, pg_crc32 valcrc, int distance, int
{
QueryOperand *tmp;
- if (distance >= MAXSTRPOS)
+ if (distance > MAXSTRPOS)
ereturn(state->escontext,,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("value is too big in tsquery: \"%s\"",
state->buffer)));
- if (lenval >= MAXSTRLEN)
+ if (lenval > MAXSTRLEN)
ereturn(state->escontext,,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("operand is too long in tsquery: \"%s\"",
@@ -581,7 +581,7 @@ pushValue(TSQueryParserState state, char *strval, int lenval, int16 weight, bool
{
pg_crc32 valcrc;
- if (lenval >= MAXSTRLEN)
+ if (lenval > MAXSTRLEN)
ereturn(state->escontext,,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("word is too long in tsquery: \"%s\"",
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index 40be82b890a..b99878d0ca2 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -207,12 +207,12 @@ tsvectorin(PG_FUNCTION_ARGS)
while (gettoken_tsvector(state, &token, &toklen, &pos, &poslen, NULL))
{
- if (toklen >= MAXSTRLEN)
+ if (toklen > MAXSTRLEN)
ereturn(escontext, (Datum) 0,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("word is too long (%d bytes, max %d bytes)",
toklen,
- MAXSTRLEN - 1)));
+ MAXSTRLEN)));
if (cur - tmpbuf > MAXSTRPOS)
ereturn(escontext, (Datum) 0,
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 782b91f701e..d5336774b22 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -776,12 +776,12 @@ array_to_tsvector(PG_FUNCTION_ARGS)
ereport(ERROR,
(errcode(ERRCODE_ZERO_LENGTH_CHARACTER_STRING),
errmsg("lexeme array may not contain empty strings")));
- if (toklen >= MAXSTRLEN)
+ if (toklen > MAXSTRLEN)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("word is too long (%d bytes, max %d bytes)",
toklen,
- MAXSTRLEN - 1)));
+ MAXSTRLEN)));
}
/* Sort and de-dup, because this is required for a valid tsvector. */
--
2.52.0
