On Tue Oct 6, 2026 at 9:22 PM UTC, Andrew Dunstan wrote: > > On 2026-10-04 Su 12:42 PM, Rui Zhao wrote: >> Hi Andrew, >> >> Thanks for working on this. I tried two further optimizations on top >> of v4. They can be applied independently: >> >> 1. unicode_assigned() still calls pg_mbstrlen_with_len() on the >> remaining input, then walks it again to check each code point. 0001 >> uses the byte length to walk to the end directly. This also avoids >> scanning the whole suffix before returning false for an unassigned >> code point near the beginning. >> >> 2. unicode_normalize_func() and unicode_is_normalized() allocate a C >> string for the normalization form on every call, even when the input >> is all ASCII. 0002 compares the form directly in the text value, >> preserving case-insensitive matching and rejecting extra bytes. >> It only builds a C string when reporting an invalid form. >> >> 0003 replaces the constant-folded VALUES test with stored short and >> compressed text values. >> >> Core regression and the Unicode normalization checks passed with these >> patches. >> > > OK, that all looks good. The attached combines all this.
Thanks for the updates Andrew. They looks pretty good to me. A few nitpicks: > +/* > + * Return the length of an initial portion of s[0..len) that is valid ASCII, > + * that is, contains no zero bytes and no bytes with the high bit set, or > + * len if all of it is. > + * > + * is_valid_ascii() requires a length that is a multiple of sizeof(Vector8), > + * so check one chunk at a time with it and then the remainder byte by byte. > + * A failure inside a chunk is reported at the start of that chunk, so the > + * result can be up to sizeof(Vector8) - 1 bytes short of the full valid > + * ASCII prefix. > + */ I would relegate that second paragraph to a comment within the function itself. I am not sure it is valuable to callers, only to those reading the actual implementation. It also may be a bit verbose, but that isjust an opinion. > + /* > + * Check the remaining code points without first counting them, stopping > + * at a NUL as pg_mbstrlen_with_len() would. > + */ > + p = (unsigned char *) VARDATA_ANY(input) + start; > + end = (unsigned char *) VARDATA_ANY(input) + len; I am not sure that it makes sense to mention pg_mbstrlen_with_len() here. I would be more explicit as to why we are stopping at a NUL. -- Tristan Partin PostgreSQL Contributors Team AWS (https://aws.amazon.com)
