Hi, > On Nov 19, 2026, at 09:25, Tom Lane <[email protected]> wrote: > > I wrote: >> I'm not sure what I think about this proposal. I agree it looks dumb >> to estimate the selectivity of "é" differently from "e", and even more >> so for it to depend on the encoding. However, I believe this per-byte >> counting was intentional, with an eye to Far Eastern character sets. >> A single ideogram of Chinese or Japanese probably carries about the >> same selectivity as a word of English, so giving it the same >> selectivity as "e" likewise doesn't sound great. I'm not trying to >> claim that per-byte counting is ideal, but there is more here than >> meets the eye, so I'm loath to change a longstanding behavior on >> the basis of no discussion or evidence. How can we arrive at a >> behavior that is sensible for both of those cases? > > After noodling about that for awhile, I have a potential answer. > Basically we need (I think) to distinguish letter-y characters > from ideogram-y characters. But there is a serviceable, and > fast, approximation to that in our infrastructure already: > check pg_encoding_dsplen() and see if it's 1 or 2. Double-width > characters can be counted on to carry more information than > regular-width ones. I don't have an immediate proposal about > what selectivity multiplier to assign to double-width characters, > but this approach seems like there's potential to accomplish > what the original hack hoped for without the encoding dependency. > > regards, tom lane
Agreed, and v2 takes that approach. dsplen describes the character rather than its storage: "é" is narrow in both LATIN1 and UTF-8, and a double-width character gets extra weight, so an ideograph is estimated to match far fewer rows than a letter. What the estimate depends on is how wide the character is, not how many bytes the encoding needs. The multiplier is the part I can't justify rigorously: how much a double-width character narrows a match probably differs between scripts, and I saw no way to derive one value from data. I picked 3 because wide characters come mostly from the CJK scripts and their common ones take three bytes in UTF-8, so the estimates in the usual case stay as they are. Happy to change it. Your other comments are all in v2. Thanks for the review and the dsplen suggestion. -- Best regards, Chengpeng Yan
v2-0001-Charge-pattern-selectivity-per-character-not-per-.patch
Description: v2-0001-Charge-pattern-selectivity-per-character-not-per-.patch
