On Wed, Oct 14, 2026 at 3:20 AM Peter Smith <[email protected]> wrote: > > On Tue, Oct 13, 2026 at 7:05 PM Nisha Moond <[email protected]> wrote: > > > > On Tue, Oct 13, 2026 at 12:31 PM Peter Smith <[email protected]> wrote: > > > > > > Shveta pointed me to this post explaining about why the oversized > > > columns cannot simply be truncated. > > > > > > I was wondering if a middle-ground solution would be useful. One where > > > the data is still flagged as truncated has the advantages of Amit's > > > proposal in that it cannot be confused with byte-exact data, but it > > > still carries some truncated value which might (?) be helpful to a > > > human reader. > > > > > > e.g. > > > > > > Option1 - This option is no good because JSON is too large > > > "b" : "some really long value that it too long and would exceed limits" > > > > > > Option 2 - This option is no good because SQL cannot tell if it is > > > really truncated or not. > > > "b" : "some really long value that ..." > > > > > > Option 3. This option is what Ami proposed in the above mail. > > > "b" : {"omitted": true, "length": 190000000} > > > > > > Option 4 - This option is the middle-ground that I was wondering about. > > > "b" : {"truncated": true, "length": 190000000, "prefix": "some really > > > long value that ..." } > > > > > > > Hi Peter, > > > > Thanks for the suggestions. I looked at this same option (Option 4) a > > couple of weeks ago in [1] and argued against it then, but that > > reasoning doesn't hold since v80. > > > > My concern was that building a prefix for arrays, composites or jsonb > > etc. means rendering the whole value first, which could itself go past > > 1GB. But we no longer check the stored size up front. The output > > function is always called, and only the length of its result is > > compared with the cap. So the text is already in hand when we write > > the marker, and keeping its first few bytes costs almost nothing and > > adds no new path to the 1GB limit error. So it might be worth > > discussing again. > > > > However, if we bring back the stored-size pre-check that was suggested > > as an optimization[2], oversized values would be skipped without > > rendering, and a prefix would again need a full render. In that case, > > IMO, we should not go in this direction. > > > > Thanks for the info. > > > I'm not sure what length would be useful. For text-like keys even ~100 > > bytes would help. For large arrays or nested composites, the first > > bytes are mostly structure like ("(""(""((..., so the prefix wouldn't > > tell the user much. > > Yes. I understood non text fields may not be readable. Perhaps the > "prefix" is optional only and shown only for text fields? > > > > > Thoughts? > > I was not aware this option was already previously discussed.and > dismissed so I am happy to drop the suggestion. OTOH, since this same > question has been raised separately a couple of times [1], maybe it > warrants a comment in the code to explain that the prefix is not > stored deliberately and the reason why. >
Okay. I added a comment at the marker code saying the prefix is left out deliberately. -- Thanks, Nisha
