Greetings, * Alvaro Herrera (alvhe...@2ndquadrant.com) wrote: > On 2018-Dec-10, Paul Ramsey wrote: > > Your analysis looks correct to me, I'm pretty sure I had the same reaction, > > first time I read through. It would be nice to handle partial decompression > > all the way down at this level, but unfortunately the comment at the > > Assert() is right: there's no way to know how many of the toasted pieces > > need to be read in order to have enough compressed input to create the > > desired amount of decompressed output, so there's no choice except to read > > the whole compressed thing, even in a slicing context. > > It'd be useful to have some sort of iterator-style API for detoasting. > If you need more data, just call it again. It's more wasteful if you > end up retrieving all of the toasted data, but if you just need a > fraction it's obviously a win.
I was wondering about that myself. I was looking this area with the idea of pushing Paul's patch here: https://www.postgresql.org/message-id/cacowwr1vbmmje1hdzgdxwx_z5mkypqa1jyw8xxunyjq1mri...@mail.gmail.com but that's just "give me all the data from the front to X point." Paul, what do you think about implementing an iterator for decompressing PGLZ data, and then using that? For your use-case, it'd be just one call since we know how much we want, but for other use-cases (such as searching a compressed TOAST item for something), it'd be an actual iteration and we could potentially eliminate a lot of work in those cases where we just need a boolean yes/no the TOAST'd data matches the query. Thanks! Stephen
signature.asc
Description: PGP signature
