This should have gone to hackers as well ---------- Forwarded message --------- From: Dave Cramer <davecra...@gmail.com> Date: Sat, Jun 8, 2019, 6:41 PM Subject: Re: Binary support for pgoutput plugin To: Tomas Vondra <tomas.von...@2ndquadrant.com>
On Sat, Jun 8, 2019, 6:27 PM Tomas Vondra, <tomas.von...@2ndquadrant.com> wrote: > On Fri, Jun 07, 2019 at 06:01:12PM -0700, Andres Freund wrote: > >Hi, > > > >On 2019-06-07 20:52:38 -0400, Chapman Flack wrote: > >> It seems they had ended up designing a whole 'nother "protocol level" > >> involving queries wrapping their results as JSON and an app layer that > >> unwraps again, after trying a simpler first approach that was foiled by > the > >> inability to see into arrays and anonymous record types in the > 'describe' > >> response. > > > >I suspect quite a few people would have to have left the projectbefore > >this would happen. > > > > > >> I thought, in a new protocol rev, why not let the driver send additional > >> 'describe' messages after the first one, to drill into structure of > >> individual columns mentioned in the first response, before sending the > >> 'execute' message? > >> > >> If it doesn't want the further detail, it doesn't have to ask. > >> > >> > And then we suddenly need tracking for all these, so we don't always > >> > send out that information when we previously already did > >> > >> If it's up to the client driver, it can track what it needs or already > has. > > > >> I haven't looked too deeply into the replication protocol ... it happens > >> under a kind of copy-both, right?, so maybe there's a way for the > receiver > >> to send some inquiries back, but maybe in a windowed, full-duplex way > where > >> it might have to buffer some incoming messages before getting the > response > >> to an inquiry message it sent. > > > >That'd be a *lot* of additional complexity, and pretty much prohibitive > >from a performance POV. We'd have to not continue decoding on the server > >side *all* the time to give the client a chance to inquire additional > >information. > > > > I kinda agree with this, and I think it's an argument why replication > solutions that need such additional metadata (e.g. because they have no > database to query) should not rely on pgoutput but should invent their own > decoding plugin. Which is why it's a plugin. > So the reason we are discussing using pgoutput plugin is because it is part of core and guaranteed to be in cloud providers solutions. I'm trying to find a balance here of using what we have as opposed to burdening core to take on additional code to take care of. Not sending the metadata is not a deal breaker but i can see some value in it. Dave > >
