Hi, On Wed, 5 Jun 2019 at 12:01, Andres Freund <and...@anarazel.de> wrote:
> Hi > > On June 5, 2019 8:51:10 AM PDT, Dave Cramer <davecra...@gmail.com> wrote: > >On Wed, 5 Jun 2019 at 07:21, Dave Cramer <davecra...@gmail.com> wrote: > > > >> Hi, > >> > >> > >> On Wed, 5 Jun 2019 at 07:18, Petr Jelinek > ><petr.jeli...@2ndquadrant.com> > >> wrote: > >> > >>> Hi, > >>> > >>> On 05/06/2019 00:08, Andres Freund wrote: > >>> > Hi, > >>> > > >>> > On 2019-06-05 00:05:02 +0200, David Fetter wrote: > >>> >> Would it make sense to work toward a binary format that's not > >>> >> architecture-specific? I recall from COPY that our binary format > >is > >>> >> not standardized across, for example, big- and little-endian > >machines. > >>> > > >>> > I think you recall wrongly. It's obviously possible that we have > >bugs > >>> > around this, but output/input routines are supposed to handle a > >>> > endianess independent format. That usually means that you have to > >do > >>> > endianess conversions, but that doesn't make it non-standardized. > >>> > > >>> > >>> Yeah, there are really 3 formats of data we have, text protocol, > >binary > >>> network protocol and internal on disk format. The internal on disk > >>> format will not work across big/little-endian but network binary > >>> protocol will. > >>> > >>> FWIW I don't think the code for binary format was included in > >original > >>> logical replication patch (I really tried to keep it as minimal as > >>> possible), but the code and protocol is pretty much ready for adding > >that. > >>> > >> Yes, I looked through the public history and could not find it. > >Thanks for > >> confirming. > >> > >>> > >>> That said, pglogical has code which handles this (I guess Andres > >means > >>> that by predecessor of pgoutput) so if you look for example at the > >>> write_tuple/read_tuple/decide_datum_transfer in > >>> > >>> > > > https://github.com/2ndQuadrant/pglogical/blob/REL2_x_STABLE/pglogical_proto_native.c > >>> that can help you give some ideas on how to approach this. > >>> > >> > >> Thanks for the tip! > >> > > > >Looking at: > > > https://github.com/postgres/postgres/blob/8255c7a5eeba8f1a38b7a431c04909bde4f5e67d/src/backend/replication/pgoutput/pgoutput.c#L163 > > > >this seems completely ignored. What was the intent? > > That's about the output of the plugin, not the datatypes. And independent > of text/binary output, the protocol contains non-printable chars. > > Andres > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > So one of the things they would like added is to get not null information in the schema record. This is so they can mark the field Optional in Java. I presume this would also have some uses in other languages. As I understand it this would require a protocol bump. If this were to be accepted are there any outstanding asks that would useful to add if we were going to bump the protocol? Dave
