On Wed, Dec 20, 2017 at 11:25 AM, Josh Haberman <[email protected]> wrote:
> Thanks Feng and sorry for the confusion everyone! > > Feng quoted my original email, but I'll repeat it here for clarity. > Hopefully everyone will be able to reply to messages properly this time. :) > > -- > > Hi all, > > I wanted to kick off a conversation about the future of Protobuf and > Abseil/ABSL in Debian/Ubuntu. Laszlo is the Debian maintainer for the > Protobuf package, and has also expressed interest in packaging Abseil. > > Debian currently distributes a binary shared library for protobuf > (libprotobuf.so). Debian policy requires that the SONAME is bumped every > time there is an ABI-incompatible change to the library: > > https://www.debian.org/doc/debian-policy/#shared-libraries > > Since protobuf does not guarantee any kind of binary compatibility, we > have traditionally updated our SONAME for every minor release of protobuf. > We are currently at major version 15: > > https://github.com/google/protobuf/blob/0ba8eea655a5e40d19ab95c773192b > 5d908c5a61/src/Makefile.am#L186 > > This has worked reasonably well so far. But we want to start depending on > Abseil (https://abseil.io) which does not do releases at all (so doesn't > even have a major version number to track). How would this work for Debian? > > Abseil's philosophy is that users will build from source. But Debian is > inherently a binary distribution. We have to figure out how to reconcile > these two philosophies. I don't know the answer, but I'm going to throw out > a few straw man ideas as a starting point for discussion. > > 1. Distribute libprotobuf and libabsl as static libraries only. > If protobuf is static only, could other .so libraries still link against protobuf? If multiple .so libraries has linked-in protobuf statically, would it cause symbol conflicts when they are used together? It seems to me if protobuf goes static only, every other library that depends on protobuf must be static only as well. > > Debian policy says that libraries can be distributed in static form only > if that is what is intended by upstream: > > https://www.debian.org/doc/debian-policy/#static-libraries > > Since both libprotobuf and libabsl are disavowing any binary compatibility > guarantees, that implies that they should not be distributed as .so's. Any > Debian packages or user programs that want to use protobuf have to link it > statically. > > The tricky part is that libprotobuf.a and libabsl.a would need to match > each other exactly. They would need to be compiled from exactly the same > ABSL headers. This implies that libprotobuf-dev would need to pin itself to > one specific libabsl-dev version -- the one it was compiled against. Other > libraries that depend on ABSL (like TensorFlow) would also need to be > pinned to exactly the same libabsl-dev version, otherwise it would be > impossible to install them all at the same time. So realistically all > packages that depend on ABSL would need to be updated at the same time. > > 2. Distribute libprotobuf and libabsl as shared libraries. > > This is against the philosophy of ABSL. But if Debian updated its ABSL > infrequently enough, it could just treat each ABSL update as a separate > major version. Sort of like we do with protobuf now. > > The tricky part is if two different libraries pass ABSL types across > library boundaries. For example, imagine that a user program was developing > against libabsl.so.2 but also using libprotobuf.so.15 that was linked > against libabsl.so.1. If a user got an absl::string_view from protobuf, > this could be unsafe for the user to pass directly to ABSL functions. The > representation of absl::string_view could have changed between the two > versions of libabsl. > I'm not so worried about this because this problem is not new. With or without absl, there are libraries depending on different versions of protobuf.so and there must already exists an established solution to deal with this. Adding absl to the picture won't make it any trickier than it is before. > > The all-static option (1) seems to solve this problem by pinning all of > the ABSL dependencies so that they all rev together. But this solution > could become unwieldy if too many packages depend on ABSL. Also any library > that exposes ABSL types in its API would have to be part of this lock-step > ABSL upgrade. > > 3. Remove protobuf from Debian completely, ask users to build from source. > It seems very unlikely we can take down all the protobuf.so distributed on the internet... We can make it harder but people will get a way around it. > > Since the philosophy of ABSL is that users will always build from source, > we could ask that users do this if they want to develop against protobuf. > Debian packages that depend on protobuf could build protobuf (and ABSL) and > statically link against it to build runnable binaries. It would not be > possible to have library packages in Debian that depend on protobuf. > > Thoughts? > Josh > > On Wed, Dec 20, 2017 at 10:58 AM, Feng Xiao <[email protected]> wrote: > >> [email protected], [email protected] >> >> CCed public google groups so László can reply to the group. BCCed >> internal mailing lists. >> >> On Tue, Dec 19, 2017 at 8:24 PM, Josh Haberman <[email protected]> >> wrote: >> >>> Hi all, >>> >>> I wanted to kick off a conversation about the future of Protobuf and >>> Abseil/ABSL in Debian/Ubuntu. Laszlo is the Debian maintainer for the >>> Protobuf package, and has also expressed interest in packaging Abseil. >>> >>> Debian currently distributes a binary shared library for protobuf >>> (libprotobuf.so). Debian policy requires that the SONAME is bumped every >>> time there is an ABI-incompatible change to the library: >>> >>> https://www.debian.org/doc/debian-policy/#shared-libraries >>> >>> Since protobuf does not guarantee any kind of binary compatibility, we >>> have traditionally updated our SONAME for every minor release of protobuf. >>> We are currently at major version 15: >>> >>> https://github.com/google/protobuf/blob/0ba8eea655a5e40d19ab >>> 95c773192b5d908c5a61/src/Makefile.am#L186 >>> >>> This has worked reasonably well so far. But we want to start depending >>> on Abseil (https://abseil.io) which does not do releases at all (so >>> doesn't even have a major version number to track). How would this work for >>> Debian? >>> >>> Abseil's philosophy is that users will build from source. But Debian is >>> inherently a binary distribution. We have to figure out how to reconcile >>> these two philosophies. I don't know the answer, but I'm going to throw out >>> a few straw man ideas as a starting point for discussion. >>> >>> 1. Distribute libprotobuf and libabsl as static libraries only. >>> >>> Debian policy says that libraries can be distributed in static form only >>> if that is what is intended by upstream: >>> >>> https://www.debian.org/doc/debian-policy/#static-libraries >>> >>> Since both libprotobuf and libabsl are disavowing any binary >>> compatibility guarantees, that implies that they should not be distributed >>> as .so's. Any Debian packages or user programs that want to use protobuf >>> have to link it statically. >>> >>> The tricky part is that libprotobuf.a and libabsl.a would need to match >>> each other exactly. They would need to be compiled from exactly the same >>> ABSL headers. This implies that libprotobuf-dev would need to pin itself to >>> one specific libabsl-dev version -- the one it was compiled against. Other >>> libraries that depend on ABSL (like TensorFlow) would also need to be >>> pinned to exactly the same libabsl-dev version, otherwise it would be >>> impossible to install them all at the same time. So realistically all >>> packages that depend on ABSL would need to be updated at the same time. >>> >>> 2. Distribute libprotobuf and libabsl as shared libraries. >>> >>> This is against the philosophy of ABSL. But if Debian updated its ABSL >>> infrequently enough, it could just treat each ABSL update as a separate >>> major version. Sort of like we do with protobuf now. >>> >>> The tricky part is if two different libraries pass ABSL types across >>> library boundaries. For example, imagine that a user program was developing >>> against libabsl.so.2 but also using libprotobuf.so.15 that was linked >>> against libabsl.so.1. If a user got an absl::string_view from protobuf, >>> this could be unsafe for the user to pass directly to ABSL functions. The >>> representation of absl::string_view could have changed between the two >>> versions of libabsl. >>> >>> The all-static option (1) seems to solve this problem by pinning all of >>> the ABSL dependencies so that they all rev together. But this solution >>> could become unwieldy if too many packages depend on ABSL. Also any library >>> that exposes ABSL types in its API would have to be part of this lock-step >>> ABSL upgrade. >>> >>> 3. Remove protobuf from Debian completely, ask users to build from >>> source. >>> >>> Since the philosophy of ABSL is that users will always build from >>> source, we could ask that users do this if they want to develop against >>> protobuf. Debian packages that depend on protobuf could build protobuf (and >>> ABSL) and statically link against it to build runnable binaries. It would >>> not be possible to have library packages in Debian that depend on protobuf. >>> >>> Thoughts? >>> Josh >>> >> >> > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
