Bug#888705: abseil-cpp packaging
On Sat, May 23, 2020 at 2:39 PM Benjamin Barenblat wrote: > This is now in the NEW queue. On Friday, June 19, 2020, at 8:07 AM +0200, László Böszörményi (GCS) wrote: > Not anymore and not in the archives. What happened? Can I help? ftp-master rejected the upload with concerns that changing the names of all those small binary packages every time an ABI break occurs could cause too much churn in the archive. That’s an easy fix, though – just consolidate the shared libraries into a single binary package. You actually caught me at an opportune time; I just redid the upload, and it should reappear in NEW shortly. So there’s no work to be done at present, but I’ll let you know if ftp-master has further concerns. Benjamin
Bug#888705: abseil-cpp packaging
On Sat, May 23, 2020 at 2:39 PM Benjamin Barenblat wrote: > This is now in the NEW queue. Not anymore and not in the archives. What happened? Can I help? Cheers, Laszlo/GCS
Bug#888705: abseil-cpp packaging
This is now in the NEW queue.
Bug#888705: abseil-cpp packaging
On Wed, May 20, 2020 at 5:56 PM Benjamin Barenblat wrote: > On Tuesday, May 19, 2020, at 8:59 PM +0200, László Böszörményi (GCS) wrote: > > Doesn't build with GCC 10 due to symbol changes. > > Good point. Is there an established way to deal with this? Or should I > just upload this as-is to unstable and then upload a GCC-10-compatible > version to experimental? You don't have to - but please be prepared that when GCC 10 will be the default in Sid, Abseil will FTBFS. > On Wednesday, May 20, 2020, at 12:03 PM +0200, László Böszörményi (GCS) wrote: > > Please do build static libraries with the following patch. > > This is great – thanks! I’m still unfamiliar with CMake; I really > appreciate you figuring out the incantations to get it to build both. > > As written, this patch builds shared libraries first and then archives, > which causes abslTargets.cmake to prefer archives over shared objects > when linking against Abseil. I’d like to modify it to go in the other > order, thus preferring shared libraries over archives. Requiring an > explicit opt-in before using Abseil archives would make it more > difficult to accidentally violate the One Definition Rule by > accidentally double-linking an Abseil archive. Furthermore, it would > create less work for packagers trying to use the shared Abseil > libraries, which I expect will be the usual case inside Debian. Does > that sound reasonable? Yup, please do so. Thanks go to you for packaging it and hopefully uploaded soon. Laszlo/GCS
Bug#888705: abseil-cpp packaging
On Tuesday, May 19, 2020, at 8:59 PM +0200, László Böszörményi (GCS) wrote: > Doesn't build with GCC 10 due to symbol changes. Good point. Is there an established way to deal with this? Or should I just upload this as-is to unstable and then upload a GCC-10-compatible version to experimental? > How do you plan Git tagging? Might be better to prefix current tags with > 'upstream/' and have Debian releases under 'debian/'. I generally try to keep upstream’s tags intact so that no signatures get broken. Abseil doesn’t currently sign tags, but they might in the future, and I’d be a bit bummed if we couldn’t benefit from that. I think it’s totally reasonable to put Debian releases under debian/, so if you prefer that approach, I’ll make sure to prefix my tags. > Rules-Requires-Root might be set to 'no' I guess, but that's nitpicking. Today I learned about Rules-Requires-Root! I’ve fixed that and pushed the change. On Wednesday, May 20, 2020, at 12:03 PM +0200, László Böszörményi (GCS) wrote: > Please do build static libraries with the following patch. This is great – thanks! I’m still unfamiliar with CMake; I really appreciate you figuring out the incantations to get it to build both. As written, this patch builds shared libraries first and then archives, which causes abslTargets.cmake to prefer archives over shared objects when linking against Abseil. I’d like to modify it to go in the other order, thus preferring shared libraries over archives. Requiring an explicit opt-in before using Abseil archives would make it more difficult to accidentally violate the One Definition Rule by accidentally double-linking an Abseil archive. Furthermore, it would create less work for packagers trying to use the shared Abseil libraries, which I expect will be the usual case inside Debian. Does that sound reasonable?
Bug#888705: abseil-cpp packaging
On Tue, May 19, 2020 at 6:28 PM Benjamin Barenblat wrote: > Okay, we’re all set. I’ve pushed my work to > https://salsa.debian.org/debian/abseil, and both command-line linking > and CMake integration work. Comments and suggestions are welcome – if I > don’t hear anything in the next day or two, I’ll go ahead and upload to > NEW. Please do build static libraries with the following patch. Even Google projects link with static Abseil libraries and the project itself also allow it[1]: "ALLOWED: Distribute a static library built with an LTS branch of Abseil. Because LTS branches use inline namespaces for all absl:: symbols, collisions between potential Abseil “versions” should not occur, though your library may incur code bloat." I do not want to push it without being a co-maintainer or without discussion, but please do it yourself. Thanks, Laszlo/GCS [1] https://abseil.io/about/releases diff -Nru abseil-0~20200225.2/debian/libabsl-dev.install abseil-0~20200225.2/debian/libabsl-dev.install --- abseil-0~20200225.2/debian/libabsl-dev.install 2020-05-12 22:16:31.0 + +++ abseil-0~20200225.2/debian/libabsl-dev.install 2020-05-20 09:43:24.0 + @@ -13,5 +13,6 @@ # the License. usr/include/absl +usr/lib/*/*.a usr/lib/*/*.so usr/lib/*/cmake diff -Nru abseil-0~20200225.2/debian/rules abseil-0~20200225.2/debian/rules --- abseil-0~20200225.2/debian/rules 2020-05-12 22:16:31.0 + +++ abseil-0~20200225.2/debian/rules 2020-05-20 09:43:24.0 + @@ -15,12 +15,24 @@ export DEB_BUILD_MAINT_OPTIONS = hardening=+bindnow -%: - dh $@ +override_dh_auto_clean: + $(RM) -r $(CURDIR)/shared + $(RM) -r $(CURDIR)/static override_dh_auto_configure: - dh_auto_configure -- -DCMAKE_CXX_STANDARD=14 -DBUILD_SHARED_LIBS=ON + dh_auto_configure -Bshared \ + -- -DCMAKE_CXX_STANDARD=14 -DBUILD_SHARED_LIBS=ON + dh_auto_configure -Bstatic \ + -- -DCMAKE_CXX_STANDARD=14 -DBUILD_SHARED_LIBS=OFF + +override_dh_auto_build: + dh_auto_build -Bshared + dh_auto_build -Bstatic override_dh_auto_install: - dh_auto_install + dh_auto_install -Bshared + dh_auto_install -Bstatic find debian/tmp -type d -empty -delete + +%: + dh $@
Bug#888705: abseil-cpp packaging
On Tue, May 19, 2020 at 6:28 PM Benjamin Barenblat wrote: > Okay, we’re all set. I’ve pushed my work to > https://salsa.debian.org/debian/abseil, and both command-line linking > and CMake integration work. Comments and suggestions are welcome – if I > don’t hear anything in the next day or two, I’ll go ahead and upload to > NEW. Just some points that come to mind. Backport to oldstable will not be easily possible, but not a problem for me. File 'control' might be better autogenerated with the ABI string easily changeable, but well 'sed s///' works as well. Doesn't build with GCC 10 due to symbol changes. How do you plan Git tagging? Might be better to prefix current tags with 'upstream/' and have Debian releases under 'debian/'. Rules-Requires-Root might be set to 'no' I guess, but that's nitpicking. Very well done packaging otherwise. Thanks, Laszlo/GCS
Bug#888705: abseil-cpp packaging
Okay, we’re all set. I’ve pushed my work to https://salsa.debian.org/debian/abseil, and both command-line linking and CMake integration work. Comments and suggestions are welcome – if I don’t hear anything in the next day or two, I’ll go ahead and upload to NEW. I had to disable the Abseil testsuite to get CMake integration to work. The CMake files that Abseil installs are autogenerated, and if you enable the testsuite, the generator adds a bunch of googletest dependencies. This is unfortunate, and I’ll file a bug with upstream to try to get it fixed. In the meantime, I’ve manually verified that the tests pass. Benjamin
Bug#888705: abseil-cpp packaging
On Wed, May 13, 2020 at 12:17 AM Benjamin Barenblat wrote: > If it’s all right with you, I’d prefer to stick with src:abseil. That > more closely matches the way the terminology is used within Google – > “Abseil” is unambiguously the C++ project, and “Abseil Python” is its > Python counterpart. It's OK. You can do the following in your repository: git remote add origin https://salsa.debian.org/debian/abseil git push --force origin master That should be it. Regards, Laszlo/GCS
Bug#888705: abseil-cpp packaging
On Thursday, May 7, 2020, at 6:32 PM +0200, László Böszörményi (GCS) wrote: > If I understand correctly, you retained src:abseil. If not and using > src:abseil-cpp then you need a new repository named after that. Which > way should I go? If it’s all right with you, I’d prefer to stick with src:abseil. That more closely matches the way the terminology is used within Google – “Abseil” is unambiguously the C++ project, and “Abseil Python” is its Python counterpart. Benjamin
Bug#888705: abseil-cpp packaging
On Thu, May 7, 2020 at 5:57 PM Benjamin Barenblat wrote: > I’m getting very close to an Abseil upload. The CMake integration > doesn’t work yet, but I can install the binary packages and build > software that links Abseil. Good news. > I’m going to keep working on CMake support, but I’d love to upload what > I have to Salsa. Would somebody be willing to reset the > https://salsa.debian.org/debian/abseil repository to a pristine state so > I can push to it without any of the old history in that repo? I’d do it > myself, but Google has some weird rules about creating new repositories; > it would be much easier for me if someone else could create the repo and > give me push access. All Debian Developers have push access under /debian but only admins can delete / create repositories there. If I understand correctly, you retained src:abseil. If not and using src:abseil-cpp then you need a new repository named after that. Which way should I go? Cheers, Laszlo/GCS
Bug#888705: abseil-cpp packaging
I’m getting very close to an Abseil upload. The CMake integration doesn’t work yet, but I can install the binary packages and build software that links Abseil. I’m going to keep working on CMake support, but I’d love to upload what I have to Salsa. Would somebody be willing to reset the https://salsa.debian.org/debian/abseil repository to a pristine state so I can push to it without any of the old history in that repo? I’d do it myself, but Google has some weird rules about creating new repositories; it would be much easier for me if someone else could create the repo and give me push access.
Bug#888705: abseil-cpp packaging
On Thu, Feb 27, 2020 at 9:51 AM Olaf van der Spek
wrote:
> Op do 27 feb. 2020 om 06:54 schreef László Böszörményi :
> > Are you going to rename it to abseil-cpp (as Google has abseil-python
> > as well, make a disparity between the two)?
>
> Python libraries use a "python3-" prefix while C/C++ libraries use a
> "lib" prefix, so I don't think it makes sense to change the name.
Libraries use lib, python packages use python{,3}- as prefix, yes.
But I mean src:abseil should be src:abseil-cpp to be the same upstream
uses[1].
Laszlo/GCS
[1] https://github.com/abseil/abseil-cpp
Bug#888705: abseil-cpp packaging
Op do 27 feb. 2020 om 06:54 schreef László Böszörményi : > Are you going to rename it to abseil-cpp (as Google has abseil-python > as well, make a disparity between the two)? Python libraries use a "python3-" prefix while C/C++ libraries use a "lib" prefix, so I don't think it makes sense to change the name.
Bug#888705: abseil-cpp packaging
On Wed, Feb 26, 2020 at 10:28 PM Benjamin Barenblat wrote: > https://github.com/abseil/abseil-cpp/pull/628 will let you test against > the gtest in Debian simply by passing `-DABSL_RUN_TESTS=ON > -DABSL_USE_GOOGLETEST_HEAD=OFF` to CMake. It looks like I just missed > the LTS release of a few hours ago, but just patching this in should be > fine. That's very nice of you. > I’ll try to have a look at the packaging in Salsa before the end of this > week. Are you going to rename it to abseil-cpp (as Google has abseil-python as well, make a disparity between the two)? @Anton: No one would like to step on your toes - definitely not me - still your last visible activity was two years ago. Please push your changes if you may any ready. Thanks, Laszlo/GCS
Bug#888705: abseil-cpp packaging
On Sunday, February 16, 2020, at 10:48 PM +0100, László Böszörményi (GCS) wrote: > @Benjamin: may you ask its developers to use the system gtest libraries > if only ABSL_RUN_TESTS set to ON? On Tuesday, February 18, 2020, at 6:02 PM -0500, Benjamin Barenblat wrote: > I have a preliminary version of such a patch, and I’ll try to get it > finished and sent in the next day or two; that way, even if upstream > lags taking it, we can import it and get the support we need. It took me a bit longer than I’d hoped, but https://github.com/abseil/abseil-cpp/pull/628 will let you test against the gtest in Debian simply by passing `-DABSL_RUN_TESTS=ON -DABSL_USE_GOOGLETEST_HEAD=OFF` to CMake. It looks like I just missed the LTS release of a few hours ago, but just patching this in should be fine. I’ll try to have a look at the packaging in Salsa before the end of this week.
Bug#888705: abseil-cpp packaging
On Tuesday, February 18, 2020, at 9:01 PM +0100, László Böszörményi (GCS) wrote: > There's a compatibility page[1] what Abseil is and isn't. It states > the following things. > "We do not promise any ABI compatibility — we intend for Abseil to be > built from source, hopefully from head. The internal layout of our > types may change at any point, without notice." > As I read waiting for LTS releases might be late (its head commit > version advised to be used). I guess other Google applications other > libraries commonly wants newer Abseil releases than LTS ones. I think that’s accurate. The culture at Google emphasizes continuous integration from head rather than coding against releases. However, this isn’t just about Google applications. There are other F/OSS projects that want to take an Abseil dependency and aren’t ready to commit to that sort of development model – see, for instance, https://github.com/darktable-org/rawspeed/issues/122 (“Stop reinventing the wheel”), in which Darktable investigates taking an Abseil dependency and decides to wait until the LTS story is fully hammered out. The Abseil team understands that the F/OSS world expects stable ABIs; they’re going to break ABI all over the place at head, but they’re not going to go out of their way to break it within an LTS release. > "Not all Abseil libraries are suitable for dynamic loading. Some > libraries have startup/initialization requirements and may not be > suitable for dynamic loading. We will try to be clear about which > libraries are OK for dynamic loading. However we don’t generally > deploy code in this fashion, mistakes are possible, [...]". > That is, even shared libraries can be built, those are not guaranteed to work. I don’t think we should shy away from providing functionality just because upstream doesn’t guarantee it to work. It’s true that the Abseil developers don’t test Abseil as .so’s, but testing exists in Debian for a reason. If Debian ships an Abseil .so, and bugs appear in testing, we can work with upstream to fix them or patch them ourselves if upstream is nonresponsive. I did discuss the initialization issues with an Abseiler today; he doesn’t know the full story, but he knows somebody within Abseil is working on making initialization more lazy (and therefore more .so-compatible). If you’re interested, I can discuss that with them at our next meeting and let you know what the story is. > I think there should be a compatibility matrix or test if the latest > LTS release is enough for most Google applications or those need newer > versions (no new API added for recent application development). Agreed – there should definitely be some testing involved. For what it’s worth, the next LTS is likely to be cut from head before the end of the week. For a little while afterward, at least, nobody should need anything newer than what’s in the LTS. On Sunday, February 16, 2020, at 10:48 PM +0100, László Böszörményi (GCS) wrote: >>> @Benjamin: may you ask its developers to use the system gtest libraries >>> if only ABSL_RUN_TESTS set to ON? On Monday, February 17, 2020, at 8:21 PM -0500, Benjamin Barenblat wrote: >> Absolutely. I’ll bring it up with them at work tomorrow (today was a US >> holiday). I spoke with an Abseiler today about this. He believes that Abseil definitely needs to support that, but there’s still some consensus-building that needs to happen within the team before we can guarantee that upstream will take a patch for it. I have a preliminary version of such a patch, and I’ll try to get it finished and sent in the next day or two; that way, even if upstream lags taking it, we can import it and get the support we need.
Bug#888705: abseil-cpp packaging
On Tuesday, February 18, 2020, at 9:25 AM +0100, Olaf van der Spek wrote: > What about the C++ std version? Abseil / C++14 isn't the same as Abseil / > C++17. This is true on two levels: 1. By default, Abseil detects what standard version you’re building with and conditionally defines its types to be type aliases when appropriate. For instance, in C++11, `absl::make_unique` is an actual function; in C++14 and later, `absl::make_unique` is an alias for `std::make_unique`. The next LTS release will have a CMake toggle you can set to disable this behavior, I think it would be most user-friendly for us to set it. It’s less efficient to ship an Abseil in which `absl::make_unique` is always an actual function, but I think it’s better than shipping an Abseil that can only be used with one version of the C++ standard. 2. Abseil may use some language features that changed semantics in C++17 and are therefore ABI-incompatible with translation units compiled against a different standard. I spoke with an Abseil developer about this today, and he believes it’s not likely to be an issue. Abseil does not plumb the depths of the C++ language spec (except possibly the template engine), so provided (1) is resolved, it’s entirely possible that a binary Abseil will work everywhere. We won’t really know until we package it and let it soak in testing for a while.
Bug#888705: abseil-cpp packaging
Hi all, I have dropped almost all static (*.a)-files from libraries, which I maintain(ed). The main reason is the easier security fixes if any appear in the library. One need then basically rebuild the so-files. With statically linked code the security fixes are much harder: one need to rebuild all build-depends. Some more info [1]. ABI-breakage between LTS-releases is not a big deal. We should just change the SO-name of the library and force transition-process. [1] https://wiki.debian.org/StaticLinking#downsides Regards Anton Am Di., 18. Feb. 2020 um 02:46 Uhr schrieb Benjamin Barenblat : > > On Sunday, February 16, 2020, at 10:48 PM +0100, László Böszörményi (GCS) > wrote: > > In my reading abseil is _not_ guaranteed to have ABI compatibility at > > all times. That's why it meant to be a static library collection only. > > Forcing it to build shared libraries and have other packages than > > libabseil-cpp-dev is no sense I think. > > Abseil does reserve the right to break ABI at any time, and I can’t > speak to their future plans. But I think ABI breakages are unlikely to > occur in practice within an LTS release. If we wait and then package an > LTS release, I it’ll be completely reasonable to ship .so’s and .a’s. > > Relatedly, I think we should only package LTS Abseil for Debian. If > someone finds a CVE in Abseil, the Abseil team are going to want to > backport the fix to LTS releases; they’re not going to want to backport > it everywhere else. > > > @Benjamin: may you ask its developers to use the system gtest libraries > > if only ABSL_RUN_TESTS set to ON? > > Absolutely. I’ll bring it up with them at work tomorrow (today was a US > holiday).
Bug#888705: abseil-cpp packaging
On Tue, Feb 18, 2020 at 2:31 AM Benjamin Barenblat wrote: > On Sunday, February 16, 2020, at 10:48 PM +0100, László Böszörményi (GCS) > wrote: > > In my reading abseil is _not_ guaranteed to have ABI compatibility at > > all times. That's why it meant to be a static library collection only. > > Forcing it to build shared libraries and have other packages than > > libabseil-cpp-dev is no sense I think. > > Abseil does reserve the right to break ABI at any time, and I can’t > speak to their future plans. But I think ABI breakages are unlikely to > occur in practice within an LTS release. If we wait and then package an > LTS release, I it’ll be completely reasonable to ship .so’s and .a’s. There's a compatibility page[1] what Abseil is and isn't. It states the following things. "We do not promise any ABI compatibility — we intend for Abseil to be built from source, hopefully from head. The internal layout of our types may change at any point, without notice." As I read waiting for LTS releases might be late (its head commit version advised to be used). I guess other Google applications other libraries commonly wants newer Abseil releases than LTS ones. "Not all Abseil libraries are suitable for dynamic loading. Some libraries have startup/initialization requirements and may not be suitable for dynamic loading. We will try to be clear about which libraries are OK for dynamic loading. However we don’t generally deploy code in this fashion, mistakes are possible, [...]". That is, even shared libraries can be built, those are not guaranteed to work. "We will not break API compatibility. If we must, we will ship a tool to automate the upgrade to a preferred API." Seems to suggest using the head (latest commit) freely and link with the static libraries of the applications. > Relatedly, I think we should only package LTS Abseil for Debian. If > someone finds a CVE in Abseil, the Abseil team are going to want to > backport the fix to LTS releases; they’re not going to want to backport > it everywhere else. I think there should be a compatibility matrix or test if the latest LTS release is enough for most Google applications or those need newer versions (no new API added for recent application development). Even if I agree that LTS releases are better for long time use cases and from security point of view. > > @Benjamin: may you ask its developers to use the system gtest libraries > > if only ABSL_RUN_TESTS set to ON? > > Absolutely. I’ll bring it up with them at work tomorrow (today was a US > holiday). Thanks. Please keep us posted how it goes. Regards, Laszlo/GCS [1] https://abseil.io/about/compatibility
Bug#888705: abseil-cpp packaging
What about the C++ std version? Abseil / C++14 isn't the same as Abseil / C++17. This applies to other C++ libraries as well.
Bug#888705: abseil-cpp packaging
On Sunday, February 16, 2020, at 10:48 PM +0100, László Böszörményi (GCS) wrote: > In my reading abseil is _not_ guaranteed to have ABI compatibility at > all times. That's why it meant to be a static library collection only. > Forcing it to build shared libraries and have other packages than > libabseil-cpp-dev is no sense I think. Abseil does reserve the right to break ABI at any time, and I can’t speak to their future plans. But I think ABI breakages are unlikely to occur in practice within an LTS release. If we wait and then package an LTS release, I it’ll be completely reasonable to ship .so’s and .a’s. Relatedly, I think we should only package LTS Abseil for Debian. If someone finds a CVE in Abseil, the Abseil team are going to want to backport the fix to LTS releases; they’re not going to want to backport it everywhere else. > @Benjamin: may you ask its developers to use the system gtest libraries > if only ABSL_RUN_TESTS set to ON? Absolutely. I’ll bring it up with them at work tomorrow (today was a US holiday).
Bug#888705: abseil-cpp packaging
Hi Anton, Benjamin, On Sun, Feb 16, 2020 at 9:53 PM Anton Gladky wrote: > feel free to check the package on salsa > and prepare merge request. I don't send a merge request as while it's a very little package we do it very differently. In my reading abseil is _not_ guaranteed to have ABI compatibility at all times. That's why it meant to be a static library collection only. Forcing it to build shared libraries and have other packages than libabseil-cpp-dev is no sense I think. Benjamin may or may not confute this, let's hear him. Then the self-testing can be disabled now, no need to comment that out. Pass the following to dh_auto_configure: -DABSL_RUN_TESTS=OFF \ -DABSL_USE_GOOGLETEST_HEAD=OFF Of course it would be encouraged to have package testing. But it works only with both options set to ON and have internet access to git clone google test to the source tree. @Benjamin: may you ask its developers to use the system gtest libraries if only ABSL_RUN_TESTS set to ON? Regards, Laszlo/GCS
Bug#888705: abseil-cpp packaging
Hi László, feel free to check the package on salsa and prepare merge request. I am able to have a look at it on Thursday. https://salsa.debian.org/debian/abseil Regards Anton László Böszörményi (GCS) schrieb am So., 16. Feb. 2020, 14:14: > Hi Anton, Benjamin, > > I also need abseil, to be precise abseil-cpp (as there's abseil-py as > well). Even contacted by some Google employee if I have time to > package it. Done it and ready to be uploaded. Do you allow me to take > over (and rename the package to abseil-cpp)? Feel free to do it > yourself, but please then do it in the next few days - or let me do > it. > > Regards, > Laszlo/GCS >
Bug#888705: abseil-cpp packaging
Hi Anton, Benjamin, I also need abseil, to be precise abseil-cpp (as there's abseil-py as well). Even contacted by some Google employee if I have time to package it. Done it and ready to be uploaded. Do you allow me to take over (and rename the package to abseil-cpp)? Feel free to do it yourself, but please then do it in the next few days - or let me do it. Regards, Laszlo/GCS
