Bug#340904: [Steve Langasek] Bug#340904: Bug#349318: fixed in xft 2.1.8.2-1
On Thu, Feb 09, 2006 at 06:59:02PM +0800, James Henstridge wrote: Steve Langasek wrote: If Xft is updated to a new version of either of those libraries such that those types are defined differently (altered struct layout, different type sizes, etc), then the app also needs to be updated to the new version. Ok, here's the problem with this argument. Yes, if one of the freetype types that's used in Xft/Xft.h changes, that is an ABI change... *in libXft* -- that's why we care about it! Because it's an ABI change in libXft, the soname of *libXft* should change. With the change of libXft's soname, there's no reason an application that is a consumer of libXft, but *not* a consumer of freetype, should care about libfreetype *at all*.[1] The simple matter of fact is that when you ask for the Xft API in your app (#include Xft/Xft.h), you also get the freetype API at the same time. While some apps don't use it, we can't say that apps don't use those APIs in the general case. You're right, we can't. But as library authors/maintainers, we can say whether we intend to *support* such abuses of the API. You can get a valid prototype of fprintf() on Linux by doing #include sql.h (from UnixODBC) instead of #include stdio.h, too, and we don't know that there isn't anybody relying on that -- but if they do, and they complain about it breaking, they are cordially invited to blow it out their ass. ;) The current policy of pkg-config prevents library authors from making their own choices about whether to support such API abuses on the part of applications. I don't think pkg-config should be imposing policies which prevent library authors from deciding what uses of their headers they intend to support. The current behavior of pkg-config *ensures* that the API abuses you describe will work, because use of Requires means they'll get the ldflags as well as the cflags; that makes pkg-config as it stands pretty suboptimal for anyone who *doesn't* want to get supporting their lib dependencies as part of the public API. On the other hand, there are plenty of cases in which an ABI change in libfreetype will *not* cause an ABI change in libXft. Addition or removal of functions, addition of typedefs, or removal or changing of any typedefs not used in libXft's ABI are all changes that should require an soname change in libfreetype but not an soname change in libXft. The present ABI transition in libfreetype is such a case. But because these applications are being encouraged to link directly to libfreetype, *even though they don't use it*, they have to care about ABI changes that should not affect them. They are using it in conjunction with the Xft API. There is no clear separation between the Xft APIs that use it and those that don't. Of course, an app that only depends on Xft indirectly and doesn't use Xft's APIs doesn't need direct linkage to freetype. They are using data types defined by freetype in conjunction with the Xft API. If they're using function calls from freetype without including freetype headers, they didn't get that idea from Xft's documentation or by reading the Xft headers themselves. Xft's maintainers should not be constrained by a pkg-config design decision to support people who think #include Xft/Xft.h is an acceptable substitute for #include freetype/freetype.h. The net result is that pkg-config's handling of Requires/Requires.private is directly causing churn in response to ABI changes in any indirect library dependencies, where this should be completely unnecessary on GNU/Linux platforms. It increases the chances of segfaults or other failures from loading two different versions of a library into memory, and correspondingly increases the frequency with which binaries need to be rebuilt in response to ABI changes that don't actually concern them. And it does this entirely to support a use case which, as explained above, should not actually exist. The app links to multiple versions of library A problem is quite easy to diagnose, and also makes it clear that an app needs to be rebuilt if the old library version is removed (i.e. the app doesn't start). The app depends on the structure layout of the old version of library A but doesn't directly link to library A is more difficult to catch. The app doesn't depend on the structure layout of the old version of library A. It depends on the structure layout of *library B*, which happens to have *inherited* that structure from the *headers* of library A. It's entirely feasible for a library to do this without actually using library A at runtime; so then you get library B not linking to library A at all, but because it uses pkg-config to resolve the header reference, app C *does* get linked against it for no reason... It sounds like your argument is that Xft shouldn't expose the freetype API. Given that this isn't the case, the direct linkage makes sense. The freetype change in this case might not
Bug#340904: [Steve Langasek] Bug#340904: Bug#349318: fixed in xft 2.1.8.2-1
Steve Langasek wrote: If Xft is updated to a new version of either of those libraries such that those types are defined differently (altered struct layout, different type sizes, etc), then the app also needs to be updated to the new version. Ok, here's the problem with this argument. Yes, if one of the freetype types that's used in Xft/Xft.h changes, that is an ABI change... *in libXft* -- that's why we care about it! Because it's an ABI change in libXft, the soname of *libXft* should change. With the change of libXft's soname, there's no reason an application that is a consumer of libXft, but *not* a consumer of freetype, should care about libfreetype *at all*.[1] The simple matter of fact is that when you ask for the Xft API in your app (#include Xft/Xft.h), you also get the freetype API at the same time. While some apps don't use it, we can't say that apps don't use those APIs in the general case. Until we have a widely used way to express dependencies at that granularity, you need to look at the library level. On the other hand, there are plenty of cases in which an ABI change in libfreetype will *not* cause an ABI change in libXft. Addition or removal of functions, addition of typedefs, or removal or changing of any typedefs not used in libXft's ABI are all changes that should require an soname change in libfreetype but not an soname change in libXft. The present ABI transition in libfreetype is such a case. But because these applications are being encouraged to link directly to libfreetype, *even though they don't use it*, they have to care about ABI changes that should not affect them. They are using it in conjunction with the Xft API. There is no clear separation between the Xft APIs that use it and those that don't. Of course, an app that only depends on Xft indirectly and doesn't use Xft's APIs doesn't need direct linkage to freetype. The net result is that pkg-config's handling of Requires/Requires.private is directly causing churn in response to ABI changes in any indirect library dependencies, where this should be completely unnecessary on GNU/Linux platforms. It increases the chances of segfaults or other failures from loading two different versions of a library into memory, and correspondingly increases the frequency with which binaries need to be rebuilt in response to ABI changes that don't actually concern them. And it does this entirely to support a use case which, as explained above, should not actually exist. The app links to multiple versions of library A problem is quite easy to diagnose, and also makes it clear that an app needs to be rebuilt if the old library version is removed (i.e. the app doesn't start). The app depends on the structure layout of the old version of library A but doesn't directly link to library A is more difficult to catch. It sounds like your argument is that Xft shouldn't expose the freetype API. Given that this isn't the case, the direct linkage makes sense. The freetype change in this case might not break many Xft using apps, but that won't necessarily be the case next time this type of situation occurs. Changes to type definitions _do_ change the ABI. If library A uses library B's types in its ABI, then it's ABI will break if library B changes those types. An app using library A should definitely record the version of library B being used. However: - If library A's soname is correctly changed in sync with library B's, linking application C to library B is redundant. - If library A's soname is not changed when library B's soname changes, pkg-config's behavior does not prevent applications from being broken by the ABI change in libA. At most, it makes it easier to detect such breakage due to double-linkage of libB. I'd argue that this is a useful feature to have, rather than having apps break with no indication of the cause. The cflags of dependencies listed in Requires.private are not included for either dynamic or static modes of pkg-config, so maps to case 3 quite well. Ah, guess I should have looked a bit closer at the behavior in that case. Case 3 was exactly the sort of use case I was trying to model with how Requires.private functions. This is unfortunate, because there are a great many packages that are inheriting dependencies this way on libraries they don't use. While it is true that an ABI change in the dependent library will *sometimes* mean an ABI change in the depending library, this is not always the case. As a result, this behavior of pkg-config causes unnecessary churn for packages depending on libraries in this scenario. In the case of libxft2, that's over 400 packages in Debian that are potentially affected. Sure, unnecessary churn is bad and should be avoided. But you do want to make sure that the churn happens when required. Er, the whole reason I'm objecting to the current behavior is that it *is* unnecessary churn. If the libfreetype ABI change
Bug#340904: [Steve Langasek] Bug#340904: Bug#349318: fixed in xft 2.1.8.2-1
Tollef Fog Heen wrote: Hi, any chance you could comment on this? I'm somewhat inclined to agree with Steve, but you know this stuff better than I do, so I'd value a second opinion. Also, if there's anything unclear or anything, please do mail Steve (and the bug) with me in Cc. [snip] Both the xrender and freetype2 headers are included in Xft/Xft.h, and types from both of these headers are exported in the Xft API -- so any software using Xft.h needs to be able to find these other headers as well. Currently, pkg-config does *not* pass cflags from packages listed in Requires.private when called as pkg-config --cflags foo. The fact that the headers are included makes them part of the _public_ ABI of Xft. There are Xft functions that accept or return types defined in the Xrender and freetype2 headers. If Xft is updated to a new version of either of those libraries such that those types are defined differently (altered struct layout, different type sizes, etc), then the app also needs to be updated to the new version. So far, the maintainer has resisted changing pkg-config to export cflags (which should really be called cppflags...) from Requires.private due to a fallacious argument regarding the nature of private dependencies. There are three real use cases for libraries which depend on other libraries: - the library intentionally exports the API and ABI of its dependencies when linked to, and therefore both the ldflags and the cflags of its dependencies should be exported by pkg-config in all cases[1] Yep, this is the use case addressed by Requires. - the library intentionally includes headers from dependencies in its own headers in order to inherit type definitions, but these definitions are not intended for direct consumption by users of this library alone; therefore pkg-config must export the cflags from dependencies in all cases, but the ldflags only when linking statically Changes to type definitions _do_ change the ABI. If library A uses library B's types in its ABI, then it's ABI will break if library B changes those types. An app using library A should definitely record the version of library B being used. - the library's API includes no headers from its dependencies; pkg-config needs to export the ldflags of private dependencies when statically linking but not when dynamically linking, and should *never* need to export the cflags of these headers. This is the use case for Requires.private. If the dependency doesn't form part of the library's API, then it obviously doesn't require direct linkage and is private. Please note that today, the handling of Requires.private in pkg-config maps to *none* of these cases -- I can't think of a single situation in which cflags of dependencies are needed when statically linking, and not needed when dynamically linking! The cflags of dependencies listed in Requires.private are not included for either dynamic or static modes of pkg-config, so maps to case 3 quite well. Consider the following two .pc files. First foo.pc: Name: foo Description: foo Version: 1.0 Cflags: foo-cflags Libs: foo-libs And bar.pc: Name: bar Description: bar Version: 1.0 Cflags: bar-cflags Libs: bar-libs Requires.private: foo For the dynamic linking case: $ pkg-config --cflags bar bar-cflags $ pkg-config --libs bar bar-libs For the static linking case: $ pkg-config --static --cflags bar bar-cflags $ pkg-config --static --libs bar bar-libs foo-libs Instead, the Requires.private handling is adequate for the third case; handling of Requires is correct for the first case; and there is no combination of options that is appropriate for the second case. I'd argue that the second case is a strawman. If a particular library exposes another in its API/ABI, then it is part of case 1. This is unfortunate, because there are a great many packages that are inheriting dependencies this way on libraries they don't use. While it is true that an ABI change in the dependent library will *sometimes* mean an ABI change in the depending library, this is not always the case. As a result, this behavior of pkg-config causes unnecessary churn for packages depending on libraries in this scenario. In the case of libxft2, that's over 400 packages in Debian that are potentially affected. Sure, unnecessary churn is bad and should be avoided. But you do want to make sure that the churn happens when required. Relying on indirect linkage in a lot of cases just results in more fragile applications. There are cases where it might make sense to move a library to Requires.private even when it is used in some headers, but I don't think this is one of them. Some examples where it would be appropriate include: * Cairo's dependency on glitz, xlib and freetype (this one has been done upstream). Apps only depend on glitz if they include cairo-glitz.h and use those APIs. Apps that don't use those
Bug#340904: [Steve Langasek] Bug#340904: Bug#349318: fixed in xft 2.1.8.2-1
Hi James, Nice to meet you! On Wed, Feb 08, 2006 at 07:28:39PM +0800, James Henstridge wrote: Tollef Fog Heen wrote: Hi, any chance you could comment on this? I'm somewhat inclined to agree with Steve, but you know this stuff better than I do, so I'd value a second opinion. Also, if there's anything unclear or anything, please do mail Steve (and the bug) with me in Cc. [snip] Both the xrender and freetype2 headers are included in Xft/Xft.h, and types from both of these headers are exported in the Xft API -- so any software using Xft.h needs to be able to find these other headers as well. Currently, pkg-config does *not* pass cflags from packages listed in Requires.private when called as pkg-config --cflags foo. The fact that the headers are included makes them part of the _public_ ABI of Xft. There are Xft functions that accept or return types defined in the Xrender and freetype2 headers. If Xft is updated to a new version of either of those libraries such that those types are defined differently (altered struct layout, different type sizes, etc), then the app also needs to be updated to the new version. Ok, here's the problem with this argument. Yes, if one of the freetype types that's used in Xft/Xft.h changes, that is an ABI change... *in libXft* -- that's why we care about it! Because it's an ABI change in libXft, the soname of *libXft* should change. With the change of libXft's soname, there's no reason an application that is a consumer of libXft, but *not* a consumer of freetype, should care about libfreetype *at all*.[1] On the other hand, there are plenty of cases in which an ABI change in libfreetype will *not* cause an ABI change in libXft. Addition or removal of functions, addition of typedefs, or removal or changing of any typedefs not used in libXft's ABI are all changes that should require an soname change in libfreetype but not an soname change in libXft. The present ABI transition in libfreetype is such a case. But because these applications are being encouraged to link directly to libfreetype, *even though they don't use it*, they have to care about ABI changes that should not affect them. The net result is that pkg-config's handling of Requires/Requires.private is directly causing churn in response to ABI changes in any indirect library dependencies, where this should be completely unnecessary on GNU/Linux platforms. It increases the chances of segfaults or other failures from loading two different versions of a library into memory, and correspondingly increases the frequency with which binaries need to be rebuilt in response to ABI changes that don't actually concern them. And it does this entirely to support a use case which, as explained above, should not actually exist. - the library intentionally includes headers from dependencies in its own headers in order to inherit type definitions, but these definitions are not intended for direct consumption by users of this library alone; therefore pkg-config must export the cflags from dependencies in all cases, but the ldflags only when linking statically Changes to type definitions _do_ change the ABI. If library A uses library B's types in its ABI, then it's ABI will break if library B changes those types. An app using library A should definitely record the version of library B being used. However: - If library A's soname is correctly changed in sync with library B's, linking application C to library B is redundant. - If library A's soname is not changed when library B's soname changes, pkg-config's behavior does not prevent applications from being broken by the ABI change in libA. At most, it makes it easier to detect such breakage due to double-linkage of libB. Please note that today, the handling of Requires.private in pkg-config maps to *none* of these cases -- I can't think of a single situation in which cflags of dependencies are needed when statically linking, and not needed when dynamically linking! The cflags of dependencies listed in Requires.private are not included for either dynamic or static modes of pkg-config, so maps to case 3 quite well. Ah, guess I should have looked a bit closer at the behavior in that case. This is unfortunate, because there are a great many packages that are inheriting dependencies this way on libraries they don't use. While it is true that an ABI change in the dependent library will *sometimes* mean an ABI change in the depending library, this is not always the case. As a result, this behavior of pkg-config causes unnecessary churn for packages depending on libraries in this scenario. In the case of libxft2, that's over 400 packages in Debian that are potentially affected. Sure, unnecessary churn is bad and should be avoided. But you do want to make sure that the churn happens when required. Er, the whole reason I'm objecting to the current behavior is that it *is* unnecessary churn. If the libfreetype
