On 12/29/2014 03:28 AM, Matthias Maier wrote: > > Am 29. Dec 2014, 09:46 schrieb Zac Medico <[email protected]>: > >> In order to implement something like this for Gentoo, we could add >> PROVIDES_EXCLUDE and REQUIRES_EXCLUDE ebuild metadata variables. These >> variables would be used to automatically generate PROVIDES and REQUIRES >> data for binary packages, based on the output of scanelf. The resulting >> PROVIDES and REQUIRES data would simply contain lists of sonames (and >> their corresponding ABIs, for multilib support), which would be used to >> resolve dependencies of binary packages. For example, the resulting >> REQUIRES and PROVIDES data for sys-libs/zlib-1.2.8-r1 built with >> ABI_X86="32 64" might look something like this: >> >> REQUIRES: X86_64: libc.so.6 386: libc.so.6 >> >> PROVIDES: X86_64: libz.so.1 386: libz.so.1 > > So this basically shifts information that is available only after a > merge (and e.g. checked in the preserved-libs feature) to the metadata > of the binary package?
Right. >> For libz, the new PROVIDES_EXCLUDE and REQUIRES_EXCLUDE ebuild variables >> would not be needed. However, for things like python and perl modules, >> these variables would be used to exclude unwanted sonames from REQUIRES >> and PROVIDES. > > How do you intend to use the new metadata in detail? The REQUIRES and PROVIDES data will supplement *DEPEND when resolving dependencies for binary packages. There is some overlap in information between EAPI 5 slot-operator dependencies and REQUIRES/PROVIDES. That's a good thing though, for at least the following 3 reasons: 1) Slot-operator dependencies have not reached 100% adoption, so REQUIRES/PROVIDES soname dependencies will provide information for all packages, regardless of EAPI 5 slot-operator and sub-slot adoption. We can enable PROVIDES_EXCLUDE and REQUIRES_EXCLUDE support for all existing EAPIs, so that migrating packages to a new EAPI will not be a barrier to adoption of PROVIDES_EXCLUDE and REQUIRES_EXCLUDE. 2) Gentoo has a long-standing policy of omitting system packages from dependencies. This leads to unspecified dependencies that might never be specified with slot-operators and sub-slots. For these unspecified dependencies, REQUIRES/PROVIDES soname dependencies will provide dependency information that is not currently available by any other means. 3) REQUIRES/PROVIDES soname dependencies can be used to implement QA checks for dependencies that have been accidentally omitted from *DEPEND variables. > Is it planned to introduce this as another dependency constraint on > binary merges? Will this become a 'hard' or 'soft' dependency? It should be considered a 'hard' dependency. Of course, we could provide an option to make it 'soft', but I do not anticipate a need for this. I think we should add something like the --ignore-built-slot-operator-deps option which will allow the user to explicitly disable rebuilds triggered by soname changes. We could even re-purpose --ignore-built-slot-operator-deps so that it also applies to rebuilds triggered by REQUIRES/PROVIDES data, but it's probably better to use a separate option. > E.g. consider two packages, libfoo-1 (soname libfoo-1) and bar linking > against libfoo. And libfoo-2 (with new soname libfoo-2) emerged > replacing version 1 on the binhost replacing libfoo-1, bar not rebuild. > We now have on the binhost: > libfoo-1 (soname libfoo-1) > libfoo-2 (soname libfoo-2) > bar (dynamically linked to libfoo-1) > > (i) Currently, an update on a client (using the binhost) will upgrade > libfoo-1 to libfoo-2 and the preserved-libs feature will preserve > the necessary shared library libfoo-1 for bar. This behavior will remain the same as long as FEATURES=preserve-libs is enabled on the client. > (ii) What should happen with above metadata? Any libraries preserved by preserve-libs will have no effect on the REQUIRES/PROVIDES data. So, the metadata will be exactly the same, regardless of preserve-libs settings on either the client or server. You can consider preserve-libs as being beyond the scope of the whole proposal, and essentially irrelevant to it. > Shall an update only resolve to libfoo-1 and bar? It depends on the state of the client and its configuration settings. Under the most common usage scenario, the client should be configured to skip the libfoo-2 update until the binhost has rebuilt bar to link against libfoo-2. Of course, the client would also have the option to pull in the libfoo-1 update and rebuild bar itself. If bar uses a slot-operator dependency and libfoo uses sub-slots, the usual slot-operator mechanism would trigger rebuild on the client. If libfoo has not adopted sub-slots, then the PROVIDES data of libfoo-2 and the REQUIRES data of the installed bar instance would serve as an alternative mechanism to trigger rebuild on the client. > (iii) What if libfoo-1 is removed from the binhost? If libfoo-1 is > already merged on the client an update wrt (i) is still possible. > So, shall the preserved-libs feature be used as a fall back then? The client should try to eliminate any preserved-libs at the earliest opportunity. Therefore the most common usage scenario would be essentially the same as described above with respect to (i). > (iv) If bar is rebuild on the binhost - which it should be nowadays - > above metadata information could be used to automatically remerge > bar on the client - but couldn't this already be done with a > slot-operator dependency? Yes, but we should also consider the 3 reasons that I have enumerated above, which make REQUIRES/PROVIDES data extremely useful for plenty of cases that are not currently covered by slot-operator dependencies (and will not be covered for the foreseeable future). > Further, we currently have multiple approaches for ensuring consistency > of dynamic linkage, the "poor man's approach" revdep-rebuild, language > specific update scripts (for python, perl, haskell,...), the > preserved-libs feature and, finally, slot-operators. > This would be another iteration of a solution, this time for binary > merges. Can you give me an example where your proposed approach is the > only way to ensure consistency of dynamic linkage, or is conceptually > cleaner? Conceptually, slot-operators are the cleanest solution. The preserve-libs feature complements slot-operators by temporarily satisfying soname dependencies during the process of updating packages. However, due to the 3 reasons that I have enumerated above, REQUIRES/PROVIDES data is extremely useful for plenty of cases that are not currently covered by slot-operator dependencies (and will not be covered for the foreseeable future). > I, for my part, like the fact that such metadata generation would be > completely automatic. Yes, I think that this proposal represents a very practical way to integrate binary soname dependencies with Gentoo, so that binary package support will be more competitive with binary distros. -- Thanks, Zac
