Re: [gentoo-user] Weird portage behaviour
On Sunday 13 Apr 2014 22:48:43 Rick Zero_Chaos Farina wrote: The real answer is almost certainly dynamic-deps. The binary packages store the deps from BUILD time, while when you are installing things live, it allows the current ebuilds to update the deps even for installed packages. That means, after you install a package, if the deps in the ebuild are changed, portage will take that into account for source builds, but binary packages are LOCKED to the deps from the time they were built. Why dynamic-deps aren't available no matter what? Now that's a great question... But I'd only just finished an emerge -eb world on the build host to re-create the packages. Nothing could have changed in the meantime. -- Regards Peter
Re: [gentoo-user] Weird portage behaviour
On Monday 14 Apr 2014 04:41:05 Tom Wijsman wrote: On Sun, 13 Apr 2014 22:00:10 +0100 Peter Humphrey pe...@prh.myzen.co.uk wrote: How is it possible for portage to emerge packages differently when it's installing from packages? The difference is mostly accountable to build-time dependencies; without binpkg they need to be pulled in for the build, with binpkg there is no build thus those build-time dependencies aren't needed. The rest (only a small few) are accountable to the recorded dependencies in the binpkg being different from the dependencies of the ebuilds in the Portage tree; as dynamic dependencies* don't work entirely well with binpkgs and thus there is this small difference. * These propagate ebuild dependency changes to /var/db/pkg/. Here is a list of those that aren't installed along the binpkgs: $ diff -u list listk | sort -k4 | pcregrep -Mv '^\+\N*\n-' | grep '^-\[' ---8 Many thanks for that little bit of magic. I've incorporated it into a simple script to find all the missed packages and list their atoms in a form suitable for piping into emerge. I still think this is bad behaviour by portage. If portage's environment is identical in the two cases and I tell it to emerge -ek, the results should be identical to those if I tell it to emerge -e. I know that -e means empty- tree but that sounds to me exactly like everything, so that's what I expect to get. Thanks again to both of you - at least I can now be sure that I've rebuilt everything when I think I have. I hate to think how many things I've missed in the past. -- Regards Peter
Re: [gentoo-user] Weird portage behaviour
On Mon, 14 Apr 2014 14:43:33 +0100 Peter Humphrey pe...@prh.myzen.co.uk wrote: Many thanks for that little bit of magic. I've incorporated it into a simple script to find all the missed packages and list their atoms in a form suitable for piping into emerge. You might not want to do that; those are _build_ dependencies, not _runtime_ dependencies. You don't need them for the package to run. I still think this is bad behaviour by portage. If portage's environment is identical in the two cases and I tell it to emerge -ek, the results should be identical to those if I tell it to emerge -e. I know that -e means empty- tree but that sounds to me exactly like everything, so that's what I expect to get. It is correct behaviour; `emerge -ek` is similar to `emerge -e ; emerge -c --with-bdeps=n`, where the `emerge -c --with-bdeps=n` unmerges the build dependencies after the packages were build. As they are no longer needed after that part; that is, if you don't intend to rebuild them. Thanks again to both of you - at least I can now be sure that I've rebuilt everything when I think I have. I hate to think how many things I've missed in the past. No problem; be happy to now know you don't need that what misses. :) -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D
Re: [gentoo-user] Weird portage behaviour
On Mon, 14 Apr 2014 14:32:49 +0100, Peter Humphrey wrote: That means, after you install a package, if the deps in the ebuild are changed, portage will take that into account for source builds, but binary packages are LOCKED to the deps from the time they were built. Why dynamic-deps aren't available no matter what? Now that's a great question... But I'd only just finished an emerge -eb world on the build host to re-create the packages. Nothing could have changed in the meantime. Those extra packages are build deps. When you emerge -k, you aren't doing any building, so the build deps aren't needed. They need not even be installed on that box, only on the one doing the building. -- Neil Bothwick Barnum was wrongit's more like every 30 seconds! signature.asc Description: PGP signature
Re: [gentoo-user] Weird portage behaviour
On Monday 14 Apr 2014 16:15:57 Tom Wijsman wrote: On Mon, 14 Apr 2014 14:43:33 +0100 Peter Humphrey pe...@prh.myzen.co.uk wrote: Many thanks for that little bit of magic. I've incorporated it into a simple script to find all the missed packages and list their atoms in a form suitable for piping into emerge. You might not want to do that; those are _build_ dependencies, not _runtime_ dependencies. You don't need them for the package to run. At last, I understand the difference. I might even pipe the list into emerge -cv for kicks. Pity I needed a full-size clue-stick to beat the lesson in :-( Thanks again - also to Neil. -- Regards Peter
Re: [gentoo-user] Weird portage behaviour
On 13/04/2014 23:00, Peter Humphrey wrote: Hello list, me again :-( (Apologies if I'm trying your patience - my own is wearing pretty thin.) I've discovered a problem in portage on the same Atom mini-server as I've mentioned here recently. # emerge -epv world ---8 Total: 329 packages (329 reinstalls), Size of downloads: 595,566 kB # emerge -epvk world ---8 Total: 240 packages (240 reinstalls, 240 binaries), Size of downloads: 0 kB # emerge -epv world list # emerge -epvk world listk # diff list listk ...merely shows the contents of list, then the contents of listk, so clearly the packages are being evaluated in a different order. # find /usr/portage/packages -type f | wc 330 330 17879 (The extra file is /usr/portage/packages/Packages, i.e. not a portage package.) # alias emerge -bash: alias: emerge: not found How is it possible for portage to emerge packages differently when it's installing from packages? A proper answer involves posting the full verbose output of those emerge commands. Without that we can only guess. My guess is that USE is different between your Atom and your build host, so using -k causes a different set of package deps to be pulled in. -- Alan McKinnon alan.mckin...@gmail.com
Re: [gentoo-user] Weird portage behaviour
On Monday 14 Apr 2014 00:30:44 Alan McKinnon wrote: A proper answer involves posting the full verbose output of those emerge commands. OK. Attached; list and listk are as in my first message. Without that we can only guess. My guess is that USE is different between your Atom and your build host, so using -k causes a different set of package deps to be pulled in. I've checked - for the umpteenth time - that package.use, package.keywords and package.mask are identical; also the portage profile and the world file. And here are the differences between the two make.confs (portage.serv is the Atom's etc/portage and /etc/portage is the build host's): # diff portage.serv/make.conf /etc/portage/make.conf 8c8 FEATURES=parallel-fetch userfetch --- FEATURES=buildpkg parallel-fetch userfetch 10c10 ftp_proxy=ftp://localhost:8080/; --- ftp_proxy=ftp://serv.prhnet:8080/; 14c14 http_proxy=http://localhost:8080/; --- http_proxy=http://serv.prhnet:8080/; 17c17 MAKEOPTS=-j3 --- MAKEOPTS=-j5 24c24,25 SYNC=rsync://rsync.uk.gentoo.org/gentoo-portage --- SYNC=rsync://serv.prhnet/gentoo-portage The machine serv.prhnet is a portage mirror for the rest of the LAN (it's the Atom box); besides rsyncd it runs http-replicator listening to port 8080 to serve the distfiles etc. Sometimes I have to think uncomfortably hard to remember what's where, doing what to whom :-( -- Regards Peter list.gz Description: application/gzip listk.gz Description: application/gzip
Re: [gentoo-user] Weird portage behaviour
On Sun, 13 Apr 2014 22:00:10 +0100
Peter Humphrey pe...@prh.myzen.co.uk wrote:
How is it possible for portage to emerge packages differently when
it's installing from packages?
The difference is mostly accountable to build-time dependencies;
without binpkg they need to be pulled in for the build, with binpkg
there is no build thus those build-time dependencies aren't needed.
The rest (only a small few) are accountable to the recorded
dependencies in the binpkg being different from the dependencies of
the ebuilds in the Portage tree; as dynamic dependencies* don't work
entirely well with binpkgs and thus there is this small difference.
* These propagate ebuild dependency changes to /var/db/pkg/.
Here is a list of those that aren't installed along the binpkgs:
$ diff -u list listk | sort -k4 | pcregrep -Mv '^\+\N*\n-' | grep '^-\['
-[ebuild R] app-arch/libarchive-3.1.2-r1:0/13 USE=acl bzip2 e2fsprogs
iconv lzma zlib -expat -lzo -nettle -static-libs -xattr 4,422 kB
-[ebuild R] app-arch/unzip-6.0-r3 USE=bzip2 unicode -natspec 1,345 kB
-[ebuild R] app-text/build-docbook-catalog-1.19.1 5 kB
-[ebuild R] app-text/docbook-xml-dtd-4.1.2-r6:4.1.2 74 kB
-[ebuild R] app-text/docbook-xml-dtd-4.2-r2:4.2 77 kB
-[ebuild R] app-text/docbook-xml-dtd-4.4-r2:4.4 94 kB
-[ebuild R] app-text/docbook-xsl-stylesheets-1.78.0 USE=-ruby 4,894 kB
-[ebuild R] app-text/docbook2X-0.8.8-r2 USE={-test} 707 kB
-[ebuild R] app-text/openjade-1.3.2-r6 USE=-static-libs 874 kB
-[ebuild R] app-text/opensp-1.5.2-r3 USE=nls -doc -static-libs {-test}
1,486 kB
-[ebuild R] app-text/po4a-0.42 USE={-test} 1,957 kB
-[ebuild R] app-text/sgml-common-0.6.3-r5 126 kB
-[ebuild R] dev-lang/nasm-2.10.07 USE=-doc 665 kB
-[ebuild R] dev-libs/libgcrypt-1.5.3:0/11 USE=-static-libs 1,474 kB
-[ebuild R] dev-libs/libgpg-error-1.12 USE=nls -common-lisp
-static-libs 478 kB
-[ebuild R] dev-libs/libxslt-1.1.28-r1 USE=crypt -debug -python
-static-libs PYTHON_TARGETS=python2_7 -python2_6 3,356 kB
-[ebuild R] dev-perl/Locale-gettext-1.50.0 8 kB
-[ebuild R] dev-perl/SGMLSpm-1.03-r7 92 kB
-[ebuild R] dev-perl/Text-CharWidth-0.40.0 9 kB
-[ebuild R] dev-perl/Text-WrapI18N-0.60.0 4 kB
-[ebuild R] dev-perl/XML-LibXML-2.1.400 USE={-test} 442 kB
-[ebuild R] dev-perl/XML-NamespaceSupport-1.110.0 51 kB
-[ebuild R] dev-perl/XML-Parser-2.410.0 233 kB
-[ebuild R] dev-perl/XML-SAX-0.990.0 46 kB
-[ebuild R] dev-perl/XML-SAX-Base-1.80.0 28 kB
-[ebuild R] dev-util/cmake-2.8.12.2 USE=ncurses -emacs -qt4 (-qt5)
{-test} 5,927 kB
-[ebuild R] dev-util/gperf-3.0.4 961 kB
-[ebuild R] dev-util/gtk-doc-am-1.20 629 kB
-[ebuild R] dev-util/intltool-0.50.2-r1 186 kB
-[ebuild R] dev-util/pkgconfig-0.28 USE=-hardened -internal-glib 0 kB
-[ebuild R] perl-core/Archive-Tar-1.900.0 57 kB
-[ebuild R] perl-core/CPAN-Meta-2.120.921 75 kB
-[ebuild R] perl-core/CPAN-Meta-Requirements-2.122.0 21 kB
-[ebuild R] perl-core/CPAN-Meta-YAML-0.8.0 31 kB
-[ebuild R] perl-core/ExtUtils-CBuilder-0.280.205 34 kB
-[ebuild R] perl-core/ExtUtils-Install-1.540.0 66 kB
-[ebuild R] perl-core/ExtUtils-MakeMaker-6.640.0 412 kB
-[ebuild R] perl-core/ExtUtils-Manifest-1.610.0 29 kB
-[ebuild R] perl-core/ExtUtils-ParseXS-3.180.0 62 kB
-[ebuild R] perl-core/IPC-Cmd-0.780.0 27 kB
-[ebuild R] perl-core/Module-Build-0.400.300 USE={-test} 301 kB
-[ebuild R] perl-core/Module-CoreList-2.840.0 59 kB
-[ebuild R] perl-core/Module-Load-0.240.0 6 kB
-[ebuild R] perl-core/Module-Load-Conditional-0.540.0 13 kB
-[ebuild R] perl-core/Module-Metadata-1.0.11 27 kB
-[ebuild R] perl-core/Params-Check-0.360.0 12 kB
-[ebuild R] perl-core/Parse-CPAN-Meta-1.440.400 8 kB
-[ebuild R] perl-core/Test-Harness-3.260.0 288 kB
-[ebuild R] perl-core/version-0.990.100 105 kB
-[ebuild R] sys-apps/ed-1.6 85 kB
-[ebuild R] sys-apps/help2man-1.43.3 USE=nls 195 kB
-[ebuild R] sys-devel/autoconf-2.69:2.5 USE=-emacs -multislot 1,187 kB
-[ebuild R] sys-devel/autoconf-wrapper-13 0 kB
-[ebuild R] sys-devel/automake-1.11.6:1.11 1,068 kB
-[ebuild R] sys-devel/automake-1.12.6:1.12 1,368 kB
-[ebuild R] sys-devel/automake-1.13.4:1.13 1,416 kB
-[ebuild R] sys-devel/automake-wrapper-9 0 kB
-[ebuild R] sys-devel/bison-2.4.3 USE=nls -static 1,614 kB
-[ebuild R] sys-devel/flex-2.5.37 USE=nls -static {-test} 1,273 kB
-[ebuild R] sys-devel/m4-1.4.16 USE=-examples 982 kB
-[ebuild R] virtual/perl-Archive-Tar-1.900.0 0 kB
-[ebuild R] virtual/perl-CPAN-Meta-2.120.921 0 kB
-[ebuild R] virtual/perl-CPAN-Meta-Requirements-2.122.0 0 kB
-[ebuild R] virtual/perl-CPAN-Meta-YAML-0.8.0 0 kB
-[ebuild R] virtual/perl-ExtUtils-CBuilder-0.280.205
Re: [gentoo-user] Weird portage behaviour
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/13/2014 08:20 PM, Peter Humphrey wrote: On Monday 14 Apr 2014 00:30:44 Alan McKinnon wrote: A proper answer involves posting the full verbose output of those emerge commands. OK. Attached; list and listk are as in my first message. Without that we can only guess. My guess is that USE is different between your Atom and your build host, so using -k causes a different set of package deps to be pulled in. I've checked - for the umpteenth time - that package.use, package.keywords and package.mask are identical; also the portage profile and the world file. And here are the differences between the two make.confs (portage.serv is the Atom's etc/portage and /etc/portage is the build host's): The real answer is almost certainly dynamic-deps. The binary packages store the deps from BUILD time, while when you are installing things live, it allows the current ebuilds to update the deps even for installed packages. That means, after you install a package, if the deps in the ebuild are changed, portage will take that into account for source builds, but binary packages are LOCKED to the deps from the time they were built. Why dynamic-deps aren't available no matter what? Now that's a great question... - -Zero # diff portage.serv/make.conf /etc/portage/make.conf 8c8 FEATURES=parallel-fetch userfetch --- FEATURES=buildpkg parallel-fetch userfetch 10c10 ftp_proxy=ftp://localhost:8080/; --- ftp_proxy=ftp://serv.prhnet:8080/; 14c14 http_proxy=http://localhost:8080/; --- http_proxy=http://serv.prhnet:8080/; 17c17 MAKEOPTS=-j3 --- MAKEOPTS=-j5 24c24,25 SYNC=rsync://rsync.uk.gentoo.org/gentoo-portage --- SYNC=rsync://serv.prhnet/gentoo-portage The machine serv.prhnet is a portage mirror for the rest of the LAN (it's the Atom box); besides rsyncd it runs http-replicator listening to port 8080 to serve the distfiles etc. Sometimes I have to think uncomfortably hard to remember what's where, doing what to whom :-( -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTS0yLAAoJEKXdFCfdEflKOQIP/3RSALd4/ArvT9nFN3IzPjHT zVw9iimHMTOOeec4c4dhw6URCMNlOzeA2XL4cAdhiPM+ufHdJUWuR67tu78qD44S /p9UjBppBU2pTjpD/b0XcXXoER+D0uLmcnOzV4AZ0aTxVic1mYGR2PUnzFmZGYYd /+EM8ohTyiDyS9AANSzkqZsMS0NQpfmvVy3aqL6CjZt7zG92IOAYfXFysedh4Y7p aOordofyUXdMsooiajM9KKF5kgyUW/S0OuS6yAncWAgb38EqXlrwgbqQlLbvZaDp WSYkQy9HGfRxgNP3K2kEWJOrfjxQz/NoFwffaaPcgFPRRQ1SN9g7LkheZtCYhhV+ VAUXDjMk3NswcS36hiVqPqb87/XFlowkuqyjP/xCD0GdotY8CYblhr3lpFXZ/ucL VE6IpHRXd+JknK3h2N57MIbkxReW3KKBJ6TWnZBcVYsWcnu0gTf40babpzYvmVz6 dBAZ9ID/8BAkIUCdLsQcGPNcl1VUOMcJC19S7RZLg5sbX6pwvwlGmFuz5Kqchx3L W53e7js0c1PnWp505vPGnvfmXuYGQapRJ1JUQy6ERsST3y7DeWYn3gR0sSnp4VX2 9ymJHUytULUX51fc9YUbWTJfrG02J3j3kycF6PRz6M9BrBBXnNsQQ4HJaaBqTfrr 9l9gCfTtQAJcRuF21WvW =9tei -END PGP SIGNATURE-
