Re: [gentoo-dev] Re: don't rely on dynamic deps
On Wed, 30 Jul 2014 15:38:43 +0300 Samuli Suominen ssuomi...@gentoo.org wrote: That's what I've been trying to point out, people are seriously suggesting disabling dynamic deps for race conditions It's like fixing one audio driver in the kernel by deleting whole ALSA block It is more like fixing multiple broken audio drivers; if there is only a single problem with dynamic deps, it wouldn't receive much attention. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sun, 27 Jul 2014 05:30:26 +1000 Michael Palimaka kensing...@gentoo.org wrote: On 07/27/2014 05:21 AM, Tom Wijsman wrote: On Sun, 27 Jul 2014 03:12:07 +1000 Michael Palimaka kensing...@gentoo.org wrote: On 07/26/2014 07:59 AM, Tom Wijsman wrote: On Wed, 23 Jul 2014 22:14:41 +1000 Michael Palimaka kensing...@gentoo.org wrote: Shouldn't we strive to avoid the unnecessary rebuilds in the first place? Doing updates on your schedule only avoids the symptom, not the problem. We should strive to do both; cause less rebuilds, update less often. It is comparable to flooding on IRC channels; if you send much more messages, you are much more likely to experience a kick and/or a ban. It is easier not to flood than to convince people there is no problem with you flooding the channel; out of all the IRC channels I know of, I've only come across one where they don't mind pasted long code blocks but that's mostly because of the lack of active moderation and people. (With flooding as updating and kick/ban as rebuilds) Each person should update at a frequency that suits them. Recommending to update every $period is not a valid solution to unnecessary rebuilds. The more one floods, the more one accepts kicks and/or bans; expected. How about just not causing the problem in the first place? :-) That's the ideal, no revision bumps needed at all; though, the lack of resources doesn't make that possible. Attempts to do it stall the introduction of the ebuild; so, that's why we release and revbump it. This story goes further upstream; if they would list deps right, we wouldn't need to revbump. So; if we want to fix the cause, we would need to fix it upstream although they experience a lack of resources. TL;DR: With the water tap wide open, we'll keep mopping. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 14:25:23 + (UTC) Martin Vaeth mar...@mvath.de wrote: Tom Wijsman tom...@gentoo.org wrote: Michael Palimaka kensing...@gentoo.org wrote: What a great way to kill the distro. I can already heat my house with the number of unnecessary rebuilds Do you upgrade @world every hour and thus have it cause excessive heat? If I upgrade every X weeks they become much more cool and necessary... One of the main advantages of gentoo is the flowing upgrade, especially since this can only be very poorly emulated by a binary distro. If you really suggest that the user waits one month and then recompiles the whole installation, you give up this advantage of gentoo: The user is not up-to-date for a long time, and moreover, then needs practically a full reinstall; both are things which he wants to avoid and why perhaps he has chosen gentoo in the first place. At least, for me it is the case: if I have to reinstall all packages every months - and even have delay in security updates for a month - I will certainly switch the distribution. I guess many others think similarly. Simple equation: The more frequent the user updates, the more frequent the user will experience the minor inconveniences by upstream and distribution maintainers. Otherwise we'd be using a -only system. Dynamic deps, as well as rev bumps, alter this equation; the problem with that is that such alterations don't come free and without flaws, which is essentially where you get to reconsider how you alter it. In a similar way the user has to reconsider whether updating less is acceptable compared to compiling an occasional inconvenience. Choosing between a stable and non-stable tree is a big gap of difference in convenience, choosing how often you update is fine tuning. To get the idea: Upstream released W.X.Y.Z+1; it was only yesterday I've compiled W.X.Y.Z, turns out the difference is not so important. Agreed that this can very well be an important security update; but if you go back to the equation, that still is a minor inconvenience. PS: Not suggesting 1 month; but rather that updating not enough, or too much, can make one experience serious effects that such choices imply. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Samuli Suominen wrote: let users do the rebuild (which is the obvious answer to the output you posted) Reality check time, Samuli. Unless emerge says Your dependencies are b0rk, please rebuild $P to fix it. that answer is nowhere near obvious. Watch out with the tunnel vision. //Peter
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 7/30/14, 7:36 AM, Samuli Suominen wrote: If it's 2-3 packages out of ~300, I'd rather pick them out than revision bump all ~300 for the 2-3. Or not pick them out at all and let users do the rebuild (which is the obvious answer to the output you posted) Peter Stuge pointed it out already, but I also wanted to say rebuilding the affected packages is not obvious to me either. Paweł !!! Multiple package instances within a single package slot have been pulled !!! into the dependency graph, resulting in a slot conflict: virtual/udev:0 (virtual/udev-208-r2::gentoo, installed) pulled in by =virtual/udev-171:0/0=[gudev] required by (media-video/cheese-3.12.2::gentoo, installed) virtual/udev:0/0=[gudev] required by (x11-misc/colord-1.2.1::gentoo, installed) (virtual/udev-215::gentoo, ebuild scheduled for merge) pulled in by =virtual/udev-215 required by (games-util/xboxdrv-0.8.5-r1::gentoo, installed) (and 22 more with the same problem) signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Wed, Jul 30, 2014 at 3:38 AM, Paweł Hajdan, Jr. phajdan...@gentoo.org wrote: On 7/30/14, 7:36 AM, Samuli Suominen wrote: If it's 2-3 packages out of ~300, I'd rather pick them out than revision bump all ~300 for the 2-3. Or not pick them out at all and let users do the rebuild (which is the obvious answer to the output you posted) Peter Stuge pointed it out already, but I also wanted to say rebuilding the affected packages is not obvious to me either. Sure, but this seems more like a portage bug (or at least a portage output bug) rather than a fundamental issue. After all, there was no true block - just a need for a rebuild. I heard prerm as a reason why dynamic deps can break (especially with slot operator deps, though obviously it also breaks for non-slot-operator deps that should be expressed as such), though as has been pointed out those will break unless we unmerge and remerge all reverse-deps on every upgrade. Are there other issues. To be honest I was expecting a plethora of issues that can go wrong with dynamic deps, but so far I'm hearing something like 2-3, and if that really is all that there is then this may be a solvable issue. Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Wed, 30 Jul 2014 07:18:22 -0400 Rich Freeman ri...@gentoo.org wrote: Sure, but this seems more like a portage bug (or at least a portage output bug) rather than a fundamental issue. After all, there was no true block - just a need for a rebuild. It's often not possible to produce a decent error message, given the limited information that ebuilds supply and the unnecessary pseudocleverness they employ to do it. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 30/07/14 14:18, Rich Freeman wrote: On Wed, Jul 30, 2014 at 3:38 AM, Paweł Hajdan, Jr. phajdan...@gentoo.org wrote: On 7/30/14, 7:36 AM, Samuli Suominen wrote: If it's 2-3 packages out of ~300, I'd rather pick them out than revision bump all ~300 for the 2-3. Or not pick them out at all and let users do the rebuild (which is the obvious answer to the output you posted) Peter Stuge pointed it out already, but I also wanted to say rebuilding the affected packages is not obvious to me either. Sure, but this seems more like a portage bug (or at least a portage output bug) rather than a fundamental issue. After all, there was no true block - just a need for a rebuild. I heard prerm as a reason why dynamic deps can break (especially with slot operator deps, though obviously it also breaks for non-slot-operator deps that should be expressed as such), though as has been pointed out those will break unless we unmerge and remerge all reverse-deps on every upgrade. Are there other issues. To be honest I was expecting a plethora of issues that can go wrong with dynamic deps, but so far I'm hearing something like 2-3, and if that really is all that there is then this may be a solvable issue. That's what I've been trying to point out, people are seriously suggesting disabling dynamic deps for race conditions It's like fixing one audio driver in the kernel by deleting whole ALSA block :-( - Samuli
Re: [gentoo-dev] Re: don't rely on dynamic deps
Martin Vaeth wrote:
The user's vardb could then automatically receive the last state of
the ebuild, before it was removed.
It doesn't help reliably, either, since the last state of the ebuild,
before it was removed, will be outdated at some point, too.
Sorry, I don't see how. Can you give an example? Thanks!
1. foo depends on bar
2. user installs foo
3. foo is treecleaned
4. bar ebuild is replaced by the pair bar-base and bar-gui to
allow for finer dependency. All ebuilds in the tree take
care about this new structure (possibly with revbumps).
Of course, the dependency for an already removed package
is not fixed.
5. bar{-base,gui} gets several upgrades.
6. foo on user's system still blocks bar from being removed
and thus blocks the installation of bar-base and bar-gui
forever.
Thanks for spelling it out!
This suggests another data model bug to me: that dependencies belong
to the dependent packages, rather than to dependency providers.
What I mean is that in the above example, bar knows that bar has
turned into bar-{base,gui}, but foo doesn't.
Note that 4. cannot be replaced by any pushing mechanism,
since only the maintainer of the ebuild can know which is
the right dependency for the new tree structure. Such a
maintainer does not exist for treecleaned packages (which
is the purpose of treecleaning, after all...)
If the user updates their package database things would still work
if 4 modifies the effective dependencies for foo to reflect the new
reality of bar - bar-{base,gui}.
foo is not being maintained, but bar is. It might be incorrect to say
that foo depends on both bar-base and -gui (foo might just need -base)
but it is perfectly safe to automatically make the most pessimistic
assumption when upgrading the outdated dependency on bar in all
installed-but-outdated-ebuilds.
The code required for this would even allow to partially automate
dependency changes like bar - bar-{base,gui} across the tree.
Maintainers could get notified when a package they depend on change,
and the safe default is to just roll along. Less dev work! \o/
The more I think about dependencies the more convinced I am that
dependency information must be versioned, ie. dependencies only
matter in the context of the particular package database snapshot
they came from, and that installed dependencies must be kept up to
date as the package database evolves.
Otherwise there's just a pile of heuristics, which throw people,
which I guess is why dynamic-deps cause problems and ire.
Rich Freeman wrote:
This is really the crux of these sorts of issues. It doesn't matter
if dependencies are static or dynamic - if you hang onto orphans then
you're going to have cruft in your vdb which is going to lead to
blockers of some kind eventually.
I think the vdb can and should be updated according to portage changes.
Someone just needs to code it. ;)
//Peter
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 29 July 2014 19:33, Peter Stuge pe...@stuge.se wrote: I think the vdb can and should be updated according to portage changes. Someone just needs to code it. ;) And an appropriate method for doing this must be decided upon. And that part entails 70% of the discussion dispute :) -- Kent *KENTNL* - https://metacpan.org/author/KENTNL
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Tue, Jul 29, 2014 at 3:33 AM, Peter Stuge pe...@stuge.se wrote: Rich Freeman wrote: This is really the crux of these sorts of issues. It doesn't matter if dependencies are static or dynamic - if you hang onto orphans then you're going to have cruft in your vdb which is going to lead to blockers of some kind eventually. I think the vdb can and should be updated according to portage changes. Someone just needs to code it. ;) So, I'll agree that vdb should change when portage changes (and we should manage portage changes so that this can be done reliably), but we're talking about orphans here. Portage is only going to get one side of the story when dealing with an orphan. In your example of a package split the original package went away, and perhaps with some mechanism we could get portage to update all former dependencies to use both sides of the split. But, how about virtualization of a package? Your orphan depends on non-virtual udev, but now you want to install systemd which of course blocks udev. Maybe your package really does depend on the real udev (probably not a good example here - think ffmpeg instead perhaps), or maybe it can use the virtual. Just telling portage that the virtual replacement has been made is one problem, but figuring out whether to use it is going to be a wild guess for a PM. And there are likely other variations as well. Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
В Sun, 27 Jul 2014 14:42:24 +0300 Samuli Suominen ssuomi...@gentoo.org пишет: On 26/07/14 15:49, Ciaran McCreesh wrote: On Sat, 26 Jul 2014 12:41:16 + (UTC) Martin Vaeth mar...@mvath.de wrote: hasufell hasuf...@gentoo.org wrote: Dynamics deps are already broken, not consistently enabled (e.g. when subslots are in use) Just to make it clear: No, dynamic deps are not broken. Yes they are. We just succesfully converted ~300 ebuilds in tree without revision bumps from virtual/udev[gudev,introspection,static-libs] to virtual/libudev and virtual/libgudev Tested it on multiple boxes, went fine. Nobody has filed bugs at http://bugs.gentoo.org/, nobody has filed a single forums post, nobody has said anything at #gentoo, Freenode Only one person said he had to manually build 2 GNOME related packages, simple-scan and something else As Michał already noted in this thread, dynamic deps does not play nice with slot operators. So I had the same problem with 2 GNOME related packages: !!! Multiple package instances within a single package slot have been pulled !!! into the dependency graph, resulting in a slot conflict: virtual/udev:0 (virtual/udev-208-r2::gentoo, installed) pulled in by =virtual/udev-171:0/0=[gudev] required by (media-video/cheese-3.12.2::gentoo, installed) virtual/udev:0/0=[gudev] required by (x11-misc/colord-1.2.1::gentoo, installed) (virtual/udev-215::gentoo, ebuild scheduled for merge) pulled in by =virtual/udev-215 required by (games-util/xboxdrv-0.8.5-r1::gentoo, installed) (and 22 more with the same problem) So, broken? Far from it. More like essential feature. People have just listed some known races dynamic deps have, and I take those races anyday over an regression that causes endless rebuilding... - Samuli -- Alexander Tsoy
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 30/07/14 07:45, Alexander Tsoy wrote: В Sun, 27 Jul 2014 14:42:24 +0300 Samuli Suominen ssuomi...@gentoo.org пишет: On 26/07/14 15:49, Ciaran McCreesh wrote: On Sat, 26 Jul 2014 12:41:16 + (UTC) Martin Vaeth mar...@mvath.de wrote: hasufell hasuf...@gentoo.org wrote: Dynamics deps are already broken, not consistently enabled (e.g. when subslots are in use) Just to make it clear: No, dynamic deps are not broken. Yes they are. We just succesfully converted ~300 ebuilds in tree without revision bumps from virtual/udev[gudev,introspection,static-libs] to virtual/libudev and virtual/libgudev Tested it on multiple boxes, went fine. Nobody has filed bugs at http://bugs.gentoo.org/, nobody has filed a single forums post, nobody has said anything at #gentoo, Freenode Only one person said he had to manually build 2 GNOME related packages, simple-scan and something else As Michał already noted in this thread, dynamic deps does not play nice with slot operators. So I had the same problem with 2 GNOME related packages: I've revision bumped x11-misc/colord and media-gfx/simple-scan because of this reason, I'll do media-video/cheese as well If it's 2-3 packages out of ~300, I'd rather pick them out than revision bump all ~300 for the 2-3. Or not pick them out at all and let users do the rebuild (which is the obvious answer to the output you posted) !!! Multiple package instances within a single package slot have been pulled !!! into the dependency graph, resulting in a slot conflict: virtual/udev:0 (virtual/udev-208-r2::gentoo, installed) pulled in by =virtual/udev-171:0/0=[gudev] required by (media-video/cheese-3.12.2::gentoo, installed) virtual/udev:0/0=[gudev] required by (x11-misc/colord-1.2.1::gentoo, installed) (virtual/udev-215::gentoo, ebuild scheduled for merge) pulled in by =virtual/udev-215 required by (games-util/xboxdrv-0.8.5-r1::gentoo, installed) (and 22 more with the same problem)
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 27/07/14 16:47, Michał Górny wrote:
Dnia 2014-07-27, o godz. 14:42:24
Samuli Suominen ssuomi...@gentoo.org napisał(a):
On 26/07/14 15:49, Ciaran McCreesh wrote:
On Sat, 26 Jul 2014 12:41:16 + (UTC)
Martin Vaeth mar...@mvath.de wrote:
hasufell hasuf...@gentoo.org wrote:
Dynamics deps are already broken, not consistently enabled (e.g.
when subslots are in use)
Just to make it clear: No, dynamic deps are not broken.
Yes they are.
We just succesfully converted ~300 ebuilds in tree without revision
bumps from virtual/udev[gudev,introspection,static-libs]
to virtual/libudev and virtual/libgudev
Tested it on multiple boxes, went fine.
How did you exactly test them?
That you could still emerge packages, even world, without Portage
complaining about unsatisfied
deps
Did you at least bother checking if
portage actually uses the deps you added?
When you `cd /var/db/pkg` and run `grep virtual/udev.*gudev
*/*/*.ebuild`, you can indeed still see some:
app-cdr/xfburn-0.5.2/xfburn-0.5.2.ebuild:udev? ( virtual/udev[gudev] )
gnome-base/gvfs-1.20.2/gvfs-1.20.2.ebuild:virtual/udev[gudev] )
media-gfx/gimp-2.8.10-r1/gimp-2.8.10-r1.ebuild:udev? (
virtual/udev[gudev] )
sys-fs/udisks-1.0.5-r1/udisks-1.0.5-r1.ebuild:=virtual/udev-208[gudev]
sys-fs/udisks-2.1.3/udisks-2.1.3.ebuild:
=virtual/udev-${UDEV_VERSION}[gudev]
virtual/libgudev-208/libgudev-208.ebuild:# $Header:
/var/cvsroot/gentoo-x86/virtual/libgudev/libgudev-208.ebuild,v 1.7
2014/06/18 20:55:21 mgorny Exp $
xfce-extra/thunar-volman-0.8.0/thunar-volman-0.8.0.ebuild:
virtual/udev[gudev]
But if you try to emerge these packages, like, for example:
$ sudo emerge -pv xfburn thunar-volman
These are the packages that would be merged, in reverse order:
Calculating dependencies... done!
[ebuild R] xfce-extra/thunar-volman-0.8.0 USE=libnotify -debug 0 kB
[ebuild R] app-cdr/xfburn-0.5.2 USE=udev -debug -gstreamer 0 kB
Portage is using the fresh copies from gentoo-x86
I'm _not_ a Portage, the package manager, developer, so I'd really
appericiate some *examples* where this would become a problem, binary
packages is known one, we have lived with that problem since forever, so
something else, please.
For everything I do with Portage, this is fine with me, and I expect
it's fine for the vast majority of users as well.
And this majority of users won't appericiate the suggested solution of
lets always revision bump, despite of triggering rebuild for everyone
To clarify, I know dynamic deps is not perfect, but it's the best
solution we have implemented to avoid the rebuilding problem, and long
as that's true... And seems like you, yourself, have thought about the
same issue:
http://bugs.gentoo.org/show_bug.cgi?id=486778
As in, you can't skip that bug, and go directly to
http://bugs.gentoo.org/show_bug.cgi?id=516612
- Samuli
(Excuse my grammar, woke up five minutes ago, to make some coffee now...)
Re: [gentoo-dev] Re: don't rely on dynamic deps
Martin Vaeth wrote: The user has to put a corrected ebuild into his overlay and must reemerge the package (currently, the latter could be skipped with dynamic deps). In fact, no matter whether you have static or dynamic deps, this is the only way to cleanly avoid the problems if you want to keep a package installed which is not maintained anymore: *You* must maintain it. There simply is no magic which can avoid this. It could be avoided if portage tree sync applied each tree change locally rather than jump from old to new. There was a delta idea discussed a year or so ago in connection with some git discussions. The user's vardb could then automatically receive the last state of the ebuild, before it was removed. That's no small change though. //Peter
Re: [gentoo-dev] Re: don't rely on dynamic deps
Martin Vaeth wrote: The user's vardb could then automatically receive the last state of the ebuild, before it was removed. It doesn't help reliably, either, since the last state of the ebuild, before it was removed, will be outdated at some point, too. Sorry, I don't see how. Can you give an example? Thanks! //Peter
Re: [gentoo-dev] Re: don't rely on dynamic deps
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On 26/07/14 11:22 AM, Ciaran McCreesh wrote:
Let's start with the easiest issue: please point us all to the
place where you proved how dynamic dependencies still work in the
face of ebuild removals. Your solution to this problem will be of
great benefit to all of us.
This is something I personally don't understand. If an ebuild for a
package installed on the system has been removed from the tree, but
newer and/or older ebuilds exist in the tree, then the installed
package can #1 only be trusted in accordance with the ebuild copy
enbedded in VDB (that i get), BUT, #2 should be forced to either
upgrade or downgrade so that it matches what *is* in the tree anyhow,
and that's done via a standard ${PV} comparison that should happen
regardless of whether static or dynamic deps methods are in place.
IMO, if currently-installed versions of packages are satisfying
dependencies after those packages have been removed from the tree, I
don't see this as being particularly valid anyhow. Sure, end-users
should be able to force this using masks or whatnot in the particular
cases they need to do this, but i don't think this should be in any
way a default behaviour, should it?? Ebuilds are removed for a reason...
-BEGIN PGP SIGNATURE-
Version: GnuPG v2
iF4EAREIAAYFAlPWXncACgkQ2ugaI38ACPBWLQEAp3sB8lfdZ8FYmXRsxNy6SlHE
AR40+p+/x6J5+m4D618BAK4XKG64W92WFWne2rn3cDtdKuoQ+wwN2RBw066MoPJQ
=TyGx
-END PGP SIGNATURE-
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Mon, 28 Jul 2014 10:30:15 -0400
Ian Stakenvicius a...@gentoo.org wrote:
On 26/07/14 11:22 AM, Ciaran McCreesh wrote:
Let's start with the easiest issue: please point us all to the
place where you proved how dynamic dependencies still work in the
face of ebuild removals. Your solution to this problem will be of
great benefit to all of us.
This is something I personally don't understand. If an ebuild for a
package installed on the system has been removed from the tree, but
newer and/or older ebuilds exist in the tree, then the installed
package can #1 only be trusted in accordance with the ebuild copy
enbedded in VDB (that i get), BUT, #2 should be forced to either
upgrade or downgrade so that it matches what *is* in the tree anyhow,
and that's done via a standard ${PV} comparison that should happen
regardless of whether static or dynamic deps methods are in place.
But you can't run pkg_prerm unless a package's dependencies are
satisfied. How do you know what those dependencies are, if you don't
use VDB and if the ebuild isn't there?
(This is a real issue: see the botched ruby-config switch.)
--
Ciaran McCreesh
signature.asc
Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256
On 28/07/14 10:43 AM, Ciaran McCreesh wrote:
On Mon, 28 Jul 2014 10:30:15 -0400 Ian Stakenvicius
a...@gentoo.org wrote:
On 26/07/14 11:22 AM, Ciaran McCreesh wrote:
Let's start with the easiest issue: please point us all to the
place where you proved how dynamic dependencies still work in
the face of ebuild removals. Your solution to this problem will
be of great benefit to all of us.
This is something I personally don't understand. If an ebuild
for a package installed on the system has been removed from the
tree, but newer and/or older ebuilds exist in the tree, then the
installed package can #1 only be trusted in accordance with the
ebuild copy enbedded in VDB (that i get), BUT, #2 should be
forced to either upgrade or downgrade so that it matches what
*is* in the tree anyhow, and that's done via a standard ${PV}
comparison that should happen regardless of whether static or
dynamic deps methods are in place.
But you can't run pkg_prerm unless a package's dependencies are
satisfied. How do you know what those dependencies are, if you
don't use VDB and if the ebuild isn't there?
(This is a real issue: see the botched ruby-config switch.)
Yes, that's an issue -- I thought the pkg-*rm case had to do with the
ebuild's phase definition itself being (or not being) updated, I
haddn't thought about it in terms of dependencies being unsatisfied.
Keeping every single dependency around and valid just so that pkg_*rm
can completely cleanly seems like so much overkill, though..
Unfortunately the only way I see around that would be to have some
form of reduced subset, which would mean yet-another-*DEPEND, and we
all know that's not going to happen..
I wonder if there would be a way to somehow add restrictions to
pkg_*rm phases s.t. either REPLACED_BY_VERSION's dependencies must be
satisfied at the time the phases are run, or REPLACED_BY_VERSION is
empty and the in-VDB ones must be satisfied. It'll be a pain for
dev's to make sure their stuff works within these restrictions, and
the likeliness of repoman being able to enforce any sort of QA on this
probably so close to zero it doesn't matter, but i'm not seeing
another way.
(outside of forcing the in-vdb deps to always remain satisfied, of
course; however i think the dependencies-get-upgraded-first approach
which is generally necessary for all but PDEPEND can still cause
breakage here too, no??)
-BEGIN PGP SIGNATURE-
Version: GnuPG v2
iF4EAREIAAYFAlPWbpMACgkQ2ugaI38ACPBkdwEAsPg3Gu4I3LuXfBuvrxfGJ3D6
sv/ZOROo0a0xPzQ8IZgA/icJDURR+a0mrvT2fwSzXNfd+azvaaKxjNOcRPHOcSYE
=YElO
-END PGP SIGNATURE-
Re: [gentoo-dev] Re: don't rely on dynamic deps
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 27/07/14 08:04 AM, Rich Freeman wrote: On Sun, Jul 27, 2014 at 6:43 AM, Kent Fredric kentfred...@gmail.com wrote: In a no dynamic deps, period scenario, this just strikes me as 2 flavours of the same weirdness, -r2 and -r1.1 are just equally weird choices to make if the ebuild itself doesn't change at all. You have a good point here. With this proposal we have three kinds of ebuild changes: 1. Those that do not involve any revbump. 2. Those with a minor revbump (ie -r1 - -r1.1). 3. Those with a normal revbump (ie -r1 - -r2). What is the purpose of allowing the first? Presumably that should be used in even more limited circumstances than a minor revbump, so why allow it at all? Why not just make an in-place change equivalent to a minor revbump and ditch the minor revbump entirely? Devs would still need to distinguish between what requires a revbump and what does not. Doing this would require having portage cache a hash of whatever ebuild it last parsed, and perhaps its eclasses as well if we permit revbump-less eclass changes. Then it would have to check for when these change, perhaps this might be stored in the metadata to speed things up. You could view such an approach as being equivalent to just sticking a .hash on every ebuild in the tree as a minor revision tag, so that any change triggers a minor revision. The only difference between that and the proposal that I can see is that the minor revisions would be unordered. However, if we go with the theory that defective ebuilds get removed immediately then there is no point in ordering minor revisions because there will only be one in the tree at any time for a given major revision, so the package manager couldn't take advantage of any information conveyed by ordering. This probably isn't too different from what portage is doing already, except: 1. Portage is only looking for dependency changes when it has another reason to look closely at the ebuild - it has no mechanism to detect that an ebuild changes, and this would add one. 2. We don't have any clear policies today on when ebuilds should be revbumped or not as the result of things like dependency changes, and this would cause us to make some. The fact that this isn't a big change does concern me that it may not fix all of the underlying problems. However, some of those problems might not actually have clean solutions other than never committing mistakes in the first place. For example, if a prerm dependency was missing then removing the ebuild can potentially fail whether you use dynamic deps or not until it is satisfied. The primary underlying problem I see about this is that it doesn't force devs to start doing something to the tree that will suddenly help make all of the static-deps-only PMs (ie, those that aren't going to implement this new hash-changed-so-re-evaluate-ebuild method) suddenly work in a more consistent fashion. IIRC, the very first post of this thread was a reminder to dev's to revbump so that static-deps behaviour is more correct/consistent. However, if we put something into the next EAPI about this and make it a requirement for all PMs (although I have no idea how we would roll this out; maybe make it a profile-level requirement instead of an ebuild-level one, if there is such a thing??) then it would become much less of an issue.. Mind you, we need to solve all of the problems first and make sure PMS documents all of the requirements in an appropriately complete and ambiguity-preventing manner that everyone agrees with.. Easy, right? :) -BEGIN PGP SIGNATURE- Version: GnuPG v2 iF4EAREIAAYFAlPWlfYACgkQ2ugaI38ACPApTwD9H+PX4f1XGtauwbjfXczPqAYf yBqDW9MOwIlWPCqeu6IA/ipySyWxA/J12RSuLTNyj98li9Qmeq0GLR37KSZ2Cc/p =05lZ -END PGP SIGNATURE-
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Mon, Jul 28, 2014 at 2:27 PM, Ian Stakenvicius a...@gentoo.org wrote: The primary underlying problem I see about this is that it doesn't force devs to start doing something to the tree that will suddenly help make all of the static-deps-only PMs (ie, those that aren't going to implement this new hash-changed-so-re-evaluate-ebuild method) suddenly work in a more consistent fashion. IIRC, the very first post of this thread was a reminder to dev's to revbump so that static-deps behaviour is more correct/consistent. I think the intent here is to define how we want the PM to behave, and what kinds of changes should require revbumps (ie those the PM can't handle otherwise). Obvious a side-effect of this will be that PMs that don't behave as we intend them to may have issues. However, if we put something into the next EAPI about this and make it a requirement for all PMs (although I have no idea how we would roll this out; maybe make it a profile-level requirement instead of an ebuild-level one, if there is such a thing??) It may make sense to do this via a new EAPI, though I think figuring out what we want to do comes first. That is, I want to ask the question if no PMs existed and we were writing our first one, how would we want it to behave? Getting from here to there is the next problem. Really the issue comes down to how we maintain ebuilds. If we aren't revbumping for dependency errors, then PMs that don't handle dynamic deps wouldn't update their dependencies. That certainly has consequences, but whether they're considered bugs/problems/etc is a bit up for debate. I'm not convinced that it makes sense to do micro-revbumps just to force PMs that don't have any concept of dynamic dependencies to treat them as full revbumps. Devs can still forget to do them, and it results in churn that doesn't seem necessary to me. On the other hand I don't want to make life even more difficult on those using alternative PMs (though it sounds like we're doing this already). It seems like we aren't getting many more new options here. Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Mon, Jul 28, 2014 at 2:50 PM, Martin Vaeth mar...@mvath.de wrote: In both cases of 6., the user is not even aware that he uses long obsolete packages unless portage prints a big fat warning for orphaned packages (which currently is not the case. Well, at least eix -t will be print a message.) This is really the crux of these sorts of issues. It doesn't matter if dependencies are static or dynamic - if you hang onto orphans then you're going to have cruft in your vdb which is going to lead to blockers of some kind eventually. Portage should probably generate a warning when there are orphan packages. The same is true if you keep cruft in a local overlay or such. We can have all the pretty virtuals/etc we want, but if users stick hard-coded obsolete package names in their overlays or have them in their vdb, then they're going to get blockers. Though, we could do a better job with the error messages even when that happens... Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 27 July 2014 19:16, Martin Vaeth mar...@mvath.de wrote: Not at all, it is completely identical to a revision bump: If you would use -r2 instead of -r1.1, you also would end up in -r1 and -r2 being identical. Actually, in both cases, you would *remove* -r1, since -r1 is incorrect since it should have been bumped. It just seems strange to me to go round a large tree and bump every ebuild affected by an eclass change, simply not modifiying the code, buy changing the -r value on the end of it. In a no dynamic deps, period scenario, this just strikes me as 2 flavours of the same weirdness, -r2 and -r1.1 are just equally weird choices to make if the ebuild itself doesn't change at all. For some things it would be a nightmare of impossibility for even a trivial change if some eclass changed to have one new dependency/one less dependency. You'd need some system in place to go around the tree -r bumping things, and the system would involve humans who are prone to making mistakes, and create commit churn to make it happen. The same problem would still persist with people forgetting to -r bump, or missing one ebuild in the ~200 affected, but with dynamic deps off, those bugs would lie in wait and confuse portage until somebody filed a bug and it got responded to. Sure, having an approved system to ship dependency-only changes to the end users tree is something we do need, I'll grant that, but the point of failure is already significantly weighing on developer time, and this would see a growth in developer time to manage this ( I could be over estimating how much, but -r bumping everything that used a specific eclass is something I very much do not envy ). However, if there was a system in place such that developers didn't have to manually do any -r bumping , but some system acknowledge the changes and scheduled some kind of VDB update in response to some kind of data that was transmitted as part of the sync, that would be nice. -r bumping is of course *a* way to transmit the data. Just I feel strongly inclined that we shouldn't be making developers expend *more* effort to solve this problem if there's a way we can make them spend *less*. -- Kent *KENTNL* - https://metacpan.org/author/KENTNL
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 26/07/14 15:49, Ciaran McCreesh wrote: On Sat, 26 Jul 2014 12:41:16 + (UTC) Martin Vaeth mar...@mvath.de wrote: hasufell hasuf...@gentoo.org wrote: Dynamics deps are already broken, not consistently enabled (e.g. when subslots are in use) Just to make it clear: No, dynamic deps are not broken. Yes they are. We just succesfully converted ~300 ebuilds in tree without revision bumps from virtual/udev[gudev,introspection,static-libs] to virtual/libudev and virtual/libgudev Tested it on multiple boxes, went fine. Nobody has filed bugs at http://bugs.gentoo.org/, nobody has filed a single forums post, nobody has said anything at #gentoo, Freenode Only one person said he had to manually build 2 GNOME related packages, simple-scan and something else So, broken? Far from it. More like essential feature. People have just listed some known races dynamic deps have, and I take those races anyday over an regression that causes endless rebuilding... - Samuli
Re: [gentoo-dev] Re: don't rely on dynamic deps
Samuli Suominen: On 26/07/14 15:49, Ciaran McCreesh wrote: On Sat, 26 Jul 2014 12:41:16 + (UTC) Martin Vaeth mar...@mvath.de wrote: hasufell hasuf...@gentoo.org wrote: Dynamics deps are already broken, not consistently enabled (e.g. when subslots are in use) Just to make it clear: No, dynamic deps are not broken. Yes they are. We just succesfully converted ~300 ebuilds in tree without revision bumps from virtual/udev[gudev,introspection,static-libs] to virtual/libudev and virtual/libgudev Tested it on multiple boxes, went fine. Nobody has filed bugs at http://bugs.gentoo.org/, nobody has filed a single forums post, nobody has said anything at #gentoo, Freenode Only one person said he had to manually build 2 GNOME related packages, simple-scan and something else So, broken? Far from it. More like essential feature. People have just listed some known races dynamic deps have, and I take those races anyday over an regression that causes endless rebuilding... I'm not sure if you realize that you just disabled dynamic deps support on most of those converted ebuilds.
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 7/27/14, 1:42 PM, Samuli Suominen wrote: Only one person said he had to manually build 2 GNOME related packages, simple-scan and something else So, broken? Far from it. More like essential feature. People have just listed some known races dynamic deps have, and I take those races anyday over an regression that causes endless rebuilding... I think this really boils down to deciding on the tradeoff between correctness and speed. Even the example above shows that with dynamic rebuilds some manual rebuilds were needed, which I interpret as dynamic deps not being correct. Static deps go the other way around: they do lead to more rebuilds, and I think no one is denying that. However, they seem more predictable to me. Paweł signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Paweł Hajdan, Jr.: On 7/27/14, 1:42 PM, Samuli Suominen wrote: Only one person said he had to manually build 2 GNOME related packages, simple-scan and something else So, broken? Far from it. More like essential feature. People have just listed some known races dynamic deps have, and I take those races anyday over an regression that causes endless rebuilding... I think this really boils down to deciding on the tradeoff between correctness and speed. Even the example above shows that with dynamic rebuilds some manual rebuilds were needed, which I interpret as dynamic deps not being correct. Static deps go the other way around: they do lead to more rebuilds, and I think no one is denying that. However, they seem more predictable to me. Paweł his argument is pointless, because he got static deps on all those ebuilds now anyway
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sun, Jul 27, 2014 at 6:43 AM, Kent Fredric kentfred...@gmail.com wrote: In a no dynamic deps, period scenario, this just strikes me as 2 flavours of the same weirdness, -r2 and -r1.1 are just equally weird choices to make if the ebuild itself doesn't change at all. You have a good point here. With this proposal we have three kinds of ebuild changes: 1. Those that do not involve any revbump. 2. Those with a minor revbump (ie -r1 - -r1.1). 3. Those with a normal revbump (ie -r1 - -r2). What is the purpose of allowing the first? Presumably that should be used in even more limited circumstances than a minor revbump, so why allow it at all? Why not just make an in-place change equivalent to a minor revbump and ditch the minor revbump entirely? Devs would still need to distinguish between what requires a revbump and what does not. Doing this would require having portage cache a hash of whatever ebuild it last parsed, and perhaps its eclasses as well if we permit revbump-less eclass changes. Then it would have to check for when these change, perhaps this might be stored in the metadata to speed things up. You could view such an approach as being equivalent to just sticking a .hash on every ebuild in the tree as a minor revision tag, so that any change triggers a minor revision. The only difference between that and the proposal that I can see is that the minor revisions would be unordered. However, if we go with the theory that defective ebuilds get removed immediately then there is no point in ordering minor revisions because there will only be one in the tree at any time for a given major revision, so the package manager couldn't take advantage of any information conveyed by ordering. This probably isn't too different from what portage is doing already, except: 1. Portage is only looking for dependency changes when it has another reason to look closely at the ebuild - it has no mechanism to detect that an ebuild changes, and this would add one. 2. We don't have any clear policies today on when ebuilds should be revbumped or not as the result of things like dependency changes, and this would cause us to make some. The fact that this isn't a big change does concern me that it may not fix all of the underlying problems. However, some of those problems might not actually have clean solutions other than never committing mistakes in the first place. For example, if a prerm dependency was missing then removing the ebuild can potentially fail whether you use dynamic deps or not until it is satisfied. Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 27/07/14 14:50, hasufell wrote: Samuli Suominen: On 26/07/14 15:49, Ciaran McCreesh wrote: On Sat, 26 Jul 2014 12:41:16 + (UTC) Martin Vaeth mar...@mvath.de wrote: hasufell hasuf...@gentoo.org wrote: Dynamics deps are already broken, not consistently enabled (e.g. when subslots are in use) Just to make it clear: No, dynamic deps are not broken. Yes they are. We just succesfully converted ~300 ebuilds in tree without revision bumps from virtual/udev[gudev,introspection,static-libs] to virtual/libudev and virtual/libgudev Tested it on multiple boxes, went fine. Nobody has filed bugs at http://bugs.gentoo.org/, nobody has filed a single forums post, nobody has said anything at #gentoo, Freenode Only one person said he had to manually build 2 GNOME related packages, simple-scan and something else So, broken? Far from it. More like essential feature. People have just listed some known races dynamic deps have, and I take those races anyday over an regression that causes endless rebuilding... I'm not sure if you realize that you just disabled dynamic deps support on most of those converted ebuilds. There is a bug in the package manager, you mean.
Re: [gentoo-dev] Re: don't rely on dynamic deps
Samuli Suominen:
On 27/07/14 14:50, hasufell wrote:
Samuli Suominen:
On 26/07/14 15:49, Ciaran McCreesh wrote:
On Sat, 26 Jul 2014 12:41:16 + (UTC)
Martin Vaeth mar...@mvath.de wrote:
hasufell hasuf...@gentoo.org wrote:
Dynamics deps are already broken, not consistently enabled (e.g.
when subslots are in use)
Just to make it clear: No, dynamic deps are not broken.
Yes they are.
We just succesfully converted ~300 ebuilds in tree without revision
bumps from virtual/udev[gudev,introspection,static-libs]
to virtual/libudev and virtual/libgudev
Tested it on multiple boxes, went fine. Nobody has filed bugs at
http://bugs.gentoo.org/, nobody has filed a single forums post,
nobody has said anything at #gentoo, Freenode
Only one person said he had to manually build 2 GNOME related packages,
simple-scan and something else
So, broken? Far from it. More like essential feature.
People have just listed some known races dynamic deps have, and I take
those races anyday over an regression that causes
endless rebuilding...
I'm not sure if you realize that you just disabled dynamic deps support
on most of those converted ebuilds.
There is a bug in the package manager, you mean.
I'm eager to hear how you want to make subslots work with dynamic deps.
:= gets converted to :${SLOT}/${SUBSLOT} in vardb and this is used to
trigger the rebuilds.
How do you record the subslot a package was built against in the live tree?
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sun, Jul 27, 2014 at 8:31 AM, hasufell hasuf...@gentoo.org wrote:
I'm eager to hear how you want to make subslots work with dynamic deps.
:= gets converted to :${SLOT}/${SUBSLOT} in vardb and this is used to
trigger the rebuilds.
How do you record the subslot a package was built against in the live tree?
Well, suppose the dependency is removed because it never was a true
dependency to begin with. Portage can handle this by deleting the
corresponding entry from vardb.
That is a dynamic dependency change, and offhand I don't see how it
breaks with subslots.
This is why we have to be careful about tossing around phrases like
dynamic deps don't work - they don't work in particular
circumstances, and it is helpful to the discussion if we try to
characterize when they do/don't work rather than painting with broad
strokes.
I do think that this needs some attention so that we can make portage
more predictable, but I think the argument has been made that we have
a LOT of changes in the tree today which don't involve revbumps, and
turning them into revbumps could cause a lot of turmoil for users.
So, I'm interested in seeing if there is a better compromise to be
found.
Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sun, 27 Jul 2014 14:42:24 +0300 Samuli Suominen ssuomi...@gentoo.org wrote: We just succesfully converted ~300 ebuilds in tree without revision bumps from virtual/udev[gudev,introspection,static-libs] to virtual/libudev and virtual/libgudev Tested it on multiple boxes, went fine. Testing can't prove that it's correct, only that it's incorrect... All Aside from all the way you've disabled dynamic dependencies for a whole bunch of packages without realising it, your other problem is that some of the breakage won't show up until later, when people start bumping and removing ebuilds. And this is the problem: you need to think carefully about dynamic dependencies and fully understand the problem. Superficial testing won't give you the whole story. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Dnia 2014-07-27, o godz. 14:42:24 Samuli Suominen ssuomi...@gentoo.org napisał(a): On 26/07/14 15:49, Ciaran McCreesh wrote: On Sat, 26 Jul 2014 12:41:16 + (UTC) Martin Vaeth mar...@mvath.de wrote: hasufell hasuf...@gentoo.org wrote: Dynamics deps are already broken, not consistently enabled (e.g. when subslots are in use) Just to make it clear: No, dynamic deps are not broken. Yes they are. We just succesfully converted ~300 ebuilds in tree without revision bumps from virtual/udev[gudev,introspection,static-libs] to virtual/libudev and virtual/libgudev Tested it on multiple boxes, went fine. How did you exactly test them? Did you at least bother checking if portage actually uses the deps you added? -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Ciaran McCreesh wrote: Uh huh, so you add an overlay, and suddenly the dependencies for a random subset of your installed packages change in ways that don't in any way reflect what you have installed. How is this the desired behaviour? There are several different cases of dependency data which apply in different ways for different operations. For a model to be accurate it needs to consider all cases, and treat them differently where neccessary. Adding an overlay doesn't change anything about what is installed. Installing one version of an ebuild from an overlay, with another version of the ebuild (from same overlay, or not) already installed, clearly needs the package manager to consider what the overlay ebuild says. That's the purpose of the overlay. Overlay maintainers already have the responsibility to create well-formed ebuilds, and there are even tools like overlint to help with the task. If they don't, the applicability of their overlay decreases. Sometimes this is on purpose, and it can be completely unproblematic. It's a little like rewriting public git history. Many people cry bloody murder about that, but in lots of cases it's perfectly fine. So are overlay dependencies which aren't perfectly well-formed. That's probably why they're in an overlay in the first place. It's not the responsibility of the package manager to magically resolve dependencies without sufficient information. It's the responsibility of ebuild authors to provide such information, so that dependencies can be resolved accurately for all package manager operations. It seems that this may be a case of trying to do way too much, and as a result doing a bad job at everything. I agree firmly with Rich that it is much better to do a good job at some (well-known, perhaps even documented) things, than a bad job at all things. It's fine to simply error out with a message saying that the encountered situation is not supported. //Peter pgpGNpzVh0wY0.pgp Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sun, Jul 27, 2014 at 8:04 AM, Rich Freeman ri...@gentoo.org wrote: Doing this would require having portage cache a hash of whatever ebuild it last parsed, and perhaps its eclasses as well if we permit revbump-less eclass changes. Then it would have to check for when these change, perhaps this might be stored in the metadata to speed things up. I was thinking about this a little more, and the way I'd do it is ID whatever ebuild variables we want to allow to be dynamic. Then the ebuild would be sourced, and then those variables would be extracted and hashed, and that would be treated as the ebuilds dependency state. That would be stored in the metadata, and portage would store it in vdb when a package is installed. Then portage could look for any change in state and that would trigger a build-less re-merge, which would update vdb with the new state (including the new hash). With this approach you don't need minor revision numbers - any change to an ebuild would be considered a minor revision, and when that is inappropriate a full revbump should be used instead. Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 28 July 2014 09:46, Rich Freeman ri...@gentoo.org wrote: Then portage could look for any change in state and that would trigger a build-less re-merge, which would update vdb with the new state (including the new hash). If we're scared about this being worse than what we have, I notice there's a bunch of --rebuild-if-whaetever variables. For some reason I was under the impression there was already a --rebuild-if-deps-changed flag, but I seem to have been wrong about that. --rebuild-if-deps-changed=fast # VDB only updates --rebuild-if-deps-changed=full # complete reinstall if vdb != tree --rebuild-if-deps-changed=n# current behaviour. -- Kent *KENTNL* - https://metacpan.org/author/KENTNL
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 12:32:20 + (UTC) Martin Vaeth mar...@mvath.de wrote: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: User installs foo-1.1-r1 Developer makes foo-1.1-r1.1 foo-1.1* is removed from the tree User syncs How is this different from your suggestion (which you *claim* to be non-broken): User installs foo-1.1-r1 Developer makes foo-1.1-r2 foo-1.1* is removed from the tree User syncs In fact, the result is completely the same, no matter whether you have minor revisions or not, and no matter whether you have static or dynamic deps. What is *actually* broken here is that the user has installed a package which is not maintained anymore: *This* is what needs to be fixed. This issue is completely independent of static vs. dynamic deps. You misuse this problem as a strawman, only. Uhm. That works just fine... I don't think you understand how this works: we can always use the metadata that's in VDB for dealing with the installed package. The issue is that sometimes Portage tries to guess that it's better to use the metadata from an ebuild instead of what's in VDB when dealing with an installed package. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 12:41:16 + (UTC) Martin Vaeth mar...@mvath.de wrote: hasufell hasuf...@gentoo.org wrote: Dynamics deps are already broken, not consistently enabled (e.g. when subslots are in use) Just to make it clear: No, dynamic deps are not broken. Yes they are. What is broken is that portage does not use them consistently. Because using them consistently is impossible by design. It would be a rather bad idea to change policy just because of this portage bug and force users to permanently do unnecessary recompilations. At least, for me, it would mean that I have to change distribution, since I cannot afford this. This is not a Portage bug. optional and not defined in PMS. Static deps are also optional and not defined in PMS. In fact, PMS makes no claim *where* to read the DEP strings from; it only specified how they are to be stored in the tree. Incorrect. Quite the opposite, PMS claims that one cannot rely on anything stored in /var/db Incorrect. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 12:54:08 + (UTC) Martin Vaeth mar...@mvath.de wrote: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: Jeroen Roovers j...@gentoo.org wrote: On Mon, 21 Jul 2014 23:06:07 +0200 Pacho Ramos pa...@gentoo.org wrote: Maybe this could be solved by having two kinds of revisions: - One would rebuild all as usually (for example, -r1...) - The other one would only regenerate VDB and wouldn't change the installed files (for example, -r1.1) Or the package manager looks at changed in *DEPEND between the repo and vdb and resolves those. ...assuming that the ebuild hasn't been removed, and that it can be associated correctly when overlays are involved, and that the change wasn't a change where a saved pkg_prerm uses the old dependency, not the new one, or all the other ways this breaks. You need to think your cunning plan the whole way through. It works, since it is completely equivalent to a revbump, only that the unnecesary recompilation is avoided: All of your problems exist (or don't exist) for usual revbumps as well. At this point, I think it would be most helpful towards us reaching a conclusion if you agreed to refrain from commenting further until you've understood the problem at hand. You see, the rest of us are using broken to mean broken in a technical sense, based upon our understanding of how ebuilds, the VDB and metadata work. You seem to be using it to mean does something you superficially or ideologically don't like. This is a technical discussion, and you need to read up on how things work before you can make a meaningful contribution. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 13:00:31 + (UTC) Martin Vaeth mar...@mvath.de wrote: Both, dynamic and static deps are broken. They are broken in different ways, but both are broken. You keep using that word. I do not think it means what you think it means. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 13:16:13 + (UTC) Martin Vaeth mar...@mvath.de wrote: But, OK, so I will use your strawman to prove how static deps are broken: This is not broken. This is exactly what is supposed to happen, and it is exactly what *does* happen some of the time with dynamic dependencies too. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
El sáb, 26-07-2014 a las 12:00 +, Martin Vaeth escribió: [...] Probably there are many more examples than 1.-4, but I hope that the point becomes clear: Whenever packages split, merge, or can substitute each other, dependency changes are necessary, and rebuilds caused by these are unnecessary. Unfortunately, such things happen *very* frequently... Nobody is to blame for this; the PM just should be ready to deal with such situations without unnecessary rebuilds, be it by dynamic deps or by a mechanism to avoid recompilation. I have also seen the need of add slots to DEPEND/RDEPEND due the inclusion of an updated package like gstreamer:1.0/gtk+:3/libpng... or needing to add forgotten USE deps (like package[introspection?] or package[X]).
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 13:41:34 + (UTC) Martin Vaeth mar...@mvath.de wrote: The idea is to act as usual, just to skip unnecessary phases... So someone adds optional selinux support to a package, and then you end up with selinux being on, despite not having it, and then another package depends upon your package with [selinux], and the dependency is mistakenly treated as met... -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 14:09:44 + (UTC) Martin Vaeth mar...@mvath.de wrote: PMS defines a static dependency model No. PMS does not specify which dependency information has to be taken. Yes it does. Please read PMS, and do not guess as to what it says. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Dnia 2014-07-26, o godz. 14:02:29 Martin Vaeth mar...@mvath.de napisał(a): Alexandre Rostovtsev tetrom...@gentoo.org wrote: rdepends-add is easy to implement [...] Deletion is trickier [...] The point is to *not* clean up these entries for months/years. So, essentially, you want the developer to do part of CVS/git's job, namely keeping a history of changes in a compressed format, keeping the history forever (or almost forever). As mentioned in another post, you highly understimate the amount of data which would have to be treated this way: For every python release and many other eclass changes, almost all packages in the tree are involved, usually several times a months. Python is irrelevant here. Our dependencies are USE-conditional, so dependencies are added and removed along with USE flags. If we add new implementation, you need to rebuild the package anyway to use it, and there is no point populating extra dependencies. Revbump isn't necessary either since --changed-use will pick it up if necessary. If we remove an implementation, PM isn't supposed to remove the dependencies until the package is rebuilt with flag disabled. If it was enabled, --changed-use is supposed to clean it up. If it was not, the extra dependencies do not matter (and are not even stored in vdb). -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Dnia 2014-07-26, o godz. 14:09:44 Martin Vaeth mar...@mvath.de napisał(a): Alexander Berntsen berna...@gentoo.org wrote: 1. Improve dynamic-deps. This is, as Michał pointed out earlier in this thread a pipe dream. Not necessarily. Just somebody with enough knowledge in portage and python has to fix it. The problem is that in gentoo there seems to be currently nobody with these skills and free time... All people with enough knowledge already know that this is technically impossible. This is not theoretical physics, ignoring the impossible won't breed anything here. -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 14:33:38 + (UTC) Martin Vaeth mar...@mvath.de wrote: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: No. PMS does not specify which dependency information has to be taken. Yes it does. Please read PMS, and do not guess as to what it says. Looking for /var/db/pkg gave exactly one hit: The assertion that this is *not* part of PMS. The section on dependencies is only about the ebuild format. So either it is very hidden or not in there. I said read PMS, not grep PMS. You're making all kinds of wild claims based upon what you guess PMS says, not what it actually says. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Ciaran McCreesh: On Sat, 26 Jul 2014 14:33:38 + (UTC) Martin Vaeth mar...@mvath.de wrote: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: No. PMS does not specify which dependency information has to be taken. Yes it does. Please read PMS, and do not guess as to what it says. Looking for /var/db/pkg gave exactly one hit: The assertion that this is *not* part of PMS. The section on dependencies is only about the ebuild format. So either it is very hidden or not in there. I said read PMS, not grep PMS. You're making all kinds of wild claims based upon what you guess PMS says, not what it actually says. This looks like a private lesson in PMS reading. Can you guys do that on IRC?
Re: [gentoo-dev] Re: don't rely on dynamic deps
Martin Vaeth: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: But, OK, so I will use your strawman to prove how static deps are broken: This is not broken. This is exactly what is supposed to happen It's not a bug it's a feature Of course, one can always close the eyes when faced with problems. and it is exactly what *does* happen some of the time with dynamic dependencies too. Yes, both concepts have problems. Since neither solution is perfect, why choose the one with unnecessary rebuilds? You are not contributing anything useful to the thread currently. Read the whole thread. Read up on dynamic deps. Read up on PMS.
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 14:46:42 + (UTC) Martin Vaeth mar...@mvath.de wrote: Yes, both concepts have problems. The problems are of a different kind. Static dependencies don't do something that you want them to do. Dynamic dependencies are outright broken. Since neither solution is perfect, why choose the one with unnecessary rebuilds? We are picking the *correct* solution, not the one that sometimes hides an occasional inconvenience (but unreliably) at the expense of being utterly broken. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 14:57:20 + (UTC) Martin Vaeth mar...@mvath.de wrote: This is a technical discussion Exactly. So instead of writing such pointless personal attacks, you should give technical arguments. The technical reasons that dynamic dependencies can never work have already been explained. However, you continue to ignore them, and you continue to refuse to read what PMS actually says on the matter. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Dnia 2014-07-26, o godz. 15:01:46 Martin Vaeth mar...@mvath.de napisał(a): Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: Martin Vaeth mar...@mvath.de wrote: The idea is to act as usual, just to skip unnecessary phases... So someone adds optional selinux support to a package, and then you end up with selinux being on, despite not having it, and then another package depends upon your package with [selinux], and the dependency is mistakenly treated as met... If the developer has added IUSE=selinux and bumps from -r1 to -r1.1, he has of course verified that this USE-change does not require recompilation either way, since otherwise he would not have been allowed to explicitly say the package manager that recompilation is unnecessary. So the dependency is *correctly* treated as met. Excuse me but are we talking about updating *DEPEND or IUSE? -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Martin Vaeth: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: Martin Vaeth mar...@mvath.de wrote: hasufell hasuf...@gentoo.org wrote: Dynamics deps are already broken, not consistently enabled (e.g. when subslots are in use) Just to make it clear: No, dynamic deps are not broken. Yes they are. Could you please stop your childish behaviour? What is broken is that portage does not use them consistently. Because using them consistently is impossible by design. It seems that *you* should take some reading before you continue with discussion. I think no one cares about this part of the thread anymore. Can you take it elsewhere?
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 15:04:31 + (UTC) Martin Vaeth mar...@mvath.de wrote: It seems that *you* should take some reading before you continue with discussion. I wrote PMS, the dev manual and a package manager... I understand the issues involved. If you want to contribute, you should at least read the spec. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 15:11:36 + (UTC) Martin Vaeth mar...@mvath.de wrote: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: Martin Vaeth mar...@mvath.de wrote: The problems are of a different kind. Static dependencies don't do something that you want them to do. Dynamic dependencies are outright broken. Please, stop your childish behaviour. You prove nothing be repeating claims which had just been disproved. Let's start with the easiest issue: please point us all to the place where you proved how dynamic dependencies still work in the face of ebuild removals. Your solution to this problem will be of great benefit to all of us. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
Martin Vaeth: Indeed, it just would just need a little programming. would you like to implement it?
Re: [gentoo-dev] Re: don't rely on dynamic deps
Dnia 2014-07-26, o godz. 15:27:51 Martin Vaeth mar...@mvath.de napisał(a): Michał Górny mgo...@gentoo.org wrote: All people with enough knowledge already know that this is technically impossible. We already discussed in the bug how it *would* be possible, just nobody implements it: Portage would have to use dynamic deps throughout, using the data stored in /var/db only to find out the correct information for := dependencies. This is only one of the issue. And what if the match for := is incompatible with new dependency atom? Like when you replace 'dev-foo/bar:=' with '=dev-foo/bar-2:=' but bar-1 is installed. What if a new := dependency is added? -- Best regards, Michał Górny signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 15:27:51 + (UTC) Martin Vaeth mar...@mvath.de wrote: Michał Górny mgo...@gentoo.org wrote: All people with enough knowledge already know that this is technically impossible. We already discussed in the bug how it *would* be possible, just nobody implements it: Portage would have to use dynamic deps throughout, using the data stored in /var/db only to find out the correct information for := dependencies. This would fix the behaviour except for some corner cases concerning orphaned packages which can lead to broken situations with any approach. Your solution fails spectacularly in the following ways: * Ebuild removal * Overlays * Introduction of := dependencies * pkg_*rm Which brings us back to the all people with enough knowledge already know that this is technically impossible thing... -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 15:40:40 + (UTC) Martin Vaeth mar...@mvath.de wrote: Let's start with the easiest issue: please point us all to the place where you proved how dynamic dependencies still work in the face of ebuild removals. *Neither* dynamic deps nor static deps solve this problem satisfactory (How often did I repeat this now?). With static dependencies, you have correct dependency information, and the worst that can happen is occasionally you might have to rebuild a package where nothing substantial has changed. However, this is a general issue with bumps (recompiling the whole thing for an init script or language file change, recompiling the whole thing for a change to only one of the binaries provided by a package, and so on), so it is not a static dependencies problem. With dynamic dependencies, you have incorrect dependency information, your system randomly breaks on a sync, you sometimes can't uninstall packages due to pkg_* breakage, uninstalling a package sometimes looks safe but isn't, overlays don't work, subslots don't work, binaries don't work, and dependencies can appear to be met when they aren't. So in summary, dynamic dependencies are broken, and static dependencies are correct, and the only issue you think you have with static dependencies isn't a problem specific to static dependencies and isn't reliably solved by dynamic dependencies. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 15:59:58 + (UTC) Martin Vaeth mar...@mvath.de wrote: And what if the match for :=3D is incompatible with new dependency atom? Like when you replace 'dev-foo/bar:=3D' with '=3Ddev-foo/bar-2:=3D' but bar-1 is installed. This is simple: The dependency is not satisfied. That isn't simple at all... It means you can't uninstall or upgrade the package... -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 16:05:58 + (UTC) Martin Vaeth mar...@mvath.de wrote: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: Your solution fails spectacularly in the following ways: * Ebuild removal Already discussed as to fail with static deps, too. Uh, static dependencies don't behave any differently when an ebuild is removed. I don't think you understand how that works. * Overlays Not an issue: Exactly the information of that ebuild which *would* be used if you reemerge contains the relevant data. The association between an installed package and the ebuild it came from doesn't work correctly when overlays around. Again, you don't understand the issue. * Introduction of :=3D dependencies This is not a minor update in dependencies and thus requires a revbump. So what is a minor update, and what are you planning to do to prevent what you call useless rebuilds when := dependencies are introduced? * pkg_*rm Not related. Yes it is. Read and understand the previous discussion about the ruby-config issue. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, Jul 26, 2014 at 12:14 PM, Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: On Sat, 26 Jul 2014 16:05:58 + (UTC) Martin Vaeth mar...@mvath.de wrote: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: Your solution fails spectacularly in the following ways: * Introduction of :=3D dependencies This is not a minor update in dependencies and thus requires a revbump. So what is a minor update, and what are you planning to do to prevent what you call useless rebuilds when := dependencies are introduced? Picking this to illustrate my point, not to endorse a particular implementation here... This is what I'm getting at when I argue that we don't need a solution to every possible problem. If we only accept solutions that handle conditional dependencies, IUSE changes, new eclass inherits, dep additions, dep removals, etc, then it does seem likely to me that we'll never find a good solution. On the other hand, if we can come up with something that RELIABLY fixes things for 3/4ths of these, then that is probably good enough. I'm not certain at this point that even such a partial solution doesn't exist, but the fact that any particular solution doesn't handle every possible case isn't automatically a reason to reject it. Preventing unnecessary rebuilds is a worthwhile goal, even if we can't get 100% of the way there. If you don't care whatsoever about unnecessary rebuilds then we can simplify things tremendously - just have the package manager default to --emptytree on all operations. Sure, it might cause a few unnecessary ebuilds but whether your package manager attempts to support dynamic deps or not you'll certainly have an up-to-date dependency cache. Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 12:28:27 -0400 Rich Freeman ri...@gentoo.org wrote: Sure, it might cause a few unnecessary ebuilds but whether your package manager attempts to support dynamic deps or not you'll certainly have an up-to-date dependency cache. VDB is not a cache. This is important. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, Jul 26, 2014 at 12:02 PM, Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: On Sat, 26 Jul 2014 15:59:58 + (UTC) Martin Vaeth mar...@mvath.de wrote: And what if the match for :=3D is incompatible with new dependency atom? Like when you replace 'dev-foo/bar:=3D' with '=3Ddev-foo/bar-2:=3D' but bar-1 is installed. This is simple: The dependency is not satisfied. That isn't simple at all... It means you can't uninstall or upgrade the package... Why not? How is this any different from unmerging bar-1 back when bar-1 satisfied the dependency (using --unmerge which breaks reverse dependencies)? If you want to upgrade or re-install the package I would expect portage to pull in the missing dependency. I'd expect the next emerge -u world to do that as well, which it already does if you --unmerge a dependency). If there would be some unintended side-effect from doing things this way I'm all ears, but as long as you don't get into @system circular dependencies issues I'd expect portage to be able to install any packages that are missing after such a dependency change. Sure, until the missing dep is installed I'd expect a risk of breakage, but that is no different than what I'd expect if the package were modified without a bump and the package manager didn't attempt to support dynamic dependencies. Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 12:36:45 -0400 Rich Freeman ri...@gentoo.org wrote: On Sat, Jul 26, 2014 at 12:02 PM, Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: On Sat, 26 Jul 2014 15:59:58 + (UTC) Martin Vaeth mar...@mvath.de wrote: And what if the match for :=3D is incompatible with new dependency atom? Like when you replace 'dev-foo/bar:=3D' with '=3Ddev-foo/bar-2:=3D' but bar-1 is installed. This is simple: The dependency is not satisfied. That isn't simple at all... It means you can't uninstall or upgrade the package... Why not? pkg_prerm. See the discussion about the ruby-config problem. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sat, 26 Jul 2014 18:36:27 + (UTC) Martin Vaeth mar...@mvath.de wrote: Ciaran McCreesh ciaran.mccre...@googlemail.com wrote: * Overlays Not an issue: Exactly the information of that ebuild which *would* be used if you reemerge contains the relevant data. The association between an installed package and the ebuild it came from doesn't work correctly when overlays around. It doesn't have to: That ebuild which would be installed *has* to carry the up-to-date information for that package. Otherwise, the overlay is broken. An overlay is *not* a different slot for a package. Uh huh, so you add an overlay, and suddenly the dependencies for a random subset of your installed packages change in ways that don't in any way reflect what you have installed. How is this the desired behaviour? -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Sun, 27 Jul 2014 03:12:07 +1000 Michael Palimaka kensing...@gentoo.org wrote: On 07/26/2014 07:59 AM, Tom Wijsman wrote: On Wed, 23 Jul 2014 22:14:41 +1000 Michael Palimaka kensing...@gentoo.org wrote: On 07/23/2014 09:36 AM, Tom Wijsman wrote: On Tue, 22 Jul 2014 18:21:00 +1000 Michael Palimaka kensing...@gentoo.org wrote: What a great way to kill the distro. I can already heat my house with the number of unnecessary rebuilds Do you upgrade @world every hour and thus have it cause excessive heat? If I upgrade every X weeks they become much more cool and necessary... Shouldn't we strive to avoid the unnecessary rebuilds in the first place? Doing updates on your schedule only avoids the symptom, not the problem. We should strive to do both; cause less rebuilds, update less often. It is comparable to flooding on IRC channels; if you send much more messages, you are much more likely to experience a kick and/or a ban. It is easier not to flood than to convince people there is no problem with you flooding the channel; out of all the IRC channels I know of, I've only come across one where they don't mind pasted long code blocks but that's mostly because of the lack of active moderation and people. (With flooding as updating and kick/ban as rebuilds) Each person should update at a frequency that suits them. Recommending to update every $period is not a valid solution to unnecessary rebuilds. The more one floods, the more one accepts kicks and/or bans; expected. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 27 July 2014 02:12, Martin Vaeth mar...@mvath.de wrote: Do not forget modification of eclasses which then require mass bumps! I'm curious what the -r1.1 technique would do here. I mean, wouldn't that mean you have 2 ebuilds that are identical except for the '.1' simply due to the eclass change? That's going to be confusing. -r1.1 weirdness feels like it may cause more problems than it solves. -- Kent *KENTNL* - https://metacpan.org/author/KENTNL
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Wed, 23 Jul 2014 22:14:41 +1000 Michael Palimaka kensing...@gentoo.org wrote: On 07/23/2014 09:36 AM, Tom Wijsman wrote: On Tue, 22 Jul 2014 18:21:00 +1000 Michael Palimaka kensing...@gentoo.org wrote: What a great way to kill the distro. I can already heat my house with the number of unnecessary rebuilds Do you upgrade @world every hour and thus have it cause excessive heat? If I upgrade every X weeks they become much more cool and necessary... Shouldn't we strive to avoid the unnecessary rebuilds in the first place? Doing updates on your schedule only avoids the symptom, not the problem. We should strive to do both; cause less rebuilds, update less often. It is comparable to flooding on IRC channels; if you send much more messages, you are much more likely to experience a kick and/or a ban. It is easier not to flood than to convince people there is no problem with you flooding the channel; out of all the IRC channels I know of, I've only come across one where they don't mind pasted long code blocks but that's mostly because of the lack of active moderation and people. (With flooding as updating and kick/ban as rebuilds) -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Fri, 25 Jul 2014 05:44:34 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: How long have dynamic-deps been around? Since EAPI-0? Because if so, that interpretation must be incorrect, since EAPI-0 was defined as portage behavior at the time, and AFAIK, no EAPI since then has been approved without a working portage implementation. Good question, probably needs a dig in the old Portage history; it is something long the search terms of dynamic dependencies in FakeVarTree, given that the parameter[1] has been added later on. EAPI specifies what PMs need to conform to, not the other way around; EAPI-0 may very well be derived from Portage, that doesn't make such side features that have not been specified in EAPI-0 a part of EAPI-0. [1]: Add emerge --dynamic-deps y|n option. http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=f8e0c75e In the context of dynamic-deps I'd interpret the above to mean within a single portage session. What happens some sessions later when an ebuild's deps are dynamic-updated after a tree sync is an entirely new session, and unless I'm missing something, the above PMS requirements can be met within a single session with dynamic-deps. Apart from the words merge and batch, which are in the piece of text that I've quoted; I'm not sure how for the rest of the piece a session can be deduced or interpreted from what is specified. -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Tue, 22 Jul 2014 18:21:00 +1000 Michael Palimaka kensing...@gentoo.org wrote: What a great way to kill the distro. I can already heat my house with the number of unnecessary rebuilds Do you upgrade @world every hour and thus have it cause excessive heat? If I upgrade every X weeks they become much more cool and necessary... -- With kind regards, Tom Wijsman (TomWij) Gentoo Developer E-mail address : tom...@gentoo.org GPG Public Key : 6D34E57D GPG Fingerprint : C165 AF18 AB4C 400B C3D2 ABF0 95B2 1FCD 6D34 E57D signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 22/07/14 04:51 PM, Andreas K. Huettel wrote: Am Dienstag 22 Juli 2014, 22:40:03 schrieb Ulrich Mueller: On Tue, 22 Jul 2014, Martin Vaeth wrote: PF has to be filled correctly, of course: The versions foo-1 and foo-1-r0 are identical according to PMS and should thus lead to the same $PF. This is not so. These versions are equal in comparision, so they cannot be in the tree at the same time. However, PF will be different for them. Well we'd need a new EAPI for this anyway. So we might as well redefine -r0 there. I still don't follow why we need new EAPI for this, as presented. What we are talking about here is optional PM behaviour only, and a convention that developers will need to adopt. It doesn't much matter if a PM doesn't implement minor-revision-vdbonly-merging because that PM would just do a full re-emerge same as any other revbump. The only need for EAPI change that I can see is to allow non-integer revision values, but that wasn't on mva's list of changes from what I remember. Am I missing something else, here? -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iF4EAREIAAYFAlPPtWIACgkQ2ugaI38ACPCjBQD+K0aQW3lJqVUJTo1nO9nnFlsY NfrgaIuu6eescdN6FDkBALwizKGBI4I0iSmj2ywis/4OTNsvFBQm9sxywXq7HFz1 =3Ajb -END PGP SIGNATURE-
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 07/23/2014 09:36 AM, Tom Wijsman wrote: On Tue, 22 Jul 2014 18:21:00 +1000 Michael Palimaka kensing...@gentoo.org wrote: What a great way to kill the distro. I can already heat my house with the number of unnecessary rebuilds Do you upgrade @world every hour and thus have it cause excessive heat? If I upgrade every X weeks they become much more cool and necessary... Shouldn't we strive to avoid the unnecessary rebuilds in the first place? Doing updates on your schedule only avoids the symptom, not the problem.
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Tue, Jul 22, 2014 at 4:06 PM, Martin Vaeth mar...@mvath.de wrote: ...but by introducing all the additional complications Ian has mentioned. More precisely: What happens if new dependencies are introduced which are not satisfied? One has to face it: Portage must not just silently update the database and thus silently produce a /var/db which does not satisfy its own dependencies... While this is problematic, I think portage actually can handle this (but I haven't tested this recently). Portage already allows you to clean a package without its reverse deps leading to a system in exactly the state you describe. I believe portage will just try to bring the package back at the next emerge @world (or any other set containing the reverse dep). If there is a problem with the dependency version then the system is already broken anyway - portage just doesn't realize it. I think that allowing devs to instruct portage to update VDB with USE/dep/etc changes is potentially less problematic than having portage trying to guess what the right thing to do is. We could then either use that feature or revbump as appropriate under the specific circumstances. Ultimately I think the most important thing we need is for PMs to follow some kind of defined behavior. In our efforts to get portage to do the right thing we sometimes end up with a portage that does things that nobody really understands. Things like that tend to lead to convenience 95% of the time and head-banging the other 5%. I'm all for workarounds, but I'd advocate simple ones that lead to easily predicted behavior (and failure modes). I'd rather safely eliminate 70% of useless rebuilds than unsafely try to eliminate 90% of them. However, I do agree that we need to be sensitive to rebuilds. Rich
Re: [gentoo-dev] Re: don't rely on dynamic deps
El mar, 22-07-2014 a las 07:39 +, Martin Vaeth escribió: Pacho Ramos pa...@gentoo.org wrote: Maybe this could be solved by having two kinds of revisions: - One would rebuild all as usually (for example, -r1...) - The other one would only regenerate VDB and wouldn't change the installed files (for example, -r1.1) I made the same suggestion already on the corresponding bug https://bugs.gentoo.org/show_bug.cgi?id=516612#c33 without any response. Just CCed :) It seems to me that this could avoid the problem of useless recompilation and would allow fine-graining of the issue by the ebuild maintainer (if not the maintainer of the ebuild, who else should be able to decide whether recompilation might be necessary to handle certain exceptions?) and simultaneously allow to revbump even on presumably tiny dependency changes. I still have not seen an argument against this idea. Of course, this would need an EAPI bump and could only be used for packages which are (or switch to(?)) this new EAPI, so a few (core) packages which should stay EAPI=0 for a long time are excluded from this for still quite a while. But apart from that few exceptions...? Also, this could be a benefit in the long term if we need to spread any changes to VDB in the future.
Re: [gentoo-dev] Re: don't rely on dynamic deps
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 On 07/22/2014 10:21 AM, Michael Palimaka wrote: On 07/22/2014 07:52 AM, Alexander Berntsen wrote: To sum up: My vote is disable dynamic-deps. And I would be happy to apply a patch that does this with the information I have today. What a great way to kill the distro. I can already heat my house with the number of unnecessary rebuilds - I can't imagine how many people will be left once we have to needlessly rebuild libreoffice and half the tree every time someone makes some minor change. If developers can't revbump correctly to address the shortcomings of dynamic deps, why do we expect they will be able to do so for static deps? I find it somewhat curious that the difference between ~arch and stable hasn't been brought up in this discussion yet. IMHO a user on ~arch should expect a higher number of rebuilds, it _is_ after all testing, whereby at the point it reaches stable, the deps are hopefully more likely to be correct to begin with. Does anyone have any insight into where these changes most often occur? - -- - Kristian Fiskerstrand Blog: http://blog.sumptuouscapital.com Twitter: @krifisk - Public PGP key 0xE3EDFAE3 at hkp://pool.sks-keyservers.net fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3 - Ad astra per aspera To the stars through thorns -BEGIN PGP SIGNATURE- iQIcBAEBCgAGBQJTziGbAAoJEPw7F94F4Tag61QP/iznpmFoc2jKO1Ex8fHEFURA 8D6mgzmkBW2OYWux8JnSfmS7WBrXvs0869Ey/dTgQeMWsdaauhFSqGUTL8k2s3pD 2NZUiNM9fBDGrVR589r0EX5jpPoylsq+ViYc/GtsWfUx8QZGRQOs75ecIB3G5dfS eg15r/UI7Q5/vQv93s49Wl3EKWR8peqf46nsluXLMLZff80I6S2T0wGTZP9lMHd7 62Lwo4MxQEm+0XBqVNiY438qaF9eZBR8W14BPUwn2VWD0tL8CtNLiHZwLAAnYZrs FE4mgAFUQu+cmJow4DAPkOxMqiqFHLlyh2wVKkxNVTp4fwYdLLeQZWLVLqF3Z7BR cx75ocKCZmxcKqPA6gYj0hcl1W8uj7WyAwIOcYTzBBFf0eSaBzErtZ1WS7GVM/7z 1cauEo7DsDjiW2THZDkSy/zLn/O9XxRwMddqT7rT7IxDiy+V0n6AW4WD7mA/sAkd YfEnQmZARF0hK7msiy88ScBK73NpmAmU04kVoIV1IUaKNjaahWi4sk8MDKbV/V7z qnXvbQEejH9O9FbsY0ugWB6TL1imyfE3Va+Z63/nF3GFtO+XwJ3fqpT0VbDR3YBL sYXNQ9CHRBoemIOaCLLGPJbkwAALS2/gt9aVsxHLE75efvuGAqj2Qa9g8Paj5WBH Pq8eqnjDYOX02BBjSzSr =sYA4 -END PGP SIGNATURE-
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 22/07/14 11:21, Michael Palimaka wrote: On 07/22/2014 07:52 AM, Alexander Berntsen wrote: To sum up: My vote is disable dynamic-deps. And I would be happy to apply a patch that does this with the information I have today. What a great way to kill the distro. I can already heat my house with the number of unnecessary rebuilds - I can't imagine how many people will be left once we have to needlessly rebuild libreoffice and half the tree every time someone makes some minor change. If developers can't revbump correctly to address the shortcomings of dynamic deps, why do we expect they will be able to do so for static deps? When can we expect this issue to be brought before the Council? I look forward to seeing the specific examples of unavoidable breakage that would be required to make such a decision. +1
Re: [gentoo-dev] Re: don't rely on dynamic deps
El mar, 22-07-2014 a las 10:32 +0200, Kristian Fiskerstrand escribió: [...] I find it somewhat curious that the difference between ~arch and stable hasn't been brought up in this discussion yet. IMHO a user on ~arch should expect a higher number of rebuilds, it _is_ after all testing, whereby at the point it reaches stable, the deps are hopefully more likely to be correct to begin with. Does anyone have any insight into where these changes most often occur? Well, I have seen multiple times of this kind of fixes being noticed by people running really old stable boxes. They notice them when they update to latest stable and, then, we need to fix depends raising the versions usually :/ Maybe this discussion should be focused on trying to think about how to standardize a way for distinguish between revision bumps needing full rebuild or only VDB updates :|
Re: [gentoo-dev] Re: don't rely on dynamic deps
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 22/07/14 09:39, Martin Vaeth wrote: Pacho Ramos pa...@gentoo.org wrote: Maybe this could be solved by having two kinds of revisions: - One would rebuild all as usually (for example, -r1...) - The other one would only regenerate VDB and wouldn't change the installed files (for example, -r1.1) I made the same suggestion already on the corresponding bug https://bugs.gentoo.org/show_bug.cgi?id=516612#c33 without any response. Martin, I have no arguments against your idea to tell the PM that this bump only changes dependencies. My initial reaction is that this is a Good Thing. Would you like to try to implement it? - -- Alexander berna...@gentoo.org https://secure.plaimi.net/~alexander -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlPOM9wACgkQRtClrXBQc7WV1AD+LbojEp7TVY51WobwTflCPzQK wZbmGWiyyBhylG7AgWIBAJRKURylzKxjPWcmjGwllf2CXcublXZCmndzbHeoTm0B =doak -END PGP SIGNATURE-
Re: [gentoo-dev] Re: don't rely on dynamic deps
On July 22, 2014 11:25:05 AM CEST, Pacho Ramos pa...@gentoo.org wrote: El mar, 22-07-2014 a las 10:32 +0200, Kristian Fiskerstrand escribió: [...] I find it somewhat curious that the difference between ~arch and stable hasn't been brought up in this discussion yet. IMHO a user on ~arch should expect a higher number of rebuilds, it _is_ after all testing, whereby at the point it reaches stable, the deps are hopefully more likely to be correct to begin with. Does anyone have any insight into where these changes most often occur? Well, I have seen multiple times of this kind of fixes being noticed by people running really old stable boxes. They notice them when they update to latest stable and, then, we need to fix depends raising the versions usually :/ Maybe this discussion should be focused on trying to think about how to standardize a way for distinguish between revision bumps needing full rebuild or only VDB updates :| As someone who regularly adds in dependencies without bumping (adding USE=selinux dependencies to the proper SELinux policy) because that would trigger lots of totally unnecessary rebuilds: +1 Wkr, Sven -- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Re: [gentoo-dev] Re: don't rely on dynamic deps
On Tue, 22 Jul 2014 19:03:16 +0200 Sven Vermeulen sw...@gentoo.org wrote: As someone who regularly adds in dependencies without bumping (adding USE=selinux dependencies to the proper SELinux policy) because that would trigger lots of totally unnecessary rebuilds: Er... You do realise that doing that with dynamic dependencies leads to packages depending upon selinux that don't actually use selinux, right? Thus triggering lots of totally unnecessary rebuilds on an ABI change. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: don't rely on dynamic deps
On 22/07/14 20:11, Ciaran McCreesh wrote: On Tue, 22 Jul 2014 19:03:16 +0200 Sven Vermeulen sw...@gentoo.org wrote: As someone who regularly adds in dependencies without bumping (adding USE=selinux dependencies to the proper SELinux policy) because that would trigger lots of totally unnecessary rebuilds: Er... You do realise that doing that with dynamic dependencies leads to packages depending upon selinux that don't actually use selinux, right? Thus triggering lots of totally unnecessary rebuilds on an ABI change. Err, I believe he meant adding RDEPEND -only USE=selinux to pull in eg. sec-policy/selinux-dante from net-proxy/dante So, how does that exactly make packages depending upon selinux that don't actually use selinux Doesn't make any sense - Samuli
Re: [gentoo-dev] Re: don't rely on dynamic deps
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On 22/07/14 20:40, Martin Vaeth wrote: If there is interest, I can post my patches so far. Where? If you think these patches are useful for Portage, please send them to dev-port...@gentoo.org. - -- Alexander berna...@gentoo.org https://secure.plaimi.net/~alexander -BEGIN PGP SIGNATURE- Version: GnuPG v2 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iF4EAREIAAYFAlPOslgACgkQRtClrXBQc7X58QD7BCSHxg3yiSynXe4EKpYk8R/n pZKQPCoJqQXFtkFyU/0A/2BTRMm8T60AzHFNlaeKudRPQ4EC8ACbkP8+v4C6XVoW =0GbF -END PGP SIGNATURE-
Re: [gentoo-dev] Re: don't rely on dynamic deps
Am Dienstag 22 Juli 2014, 22:40:03 schrieb Ulrich Mueller: On Tue, 22 Jul 2014, Martin Vaeth wrote: PF has to be filled correctly, of course: The versions foo-1 and foo-1-r0 are identical according to PMS and should thus lead to the same $PF. This is not so. These versions are equal in comparision, so they cannot be in the tree at the same time. However, PF will be different for them. Well we'd need a new EAPI for this anyway. So we might as well redefine -r0 there. -- Andreas K. Huettel Gentoo Linux developer kde, council
