Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
On Wed, 21 Dec 2005 09:53:29 +0900 Jason Stubbs <[EMAIL PROTECTED]> wrote: > On Wednesday 21 December 2005 01:57, Marius Mauch wrote: > > Jason Stubbs wrote: > > > Reasoning on checking all system atoms is that other groups are > > > just as likely to need the functionality as we are. Combining > > > that with how rarely versions are actually updated for system > > > packages, it shouldn't cause any more bother to users than it > > > needs to. > > > > Well, that approach has one major problem: it won't help for profile > > changes (like the mess with the cascaded profiles). > > Making the tree DEPEND on a specific portage version won't help > either. If the user doesn't have a profile, they have no ARCH or > other important things which means they can't use emerge. There can be other potential changes that will not break portage completely. > > Also I don't see what "system" has to do with handling the tree > > itself, and for anything else people can add dependencies, or am I > > missing something here? > > Doing it via system is more generic with no loss in functionality > that I can see. As I said above, the only time that not doing it in > system would be useful is if system itself is not available but > profile problems imply that one can't emerge anyway. It has other drawbacks though. First "system" is arbitrary, imagine a "nterprise desktop" profile or so where sytem includes gnome/kde. The main issue I have with it though is that it spreads the focus too much, your example had what, 20 packages? What I'm after is a information when we *know* that the tree won't work anymore, not a "it may be better to upgrade". Did I mention that I don't really like this idea in the first place? Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
On Wednesday 21 December 2005 01:57, Marius Mauch wrote: > Jason Stubbs wrote: > > Reasoning on checking all system atoms is that other groups are just as > > likely to need the functionality as we are. Combining that with how > > rarely versions are actually updated for system packages, it shouldn't > > cause any more bother to users than it needs to. > > Well, that approach has one major problem: it won't help for profile > changes (like the mess with the cascaded profiles). Making the tree DEPEND on a specific portage version won't help either. If the user doesn't have a profile, they have no ARCH or other important things which means they can't use emerge. > Also I don't see what "system" has to do with handling the tree itself, and > for anything else people can add dependencies, or am I missing something > here? Doing it via system is more generic with no loss in functionality that I can see. As I said above, the only time that not doing it in system would be useful is if system itself is not available but profile problems imply that one can't emerge anyway. -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
Jason Stubbs wrote: Reasoning on checking all system atoms is that other groups are just as likely to need the functionality as we are. Combining that with how rarely versions are actually updated for system packages, it shouldn't cause any more bother to users than it needs to. Well, that approach has one major problem: it won't help for profile changes (like the mess with the cascaded profiles). Also I don't see what "system" has to do with handling the tree itself, and for anything else people can add dependencies, or am I missing something here? Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
On Tuesday 20 December 2005 23:15, Jason Stubbs wrote: > On Tuesday 20 December 2005 01:49, Marius Mauch wrote: > > Also not talking about implementation details yet, just after comments > > about the general idea of forced portage updates. > > I gave it a go anyway... ;) Also needed: Index: portage.py === --- portage.py (revision 2414) +++ portage.py (working copy) @@ -6742,7 +6742,7 @@ mtimedbkeys=[ "updates", "info", "version", "starttime", -"resume", "ldpath" +"resume", "ldpath", "last_system_check" ] mtimedbfile=root+"var/cache/edb/mtimedb" try: -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
On Tuesday 20 December 2005 01:49, Marius Mauch wrote:
> Also not talking about implementation details yet, just after comments
> about the general idea of forced portage updates.
I gave it a go anyway... ;)
# emerge -up kde-base/kde
Checking for mandatory system updates... Done.
The following packages can't be merged until your system is updated:
kde-base/kde
The following important packages are either not up to date or not installed:
virtual/baselayout sys-apps/coreutils sys-apps/debianutils
sys-apps/diffutils sys-apps/file sys-apps/findutils sys-apps/gawk
sys-apps/grep sys-apps/groff sys-apps/kbd sys-apps/man sys-apps/man-pages
sys-apps/net-tools >=sys-apps/portage-2.0.51.22 sys-apps/sed
sys-apps/shadow sys-apps/texinfo sys-apps/which virtual/modutils
virtual/pager sys-apps/busybox sys-apps/hdparm sys-apps/util-linux
sys-apps/linux32 >=sys-apps/baselayout-1.9.4-r7
Please retry your command after updating your system.
These are the packages that I would merge, in order:
Calculating system dependencies ...done!
[ebuild N] sys-apps/sandbox-1.2.17
[ebuild N] sys-apps/sed-4.1.4 USE="-bootstrap -build -nls -static"
[ebuild N] sys-apps/debianutils-2.15 USE="-build -static"
[ebuild N] sys-apps/portage-2.1_pre1 USE="-build"
*** Portage will stop merging at this point and reload itself,
recalculate dependencies, and complete the merge.
You may avoid the remerging of packages by updating portage on its own.
[ebuild N] sys-apps/help2man-1.35.1 USE="-nls"
[ebuild N] sys-apps/coreutils-5.3.0-r2 USE="-acl -build -nls -static"
[ebuild N] sys-apps/sysvinit-2.86-r3 USE="-bootstrap -build -ibm
-static"
[ebuild U ] sys-libs/readline-5.1 [5.0-r2]
[ebuild N] sys-apps/baselayout-1.12.0_pre11-r3 USE="unicode -bootstrap
-build -static"
[ebuild N] sys-apps/diffutils-2.8.7-r1 USE="-nls -static"
[ebuild N] sys-apps/file-4.16 USE="python -build"
...
...
* Portage is moved to the top of both system and world
* All system atoms are checked that they are matched by installed packages
and emerge refuses to do anything until unsatisfied atoms are installed
and/or updated.
* EMERGE_INSANITY_OK is provided for bootstrap.sh (and emerge itself) to
bypass the checks when the system is known to be in an incomplete state.
Reasoning on checking all system atoms is that other groups are just as likely
to need the functionality as we are. Combining that with how rarely versions
are actually updated for system packages, it shouldn't cause any more bother
to users than it needs to.
--
Jason Stubbs
Index: bin/emerge
===
--- bin/emerge (revision 2414)
+++ bin/emerge (working copy)
@@ -1418,6 +1418,14 @@
mylist = sysdict.keys()
+ newlist = []
+ for atom in mylist:
+ if portage.dep_getkey(atom).split("/")[-1] == "portage":
+newlist.insert(0, atom)
+ else:
+newlist.append(atom)
+ mylist = newlist
+
for mydep in mylist:
try:
if not self.select_dep(portage.root, mydep, raise_on_missing=True):
@@ -2501,6 +2509,73 @@
if "--debug" in myopts:
edebug=1
+
+def check_and_update_system():
+ if os.environ.get("EMERGE_INSANITY_OK") or myaction == "system":
+ return
+ rsync_timestamp = open(portage.settings["PORTDIR"]+"/metadata/timestamp").read().strip()
+ last_check = portage.mtimedb.get("last_system_check", "")
+ if rsync_timestamp == last_check:
+ return
+ print "Checking for mandatory system updates... ",
+ system_atoms = getlist("system")
+ vdb = portage.db["/"]["vartree"].dbapi # This check only applies to /
+ unsatisfied_atoms = [atom for atom in system_atoms if not vdb.match(atom)]
+ print "Done."
+ if unsatisfied_atoms:
+ if myfiles:
+ allowed_keys = [portage.dep_getkey(atom) for atom in system_atoms]
+ virtuals = portage.settings.virtuals
+ for key in allowed_keys:
+if key in virtuals:
+ allowed_keys.extend(virtuals[key])
+ allowed_files = []
+ for myfile in myfiles:
+key = portage.dep_getkey(myfile)
+if key in allowed_keys:
+ allowed_files.append(myfile)
+ continue
+if "/" in myfile:
+ continue
+for category in portage.settings.categories:
+ if category+"/"+key in allowed_keys:
+ allowed_files.append(myfile)
+ break
+ disallowed_files = [myfile for myfile in myfiles if myfile not in allowed_files]
+ if disallowed_files:
+print red("\nThe following packages can't be merged until your system is updated:")
+width = 80
+for myfile in disallowed_files:
+ width += len(myfile) + 1
+ if width >= 76:
+ width = 2
+ print "\n ",
+ print myfile,
+ myfiles.remove(myfile)
+print
+ if myfiles:
+return
+ print red("\nThe following important packages are either not up to date or not installed:")
+ width = 80
+ for atom in unsatisfied_atoms:
+ width += len(atom) + 1
+ if width >= 76:
+width = 2
+print "\n ",
+
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
Jason Stubbs wrote: > On Tuesday 20 December 2005 21:42, Francesco Riosa wrote: > >>> Whenever we want/need to make structural changes to the tree that are >>> going to break backwards compability we have a serious problem >>> >> [...] >> >> Just throwing random thoughts here, but >> >> Would a rescue-portage help on this ? >> > > It's easy enough to install portage from sources (for the time being) that > I'd > say it's not necessary at this stage. A HOWTO on the matter would be much > quicker and easier to maintain while it remains viable. > > (-: great :-) -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
On Tuesday 20 December 2005 21:42, Francesco Riosa wrote: > > Whenever we want/need to make structural changes to the tree that are > > going to break backwards compability we have a serious problem > > [...] > > Just throwing random thoughts here, but > > Would a rescue-portage help on this ? It's easy enough to install portage from sources (for the time being) that I'd say it's not necessary at this stage. A HOWTO on the matter would be much quicker and easier to maintain while it remains viable. -- Jason Stubbs -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
> Whenever we want/need to make structural changes to the tree that are > going to break backwards compability we have a serious problem [...] Just throwing random thoughts here, but Would a rescue-portage help on this ? What I'm meaning is a "binary" portage provided for each ARCH (that install in /opt), it MUST be easily downloadable from every user. May be portage in case of hard fail could do that automatically issueing a message like: * OOPS, could not include module X this mean portage system is utterly broken. * would you try to download a rescue portage? The rescue-portage does not need to be a fully functional portage, it need only to be able to install a newer portage and it's direct dependancies. Regards, Francesco -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
On Mon, Dec 19, 2005 at 05:24:06PM +, Ciaran McCreesh wrote: > On Mon, 19 Dec 2005 17:49:02 +0100 Marius Mauch <[EMAIL PROTECTED]> > wrote: > | And just in case anybody wonders: this cannot be fixed with EAPI or > | adding a portage dep on packages as those only take effect when the > | ebuild is already parsed while the mentioned problems occur much > | earlier. > > Is it too late to switch to .ebuild.2, .ebuild.3 etc instead of making > EAPI an internal variable? Yes. EAPI mechanism will work as long as the ebuild is still valid shell (eclasses complicate this, but the upcoming eclass2 switch over is the only issue). EAPI isn't going to be 5 versions in 5 years, I'd expect 2-3 version a year. Any time we extend the api of provided functions to the ebuild env, we should be bumping EAPI so the ebuild is known to work. ~harring pgpyUwVC0VNpG.pgp Description: PGP signature
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
On Mon, 19 Dec 2005 17:49:02 +0100 Marius Mauch <[EMAIL PROTECTED]> wrote: | And just in case anybody wonders: this cannot be fixed with EAPI or | adding a portage dep on packages as those only take effect when the | ebuild is already parsed while the mentioned problems occur much | earlier. Is it too late to switch to .ebuild.2, .ebuild.3 etc instead of making EAPI an internal variable? -- Ciaran McCreesh : Gentoo Developer (I can kill you with my brain) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [RFC] making the tree depend on portage
On Mon, 2005-12-19 at 17:49 +0100, Marius Mauch wrote: > Ok, the subject might be confusing, so let me explain this a bit: > > Whenever we want/need to make structural changes to the tree that are > going to break backwards compability we have a serious problem (see > GLEP 44 in case you don't know about it). To reduce the impact of that > problem I've got the idea to make the tree itself (so not any > particular ebuild or profile) DEPEND on a minimal portage version, Makes syncing a bit more complicated. Downside: potentially more problems Upside: if there's a disruptive change people will at all times have a working system (just think about stacked profiles :-) ) > which the users would be forced to upgrade to (maybe with an override) > before they can do anything else (with the exception of --sync). Or you have a "version file" transferred by rsync (like timestamp) When version > portage version: fetch a static portage tree from a known location (mirror://portage-update-version-N.tbz2 or something) (emerge-delta-webrsync to the rescue!) > Manifest2 is one example for such a situation, another one is the > request to not create manifest entries for ChangeLog and metadata.xml > anymore (needs >=2.0.51.20 on user side). > Don't really like this idea myself, but somthing needs to be done to at > least reduce the problem, having to wait years for old portage versions > to (almost) vanish can't be a permanent solution. I don't like it, but unless portage is smart enough to self-update before anything else ... > Also not talking about implementation details yet, just after comments > about the general idea of forced portage updates. If documented properly and guaranteed to work for at least n months after a format change I like it (But I guess iterative updates should work "forever" as long as all files still exist) > And just in case anybody wonders: this cannot be fixed with EAPI or > adding a portage dep on packages as those only take effect when the > ebuild is already parsed while the mentioned problems occur much > earlier. Thanks for bringing it up for discussion! Patrick -- Stand still, and let the rest of the universe move signature.asc Description: This is a digitally signed message part
[gentoo-portage-dev] [RFC] making the tree depend on portage
Ok, the subject might be confusing, so let me explain this a bit: Whenever we want/need to make structural changes to the tree that are going to break backwards compability we have a serious problem (see GLEP 44 in case you don't know about it). To reduce the impact of that problem I've got the idea to make the tree itself (so not any particular ebuild or profile) DEPEND on a minimal portage version, which the users would be forced to upgrade to (maybe with an override) before they can do anything else (with the exception of --sync). Manifest2 is one example for such a situation, another one is the request to not create manifest entries for ChangeLog and metadata.xml anymore (needs >=2.0.51.20 on user side). Don't really like this idea myself, but somthing needs to be done to at least reduce the problem, having to wait years for old portage versions to (almost) vanish can't be a permanent solution. Also not talking about implementation details yet, just after comments about the general idea of forced portage updates. And just in case anybody wonders: this cannot be fixed with EAPI or adding a portage dep on packages as those only take effect when the ebuild is already parsed while the mentioned problems occur much earlier. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
