Re: [gentoo-portage-dev] Re: equery refactorization
On Sun, 7 Dec 2008 12:44:25 +0900 Douglas Anderson [EMAIL PROTECTED] wrote: I also thought about renaming the list(l) option as search, because if you look at the help output, almost every module lists something. equery's list is actually a search, I don't see why we shouldn't name it that. I think maybe list was used because there were already two s options, stats and size. Stats is not implemented so I'm taking it out of help for now. Size can use the short z, becaues that's quite unique. That would free up s for search and it would be a whole lot clearer. Yes? No? No. search (if used at all) should be reserved for a more comprehensive search framework (though IMO a separate tool for that is more appropriate), not just a simple name match. list makes sense if you consider that the pkgspec argument is optional, and one of the main tasks of it is to simply list the packages in the given repository (that's why vardb is also the default for it) without further filtering. Also one of the main goals of equery (according to karltk, the original author) was to have a stable user interface, compared to the deprecated qpkg and etcat scripts. And while the equery interface isn't exactly the best I've seen it has been stable, so you might want to think twice before renaming options and eventually pissing off users or breaking third-party scripts. Marius
[gentoo-portage-dev] Time to say goodbye
So, time has come for me to realize that my time with Gentoo is over. I haven't actually been doing much Gentoo work over the last months due to personal reasons (nothing Gentoo related), and I don't see that situation changing in the near future. In fact I've already reassigned or dropped most of my responsibilites in Gentoo a while ago, so there are just a few pet projects left to give away: - my gentoo-stats project (in the portage/gentoo-stats svn repository). I know quite a few people are interested in the idea of collecting various statistic data from gentoo user systems, and I'd encourage everyone who wants to implement such a system to at least look at it (I may have even finished it if I wouldn't have wasted my time focusing on the wrong problems). There is quite a bit of documentation also that should help to get you started - a graphical security update tool (see bug #190397) So if anyone wants to adopt those, complete or just parts, just take them. As for Portage, Zac has practically already filled my role. So I guess that wraps it up. It's been a nice ride most of the time, but now it's time for me to leave the Gentoo train. Marius
Re: [gentoo-portage-dev] search functionality in emerge
On Sun, 23 Nov 2008 07:17:40 -0500 Emma Strubell [EMAIL PROTECTED] wrote: However, I've started looking at the code, and I must admit I'm pretty overwhelmed! I don't know where to start. I was wondering if anyone on here could give me a quick overview of how the search function currently works, an idea as to what could be modified or implemented in order to improve the running time of this code, or any tip really as to where I should start or what I should start looking at. I'd really appreciate any help or advice!! Well, it depends how much effort you want to put into this. The current interface doesn't actually provide a search interface, but merely functions to 1) list all package names - dbapi.cp_all() 2) list all package names and versions - dbapi.cpv_all() 3) list all versions for a given package name - dbapi.cp_list() 4) read metadata (like DESCRIPTION) for a given package name and version - dbapi.aux_get() One of the main performance problems of --search is that there is no persistent cache for functions 1, 2 and 3, so if you're just interested in performance aspects you might want to look into that. The issue with implementing a persistent cache is that you have to consider both cold and hot filesystem cache cases: Loading an index file with package names and versions might improve the cold-cache case, but slow things down when the filesystem cache is populated. As has been mentioned, keeping the index updated is the other major issue, especially as it has to be portable and should require little or no configuration/setup for the user (so no extra daemons or special filesystems running permanently in the background). The obvious solution would be to generate the cache after `emerge --sync` (and other sync implementations) and hope that people don't modify their tree and search for the changes in between (that's what all the external tools do). I don't know if there is actually a way to do online updates while still improving performance and not relying on custom system daemons running in the background. As for --searchdesc, one problem is that dbapi.aux_get() can only operate on a single package-version on each call (though it can read multiple metadata variables). So for description searches the control flow is like this (obviously simplified): result = [] # iterate over all packages for package in dbapi.cp_all(): # determine the current version of each package, this is # another performance issue. version = get_current_version(package) # read package description from metadata cache description = dbapi.aux_get(version, [DESCRIPTION])[0] # check if the description matches if matches(description, searchkey): result.append(package) There you see the three bottlenecks: the lack of a pregenerated package list, the version lookup for *each* package and the actual metadata read. I've already talked about the first, so lets look at the other two. The core problem there is that DESCRIPTION (like all standard metadata variables) is version specific, so to access it you need to determine a version to use, even though in almost all cases the description is the same (or very similar) for all versions. So the proper solution would be to make the description a property of the package name instead of the package version, but that's a _huge_ task you're probably not interested in. What _might_ work here is to add support for an optional package-name-description cache that can be generated offline and includes those packages where all versions have the same description, and fall back to the current method if the package is not included in the cache. (Don't think about caching the version lookup, that's system dependent and therefore not suitable for caching). Hope it has become clear that while the actual search algorithm might be simple and not very efficient, the real problem lies in getting the data to operate on. That and the somewhat limited dbapi interface. Disclaimer: The stuff below involves extending and redesigning some core portage APIs. This isn't something you can do on a weekend, only work on this if you want to commit yourself to portage development for a long time. The functions listed above are the bare minimum to perform queries on the package repositories, but they're very low-level. That means that whenever you want to select packages by name, description, license, dependencies or other variables you need quite a bit of custom code, more if you want to combine multiple searches, and much more if you want to do it efficient and flexible. See http://dev.gentoo.org/~genone/scripts/metalib.py and http://dev.gentoo.org/~genone/scripts/metascan for a somewhat flexible, but very inefficient search tool (might not work anymore due to old age). Ideally repository searches could be done without writing any application code using some kind of query language, similar to how SQL works for generic database searches (obviously not that complex). But
Re: [gentoo-portage-dev] search functionality in emerge
On Sun, 23 Nov 2008 21:01:40 -0800 (PST) devsk [EMAIL PROTECTED] wrote: not relying on custom system daemonsrunning in the background. Why is a portage daemon such a bad thing? Or hard to do? I would very much like a daemon running on my system which I can configure to sync the portage tree once a week (or month if I am lazy), give me a summary of hot fixes, security fixes in a nice email, push important announcements and of course, sync caches on detecting changes (which should be trivial with notify daemons all over the place) etc. Why is it such a bad thing? Well, as an opt-in solution it might work (though most of what you described is IMO just stuff for cron, no need to reinvent the wheel). What I was saying is that _relying_ on custom system daemons/filesystems for a _core subsystem_ of portage is the wrong way, simply because it adds a substantial amount of complexity to the whole package management architecture. It's one more thing that can (and will) break, one more layer to take into account for any design decisions, one more component that has to be secured, one more obstacle to overcome when you want to analyze/debug things. And special care must be taken if it requires special kernel support and/or external packages. Do you want to make inotify support mandatory to use portage efficiently? (btw, looks like inotify doesn't really work with NFS mounts, which would already make such a daemon completely useless for people using a NFS-shared repository) And finally, if you look at the use cases, a daemon is simply overkill for most cases, as the vast majority of people only use emerge --sync (or wrappers) and maybe layman to change the tree, usually once per day or less often. Do you really want to push another system daemon on users that isn't of use to them? Its crazy to think that security updates need to be pulled in Linux. That's IMO better be handled via an applet (bug #190397 has some code), or just check for updates after a sync (as syncing is the only way for updates to become available at this time). Maybe a message could be added after sync if there are pending GLSAs, now that the glsa support code is in portage. Marius
Re: [gentoo-portage-dev] [2/4] proto-GLEPS for Tree-signing
On Tue, 29 Jul 2008 20:51:45 +0100 Mike Auty [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Sorry, I lost my notes from when I last looked these over several months ago, and only just found them again. I haven't copied this to [EMAIL PROTECTED], so let me know if I should do that. I just had a quick couple of things I was thinking about, and one of them I figured out during my re-read, so it's only really the following... In this Glep (xx+1), in the section discussing the procedure for creating a MetaManifest file, in step 3.3, does that include verification of the manifest's signature if it has one? It would seem odd to ignore the signature if it's wrong (I'm not sure about the case if a signature isn't present). I also don't know how this would then be handled (a complete abort, or ignoring the latest changeset to that ebuild?). I don't think that verification at this stage would be a good idea. The only sane way to respond to a failed check would be to either exclude the whole package from the sync (keeping the state from the last run), leading to various problems (what if it's a critical bugfix/security bump, or breaks a the deptree of many packages?), or not record the Manifest in the Metamanifest, which hasn't any benefits over reyling on the client doing the verification. Marius
Re: [gentoo-portage-dev] portage-2.2-rc3 parallel merges quit being parallel
On Sat, 26 Jul 2008 16:56:20 -0500 Andrew Gaffney [EMAIL PROTECTED] wrote: Duncan wrote: --jobs=10 --keep-going --load-average=15 For a dual-dual-core setup, a load average of 4.0 is fully loaded. Only in ideal cases, when you have long-running processes hammering the cpu and little or no context switches. This isn't the case with builds, the actual compile processes that need cpu time usually terminate very quickly, which increases the load average. I did some benchmarking a while ago with different combinations of -j and -l in MAKEOPTS, using the kernel as testcase, and IIRC the fastest builds were around -l6.0 (on a dual-core system) and high (or unlimited) values for -j (sidenote here: some ebuilds like openoffice parse MAKEOPTS to get parameters for their own build systems, but only recognize/support -j and ignore -l, so one should still be careful with the value for -j). Marius
Re: [gentoo-portage-dev] A seemingly bug found when emerging @preserved-rebuild
On Wed, 2 Jul 2008 02:41:14 +0800 Zhang Le [EMAIL PROTECTED] wrote: Is this a bug? or did I miss something here? Thanks for your time! The 'bug' here is that USE=multislot shouldn't exist. People using it should be able to deal with resulting breakages on their own Marius -- gentoo-portage-dev@lists.gentoo.org mailing list
Re: [gentoo-portage-dev] [PATCH] Repoman subversion support
On Mon, 12 May 2008 20:01:34 +0200 Fabian Groffen [EMAIL PROTECTED] wrote: After a short discussion on #-portage, here a feature patch from the prefix branch, diffed to the trunk, irrelevant hunks removed. Antarus once was working on factoring out the vcs bits of repoman into pym/repoman/utilities.py, but he never finished it. I'm willing to give that some more time, provided it'll end up in the trunk. Sofar, here's the svn patch that we tested extensively in Prefix ;) I still believe that it has a significant value for non-Prefix users (overlays...), even though the code doesn't deserve any price for its beauty. I'd say it fits nicely with large chunks of Portage :p Merged in r10325 with some minor changes (removed the 'svn update' bit until someone remembers why exactly it's there, skip the commit optimization for svn regarding Manifest regeneration, unify isVcs and vcs variables) Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] BNF for DEPEND in EAPI0 and EAPI1
On Sun, 13 Apr 2008 20:04:41 +0200 Björn Michaelsen [EMAIL PROTECTED] wrote: Hi there, I worked through the current PMS and tried to formalize the grammar for DEPEND a little better. Maybe someone could have a look at it and check it for correctness. It could even become part of the gentoo development docs or be an appendix to the PMS. The other grammars (SRC_URI etc.) should be fairly easy. Any comments? Offhand the 'package name' part is wrong/incomplete, pretty sure even PMS says that they can't start/end with underscores or dashes (it's a real PITA to get it right). Haven't really read the non-terminals section as it's hard to read in this form, using EBNF or some markup language might help with that. Marius -- gentoo-portage-dev@lists.gentoo.org mailing list
Re: [gentoo-portage-dev] Re: querying whether pkg is installed with version X or upwards (atom syntax) (re-post)
On Wed, 26 Mar 2008 12:07:13 +0200 Amit Dor-Shifer [EMAIL PROTECTED] wrote: reposting, hoping for a reply this time. Thanks, Amit Amit Dor-Shifer wrote: emerge features syntax for installing a package according to certain bounds. This is referred to in the manpage as atom specification. I'm wondering whether equery has the same feature for querying. E.G: I'd like to be able to ask: equery l '=app-misc/my-pkg-1.1' I'd expect the listing to be empty when my-pkg is not installed, or is installed by of version less than 1.1. Well, most equery subcommands support the same atom syntax, just 'list' and 'size' apparently don't implement it, as those support regular expressions instead. Marius -- gentoo-portage-dev@lists.gentoo.org mailing list
Re: [gentoo-portage-dev] Re: [gentoo-qa] splitting up package.mask
On Sat, 15 Mar 2008 01:34:24 -0700 Alec Warner [EMAIL PROTECTED] wrote: On 3/14/08, Mike Frysinger [EMAIL PROTECTED] wrote: On Friday 14 March 2008, Alec Warner wrote: On 3/14/08, Mike Frysinger [EMAIL PROTECTED] wrote: i think the real solution here is allowing masking in a package You want to add a metadata key and cache it you mean? i dont care terribly much about the logistics, just the results. as long as an ebuild can declare itself masked, it sounds good to me. this doesnt preclude the other ideas as there are often times where you want to have 1 global package mask piece (like large package set bumps ... so X or KDE or GNOME or ...). -mike [-gentoo-qa, +gentoo-portage-dev] Original thread was splitting up package.mask entries. Genone notes the code to do this already is basically in already (we just don't invoke it for $PORTDIR/profiles afaik). Genone, do we use existing code for package.mask (ie if we switch from a file to a dir will it break existing versions? I am unsure if we used the directory code for $PORTDIR/profiles/* The handling code for all package.mask files is identical, in particular they all support recursion (just search for package.mask in portage/__init__.py). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] please explain use of hooks
On Sun, 2 Mar 2008 11:18:39 +0100 Jonas Bernoulli [EMAIL PROTECTED] wrote: And if not the handbook this should really go into the portage manpage. bashrc If needed, this file can be used to set up a special environment for ebuilds, different from the standard root environment. The syntax is the same as for any other bash script. + One thing you can do here is define phase hooks. + A phase hook function name begins with a pre_ or post_ prefix to indicate that it will be called before or after one of the ebuild phases. The prefix is followed by the name of the ebuild function that the hook will be associated with. + For example, a hook named pre_src_compile will be called before src_compile, and a hook named post_src_compile will be called after src_compile. Care to write a patch? (nobody likes to edit manpages) postsync.d should also be documented there. No, as it isn't used by portage itself. All in all now that I have looked at the documentation installed with portage I think it is obsolete. Not in the sense that it is outdated but everything in it is documented elsewhere already or should be moved there. Well, it's horribly incomplete and a work in progress. Eventually it will replace the manpages (in the sense that manpages will be generated from the same source). And authorative documentation (which isn't exactly the same as a specification) should be kept with the implementation, and that's not the case with the dev handbook/manual. If some developers an the list agree with me and welcome me to do so then I could file a bug(s) for those changes. No. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
[gentoo-portage-dev] New maintainers needed
For various reasons I can no longer maintain the following packages (in fact I haven't really maintained them for a while already), so they need a new maintainer: * app-editors/zoink - simple gtk based editor, little to no maintenance required * dev-util/gambas - Visual Basic look-alike for Linux, fairly active development (2.0 is about to come out), doesn't work on 64bit archs (only known to work on x86), has a number of open bugs Also I'm going to remove myself from the tools-portage team as I haven't done much there either, which effectively only leaves Paul (fuzzyray) on it, who is also somewhat inactive recently, so we could use a few new people there as well. Finally at the beginning of this year I started to convert packages from einfo to elog where appropriate, unfortuntely I haven't been able to complete that job, if anyone is interested in completing it please let me know (I guess it's currently about 40-60% done). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] use* cleanup
On Tue, 30 Oct 2007 19:32:52 +0100 Marijn Schouten (hkBst) [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 The purpose of this patch is to expose a generic function, namely _use, which can be used to build your own use* variant if you need that. I reimplemented all other current use function using _use (and _if) to cut out duplicate and verbose code. Comments expected. I didn't test this code. Please don't post patches inline, there are these things called attachments that work so much better for such things. Marius -- [EMAIL PROTECTED] mailing list
Re: [gentoo-portage-dev] RFC: replacing packages
On Thu, 25 Oct 2007 22:14:40 -0700 Alec Warner [EMAIL PROTECTED] wrote: Another issue that isn't directly related, but covered by the proposed solution below, is the so called implicit system dependency in ebuilds, which doesn't really exist. It only an assumption by people that the system set is satisfied before an ebuild is processed, so its contents don't have to be listed in *DEPEND. If a user breaks that assumption by not regulary ensuring that system is satisfied bugs can occur. Another issue is the informal exception when system items are allowed to/must be listed in *DEPEND, IMO that should be formalized. I'm going to discuss a bit your implementation below, since you address what you are trying to solve here. My main problem is that given a tree (such as gentoo-x86) it may have one or more profiles with one or more sub-profiles (children). These profiles may contain a set of packages that make up 'system'. The ebuilds DEPEND (or RDEPEND or whatever) on this set of packages existing. The problem I see with your solution is that it doesn't address different system sets in different profiles. Now in practice this isn't much of a problem within gentoo-x86 (most profiles have similar system sets). However it places a certain onus on any profile in a given tree, it must have a system package 'similar' enough to other sets in functioning profiles; otherwise breakage is likely to occur. How would you address this in your system? How is the problem solved now? Remember that I'm just proposing to formalize existing assumptions (packages in system should state all their dependencies explicitly). The answer is that there is no real solution to that problem, at least not without creating a huge mess by enumerating all available profiles. The real question is what system is supposed to be. Is it just a more or less arbitrary list of packages chosen by the profile maintainer, or does it have a global, predefined purpose, like defining the minimum requirements for packages in the associated tree? In the first case the second part of my proposal obviously doesn't work, as you can't make any valid assumptions about system. In the second case however the purpose already mandates that the content of system is effectively identical across different profiles. Marius -- [EMAIL PROTECTED] mailing list
Re: [gentoo-portage-dev] RFC: replacing packages
On Wed, 24 Oct 2007 13:34:44 -0500
Andrew Gaffney [EMAIL PROTECTED] wrote:
For packages that are in the system set, wouldn't adding the
contents of system.{,r}depend to {,R}DEPEND cause problems in dep
resolution? Would there be a way to prevent the contents of these
files from being added for packages that are themselves in these
files?
Did you miss the part about RESTRICT=system-deps?
How is the system set constructed from these files? Would they
simply be concatenated (minus duplicates) and treated like the old
packages file (minus the magic visibility filtering)?
Maybe, we'd have to see what works best.
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
signature.asc
Description: PGP signature
[gentoo-portage-dev] RFC: replacing packages
As package sets are mostly done now, I'm starting to think about something else. One of my pet peeves with portage is the packages file in profiles, for several reasons: 1) it has two completely independent purposes 2) it implements a redundant visibility filter as package.mask is also available in profiles 3) the syntax for defining the system set plain sucks 4) the name doesn't really say anything about the purpose Another issue that isn't directly related, but covered by the proposed solution below, is the so called implicit system dependency in ebuilds, which doesn't really exist. It only an assumption by people that the system set is satisfied before an ebuild is processed, so its contents don't have to be listed in *DEPEND. If a user breaks that assumption by not regulary ensuring that system is satisfied bugs can occur. Another issue is the informal exception when system items are allowed to/must be listed in *DEPEND, IMO that should be formalized. To solve both issues I propose the following system: Profiles can contain the files system.depend and system.rdepend whose combined contents make up the system target. If any of those two files exists the packages file is ignored. The syntax should either be the same as package.mask or, if desired, also allow complex atoms (use-conditionals, any-of constructs). The second part would be that portage implicitly adds the content of the files to the DEPEND or RDEPEND setting of each ebuild unless it contains RESTRICT=system-deps. Obviously this would have to be tied to a new EAPI version. (an undefined cornercase here is if a profile does not contain the new files, not sure how that should be handled) Benefits: - we get rid of all the issues outlined at the top - should make it easier long-term to setup a no-compile system that only installs binary packages due to separation of build- and runtime deps in system Problems: - obviously it has to be implemented first - no obvious solution for stacking rules (e.g. child uses system.*, but parent profile only has packages) - some people might rely on the packages file So, comments? Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Re: About system and world
On Mon, 22 Oct 2007 21:29:02 -0700 Zac Medico [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ryan Hill wrote: Zac Medico wrote: implement greedy atoms for the world set. I've been pondering the idea of making world non-greedy for slots by default [1], since you can add slot atoms to the world file to pull in any specific slot(s) that you want. That would rock. Portage always insisting on pulling in the highest SLOT whether needed or not is one of my biggest pet-peeves. Well, you can already use SLOT atoms in your world file if you don't want the highest available. Packages that pull in =foo are a different story though. I suppose we can add something like a - --upgrade=minimal option that prevents pulling in new slots if they aren't required. Don't restrict it to SLOTs though. minimal implies that only upgrades required to satisfy the depgraph are performed. The described behavior should be another value, e.g. no-slot-change. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] About system and world
On Sun, 21 Oct 2007 15:12:31 +0200 Marius Mauch [EMAIL PROTECTED] wrote: Well, the primary goal is to make all sets behave in a consistent way. And some sets have the explicit purpose to rebuild stuff, so making sets selective by default also has issues. The proposed change would also make sets behave in the same way as packages which is IMO another benefit. This actually has more consequences: selective is a global setting, you can't enable it just for some arguments. Therefore if packages and sets are treated differently it requires that they can't be mixed on the commandline (and if we'd make the setting configurable for each set then only one set can be used at any time). And right now I can't think of another reason why that restriction would be necessary. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Re: About system and world
On Tue, 23 Oct 2007 21:11:48 +0200
Thomas de Grenier de Latour [EMAIL PROTECTED] wrote:
On 2007/10/23, Marius Mauch [EMAIL PROTECTED] wrote:
On Mon, 22 Oct 2007 21:29:02 -0700
Zac Medico [EMAIL PROTECTED] wrote:
Well, you can already use SLOT atoms in your world file if you
don't want the highest available. Packages that pull in =foo are
a different story though. I suppose we can add something like a
- --upgrade=minimal option that prevents pulling in new slots if
they aren't required.
Don't restrict it to SLOTs though. minimal implies that only
upgrades required to satisfy the depgraph are performed.
Couldn't this minimal behavior just be triggered by lack of the
--upgrade option (emerge -D set)?
--upgrade != the current --update (at least that's what I assumed)
Actually, the current --upgrade behavior (with or without --deep)
is a bit weird imho, i would prefer something like this (with foo
being either a set or an atom):
Yes, --update has a history of being a mess of random behavior.
Personally I'd drop it completely, or alias it to --noreplace.
* emerge foo: do the minimum upgrades or new installs required to
get foo satisfied. Only its direct deps are checked (and direct
deps of the unsatisfied ones, etc.), with the minimal heuristic
(upgrade them only if stricly required).
* emerge -u foo: do all required new installs and possible upgrades
of foo. Only its direct deps are checked (and direct deps of the
unsatisfied ones, etc.), with the minimal heuristic (upgrade them
only if strictly needed).
* emerge -D foo: do the minimal upgrades or new installs required
to get foo satisfied. Also, check all of its direct and indirect
deps, with the minimal heuristic (upgrade only if strictly needed).
* emerge -uD foo: do all required new installs and possible
upgrades of foo. Also, all its direct and indirect deps are checked,
and upgraded where possible.
Maybe --upgrade as name was an unfortunate choice, --resolver might be
better. What I'd like ideally (as a user) is to replace --update,
--deep, --noreplace and --emptytree with the following options:
--resolver={minimal,noslotchange,leastchange,default}
minimal: select the lowest possible version
noslotchange: like default, but try to avoid slotchanges
leastchange: select the version with the smallest version delta
compared to the installed version, otherwise like default
default: current behavior
--rebuild={never,always,changeduse,newuse,changeddeps,selected}
never: never rebuild packages
always: always rebuild packages (like current --emtpytree)
changeduse: rebuild packages if $USE has changed
newuse: rebuild packages if $USE or $IUSE has changed
changeddeps: rebuild packages if any of its direct dependencies were
updated
selected: rebuild the selected arguments (root nodes)
--deplevel=n
check n level of dependencies for updates (with -1 == infinite)
of course all dependencies have to be satisfied, this controls only
what deps should be checked for optional updates
current defaults would be --resolver=default --rebuild=selected
--deplevel=0
--update would be --rebuild=never --deplevel=1
--deep would be --deplevel=-1
--emptytree would be --rebuild=always --deplevel=-1
But that's just me dreaming of a better world, where we don't have
to deal with legacies ;)
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
signature.asc
Description: PGP signature
Re: [gentoo-portage-dev] Re: About system and world
On Wed, 24 Oct 2007 00:03:47 +0200 Thomas de Grenier de Latour [EMAIL PROTECTED] wrote: Bah, you're introducing a new options set, and have (i think) ways to emulate all of the old ones with them, so what is stopping you? You would just have to forbid mixes of old style and new style on the same command line. Mainly the work required to implement it ;) And I'd rather finish dynoparse first so it can be used here. Oh, and ideally the mentioned values wouldn't be hardcoded lists, but come from a pluggable resolver system, but that probably counts as overengineering. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] localization.py
On Mon, 22 Oct 2007 04:54:59 +0200 Arfrever Frehtes Taifersar Arahesis [EMAIL PROTECTED] wrote: 2007-10-21 22:49:10 Marius Mauch napisał(a): On Sun, 21 Oct 2007 12:23:59 -0700 Zac Medico [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Arfrever Frehtes Taifersar Arahesis wrote: Hello, Does localization.py exist for a reason? Over the years we've had a few people express a desire for internationalization and localization support in portage (bugs #13618, #32630, and #191189 and possible others). It seems like a nice idea to me, but I recall Marius being opposed to the idea due to some potential problems that he was concerned about. Unfortunately I don't have a link to the previous discussion but maybe somebody else does. Probably not the one you meant, but you can find a related discussion on the forums: http://forums.gentoo.org/viewtopic-t-500209.html It seems that you just don't like German translation, so you could set LC_ALL=C in make.conf. I don't have an issue with localizations on my systems (I don't use them anyway), the problem is with other people reporting bugs or asking for support posting messages we can't understand, the questionable quality of translations (as we can't really control them) potentially leading to bogus bug reports and the general added complexity. It's just my opinion that system tools simply shouldn't be localized (best example: gcc error messages). Marius -- [EMAIL PROTECTED] mailing list
Re: [gentoo-portage-dev] About system and world
On Sun, 21 Oct 2007 05:23:45 -0700 Ned Ludd [EMAIL PROTECTED] wrote: On Sun, 2007-10-21 at 13:01 +0200, Marius Mauch wrote: So, what do people think about removing (some) of the special treatment for the system and world targets? Mainly I'm interested in removing the selective parameter that's currently enabled for them (so for example without that parameter `emerge world` would default to remerging packages, unless --update or --noreplace are specified). That change has already been requested a few times in the past by users, but OTOH it could probably upset people who don't use --update with world/system. What would such a disruptive change really gain us? The goal is to make package sets behave in a consistent way. I personally feel our users need consistency from Gentoo. If they grew up doing 'emerge world' and have come to expect that behavior and all of the sudden we change behavior on them.. Yeah I can see how ppl would get upset. As do I, which is why I haven't simply changed it. Perhaps a less intrusive way would be to introduce another flag to get the specified behavior you are after. Well, the primary goal is to make all sets behave in a consistent way. And some sets have the explicit purpose to rebuild stuff, so making sets selective by default also has issues. The proposed change would also make sets behave in the same way as packages which is IMO another benefit. But as I said, I can see that it could upset people. One possible solution that I've thought about would be to remove the hardcoded selective parameter and let the set configuration determine if a set is selective or not (with missing = false), and then enable it for world and system in our default config, e.g. [world] class = portage.sets.files.WorldSet selective = True and extend the --rebuild option with new arguments always and never. Don't really like the additional complexity though (and I haven't checked how much work it would be). Marius -- [EMAIL PROTECTED] mailing list
Re: [gentoo-portage-dev] localization.py
On Sun, 21 Oct 2007 12:23:59 -0700 Zac Medico [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Arfrever Frehtes Taifersar Arahesis wrote: Hello, Does localization.py exist for a reason? Over the years we've had a few people express a desire for internationalization and localization support in portage (bugs #13618, #32630, and #191189 and possible others). It seems like a nice idea to me, but I recall Marius being opposed to the idea due to some potential problems that he was concerned about. Unfortunately I don't have a link to the previous discussion but maybe somebody else does. Probably not the one you meant, but you can find a related discussion on the forums: http://forums.gentoo.org/viewtopic-t-500209.html -- [EMAIL PROTECTED] mailing list
[gentoo-portage-dev] Re: [gentoo-dev] [RFC] Properties of package sets
On Fri, 29 Jun 2007 05:07:28 +0200 Marius Mauch [EMAIL PROTECTED] wrote: Please reply on gentoo-portage-dev, _not_ on gentoo-dev, thanks. One missing feature in portage is the lack of package sets. Before we (re)start working on that however I'd like to get some feedback about what properties/features people would expect from portage package set support. Some key questions: - should they simply act like aliases for multiple packages? E.g. should `emerge -C sets/kde` be equivalent to `emerge -C kdepkg1 kdepkg2 kdepkg3 ...`? Or does the behavior need to be smarter in some ways? - what kind of atoms should be supported in sets? Simple and versioned atoms for sure, but what about complex atoms (use-conditional, any-of, blockers)? - should sets be supported everywhere, or only in selected use cases? (everywhere would include depstrings for example) - what use cases are there for package sets? Other than the established system and world, and the planned all and security sets. - how/where should sets be stored/distributed? Forgot one question: - should sets have metadata? (e.g. a description for searching) There hasn't been much feedback yet, so if you want to add anything now is your chance, otherwise I'll implement things the way that works best/is easiest for me, which might be different from what you expect. Marius PS: I also accept off-list replies -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
[gentoo-portage-dev] [RFC] Properties of package sets
Please reply on gentoo-portage-dev, _not_ on gentoo-dev, thanks. One missing feature in portage is the lack of package sets. Before we (re)start working on that however I'd like to get some feedback about what properties/features people would expect from portage package set support. Some key questions: - should they simply act like aliases for multiple packages? E.g. should `emerge -C sets/kde` be equivalent to `emerge -C kdepkg1 kdepkg2 kdepkg3 ...`? Or does the behavior need to be smarter in some ways? - what kind of atoms should be supported in sets? Simple and versioned atoms for sure, but what about complex atoms (use-conditional, any-of, blockers)? - should sets be supported everywhere, or only in selected use cases? (everywhere would include depstrings for example) - what use cases are there for package sets? Other than the established system and world, and the planned all and security sets. - how/where should sets be stored/distributed? Please reply on gentoo-portage-dev, _not_ on gentoo-dev, thanks. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [RFC] Properties of package sets
On Thu, 28 Jun 2007 21:03:54 -0700 Ned Ludd [EMAIL PROTECTED] wrote: On Fri, 2007-06-29 at 05:07 +0200, Marius Mauch wrote: Please reply on gentoo-portage-dev, _not_ on gentoo-dev, thanks. One missing feature in portage is the lack of package sets. Before we (re)start working on that however I'd like to get some feedback about what properties/features people would expect from portage package set support. Some key questions: - should they simply act like aliases for multiple packages? E.g. should `emerge -C sets/kde` be equivalent to `emerge -C kdepkg1 kdepkg2 kdepkg3 ...`? Or does the behavior need to be smarter in some ways? I like the alias way acting simply as a metapkg. Well, mind that a very simple alias implementation as outlined above would for example unmerge kdepkg3 even if it was merged explicitly. - should sets be supported everywhere, or only in selected use cases? (everywhere would include depstrings for example) Please NO. emerge.py should know about sets but ebuild.py should be oblivious to them. package sets as you are proposing imo should be limited to portage only. By putting package sets in ebuild depstrings we would be effectively forcing all other pkg managers to support the feature. I don't think we should do that. What about config files (package.*) or other sets? - what use cases are there for package sets? Other than the established system and world, and the planned all and security sets. Assuming you guys run with with the simple alias method I don't see how 'security' can fit into this. Simple: the list of atoms would be assembled dynamically by checking the available GLSAs, similar to how the affected target works in glsa-check. But lets keep the backend implementation details out for now. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] LC_ALL and friends in make.conf
On Thu, 8 Mar 2007 20:14:26 +0100
Kevin F. Quinn [EMAIL PROTECTED] wrote:
This is an old issue, but I want to suggest a re-visit :)
As we all know, setting LC_ALL and friends can cause all sorts of
trouble in package builds. However, many users really appreciate
being able to set it so that errors from the compiler etc are in their
own language.
Actually most people I've talked to prefer to have buildsystem messages
in english even if the rest of their system is localized. Reasons
include:
- many users don't understand the messages anyway, so to them language
is irrelevant
- often the translations are harder to understand than the english
version (IMO)
- when reporting a bug or asking questions on forums/irc/lists/...
they'll usually need the english messages anyway
- when searching for error messages in bugzilla/google/... the english
version will usually get more and/or better results
So going with Mike I'd invert your proposal and have portage (un)set
LC_MESSAGES to get english buildsystem messages, e.g. by adding
export LC_MESSAGES=${PORTAGE_LC_MESSAGES:-en_US}
to ebuild.sh.
Marius
--
gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] dep resolution weirdness
On Sat, 17 Feb 2007 12:30:31 +0100 George Shapovalov [EMAIL PROTECTED] wrote: Hi guys. I am quite confused by the following: aldar ~ # emerge -puD world --tree These are the packages that would be merged, in reverse order: Calculating world dependencies... done! [nomerge ] dev-ada/cbind-6.0 [nomerge ] virtual/gnat-4.1 [ebuild UD] dev-lang/gnat-gcc-4.1.1 [4.1.2] aldar ~ # emerge -puD cbind --tree These are the packages that would be merged, in reverse order: Calculating dependencies... done! (or, for that matter: aldar gnat # e -puD virtual/gnat These are the packages that would be merged, in order: Calculating dependencies... done! The relevant dependency lines are here: dev-ada/cbind: DEPEND=virtual/gnat virtual/gnat-4.1: RDEPEND==dev-lang/gnat-gcc-4.1* DEPEND= (and virtual/gnat-3.4: RDEPEND=|| ( =dev-lang/gnat-gcc-3.4* =dev-lang/gnat-gpl-3.4* ) DEPEND= although it looks like this one is not considered in this particular case) As you see emerge -uD world wants to downgrade gnat for some reason (which is wrong), while emerge -uD cbind (or any other Ada library that has similar dependencies) does not (which is right). Is there maybe a different package in world that depends on =gnat-gcc-4.1.1? Anyway, you may want to look at --debug output to get a clue why it wants to downgrade. Oh, and ~dev-lang/gnat-gcc-4.1 as RDEPEND in virtual/gnat does not work at all (just, as I understand, it should not)). It should work, but it's not the same as the dep you stated above: it would match gnat-gcc-4.1-r1, gnat-gcc-4.1-r2 and so on, but not gnat-gcc-4.1.1, gnat-gcc-4.1.2, ... Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
[gentoo-portage-dev] New preserve-libs feature
If you haven't noticed, I just added a new 'preserve-libs' feature for bug 62207 that moves shared object that are still used but would be removed on an update into the new package instance (so on a update from expat-1 to expat-2 the user would still have libexpat.so.0, owned by expat-2). The actual match criteria are a bit stricter, but you get the idea. I think this is an long overdue bugfix, but given past discussions not everyone has the same opinion, though the last discussion about it ended without a conclusion (at least none that I could see). So everyone who has valid objections to the _general idea_ of this implementation (preserving old libraries to avoid some runtime linker errors) speak up now. Just keep the following things in mind: - I'm not claiming that it's a silver bullet to all possible runtime linker problems, it's supposed to cover some of the common cases (like the expat problem) - I'm not claiming that the implementation is perfect yet - This feature is currently optional, so I'm not forcing it down on anyone (in fact it's completely hidden for now). Of course if people start using it ebuild devs will have to deal with it sooner or later, but lets discuss it here first. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] dep resolution weirdness
On Sat, 17 Feb 2007 13:42:49 +0100 George Shapovalov [EMAIL PROTECTED] wrote: Ok, found it. Thanks Marius! (for the debug hint) It was indeed forcing gnat-gcc-4.1.1 by asis-gcc-4.1.1 which has =dev-lang/gnat-4.1.1 (this is an extension to compiler and has to match it 1 for 1, forgot to bump that one :(). However, shouldn't --tree have reported the relevant package as requiring this downgrade instead of some other, pretty much arbitrary (from dependants of gnat-gcc)? In an ideal world: yes, but portage is far from being ideal (I won't pretend to know the internals of the dep resolver, so can't give a good explanation for this behavior). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] New preserve-libs feature
On Sat, 17 Feb 2007 14:55:26 +0100 Simon Stelling [EMAIL PROTECTED] wrote: Marius Mauch wrote: So everyone who has valid objections to the _general idea_ of this implementation (preserving old libraries to avoid some runtime linker errors) speak up now. For how long are these libraries preserved? This might have a security impact in cases like the recent openssl-case where you had to upgrade to an incompatible ABI because the version using the old one was vulnerable. Using preserve-libs it would leave the old lib around, making it possible for programs to link against the wrong version and ending up being vulnerable. I realize that the feature is meant to help the transitional phase until all apps are built against the new ABI, but how would you find these vulnerable apps currently? revdep-rebuild wouldn't rebuild them since they are still functional. Currently they are around as long as they are referenced by other packages or until the package is unmerged. And yes, there should be a way to tell revdep-rebuild/the user which packages should/need to be rebuilt, but I haven't made my mind up yet on how to accomplish that (in fact atm there is no separation between native and imported libs in vdb, I'm aware that needs to be added). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
[gentoo-portage-dev] Storing origin repository in VDB
There are several cases where the information from which repository (portdir, overlays, ...) a package originally came from when it was installed. Currently that information isn't really available, at most you can use some heuristics on environment.bz2 or comparing ebuilds directly, but those methods aren't very reliable (may return both false positives and false negatives). So storing the origin repo seems like a good idea, qustion is how: 1) Save (part of) the value of $EBUILD 2) Save the repository name 3) ??? Option 1) has the benefit that it would be useful from the start, but the drawback that if the path of a repository changes the information becomes invalid. Option 2) avoids that but would require that all repositories are named to be useful. Don't know if there is another option that avoids both problems. If I'd had to choose I'd probably go for option 2) and encourage people to name their repositories, but I'm open for suggestions to use option 1) or for a third option. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [RFC] Depending on active version
On Wed, 31 Jan 2007 17:47:26 + Stephen Bennett [EMAIL PROTECTED] wrote: On Tue, 30 Jan 2007 17:06:51 +0100 Marius Mauch [EMAIL PROTECTED] wrote: The idea is to add a special category (let's call it active for now) that has the following properties: - this category doesn't exist in portdir or vdb (= no ebuilds) - when portage ($pkgmanager) encounters a active/foo atom in a dependency string it executes some special code (e.g. $PORTDIR/scripts/active-check/foo =active/foo-1) to determine if that atom is satisfied Given that in the general case the package manager can't change the active provider and will have to bail with an appropriate message that the user needs to change it themselves, the obvious solution to this is the previously-discussed-somewhere-I-can't-remember ebuild function, called on depgraph creation, to check that it will be able to compile in the current system environment. It's considerably simpler and more generally useful than subverting DEPEND to add weird special-case hacks to it. Yeah, thinking about it I have to agree tat generic dep_check() support (to give the thing a name) would be a better solution here. Marius -- gentoo-portage-dev@gentoo.org mailing list
[gentoo-portage-dev] [RFC] Depending on active version
Sometimes a package has to depend on a specific version of a slotted package being the active one to build correctly, like in the current tr1 discussion on -dev [1] or with packages that depend on the running kernel. Currently this isn't really possible, however I while ago I got an idea how to solve this. Keep in mind this is just a rough idea and I'm pretty sure some people can/will point out why it is a stupid idea, but anyway: The idea is to add a special category (let's call it active for now) that has the following properties: - this category doesn't exist in portdir or vdb (= no ebuilds) - when portage ($pkgmanager) encounters a active/foo atom in a dependency string it executes some special code (e.g. $PORTDIR/scripts/active-check/foo =active/foo-1) to determine if that atom is satisfied (and yes, this kinda goes with multi-repo/multi-format support) Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [RFC] Depending on active version
On Tue, 30 Jan 2007 08:25:31 -0800 Brian Harring [EMAIL PROTECTED] wrote: On Tue, Jan 30, 2007 at 05:06:51PM +0100, Marius Mauch wrote: Sometimes a package has to depend on a specific version of a slotted package being the active one to build correctly, like in the current tr1 discussion on -dev [1] or with packages that depend on the running kernel. tr1 is partially addressed via addition of a 'binding' modifier for rdeps, to state that ||() deps are locked down after compilation. And how would that solve the actual issue of expressing I need /usr/bin/gcc to run gcc-4.1 and not gcc-3.4? The lockdown of || deps is a completely separate issue, unless I'm missing something. The idea is to add a special category (let's call it active for now) that has the following properties: - this category doesn't exist in portdir or vdb (= no ebuilds) - when portage ($pkgmanager) encounters a active/foo atom in a dependency string it executes some special code (e.g. $PORTDIR/scripts/active-check/foo =active/foo-1) to determine if that atom is satisfied Non deterministic resolution; previous steps in the graph can cause that value to flip to a different setting by the time the 'dep' is encountered. Ok, that's a problem, though for the use cases at hand (gcc and kernel) it would be mostly irrelevant. That's ignoring the kick in the nads usage of this will due to resolution... Neglectable IMO, it's not such a common use case anyway, and I don't think I have to compare it to the current solution (die in setup or compile). (and yes, this kinda goes with multi-repo/multi-format support) Don't really see how this enables multiple standalone repos in any sane way, so that one requires justification... Where did I say anything about enabling? It would need more or less a separate repository (dbapi) instance, so it would require such support. Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [RFC] Mask packages that don't cross-compile
On Wed, 22 Nov 2006 03:10:45 -0500 (EST) Daniel Barkalow [EMAIL PROTECTED] wrote: There are packages (such as perl) which work fine on both x86 and arm, but don't build on x86 cross-compiling for arm. Furthermore, pam-0.78 can be cross-compiled (with a patch available in bug comments), but pam-0.99 will require more work to get to cross-compile. It would be useful to be able to have 0.99 use a keyword (or something) such that it is excluded from cross-compiles and these builds get 0.78 instead. I've got a working patch that extends the KEYWORDS semantics slightly, such that, if KEYWORDS includes -foo, and ACCEPT_KEYWORDS includes foo but not -foo, the package is masked, even if KEYWORDS also includes bar and ACCEPT_KEYWORDS also includes bar. Then you can have the ACCEPT_KEYWORDS value $ARCH cross and packages with KEYWORDS including -cross will be masked, unless you have -cross in packages.keywords. This probably horribly mangles the concept of KEYWORDS, but I can't really tell from reading the documentation that it doesn't work this way. (Of course, I know it doesn't from trying things and reading the source, but...) It doesn't specify whether foo or -bar on a package takes precedence if foo and bar both apply to the system. (Of course, it does say that keywords are always ARCHes, which would imply that only one could apply.) I think this all works as expected, however, except that a package having cross means that it is known to automatically work if you're cross-compiling, rather than simply meaning that it is known to work if you're cross-compiling and it would work if you were building natively on your target. No. -foo is reserved for incremental negation. Maybe that isn't widely used in ACCEPT_KEYWORDS, but it has valid uses there and there can't be repurposed. Is there a particular reason why you don't use the normal keyword logic (KEYWORDS=cross and setting -cross in p.keywords to mask a package), or simply a package.mask entry? Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [PATCH] --config-root command-line option
On Thu, 16 Nov 2006 09:33:05 -0800 Zac Medico [EMAIL PROTECTED] wrote: Sure, that's one way to solve that particular problem. However, I feel that command line options are more aesthetically appealing than environment variables (maybe it's just me). Zac It's just you ;) Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [PATCH] --config-root command-line option
On Thu, 16 Nov 2006 15:43:48 -0800 Zac Medico [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Marius Mauch wrote: On Thu, 16 Nov 2006 09:33:05 -0800 Zac Medico [EMAIL PROTECTED] wrote: Sure, that's one way to solve that particular problem. However, I feel that command line options are more aesthetically appealing than environment variables (maybe it's just me). Zac It's just you ;) Marius Do you honestly prefer environment variables over command line options? I prefer command line options because they are part of a clearly defined interface. With environment variables, the interface may not be clearly defined and it's easy for variables to accidentally leak in, affecting internals in unexpected ways. Zac I prefer to have one way to do things, and the system for variables worked quite well so far. Adding CLI overrides on top of that seems not only redundant to me but also potentially confusing (which vars have CLI overrides? where in the incremental stack does the override fit in?). Add that to the already overloaded option system. Besides, if you need this feature on a somewhat regular base it seems easier to me to just export the var once instead of specifying the option on the command line all the time (ok, using DEFAULT_OPTS avoids that, but then you have the same issues you listed above). And last but not least a CLI option is bound to emerge, but this feature can also be useful for other tools. Checking an env var in the config class would enable it implicitly for all users of portage.py, without it everyone would have to basically duplicate the relevant code from emerge. I'm not generally against using CLI options for passing in arguments, but in this case I think it's wrong. Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] GPG support for mod_mail
On Sat, 28 Oct 2006 15:42:50 +0300 Ali Polatel [EMAIL PROTECTED] wrote: Hi everyone, I wonder if GPG support for mod_mail would be useful.I wrote some basic code that works.The user is required to export his public key and put the path to PORTAGE_ELOG_MAILGPG. Attached is the patch for mod_mail.py.It won't work with armored key files because gpg can't directly encrypt messages with ascii armored keys , they need to be --dearmor'ed first. Thoughts? Well, I don't really see a reason why one would want to encrypt those mails. The only potentially sensitive information (package name/version) is already contained in the subject and therefore wouldn't be encrypted. And gpg support can be a bitch to maintain, so I'd like to see the benefit before adding that feature. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. -- gentoo-portage-dev@gentoo.org mailing list
[gentoo-portage-dev] RFC: new virtual metadata variable to list combined deps
Ok, I have to admit the subject may be a bit confusing, so let me get the motivation first: Often people want to know the dependencies of a package. Not much of a problem when they know how to use auxget or the portageq metadata commands. However these all work on the specific DEPEND, RDEPEND and PDEPEND vars directly, so to get all dependencies of a package one has to read all three variables. And as in general some entries exist in both DEPEND and RDEPEND one gets even more info than wanted. At least this me this manual parsing gets annoying over time. So in my auxget script[1] I added support for a new virtual metadata variable '*DEPEND' that basically reads and concatenates the three dep variables, then eliminates duplicates (of course only on the top level, complex atoms are treated as a single unit). This allows a quick check what dependencies a package has ignoring if it's buildtime, runtime or postbuild (or however you want translate the P in PDEPEND). So now I was wondering a) if I'm the only one who finds this feature useful and b) if adding it at the dbapi level (in dbapi.aux_get) would be considered a good idea, so it could be used by other tools? Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Max parallelization setting
On Tue, 10 Oct 2006 00:04:57 -0700
Brian Harring [EMAIL PROTECTED] wrote:
I might be daft (likely), but why not just introduce a var indicating
max parallelization instead? Tweak portage to push that setting into
MAKEOPTS=${MAKEOPTS+${MAKEOPTS} } -j${PARALLELIZATION}.
Might sound daft, but -j is hardcoded parallelization; if you're
trying to run 3 build processes, the original var of -j isn't all
that useful, nor will the hardcoded -j# var set for 3 package build
processes be useful if you're doing a single build.
Depending on number of seperate package build processes possible, the
# of build processes allocated per build process needs to vary
(essentially).
Seems useful as *alternative* to the low level vars, but shouldn't
replace them (so if the low level vars are set they override the global
setting). So the order in your MAKEOPTS assignment above would have to
be reversed (assuming sequential option parsing).
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
--
gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Atom matching behavior
On Mon, 31 Jul 2006 15:48:23 -0700 Brian Harring [EMAIL PROTECTED] wrote: but I'd suspect that many people share my original assumption and expect it to only match full version components Hear a bit of screaming from it once every 4-6 months; personally, I interpret that as devs know which to use usually- additionally, once the (bluntly) hissy fit from the dev subsides, and they're reminded yes it's annoying, but if you want it changed take it to dev to get consensus folks promptly forget about it. Either they're silently working around it, or it's not that much of an issue (I suspect the latter, but am neutral towards the change). Well, practically it's not a big issue as in 99% of all cases it works like people expect. Should be discussed on -dev, not here imo; they're the ones affected by it (it's essentially their standard after all). Changing it? Sure, but it's a required eapi bump; portage chokes on .* now. I'd also not bump eapi just for one change; there is a boatload of other stuff that's waiting for an apt time to be pushed out together. Sure going to sned this to -dev, just usually going for input from a smaller audience first. Haven't yet thought about details in case of changing it, first gonna decide what the correct behavior is. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Portage phase hooks patch
On Sat, 22 Jul 2006 16:52:38 -0700 Zac Medico [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Kelly wrote: In my case, I feel this functionality would be very useful as it allows for me to integrate my GLEP 27 implementation into portage without portage needing to worry about my implementation specifics, which may well change in later versions. I am sure there are other practical ways in which these hooks could be used. Patch at: http://soc.gentoo.org/viewcvs.py/*checkout*/glep0027/patches/portage-hooks.patch See bug# 141215. I like the idea. My main concern is that, like /etc/portage/bashrc, this creates lots of potential for foreign code to interfere with the internal workings of the ebuild environment. At a minimum, I think there should be something in the `emerge --info` output that indicates whether or not any phase hooks exist. Traditionally, configurable files that affect portage behavior go mostly in /etc/portage. I see that your intention is to make /var/libexec/portage/hooks/ a location for third-party packages to install hooks, so it makes sense to keep those separate from hooks that the user might install. I know that portage-utils currently installs a post-sync hook in /etc/portage/postsync.d/q-reinitialize, so there is some inconsistency there. We need to develop a consistent policy for appropriate locations of files like these. I still think that those hooks should go into the tree rather than being installed by packages. That's mainly so they get increased visibility and gives us (us being Gentoo, not just Portage) more access and control over them, like we have with eclasses. This also moves responsibility from hook authors to pkgmanager authors (the package mamanger has to support the hook format in the tree, not the hook has to support (all of) the specific package manager hook formats). I know Mike has taken care of Paludis support already, but look at a larger scale: a) n hook authors supporting m package manager interfaces: O(n*m) b) n hook authors and m package managers supoprting one repository interface: O(n+m) Also with a) you have to play what I call the catch up game, e.g. a new package manager gets out and all hooks have to account for a new interface (even if it's just a new path). Of course a) has the drawback that it requires a solid spec of the interface, but that's something we want anyway. However, *if* they are going to be installed by packages they should go either into /etc/portage/foobar or /usr/lib/portage/foobar (like any 3rd party elog or cache modules), foobar being something we'd have to decide on. The suggested /var/libexec/portage has several issues: - /var/libexec is AFAIK not defined by FHS - executable data doesn't belong in /var unless it's application data which isn't the case here - libexec generally sucks (even in /usr), especially as we already use /usr/lib for what generally goes there (support scripts) Independent on the location I'd like to see the ability to blacklist individual hooks for testing, troubleshooting, debugging and probably a few other reasons. And no, this should not be done with $FEATURES ;) Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [RFC] Naming Conventions
On Sat, 22 Jul 2006 16:25:01 -0700 Zac Medico [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chris White wrote: 1) Create aliases to the new functions, then at some yet-to-be-determined point, kill the aliases and bomb on the scripts (this suffers from procrastination). 2) Make an official release with the new function names and no aliases, as well as the soon to come docs. I sort of like this method because those with official portage tools can adjust their scripts, and simply alter the depend atoms for = (new API versions) and = (old versions), effectively forcing/preventing upgrades. I vote for #1 because it's smoother and easier (which is good for me especially because I do releases). The disruptive change proposed in #2 seems like it would cause unnecessary problems with no practical advantage over #1. combination of both. Phase one: change function names and add aliases Phase two: make the aliases generate warnings Phase three: drop the aliases Maybe combine phases one and two. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] 'emerge' request: make --tree not imply --pretend
On Sat, 24 Jun 2006 18:02:19 -0700 Dan Corson [EMAIL PROTECTED] wrote: I am sorry, you are indeed correct. What I should have requested is: a way to have the package list output in non-interactive, package-installing (that is, not --pretend) execution. It may be that --pretend is a little over-loaded currently. (I was confused, at any rate, as you can see.) Thanks, --Dan I think this is stuff for a local wrapper script and not yet another emerge option that is almost identical to --ask in this case (from a user POV). Also don't see much use for this feature, the overhead for calling emerge twice isn't that big, especially when talking about automated stuff (lie your case seems to be). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Manifest verification
On Sun, 18 Jun 2006 04:43:28 -0700 (PDT) Andrei Slavoiu [EMAIL PROTECTED] wrote: Hello, I want to raise a issue with the way that package integrity verification is done by portage. For an example of such an issue see bug 136742. The idea is that portage now checks the Manifest as a whole, not allowing a version of the package to be emerged if any of the files mentioned by the Manifest fail the verification. This makes unstable ebuilds wich change often and have a tendency to break from time to time prevent the emerge of a stable version of the same package. Do you think there is any way to fix this in portage? This isn't exactly new behavior with FEATURES=strict (which is the default setting). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] GWN -portage-2.1 announcement.
On Mon, 12 Jun 2006 17:18:44 -0700 Brian [EMAIL PROTECTED] wrote: On Mon, 2006-12-06 at 18:58 -0500, Andrew Gaffney wrote: Brian wrote: Did I understand it wrong or did they get this mixed up in the GWN announcement: /etc/portage/package.unmask/kde, /etc/portage/package.unmask/xorg Shouldn't it be: /etc/portage/kde/package.*, /etc/portage/xorg/package.* You understood it wrong. /etc/portage/package.unmask/foo is correct. -- Andrew Gaffney Thanks. I'm glad now I didn't have the time to add that to porthole yet. Might have been a quick fix anyway though. Haven't looked at porthole code, but if you use portage_util.grab* the fix is trivial (just add a recursive=1 parameter). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] GWN -portage-2.1 announcement.
On Mon, 12 Jun 2006 18:20:16 -0700 Brian [EMAIL PROTECTED] wrote: I also just did a man portage and the man page did not appear to be updated for this change. I guess quite a few docs aren't up to date in 2.1. Anyone wanna work on that? Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] default postsync
Zac Medico schrieb: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ned Ludd wrote: How do you think we should handle it? Should we install a post_sync in a postinst phase outside of portage's handling if and only if not post_sync already exists? Should we change it to handled a postsync.d by default? Should we do both? I'm open as heck but would like to start to finalize then document it's behavior. I feel it could be one of the next untapped really useful features of portage. glsa-checking, news handling, search db updating, and stuff etc.. Given the existing support for /etc/portage/bin/post_sync, the user has the freedom to do anything they want. If a program needs to make use of the post_sync trigger, it's documentation can simply state that a certain command needs the be run and the user can add that command to their post_sync script. Is that not easy and flexible enough? Well, it requires user intervention, pretty much the same argument as for any other foo.d change in any package (I don't have a personal preference here, though using bashrc for this definitely isn't a good idea). Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Perl, sort, and locale
[EMAIL PROTECTED] schrieb: I doubt this is the right place to bring this up, but maybe some one can tell me where to go :-) Yeah, this is definitely the wrong place, not really sure where the appropriate place for this is though. Probably a perl or coreutils related list/forum might help you. I have discovered an inconsistency in how perl and sort handle locale. Here are two commands you can run in a shell ... (echo '/'; echo '?') | sort (echo '/'; echo '?') | perl -e '@x = ; print $_ foreach sort @x' With LANG=en_US.UTF-8, sort says ? comes before /, perl says the opposite. Setting LC_COLLATE=C switches the sort behavior. I have no idea what else changes or where else perl and sort disagree, or what other programs do. Seems like something is wrong, but I don't know what. This happens on other versions of Linux. It is not a gentoo specific problem. -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] glsa implemented as a special set
On Sun, 23 Apr 2006 08:55:58 -0700 Brian [EMAIL PROTECTED] wrote: I was thinking that /etc/portage/sets/glsa could be a symlink to set list in the current metadata/glsa directory of the portage tree. That file should be relatively easy to auto-generate from the existing glsa*.xml files there already. Perhaps a FEATURES=GLSA_SET would generate that file on completion of an emerge --sync I could also then put a GLSA field into porthole's package Summary view as well as a GLSA notebook page(s) to display the appropriate glsa?.xml file(s). Too complicated. First you currently need gentoolkit for glsa.py, and portage shouldn't depend on gentoolkit. Also you can't store system-specific files in the tree. And finally using an intermediate file creates some additional issues (check for IO/FS problems, checking permissions, etc). Any reason you need a real file for this instead of just generating the list on the fly? Just check glsa-check how to do that, it's not much more than a wrapper for glsa.py. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] glsa implemented as a special set
On Mon, 24 Apr 2006 07:04:13 -0700 Brian [EMAIL PROTECTED] wrote: On Mon, 2006-24-04 at 14:20 +0200, Marius Mauch wrote: On Sun, 23 Apr 2006 08:55:58 -0700 Brian [EMAIL PROTECTED] wrote: I was thinking that /etc/portage/sets/glsa could be a symlink to set list in the current metadata/glsa directory of the portage tree. That file should be relatively easy to auto-generate from the existing glsa*.xml files there already. Perhaps a FEATURES=GLSA_SET would generate that file on completion of an emerge --sync I could also then put a GLSA field into porthole's package Summary view as well as a GLSA notebook page(s) to display the appropriate glsa?.xml file(s). Too complicated. First you currently need gentoolkit for glsa.py, and portage shouldn't depend on gentoolkit. I did not mean portage should and I din't want to depend on gentoolkit either. Not sure I understand your idea then, I was under the impression that `FEATURE=GLSA_SET emerge --sync` would create that file, is that not what you meant? Also you can't store system-specific files in the tree. Yeah, that was a bit of a thought evolution while I was typing. I realized after I hit send. At first I thought it could be included in the sync. Then thought it's only a duplication of the data already there, so why not generate it (save bandwidth), since the data will only change at sync time, just do it once then. Ehm, you couldn't include it in the sync as it's system specific. It would have to be generated locally, or you have to treat it special again (only update packages that are installed, don't install new packages). And finally using an intermediate file creates some additional issues (check for IO/FS problems, checking permissions, etc). Any reason you need a real file for this instead of just generating the list on the fly? I thought a smaller stripped down glsa.py module could generate the file at completion of the sync. Then no special code is needed internal in porthole beyond checking for set inclusion, atom matching, just a glsa_flag=True to ignore members that are not already installed. Once portage was able to handle sets, it would almost automatically be able to handle a glsa set as well. The only difference is not installing a set member that is not already installed. *Shrug*, generating the list dynamically shouldn't take more than 10 or 20 lines using glsa.py, basically it's pkg_list = [] glsa_list = [Glsa(x) for x in get_glsa_list(glsadir, glsaconfig)] for x in glsa_list: if x.isVulnerable(): pkg_list.extend(x.getMergelist()) plus some error handling. Add some dep_getkey() calls if you don't want the glsa resolver logic of minimal intrusion (might be problematic though). If you need a file interface wrap the list in a StringIO instance. It's really better to keep interdependencies to a minimum here, and when portage gets set support it will generate the glsa update list dynamically anyway, so portage wouldn't benefit from a file at all. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] 2.1 release candidate soon?
On Thu, 06 Apr 2006 19:11:49 -0700 Zac Medico [EMAIL PROTECTED] wrote: The manifest code doesn't have very many use cases so I'd expect that we would have hit most major problems by now (even with a small sample). Any necessary changes are likely to be small patches. As an alternative, we can cut the 2.1 branch at the point before manifest2 was merged (2.1_pre7, essentially). Releasing 2.1 without manifest2 is a no go, it would significantly delay the deployment and transition. I'm not requesting to delay 2.1 for another few months, just one more pre release so people get a chance to test it for one or two weeks. The remaining feature I'd like to get into 2.1 is the tree-format-check issue, but that could probably be slipped in in the rc phase (don't really like that idea, but it's an option). I don't want to rush the development of new features such as manifest2 or the tree-format-check. We have a 2.1 branch that, in it's current state (2.1_pre7-r4, for example), provides significant benefits over the 2.0.x branch. By delaying 2.1's release for the addition of _new_ features, we run the risk of the release being delayed indefinitely by just one more feature syndrome. Personally, I'd rather have shorter release periods so that just one more feature syndrome becomes less of an issue. Ehm, this is not just one more feature, both manifest2 and the tree-format-check are things to improve forward compability (or for the latter even enable forward compability at all), so delaying them will hinder future development, not only for us. Also this isn't exactly news to you all as I sent my intentions already a while ago, and last I asked you all agreed with them, so is there any reason to rush this now? Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] tree dependency check
Marius Mauch schrieb: So after manifest2 is in, I'll revive the other issue that IMO is a requirement for 2.1: enforcing dependencies needed to use the tree (see old threads or glep44 for reasoning). A patch for that is available at dev.gentoo.org/~genone/patches/treedeps.diff. Unless somebody objects I'll add that somewhere next week. Ok, from a discussion with Zac and a few others it seems that we should add a layer of indirection, so instead of specifying atoms the tree has a freeform version identifier, and portage can use it to look up the required atoms by using a (remote) mapping file. This new approach is implemented in dev.gentoo.org/~genone/pacthes/format-check.diff (it's basically a combination of strategies D and E in d.g.o/~genone/docs/treedeps.txt) This new patch also has a few other improvements: - checks overlays too - at sync time first check the tree format of the remote tree, if we can't handle it the sync is aborted - if no format specification is found warn the user about it (with the intention to fail in a future version) New intended merge date is this weekend or beginning of next week, so if you have any objections, make them now. Marius -- gentoo-portage-dev@gentoo.org mailing list
[gentoo-portage-dev] tree dependency check
So after manifest2 is in, I'll revive the other issue that IMO is a requirement for 2.1: enforcing dependencies needed to use the tree (see old threads or glep44 for reasoning). A patch for that is available at dev.gentoo.org/~genone/patches/treedeps.diff. Unless somebody objects I'll add that somewhere next week. Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [PATCH] Manifest2 reloaded
Marius Mauch schrieb: The first should be delayed until there is some consensus how the gpg stuff should work in the future, the others I don't see the use for. Also I only checked portage.py for changes, so emerge/repoman/... might still have to be fixed. Last but not least: I did some basic testing with this and the important stuff seems to work, but I'm quite sure the code still has a lot of bugs/issues, and this being a core functionality it needs a *lot* of testing, so I'd really appreciate if you could all give it a spin (but do not commit anything to the tree without manually checking it first). Does the lack of feedback (only got a reaction from Brian so far) mean that noone tried it or that it doesn't have any issues? Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Few things, which imho would make portage better
tvali wrote: Ok, i send a lot of them, but hopefully they're interesting :) I did research a bit about adding SQL support to portage -- as much as i see, mysql is smallest sql server, which could be emerged with python module. In beginning, i think that SQL database structure should be created, which supports basic fields from ebuilds (DESCRIPTION, HOMEPAGE, SRC_URI, LICENSE, SLOT, KEYWORDS, IUSE, DEPEND, RDEPEND). Then emerge --createsql command should be set up, which adds data from /usr/portage/ filetree into this sql database. You're talking about the cache, take a look at the cache subsystem and write a mysql module for it. This will never become a default though (we would get killed if portage starts to depend on mysql). I have some questions now: * Where i could find basic ebuild specification? Developer handbook, ebuild(5), bash(1) * Where i could find basic portage tree datastructure specification, if there is any? Not really. * Does portage have some code documentation for faster learning? (or should i just read the code?) code documentation ... what's that? ;) Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Few things, which imho would make portage better
tvali wrote: I did think about some priorities too, so that it could be perfect for me. It should be possible to add package with a priority. I will give you an use case and explanation how i would use portage. Heh, make the dep resolver even more complex ;) Also don't really see a need for such a feature, pretty much no benefit with a lot of additional complexity. Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Hello! portage UI
Brian wrote: On Tue, 2006-14-03 at 13:14 +0200, tvali wrote: Ok, i was, yes, speaking about kde. I will check out this Porthole :) I was actually thinking more about c ++, but nothing against python -- i was quite a fan of python when i first found it. I believe Kuroo is in C, maybe c++ For the record: Kuroo is known to be conceptually broken (uses the cache directly), but the author is looking to fix that in a future version. Generally writing portage utils in languages other than python is difficult as you can't use the primary API directly. Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] [rfc] variable naming for marking binaries as QA ignorable
On Sun, 5 Mar 2006 20:46:25 -0500 Mike Frysinger [EMAIL PROTECTED] wrote: On Sunday 05 March 2006 19:48, Kevin F. Quinn (Gentoo) wrote: Ned Ludd [EMAIL PROTECTED] wrote: On Fri, 2006-03-03 at 23:32 -0500, Mike Frysinger wrote: so we've found some cases where a package installs objects that either need to be ignored by some of the scanelf checks ... ... what this e-mail is about is naming convention ... i'm thinking that an ebuild sets up a variable with a list of relative paths to $D of files that should be skipped for various checks ... so with slmodem, we'd have like: QA_EXEC_STACK=usr/sbin/slmodemd usr/sbin/slmodem_test if, in the future, we need to add an ignore list for TEXTRELs, we'd use QA_TEXTRELS= This becomes tricky when looking at tests across all CHOSTs. What holds true for one arch defiantly is not the case for others. This could be done via the profiles, perhaps - package.qa, something like package.mask/use/keywords: i hate such things ... imo this information should stay in the ebuild and nowhere else ... be trivial to expand the support like: QA_TEXTRELS=... # for all arches QA_TEXTRELS_arch=... # for just one arch so in the case of slmodem: QA_EXEC_STACK=usr/sbin/slmodemd in the case of some other package that only has issues on x86: QA_EXEC_STACK_x86=some/foo this thread was about the naming convention :P does QA_EXEC_STACK and QA_TEXTRELS work for people ? Personally I'd call it QA_IGNORE_*, but that's just me. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. -- gentoo-portage-dev@gentoo.org mailing list
[gentoo-portage-dev] [PATCH] Manifest2 reloaded
So while on my way to FOSDEM I decided to do something useful with the
time and wrote a new manifest2 implementation. This has nothing to do
with the original prototype I posted a while ago, it's been written
completely from scratch.
Basically all functionality (creation, parsing, validation) is
encapsulated in the new portage_manifest.Manifest class, including
compability code to read/write old style digests.
The changes to portage.py only change the digest*() functions to use
this new class instead of handling the task themselves (exception:
digestCheckFiles() which apparently was only used internally by other
digest* functions), they should more or less behave like with the old
code. Any new code however should use the Manifest() class directly
however.
While this patch implements the basic functionality some extra stuff
that was in the old code isn't included yet:
- gpg verification
- FEATURES=autoaddcvs
- FEATURES=cvs (probably obsolete anyway)
- emerge --digest / FEATURES=digest (may or may not work)
The first should be delayed until there is some consensus how the gpg
stuff should work in the future, the others I don't see the use for.
Also I only checked portage.py for changes, so emerge/repoman/... might
still have to be fixed.
Last but not least: I did some basic testing with this and the
important stuff seems to work, but I'm quite sure the code still has a
lot of bugs/issues, and this being a core functionality it needs a
*lot* of testing, so I'd really appreciate if you could all give it a
spin (but do not commit anything to the tree without manually checking
it first).
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
diff -ru --exclude=CVS --exclude=.svn -N pym/portage.py.org pym/portage.py
--- pym/portage.py.org 2006-03-04 02:25:20.957635000 +
+++ pym/portage.py 2006-03-04 03:12:19.545785750 +
@@ -90,6 +90,7 @@
from portage_data import ostype, lchown, userland, secpass, uid, wheelgid, \
portage_uid, portage_gid
+ from portage_manifest import Manifest
import portage_util
from portage_util import atomic_ofstream, dump_traceback, getconfig, grabdict, \
@@ -2049,181 +2050,67 @@
return 0
return 1
-
-def digestCreate(myfiles,basedir,oldDigest={}):
- Takes a list of files and the directory they are in and returns the
- dict of dict[filename][CHECKSUM_KEY] = hash
- returns None on error.
- mydigests={}
- for x in myfiles:
- print ,x
- myfile=os.path.normpath(basedir+///+x)
- if os.path.exists(myfile):
- if not os.access(myfile, os.R_OK):
-print !!! Given file does not appear to be readable. Does it exist?
-print !!! File:,myfile
-return None
- mydigests[x] = portage_checksum.perform_multiple_checksums(myfile, hashes=portage_const.MANIFEST1_HASH_FUNCTIONS)
- mysize = os.stat(myfile)[stat.ST_SIZE]
- else:
- if x in oldDigest:
-# DeepCopy because we might not have a unique reference.
-mydigests[x] = copy.deepcopy(oldDigest[x])
-mysize = copy.deepcopy(oldDigest[x][size])
- else:
-print !!! We have a source URI, but no file...
-print !!! File:,myfile
-return None
-
- if mydigests[x].has_key(size) and (mydigests[x][size] != mysize):
- raise portage_exception.DigestException, Size mismatch during checksums
- mydigests[x][size] = copy.deepcopy(mysize)
- return mydigests
-
-def digestCreateLines(filelist, mydict):
- mylines = []
- mydigests = copy.deepcopy(mydict)
- for myarchive in filelist:
- mysize = mydigests[myarchive][size]
- if len(mydigests[myarchive]) == 0:
- raise portage_exception.DigestException, No generate digest for '%(file)s' % {file:myarchive}
- for sumName in mydigests[myarchive].keys():
- if sumName not in portage_checksum.get_valid_checksum_keys():
-continue
- mysum = mydigests[myarchive][sumName]
-
- myline = sumName[:]
- myline += +mysum
- myline += +myarchive
- myline += +str(mysize)
- mylines.append(myline)
- return mylines
-
-def digestgen(myarchives,mysettings,overwrite=1,manifestonly=0):
+def digestgen(myarchives,mysettings,db=None,overwrite=1,manifestonly=0):
generates digest file if missing. Assumes all files are available. If
- overwrite=0, the digest will only be created if it doesn't already exist.
-
- # archive files
- basedir=mysettings[DISTDIR]+/
- digestfn=mysettings[FILESDIR]+/digest-+mysettings[PF]
-
- # portage files -- p(ortagefiles)basedir
- pbasedir=mysettings[O]+/
- manifestfn=pbasedir+Manifest
-
- if not manifestonly:
- if not os.path.isdir(mysettings[FILESDIR]):
- os.makedirs(mysettings[FILESDIR])
- mycvstree=cvstree.getentries(pbasedir, recursive=1)
-
- if (cvs in features) and os.path.exists(pbasedir+/CVS):
- if not cvstree.isadded(mycvstree,files):
-if autoaddcvs in features:
- print Auto-adding files/ dir to CVS...
- spawn(cd +pbasedir+; cvs
[gentoo-portage-dev] [Patch] Restriction framework, new search code, metascan2
This is another attempt to prepare the way for including metascan, this
time based on a new restriction framework (which has absolutely nothing
to do with the one in savior/bcportage) included in the attached and
completely undocumented patch (once you get the concept it should be
self-explanatory however).
It basically consists of three class hierarchies:
- dataclasses responsibe for delivering the data to match against
- filterclasses implementing the match algorithms
- boolean connectors to combine multiple restrictions
There is a fourth type called prefilter that can adjust the match
algorithm of filter classes (e.g. a Split() class to break multi-value
metadata fields up for the string matchers).
The filterclasses and connectors implement the same interface for
performing the matches, so the most basic application would be:
for cpv in cpvlist:
if restriction.filter.PartialMatch(data.Metadata(key, db),
value).match(cpv):
matches.append(cpv)
For a more complex application see the patched emerge or metascan2
(at d.g.o/~genone/scripts).
A word of warning: I've just written all this code in the past few hours,
it is absolutely unoptimized, incomplete and certainly has bugs (none known
though).
That also means I'm not interested in comments about the code or algorithm
yet, as this needs a lot of work and so far I've only done the work to get
it working, mainly sending this mail so people can play with it a bit.
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
diff -ruN vanilla/bin/emerge portage/bin/emerge
--- vanilla/bin/emerge 2006-02-16 09:38:57.0 +0100
+++ portage/bin/emerge 2006-02-18 12:19:52.0 +0100
@@ -189,7 +189,8 @@
--tree,
--update, --upgradeonly,
--usepkg, --usepkgonly,
---verbose, --version
+--verbose, --version,
+--newsearch,--regex
]
shortmapping={
@@ -693,7 +694,86 @@
if x in myparams:
myparams.remove(x)
+class dumb_formatter(object):
+ def __init__(self):
+ pass
+
+ def generateReport(self, result):
+ rval = \n
+ for pkg in result.keys():
+ rval += str(result[pkg])+\n\n
+ return rval
+
# search functionality
+class restriction_search(object):
+ DATA_NAME = 1
+ DATA_DESC = 2
+
+ FILTER_REGEX = 1
+ FILTER_PARTIAL = 2
+ FILTER_EXACT = 3
+
+ def __init__(self, formatter):
+ global myopts
+ if --searchdesc in myopts:
+ self.key = self.DATA_DESC
+ else:
+ self.key = self.DATA_NAME
+ if --regex in myopts:
+ self.type = self.FILTER_REGEX
+ else:
+ self.type = self.FILTER_PARTIAL
+ self.formatter = formatter
+
+ def execute(self, searchkey):
+ from restrictions import data, filter
+
+ search_category = False
+ if searchkey.startswith(@):
+ search_category = True
+ searchkey = searchkey[1:]
+
+ if self.key == self.DATA_DESC:
+ primarydata = data.Metadata(DESCRIPTION, portage.portdb)
+ elif self.key == self.DATA_NAME:
+ primarydata = data.PackageName(with_category=search_category)
+ if self.type == self.FILTER_REGEX:
+ primaryfilter = filter.RegexMatch(primarydata, searchkey)
+ presearch_re = re.compile(searchkey)
+ elif self.type == self.FILTER_PARTIAL:
+ primaryfilter = filter.PartialMatch(primarydata, searchkey, ignoreCase=True)
+ presearch_re = re.compile(.*+re.escape(searchkey)+.*)
+
+ self.result = {}
+ for package in portage.portdb.cp_all():
+ update_spinner()
+ # This is required to get similar performance as the old code, if we
+ # don't then we have to do at least one xmatch() call per package
+ # slowing things down extremely
+ if self.key == self.DATA_NAME and not presearch_re.match(package):
+continue
+ masked = False
+ cpv = portage.portdb.xmatch(bestmatch-visible,package)
+ if len(cpv) == 0:
+cpv = portage.best(portage.portdb.xmatch(match-all,package))
+masked = True
+ if len(cpv) == 0:
+print package
+ if primaryfilter.match(cpv):
+update_spinner()
+self.result[package] = {}
+self.result[package][masked] = masked
+self.result[package][cpv_tree] = cpv
+self.result[package][cpv_installed] = portage.best(portage.db[/][vartree].dbapi.match(package))
+metadata_result = portage.portdb.aux_get(cpv, [DESCRIPTION, LICENSE, HOMEPAGE])
+self.result[package][desc] = metadata_result[0]
+self.result[package][license] = metadata_result[1]
+self.result[package][homepage] = metadata_result[2]
+self.result[package][size] = format_size(portage.portdb.getsize(cpv))
+
+ def output(self):
+ print self.formatter.generateReport(self.result)
+
class search:
#
@@ -3059,7 +3139,10 @@
if not myfiles:
print emerge: no search terms provided.
else:
- searchinstance = search()
+ if --newsearch in myopts:
+ searchinstance = restriction_search(dumb_formatter())
+ else:
+ searchinstance = search()
for mysearch in myfiles:
try:
[gentoo-portage-dev] Deprecating 'emerge action' syntax
Right now 'emerge action' and 'emerge --action' are both supported. But as we learned with the rsync case 'emerge action' has potential namespace conflicts with 'emerge package' I'd propose to deprecate 'emerge action' before we hit another real conflict. (The alternative would be to deprecate 'emerge package' in favor of a to-be-written 'emerge install package', but that's even more problematic) Technically it's a no-brainer, only potential problem would be user confusion. Any objections against this for pre5? Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. --- bin/emerge.org 2006-02-16 09:38:57.0 +0100 +++ bin/emerge 2006-02-16 12:21:26.0 +0100 @@ -272,7 +272,7 @@ sys.exit(1) elif (not myaction) and (x in actions): if x not in [system, world]: - #print red(*** Deprecated use of action '+x+') + print red(*** Deprecated use of action '%s', use '--%s' instead % (x, x)) if x==rsync: # emerge rsync print signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Deprecating 'emerge action' syntax
On Thu, 16 Feb 2006 13:04:20 + Ciaran McCreesh [EMAIL PROTECTED] wrote: On Thu, 16 Feb 2006 12:31:25 +0100 Marius Mauch [EMAIL PROTECTED] wrote: | Right now 'emerge action' and 'emerge --action' are both supported. | But as we learned with the rsync case 'emerge action' has potential | namespace conflicts with 'emerge package' I'd propose to deprecate | 'emerge action' before we hit another real conflict. | (The alternative would be to deprecate 'emerge package' in favor of a | to-be-written 'emerge install package', but that's even more | problematic) | Technically it's a no-brainer, only potential problem would be user | confusion. | Any objections against this for pre5? Mmm, is this taking advantage of Daniel no longer being able to revert it? :) Heh, almost forgot why the code has been commented for ages ;) Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Order of operations: buildpkg
On Tue, 24 Jan 2006 08:19:22 -0500 Mike Frysinger [EMAIL PROTECTED] wrote: On Tuesday 24 January 2006 03:43, Francesco Riosa wrote: Indeed, could someone shade a light on what happen to /var/db/pkg and world file when using ebuild this manner ? Could be rephrased as Does it act exactly the same way emerge does ? the 'qmerge' step would take care of updating /var/db/pkg IIRC the thing it won't do is auto-clenaing the old entries though, need to run a emerge --clean afterwards. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] confcache integration
On Tue, 24 Jan 2006 04:50:31 -0800 Brian Harring [EMAIL PROTECTED] wrote: Yo. Looking to integrate confcache support into trunk some time in the near future- had users testing it for about 2 months (give or take), so far it's behaved pretty decently. A few packages eat themselves when ran with --cache (bad autotooling), hence the addition of restrict=confcache being added to the patch. Either way, patch is attached, looking for any complaints regarding it (or integrating confcache)- will be sending an email off to gentoo-dev in the next few days to get feedback from the general dev populace,, but sending this email to get feedback on the implementation in the meantime. That directory stuff really needs a wrapper function, it's duplicated way too often. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] making aux_get more usable for vardbapi
On Mon, 23 Jan 2006 14:08:00 -0800
Brian Harring [EMAIL PROTECTED] wrote:
On Mon, Jan 23, 2006 at 07:44:44PM +0100, Marius Mauch wrote:
On Mon, 23 Jan 2006 03:47:11 -0800
Can't follow your thinking here. As said, the code won't corrupt any
data, at worst it will tell the user that it couldn't extract some
keys (and even that only if there would be such a stupid case which
itself has a chance of like 10^-10 or so, or can you name a single
case?).
Yeah, any extension of your code to pick up EAPI is going to false
positive on somewhere around a quarter of my vdb nodes- I use a
debug function from my bashrc for prefix testing (holdover from eapi
testing days). Why? Because the eapi var didn't exist (thus no env
assignment), the only possible match is in the middle of a here op-
debug_dump() {
...
echo HERE 2
EAPI=${EAPI-unset}
PF=$PF
CATEGORY=$CATEGORY
HERE
...
}
Setting EAPI to unset strikes me as corruption here, because now
portage will *never* do anything with that vdb node- the eapi differs
from what it supports (it's been set to effectively a random value),
thus I have to fix the vdb entry myself if I ever want to get rid of
it.
Guessing the response on that one is well, you're using the bashrc,
and yes, I am- the vdb code should be *robust*, not choking on a
users debug code.
Not even trying to comment on this one.
So what package contains filter-env then?
Portage- been in cvs/svn for over a year now :)
Not in trunk or any release (except the alpha snapshot). And I'm not
going to blindly copy unknown code from completely different codebases
and later be blamed for doing it improperly.
Aside from that, if the code is in debate (as this is), I really
don't think it should get slid into svn 2 weeks later effectively
unchanged- didn't write that original email just for the hell of
it :)
As said, I disagree with your assessment of the situation. If you
can name a single case where the code breaks or filter-env hits
the tree I might reconsider, but not before.
Well, if you disagreed with the original response, continue the
conversation prior to commiting- otherwise we see a commit, then a
rebuttal a few hours later. Not really how things should go for a
contested piece of code (at least when the only two to weigh in our
flat out opposed on it)- especially if the code's effect is
nontrivial and it hasn't had any actual peer review (only comment was
on your algo).
Interesting, you now count for two people? Or who is this second person
you're talking of? I still think you're making more out of this than it
really is. Also there really isn't a point in discussing a difference in
opinion IMO, such attempts lead nowhere. I considered your comment, but
couldn't come up with a reason why a theoretical issue should hold this
up.
Issues with the code.
Now we're getting somewhere.
1) Scans for metadata that didn't exist at the time of the ebuild's
installation, only possible match is unintended. Not good.
Not really sure what that's supposed to mean, assuming you mean that
nodes will lack SRC_URI/HOMEPAGE/... from env.bz2 is about as likely as
here doc statements containing them (and indicates a broken package
anyway).
2) your code is breaking upon *all* newlines, regardless if it's
in the middle of a variable assignment. Yes, bash does use $'' for
assignment, but you're relying (hoping really) that the variable has
been filtered by python portage and the ebuild defined var is
stomped. This is a bad idea- description comes to mind. Your code
seems to be aware of this possibility also, since it's doing rather
loose quote filtering.
Have to think about this.
3) Stop using regex all over the place. split/join is a helluva lot
faster, and collapses that nasty regex that attempts to remove
whitespace down to two lines (a one time translate, and the
split/join to kill whitespace).
Ok (cosmetics).
4) Code specifically tries to find, and remove variable chars-
this is *really* the wrong thing to do. If you're trying to work
around catching a var reference, either your parsing screwed up, or
you're mangling a DESCRIPTION value that you shouldn't be hosing.
define variable chars, otherwise doesn't make sense.
5) hardcoded vdb.
6) Non root aware VDB.
Ok (trivial)
7) Only is aware of environment.bz2- older portage versions differed
in this afaik (I *do* know PORT_ENV_FILE overloading for ebd I had to
address this case already, so it's out there).
Well, it's present as long as I can remember, but an sanity check
wouldn't hurt there.
8) perms on the new files?
Ok.
9) general code comments; 'if s != ', use 'if s:'. Catch the
exceptions for check mode, don't let vdbkeyhandler nuke world file
checking due to a potential exception. Right now, your code will TB
if ran by non-root for a check run rather then stating got to be
root.
a) hate theses
Re: [gentoo-portage-dev] making aux_get more usable for vardbapi
On Wed, 11 Jan 2006 12:39:03 -0800 Brian Harring [EMAIL PROTECTED] wrote: Regex you've got there allows for pulling the wrong text- recall, ebd originally was doing grep based filtering (regex). Had to rewrite that in a major hurry since bash syntax (specifically here ops) forces you to track state/constructs rather then just a regex... Not really an issue in this case. First the code bails out if more than one match is found, so unless the metadata assignment is NOT found by it we don't get the wrong info. Also a mismatch in this special is so extremely unlikely that honestly I don't really care about it, especially as this is a one time conversion (might be different if I'd have added the on the fly extraction). All together makes this pretty much a hypothetical issue with no practical relevance. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Plausible idea for GLEP 19?
On Sun, 22 Jan 2006 13:02:37 -0600 Mikey [EMAIL PROTECTED] wrote: On Sunday 22 January 2006 13:02, Marius Mauch wrote: Well, posting YAIP (yet another implementation plan) won't really help either. Correct, plans never seem to go anywhere in regards to this... Don't see the goal here, just the constraints. Are you after a non-moving tree, a tree with just security fixes, visibility filters in portage, a new `emerge moo` graphics, ... ? The existing tree with additional functionality. Implemented via a new revision type. No point, would rather add a RSYNC_OPTS var finally instead, which gives the same functionality (and much more). Yeah, RSYNC_OPTIONS would remove all that hardcoded stuff and put it in make.globals instead. From make.globals: #* #** DO NOT EDIT THIS FILE ** # *** # CHANGES TO make.conf *OVERRIDE* THIS FILE # *** # ** Incremental Variables Accumulate Across Files ** # ** USE, CONFIG_*, and FEATURES are incremental ** # *** Your point being? I removed the warning in gentoolkit-2.1 (wanted to do that for quite some, just didn't get around to do it). What kind of inconsistent results are you speaking of? http://www.gentoo.org/proj/en/portage/glsa-integration.xml, section on known problems. Which except for the SLOT issues are all fixed as far as I'm concernced. The plan is nice. It does not, however, address the needs of some users to have a STABLE system as well. Some users can't willy nilly upgrade to the next version of a package because they might have requirements to stay at the same version. Through something as simple as adding a new revision specifier, a framework can be in place to backport security fixes or ONLY apply revisions that are security related... I get see your point here. `glsa-check -p affected` will shouw you exactly the *minimal* updates necessary to fix all known vulnerabilities. Not that hard to isolate the version bumps from the revbumps, is it? Also your idea has another major problem that renders it more or less useless: - you have foo-1.0 installed - foo-1.0-r1 with major changes is added to the tree - a security update based on -r1 is added to the tree as foo-1.0-s2 Now what? a) install -s2 including the changes from -r1 b) add another ebuild -s1 that is foo-1.0 with the security update but without the -r1 updates With a) there is no point at all in using the -sX system, with b) you get into the equality issues (here -r1 and -s1 are not functionally equal). Well, it only needs a way to feed a set of nodes into the dep resolver. But that in turn is quite a task as the dep resolver code is nasty. I have looked at the dep resolver and don't want to go near it with a 10 foot pole. I only want to do the functional equivalent of filtering out anything but *-sX$ (pseudo regex) during the final doemerge or when displaying in case of --pretend. That's not really what you want. -s updates might (will) be overlaid with version or revision bumps from time to time, for this to be of any use it has to happen at the resolver level (visiblity filter). What functionality does it actually add? The changes you described so far just allow multiple letters as revision prefixes. The main thing of your proposal was probably to limit updates to -s updates, and that's a tricky goal. It enables ebuild maintainers to backport security patches. It allows them to fix security problems that are not glsa alerts. It does not require the use of a new tool or integration of glsa into portage. It allows users to distinguish between revbumps of a trivial nature and revbumps of a security nature. Everything but the last one is already possible. For the last one you currently have to read the changelog. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Re: Plausible idea for GLEP 19?
On Sun, 22 Jan 2006 14:55:48 -0600 Mikey [EMAIL PROTECTED] wrote: Users are discouraged from changing make.globals. It would be better to implement in make.conf or as a command line option to emerge itself. It is a configuration item for a user preference, not a (global) portage preference... Ehm, did you read that comment you posted? make.conf overrides make.globals, so you don't have a point here. Which except for the SLOT issues are all fixed as far as I'm concernced. That is not a small issue. portage itself has similar issues with SLOTs That's not really what you want. -s updates might (will) be overlaid with version or revision bumps from time to time, for this to be of any use it has to happen at the resolver level (visiblity filter). Normal emerges would take -s2 over -r1 or -s1. The change is transparent when not in glsa-only mode. You didn't understand what I said. If you just play around with the output info you'll miss updates. Everything but the last one is already possible. For the last one you currently have to read the changelog. It does not allow for backporting, it is not safe to use in an automated environment, it does not work in a production environment, and it does not scale well. Sorry? What prevents anyone from doing backports? It's as safe as your idea for both automatic and manual (unless you assume that all security updates will only ever be a -sX bump), and the last two are just random comments that don't mean anything. Anyway, showed you why it won't work and why it won't get implemented, so rather pointless to continue this. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Plausible idea for GLEP 19?
Mikey wrote: I have been emailing the published addresses for GLEP 19 for 2 months now with no success. I am very interested in any ideas or projects that might help gentoo be more server friendly, in an enterprise environment, for lack of a better term. I have an idea towards stabilizing portage for production environments, but I am not knowledgeable enough about portage to know if it would even be plausible. If this is the wrong place to ask this, please feel free to point me in a better direction. Check the archives for gentoo-dev and gentoo-server, several implementation plans have been presented in the not-so-distant past. However everyone seems to have a slightly different goal he wants to achieve, so maybe the best would be for people to make their goals very clear. Basically, add a new value for FEATURES. For lack of a better name, call it sticky. FEATURES=sticky If this flag is present in make.conf: 1) emerge --sync does only updates, not deletes (don't ditch old ebuilds). No point, would rather add a RSYNC_OPTS var finally instead, which gives the same functionality (and much more). 2) Implement a new revision numbering scheme for ebuilds, -sX. Similar to -rX, but for glsa updates only. It could be an abbreviation for sticky, security, or stable, whatever you want. For example if I am currently running mysql-4.0.25, the only candidate an emerge -u would consider would be mysql-4.0.25-s1, mysql-4.0.25-s2, etc In other words, emerge considers only -sX in its upgrade calculations, instead of -rX, and only considers the same version. No way. No visible benefit (you know about glep 14 / glsacheck, right?) and tons of issues (redundant ebuilds, ordering screwups, ...) 3) Package maintainers could create duplicate ebuilds for security-only related revisions to packages, some other team could maintain them, this be somehow automated, or this could be left up to the users to maintain through their own overlays. My idea is fuzzy here... No chance you get people to agree on something that will bloat the tree without any real benefit. Mind that we already revbump packages for security updates. Plausible? No, this includes way too many changes to core functions (version comparison, resolver) with no visible benefit (from my POV). In case you haven't done so already take a good look at glep 14 and glsa-check (which implements the least-invasive update algorithm you seem to be after). Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Fwd: /etc/portage/profile/{pmask,arch.list, categories}
On Mon, 9 Jan 2006 10:21:49 -0500
Alec Warner [EMAIL PROTECTED] wrote:
My Apologies, KMail died, and when I restarted it it had the mail but
stripped the attachment ;)
-- Forwarded Message --
Subject: /etc/portage/profile/{pmask,arch.list, categories}
Date: Monday 09 January 2006 09:59
From: Alec Warner [EMAIL PROTECTED]
To: gentoo-portage-dev@gentoo.org
Apparently these weren't working ( not being read in ) due to a lack
of /etc/portage/profile being in Locations in config in portage.py
I split a patch for hte issue by simply adding the extra location and
fixing the odd grabfile_package calls. It seems grabfile package was
stripping the
- so it wouldn't nuke existing entries in the pmask file.
Comments welcome as usual.
The profile stacking doesn't work that way ...
$PORTDIR/profiles/ and /etc/portage aren't part of the profile
stack, /etc/portage/profile is and actually works as intended. So this
patch would actually create a bug instead of fixing one
And if you think those dirs should be part of the stacking the patch is
still wrong as it has to be done the other way round.
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
signature.asc
Description: PGP signature
[gentoo-portage-dev] making aux_get more usable for vardbapi
Currently vardbapi.aux_get only works for a subset of all auxdbkeys, as
some like KEYWORDS or DESCRIPTIOn aren't stored in vdb directly.
They are however stored in environment.bz2, but not accessible
there.
This is unintuitive and limits tools like equery or my own auxget
and metascan tools in their usability.
There are two solutions to this problem:
a) enhance vardbapi.aux_get so it can use environment.bz2
b) store more keys in vdb
Now there is a tradeoff to made: a) doesn't need space but is slow
while b) is fast but needs space, both in non-trivial amounts (runtime
increased from 1s to 9s for a metascan -i run and from 0.5s to 0.8s
for auxget -i, haven't actually checked the size increase, expect
somewhere between 1 and 10 megabytes on a typical install).
I'm attaching a patch that implements both (each in it's own hunk) as
well as a new emaint option to create the missing entries offline.
A not so obvious issue with a) is that due to the recent
storage optimizations (empty entries not being stored) it's worse than
I originally expected, as any entry missing a file will be looked up in
env.bz2 instead. Only way to avoid that would be to add special casing
in aux_get which I really dislike.
Opinions?
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
--- /home/gentoo/svn/portage/main/trunk/bin/ebuild.sh 2006-01-08 04:30:16.0 +0100
+++ /usr/lib/portage/bin/ebuild.sh 2006-01-10 18:27:15.0 +0100
@@ -914,7 +914,8 @@
for f in ASFLAGS CATEGORY CBUILD CC CFLAGS CHOST CTARGET CXX \
CXXFLAGS DEPEND EXTRA_ECONF EXTRA_EINSTALL EXTRA_MAKE \
FEATURES INHERITED IUSE LDFLAGS LIBCFLAGS LIBCXXFLAGS \
- LICENSE PDEPEND PF PKGUSE PROVIDE RDEPEND RESTRICT SLOT; do
+ LICENSE PDEPEND PF PKGUSE PROVIDE RDEPEND RESTRICT SLOT \
+ KEYWORDS HOMEPAGE SRC_URI DESCRIPTION; do
[ -n ${!f} ] echo $(echo ${!f} | tr '\n,\r,\t' ' , , ' | sed s/' \+'/' '/g) ${f}
done
echo ${USE} USE
--- /home/gentoo/svn/portage/main/trunk/pym/portage.py 2006-01-08 04:30:11.0 +0100
+++ /usr/lib/portage/pym/portage.py 2006-01-10 18:28:00.0 +0100
@@ -4501,17 +4477,32 @@
def aux_get(self, mycpv, wants):
global auxdbkeys
results = []
+ envlines = None
for x in wants:
- myfn = self.root+VDB_PATH+/+str(mycpv)+/+str(x)
- if os.access(myfn,os.R_OK):
-myf = open(myfn, r)
+ mydir = self.root+VDB_PATH+/+str(mycpv)+/
+ if os.access(mydir+str(x),os.R_OK):
+myf = open(mydir+str(x), r)
myd = myf.read()
myf.close()
-myd = re.sub([\n\r\t]+, ,myd)
-myd = re.sub( +, ,myd)
-myd = string.strip(myd)
+ # Fallback to searching environment.bz2 if no key-specific file exists
+ elif os.access(mydir+environment.bz2, os.R_OK):
+if not envlines:
+ env = os.popen(bzip2 -dcq +mydir+environment.bz2, r)
+ envlines = env.read().split(\n)
+ env.close()
+myd = [l for l in envlines if l.strip(\\').startswith(x+=)]
+if len(myd) 1:
+ raise portage_exception.CorruptionError(multiple matches for +x+ found in +mydir+environment.bz2)
+elif len(myd) == 0:
+ myd =
+else:
+ myd = myd[0].split(=,1)[1]
+ myd = myd.lstrip($).strip(\'\)
+ myd = re.sub(([nrt])+, , myd)
else:
myd =
+ myd = re.sub([\n\r\t]+, ,myd)
+ myd = re.sub( +, ,myd)
+ myd = string.strip(myd)
results.append(myd)
if EAPI in wants:
idx = wants.index(EAPI)
--- /home/gentoo/svn/portage/main/trunk/bin/emaint 2006-01-08 04:30:16.0 +0100
+++ /usr/lib/portage/bin/emaint 2006-01-10 19:29:20.0 +0100
@@ -4,7 +4,7 @@
from copy import copy
from optparse import OptionParser, OptionValueError
-
+import re
import os, portage, portage_const
class WorldHandler(object):
@@ -44,8 +44,65 @@
errors.append(portage_const.WORLD_FILE + could not be opened for writing)
return errors
+class VdbKeyHandler(object):
+ def name():
+ return vdbkeys
+ name = staticmethod(name)
+
+ def __init__(self):
+ self.list = portage.db[/][vartree].dbapi.cpv_all()
+ self.missing = []
+ self.keys = [HOMEPAGE, SRC_URI, KEYWORDS, DESCRIPTION]
+
+ for p in self.list:
+ mydir = /var/db/pkg/+p
+ ismissing = True
+ for k in self.keys:
+if os.path.exists(mydir+/+k):
+ ismissing = False
+ break
+ if ismissing:
+self.missing.append(p)
+
+ def check(self):
+ return [%s has missing keys % x for x in self.missing]
+
+ def fix(self):
+
+ errors = []
+
+ for p in self.missing:
+ mydir = /var/db/pkg/+p
+ if not os.access(mydir+/environment.bz2, os.R_OK):
+errors.append(Can't access %s % (mydir+/environment.bz2))
+ elif not os.access(mydir, os.W_OK):
+errors.append(Can't create files in %s % mydir)
+ else:
+env = os.popen(bzip2 -dcq +mydir+/environment.bz2, r)
+envlines = env.read().split(\n)
+env.close()
+for k in self.keys:
+ s = [l for l in
Re: [gentoo-portage-dev] Package moves in overlays
Mike Auty wrote: Yeah, I agree that the updates should only affect that individual tree. Whilst I can imagine cases where a move in the main tree should affect overlay trees (such as recategorizing some net-misc ebuilds into a net-proxy category), that could at least be emulated with a per-tree move directive fairly easily. It sounds as though the answer is that there currently no way to do these kinds of moves, so what's the next step? Should I try and hack up a version to deal with updates on a per tree basis? Is it actively being developed and I'm just unaware of it? What can I do to help get this feature into a future version of portage? It would be very useful to have, because at the moment network administrator's can't quite use overlays as a software management mechanism... Thanks for you time, As currently vdb/pkgdir don't track the repo where a package comes from it's not possible to do it in a proper way (those are affected by moves, not the repositories themselves). Marius -- gentoo-portage-dev@gentoo.org mailing list
[gentoo-portage-dev] [RFC] making the tree depend on portage
Ok, the subject might be confusing, so let me explain this a bit: Whenever we want/need to make structural changes to the tree that are going to break backwards compability we have a serious problem (see GLEP 44 in case you don't know about it). To reduce the impact of that problem I've got the idea to make the tree itself (so not any particular ebuild or profile) DEPEND on a minimal portage version, which the users would be forced to upgrade to (maybe with an override) before they can do anything else (with the exception of --sync). Manifest2 is one example for such a situation, another one is the request to not create manifest entries for ChangeLog and metadata.xml anymore (needs =2.0.51.20 on user side). Don't really like this idea myself, but somthing needs to be done to at least reduce the problem, having to wait years for old portage versions to (almost) vanish can't be a permanent solution. Also not talking about implementation details yet, just after comments about the general idea of forced portage updates. And just in case anybody wonders: this cannot be fixed with EAPI or adding a portage dep on packages as those only take effect when the ebuild is already parsed while the mentioned problems occur much earlier. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] chunking up portage
Brian Harring wrote: On Thu, Dec 15, 2005 at 01:54:06PM +0200, Marius Mauch wrote: Brian Harring wrote: So... thoughts? I'm not much for making portage depend on tarsync just for emerge-webrsync improvements, would rather chunk the bugger out. How about runtime detection? runtime detection is questionable from my standpoint, since while coding for it is good, without hard dep pulling it in the only folks who will ever have a faster emerge-webrsync are those who happen to know the hidden trick to merge tarsync. Well, how do you plan to inform users of the splitted package for webrsync? Could just do the same for tarsync instead, or for example if no tarsync is found print a nice little message about falling back to slow default, emerge tarsync to avoid. Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] chunking up portage
Brian Harring wrote: So... thoughts? I'm not much for making portage depend on tarsync just for emerge-webrsync improvements, would rather chunk the bugger out. How about runtime detection? Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] ACCEPT_KEYWORDS depreciation
Carsten Lohrke wrote: On Thursday 15 December 2005 13:05, Marius Mauch wrote: package.keywords isn't better or worse than ACCEPT_KEYWORDS, it's just different in its behavior. I disagree. ACCEPT_KEYWORDS is problematic for (new) users, because of its behaviour. problematic != worse *shrug*, if there is a strong desire for it I'd suggest to check for existance of package.keywords and print a warning only if it doesn't exist and ACCEPT_KEYWORDS=~arch is used (as people who have a package.keywords file/dir clearly know the difference, right?) Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Tue, 6 Dec 2005 23:19:38 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: So I'm going to make a decision and offer until Friday (Saturday in my time) for opposers to solidify and state any opposition. If there's no solid opposition, Saturday I will put current trunk into ~arch as 2.1_beta20051210. I will also post on the 2.0.53 bug that fixes are available for the ldconfig bug and the tee bug stating that we'd like to also add trunk's cache subsystem to 2.0.54 and that dependening on the next council meeting(?) the SHA1 enabling as well. Doing it this way will make the ~arch users happy straight away. If we look at it as our responsibility to get fixes and new functionality into ~arch then our jobs done and we can get back to business. Well, I've already stated several times that IMO using a 2.1 branch is wrong (use 2.2 instead), but if I'm overvoted, so it shall be. As for the cache rewrite in 2.0.54, I don't really prefer one way or the other, from an engineering POV it is 2.2 material, but if it is a major improvement and well tested it can also be in .54 (just in case my previous mail was misunderstood). As for stable users? If arch teams are willing to selectively choose what fixes they want backported to stable (when they're not prepared to move the ~arch version into stable) things will go much smoother. Of course, it would still be our responsibility to get those things backported and assert some confidence that it is working. However, once the requested fixes are backported all that needs to be done is put out the patched stable version with ~arch keywords and then leave it up to the arch teams again. Except for the slight extra burden on (which I believe many would actually find to be a blessing), it should be a win-win situation. Just in case you forgot and also for general reference, when I asked the arch teams about the portage keywording policy a few months ago (wether we should stable even without testing on all archs or to delegate that to arch teams) everyone seemed to be happy with the old policy, at least nobody voted for a change. As portage doesn't really have any arch specific code and a rather short dep list IMO it also doesn't yield any real benefit other than more people testing it (which is of course always a good thing). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Latest version vs specific version
Andrea Carpani wrote: Hi all. Here's my problem. I'm using a lot binary packages of in portage and custom created ebuilds. I have a virtual ebuild I use that contains only dependencies and I use this one to merge given versions of packages all in one shot: sort of a shanpshot of a given moment in time (sort of tag). Eg: virtual-1.0.ebuild contains RDEPEND= a-2.0.ebuild b-3.0.ebuild c-4.0.ebuild d-5.0.ebuild So that emerge -K virtual-1.0 emerges all a,b,c,d on given versions and: virtual-2.0.ebuild contains RDEPEND= a-2.1.ebuild b-3.1.ebuild c-4.1.ebuild d-5.1.ebuild My problem is that if a-2.0.ebuild and a-2.1.ebuild contain RDEPEND= c-1.0 and I have binary packages for both c-4.0 and c-4.1 when I emerge virtual-2.0.ebuild i get c-4.1.ebuild but if I emerge virtual-1.0 I get both c-4.0 (required as a dependency for virtual-1.0) and c-4.1 (as a dep for a-2.0 who gets the latest c even though c-4.0 would have satisfied the dependency. Any info on how to avoid this? It might be helpful if you'd post a real example and not some invalid pseudo-ebuild syntax. Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] .53, .54 and beyond...
Jason Stubbs wrote: On Saturday 26 November 2005 11:07, Marius Mauch wrote: On Sat, 26 Nov 2005 00:01:15 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: The only other new thing in trunk that I know of is logging but there's still a question mark over the ordering of messages... Can that be resolved soon? Anything else missing? Any reasons against any of the above? Resolved how? I'm not really sure I understood the original problem (other than listdir being underdeterministic in theory). TGL suggested that all the messages go into a single file with some sort of prefix that would then be parsed on the python side. The original order of messages could then be maintained. However, seeing as there needs to be no compatibility with the temporary files it could wait for later anyway. a) haven't seen a patch for it, so no clue about how complex it is code wise and b) I generally dislike any markup/parsing in the temporary files. I'd rather get it out as-is now and incorporate any feedback later. As you said this interface doesn't have to be compatible, also I intended to add a general might change-note for elog in the release notes. - the pycrypto hash additions (for .54) This is only useful if the vote goes in favour of adding further hash types to Manfiest, right? If the vote goes that way I've got no issues with it, but if it doesn't it would essentially be dead code. Well, the vote was more for the SHA1 change actually as that's the one triggering the size increase. The pycrypto stuff itself doesn't do anything really, it would just make the size increase more apparent. - Killing off auto-use+USE_ORDER? Yep, I'd really like to see this one gone too. We should probably state the intention on -dev first though as there might be a lot of people against it. Well, Spanky liked the suggestion ;) - the recursive grab* functions I just posted (for .54) Needs a small amount of work (/etc/portage/package.mask/foo/bar would break it) but I like the general idea. How would it break? - integration of set modules, either as emerge targets (requires serious gutting of emerge) or a first-class atoms (semantically tricky, no clue about implementation yet) I'm working on this with my refactoring. Defininately a post-.54 thing unless you want to quickly hack it into getlist()? Don't think so given that offhand I don't even know what getlist() does ... Oh, btw, two things that are in trunk but weren't listed in your original mail: - the rewritten versioning code (including the cvs and mult-suffix enhancements) - finally killing of the stupid masked by -* message Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Help with KDE Arts
Robert wrote: Hey, for some reason I cannot seem to install arts (KDE). Try asking that on the gentooo-user list, it has nothing to do with portage development. Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Re: Bugzilla Bug 112779: New and Improved Way to Handle /etc/portage
On Thu, 17 Nov 2005 09:30:15 +0200 Marius Mauch [EMAIL PROTECTED] wrote: Anthony Gorecki wrote: On Wednesday, November 16, 2005 23:12, Zac Medico wrote: I wouldn't mind having a feature like this. I would provide a way for automatic unmasking tools to keep their changes separate and easily reversible. This seems to be borderlining on being unnecessary, in my opinion. A commented section in package.unmask could work just as easily, and it would likely save time for the user. kde-base/kdelibs is just as easy to find in a sorted, sectioned file as it is in multiple files: # GNOME Packages: [...] # KDE Packages: [...] I think the point is more with a) temporary enabling/disabling of sections and b) sharing sections. Having multiple files those situations are a bit easier to handle. Shouldn't be too hard offhand, basically if isdir(foo): for x in listdir(foo): mylines.extend(grabfile(x)) else: mylines = grabfile(foo) Ok, patch is now available at dev.gentoo.org/~genone/patches/portage-recursive-grab+config.diff Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] .53, .54 and beyond...
On Sat, 26 Nov 2005 00:01:15 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: Hi all, I don't think there's really anything else that can be done for 2.0.53 so am thinking that we should probably push _rc7 + docs out and let the arch teams mark it stable when they're ready (or stick with 2.0.51.22-r3 if it pleaseth them). We should put out a 2.0.54_pre1 out soon after that. What I'm wondering in is what will be going in? So far I'm thinking along the lines of: * cache rewrite * exec cleanup * ldconfig fix * sha1 enabling The only other new thing in trunk that I know of is logging but there's still a question mark over the ordering of messages... Can that be resolved soon? Anything else missing? Any reasons against any of the above? Resolved how? I'm not really sure I understood the original problem (other than listdir being underdeterministic in theory). What's on the map for after that? There's a few things listed on the new (still unreleased?) project index and I'm looking to get the dependency stuff refactored and moved out of emerge.. What are the shortterm goals? - the pycrypto hash additions (for .54) - Manifest2 verification support (need the GLEP first so the format is sanctioned). Technically it's complete, just generation is still unfinished. (so a maybe for .54 depending on the timeframe) - Killing off auto-use+USE_ORDER? - the recursive grab* functions I just posted (for .54) - addition of auxget/metascan tools (could be for .54) - integration of set modules, either as emerge targets (requires serious gutting of emerge) or a first-class atoms (semantically tricky, no clue about implementation yet) Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] multiple hash functions
On Wed, 23 Nov 2005 14:12:22 -0600 Brian Harring [EMAIL PROTECTED] wrote: Add the keys only if there is a func that can be used- list of required chksums is a config thing (and repoman thing during commiting), so I'm not seeing any reason to have None as a value in your hashfunc mapping... Yeah, makes a bit more sense. Good that nobody saw the nasty trick I had before ;) Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Display of keyword in emerge : code proposal
On Fri, 18 Nov 2005 19:13:13 +0100 jb benoit [EMAIL PROTECTED] wrote: $ diff -Naur portage.py.sav portage.py --- portage.py.sav 2005-11-17 15:32:20.0 +0100 +++ portage.py 2005-11-17 15:15:24.0 +0100 @@ -3866,6 +3866,42 @@ pmaskfile.close() return None +def getprintablemaskingstatus(mycpv): + + #checking + global portdb + mysplit = catpkgsplit(mycpv) + if not mysplit: + raise ValueError(invalid CPV: %s % mycpv) + if not portdb.cpv_exists(mycpv): + raise KeyError(CPV %s does not exist % mycpv) + mycp=mysplit[0]+/+mysplit[1] + + #gathering data + mygroups = portdb.aux_get(mycpv, [KEYWORDS])[0].split() + myarch = settings[ARCH] + maskdict=settings.pmaskdict + + rValue=[] + test=0 + + # keyword chercking + for gp in mygroups: + if gp==~+myarch: + kmask=~+myarch + rValue.append(kmask) + test = 1 + + # package.mask checking +if test == 1: + if maskdict.has_key(mycp): + for x in maskdict[mycp]: + if mycpv in portdb.xmatch(match-all, x): + rValue.append(~M) + + return rValue + + def getmaskingstatus(mycpv): global portdb mysplit = catpkgsplit(mycpv) - Doesn't work with binpkgs (though that's probably also a problem in getmaskingstatus() itself) - there is more than keyword and p.mask for masking (profiles) - the function name is misleading (you're not checking the actual masking status) - you don't check for non-~arch and package.mask'ed packages - you don't check for non-$ARCH ACCEPT_KEYWORDS/package.keywords entries - other semantic issues I' not going to repeat - completely useless without docs. We didn't say it's complicated for no reason ;) Marius PS: But thanks anyway, looking at getmaskingstatus() reminds me of removing the stupid -* message. -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Re: Bugzilla Bug 112779: New and Improved Way to Handle /etc/portage
On Fri, 18 Nov 2005 09:01:38 -0600 Brian Harring [EMAIL PROTECTED] wrote: On Thu, Nov 17, 2005 at 07:36:05PM -0800, Zac Medico wrote: Okay, I wrote a small patch that handles everything supported by /etc/portage except bashrc (package.mask, package.unmask, package.keywords, package.use, mirrors, categories). I tested it by moving /etc/portage/package* into /etc/portage/includes/test/ and it seems to have behaved expectedly. Feedback is welcome and appreciated. ;) Feedback? Well, I don't like it mainly. :) This makes portage go looking in two different locations for overrides; I know from looking through the code, /etc/portage/package.* overrides the includes, but users won't. Well, the original proposal would fix that. Actually it might be worth to implement this (optionally) in the grab* functions directly. Configuration in two seperate locations of the same thing is usually a bad idea (exempting global configuration, user configuration, which this is not). It's not intuitive, mainly. I disagree. It's mainly a matter of personal preference. If we stick with the two locations could also make them mutually exclusive. I'd argue for extending the existing files syntax rather then this tbh, tag in a source command makes a bit more sense to me. I strongly object against this. Having a source command for the shlex parsed files is one thing, having it for plain data files is simply wrong. I'm definitely against adding a language into these files. Also it kinda prevents us to support PQAs (partially qualified atoms) some day as there might be a app-foo/source package. Plus side, with a source command, you just comment it out and you've disabled that import. With your solution, have to remove the file from the directory. See above about personal preference. Also what's easier: - $EDITOR /etc/portage/package.foo - # find line that includes the file you don't want anymore - # comment that line - # save file and exit editor - cd /etc/portage/package.foo - mv somefile .somefile (the second one I'm referring to my idea, not Zacks patch which I haven't really checked) Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Display of keyword in emerge : code proposal
On Tue, 22 Nov 2005 21:47:40 +0100 jb benoit [EMAIL PROTECTED] wrote: Marius Mauch wrote: - Doesn't work with binpkgs (though that's probably also a problem in getmaskingstatus() itself) - there is more than keyword and p.mask for masking (profiles) - the function name is misleading (you're not checking the actual masking status) - you don't check for non-~arch and package.mask'ed packages - you don't check for non-$ARCH ACCEPT_KEYWORDS/package.keywords entries - other semantic issues I' not going to repeat - completely useless without docs. If this can help, i'll respond to some of your question : My aims is very different to the one of getmaskingstatus(). Not really, you basically just want to ignore parts of the local config, that's all. I don't have to check everywere for the status : The package which status I'm searching were already checked by getmaskingstatus(). This is very important. Wrong. getmaskingstatus() doesn't decide if a package is masked (gvivsible() does that), it only checks for possible reasons. And if this should be considered for inclusion you *have* to check for all possible masks (though with profiles this gets tricky). The only 2 thinks I implemented was to search : if the package beeing checked is an unstable version, which can be done only by looking at the supported keywords in the ebuild. Again wrong. Example: - ebuild foo has KEYWORDS=~x86 - my system has ARCH=amd64 - in package.keywords I have a line foo ~x86 So while this package isn't even marked as testing for my arch your patch would treat it as stable (at least that's how I would interpret the output). This example is rather common. if the package beeing checked is a masked version. I only need to check is the package was masked in packages.mask, don't mind the reason for which it's now unmasked. I didn't say anything about checking why it's unmasked, maybe you misunderstood the following comment: you don't check for non-~arch and package.mask'ed packages What I mean is when a package is marked stable but in package.mask, your patch simply ignores that (instead of adding a M status). If you thinks this can be usefull, I can made the corection needed. If you think this is just a bunch of trash, just tell me to forget, but i'll miss this feature. Well, not trash, but it has a ton of semantic issues (likely more than just the few I posted above) to resolve first making this much more complicated than you probably think. That's probably not important for you or 90% of all users, but if we add such a feature it has to work for at least 98% of all users (ideally of course for 100%). Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] Re: Bugzilla Bug 112779: New and Improved Way to Handle /etc/portage
Anthony Gorecki wrote: On Wednesday, November 16, 2005 23:12, Zac Medico wrote: I wouldn't mind having a feature like this. I would provide a way for automatic unmasking tools to keep their changes separate and easily reversible. This seems to be borderlining on being unnecessary, in my opinion. A commented section in package.unmask could work just as easily, and it would likely save time for the user. kde-base/kdelibs is just as easy to find in a sorted, sectioned file as it is in multiple files: # GNOME Packages: [...] # KDE Packages: [...] I think the point is more with a) temporary enabling/disabling of sections and b) sharing sections. Having multiple files those situations are a bit easier to handle. Shouldn't be too hard offhand, basically if isdir(foo): for x in listdir(foo): mylines.extend(grabfile(x)) else: mylines = grabfile(foo) Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] going to need a 2.0.53-rc8
On Mon, 14 Nov 2005 09:42:56 -0600 Brian Harring [EMAIL PROTECTED] wrote: On Mon, Nov 14, 2005 at 04:32:35PM +0100, Marius Mauch wrote: On Monday 14 November 2005 00:46, Jason Stubbs wrote: Replace 2.1.0 with 2.2.0 and I'll agree. Skipping 2.1 accomplishes what? Avoid any possible confusion about what 2.1 is. People asking, whoah there, it's a later version then 2.1, where's the 2.1 functionality? will still occur. I don't think so. *cough* this is why alpha/feature releases should be date tagged against the branch point instead of the potential version. :) sorry, I don't understand this sentence. How would you have named the alpha snapshot? Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] going to need a 2.0.53-rc8
On Tue, 15 Nov 2005 00:24:02 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: On Monday 14 November 2005 00:46, Jason Stubbs wrote: On Sunday 13 November 2005 11:52, Brian Harring wrote: On Sun, Nov 13, 2005 at 09:19:55AM +0900, Jason Stubbs wrote: On Sunday 13 November 2005 04:00, Brian Harring wrote: *cough* that's that funky _p1 you're using there? :) patchlevel... I think it gives a stronger impression that 2.0.53 is distinct from 2.0.54. Is distinct the right word? I mean that it kind of shows that 2.0.53 is done but there was something that needed to be fixed quickly. 2.0.53.1 vs 2.0.53_p1 vs 2.0.53.p1 ... either of the three indicates 2.0.53 had minor fix tagged onto the base 2.0.53 release... Given portage's history of using lots of dots, 2.0.53.1 doesn't have as much impact. Is the *cough* a complaint of sorts? Well, knowing what you mean by pN, I'm just going to gesture wildly at my earlier email of lets fix the whacked out versioning now. ;) So then my suggested 2.0.53_p1 should be 2.0.54 and what is currently referred to as 2.0.54 should be 2.1.0? Any thoughts on this? If we use 2.0.54 for the fix for this, that can go into ~arch before 2.0.53_pre7 goes to .53 and arch without versioning getting screwed up. I'm still pretty sure 2.0.53 will be stable (at least on some arch) in under 48 hours and the fix for this should really go out at the same time or before... Replace 2.1.0 with 2.2.0 and I'll agree. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. signature.asc Description: PGP signature
Re: [gentoo-portage-dev] handling config stuff in portage (for package compression, etc)
On Sat, 5 Nov 2005 20:58:57 -0600
Jason Pepas [EMAIL PROTECTED] wrote:
So, I have been going over how class config works in portage, but I
am still unsure of where to fit in the changes I would need.
I suppose I'll lay out the structure of what I had in mind and ask
y'all for advice.
Compression would be supported in a modular fashion. The following
config options would be supported for each type of compression:
ZEXT - the compression filename extension
TZEXT - the binary package filename extension
ZCOMPRESS - the command used to compress a file
ZDECOMPRESS - the command used to uncompress a file
ZFILTER - command used to compress in a pipeline
ZUNFILTER - command used to uncompress in a pipeline
ZRECOMPRESS - should files already compressed using another method be
uncompressed and then recompressed using the preferred method? (ie, if
manpages are shipped as foo.1.gz and you want foo.1.bz2)
I really don't like this Z prefix, should rather be
PORTAGE_COMPRESS_{EXT,BINEXT,COMPRESSCOMMAND,...}
(avoid env namespace pollution a IMO a lot cleaner)
For example, if Z=bzip2, a file would be sourced (bzip2.sh), which
would contain the these settings:
ZEXT=bz2
TZEXT=tbz2
ZCOMPRESS=bzip2
ZDECOMPRESS=bunzip2
ZFILTER=bzip2
ZUNFILTER=bunzip2
ZRECOMPRESS=no
Why this indirection? Just for convienience or are there technical
reasons? If it's just convienience then you don't need this, just
utilize the source command in make.conf. Anyway, there is no place for
one-letter vars in make.*
The following subsystems would source one of these files to get
their settings:
PKG_
DOC_
MAN_
INFO_
possibly others.
So, if PKG_Z=bzip2, bzip2.sh would be read to set PKG_ZEXT,
PKG_TZEXT, PKG_ZCOMPRESS, etc.
Why are those vars needed? Can't they be directly derived from the
global ones?
As these module files are sourced, individual config options could be
overridden via values in the environment. For example, if I wanted
gzip used across the board, but I wanted different levels of
compression for man pages and packages, I could do this in make.conf:
Z=gzip
PKG_ZCOMPRESS=gzip --best
MAN_ZCOMPRESS=gzip --fast
Somehow this part looks like overengineering to me. Doesn't seem worth
to introduce 7*4=28 new vars just for this.
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
pgpw5V5iOCAyQ.pgp
Description: PGP signature
Re: [gentoo-portage-dev] [PATCH] elog-base
Jason Stubbs wrote:
On Sunday 23 October 2005 00:08, Marius Mauch wrote:
- needs better integration of isolated-functions.sh, probably should be
a separate patch (Brian?)
Not sure what you mean by better as I'm happy with the current method. Other
than the hardcoded path, it should work fine...
The unconditional call to set_colors for example.
--- pym/portage.py (revision 2155)
+++ pym/portage.py (working copy)
+ # exploit listdir() file order so we process log entries in
cronological order
+ mylogfiles.reverse()
Exploiting listdir()'s ordering is a bad idea. A bugfix to it could quite
possibly change the ordering returned. Looking at cacheddir(), results are
current being returned in filesystem order.
Seeing that interspersed einfo and ewarn calls will be separated by class
anyway, is there any need to play with the ordering at all? As far as the
rest of the code goes, it only affects whether INFO comes before WARN or
not. Phases themselves come out in the correct order regardless.
Hmm, not sure myself, _might_ be a leftover from an older prototype
version that didn't have the separation. If someone else can verify
(can't do it myself atm) that the output is (mostly) unaffected by
removing that hack no objection here.
print !!! Error while importing logging module %s: % s
would be better in case the exception doesn't contain the module name.
Cosmetics, no objection here.
--- pym/portage_const.py(revision 2155)
+++ pym/portage_const.py(working copy)
+EBUILD_PHASES = setup unpack compile test install preinst
postinst prerm postrm
This constant is can only ever be used as a list. It should be defined
that way. It might also be worth combining this list into the keys of
actionmap_deps.
*shrug*, doesn't really matter to me.
+esyslog() {
+ local pri=
+ local tag=
+
+ if [ -x /usr/bin/logger ]
+ then
+ pri=$1
+ tag=$2
+
+ shift 2
+ [ -z $* ] return 0
+
+ /usr/bin/logger -p ${pri} -t ${tag} -- $*
+ fi
+
+ return 0
+}
What's this one for? Ebuilds should never be logging to syslog directly...
That was predefined in isolated-functions.sh, so I'd guess it's
something from the old functions.sh, it has nothing todo with elog
(another thing I meant with better isolated-functions.sh integration).
Marius
--
gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] The road ahead...
On Sat, 22 Oct 2005 00:14:40 +0900 Jason Stubbs [EMAIL PROTECTED] wrote: The cheapness is exactly why I was questioning. Consider: # svn cp tags/2.0.53 branches/2.0.53-branch # cd branches/2.0.53-branch # patch something-that-needs-fixing-now.patch # svn ci # cd ../.. # svn cp branches/2.0.53-branch tags/2.0.53-r1 # svn rm branches/2.0.53-branch # svn ci The last fives steps are easier to do: # svn ci # could even omit this, but that's bad practice # svn mv . ../../tags/2.0.53-something # cd ../.. # svn ci compared to: # svn cp tags/2.0.53 tags/2.0.53-r1 # cd tags/2.0.53-r1 # patch something-that-needs-fixing-now.patch # svn ci With the way subversion works, I would have thought the end result would be identical... It is, but the concept of a tag in vcs (as I understand it) is that it's immutable, and branches are the mutable counterpart. Technically it doesn't matter to svn. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. pgpDct4nhZVNI.pgp Description: PGP signature
[gentoo-portage-dev] [PATCH] elog-base
First patch for elog integration in 2.0.x adding the basic elog
framework without the actual modules, config samples or other docs. The
code is mostly unchanged from the 2.1 branch and only lightly tested
on 2.0. Known issues with this patch:
- needs better integration of isolated-functions.sh, probably should be
a separate patch (Brian?)
- module loader should be improved somehow (low priority)
- logging currenty only occurs after successful merges
- probably more that I can't remember atm
Later patches will add the actual logging modules and make.*
enhancements, maybe also python logging (for einfo style messages) and
support tools.
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
Index: pym/portage.py
===
--- pym/portage.py (revision 2155)
+++ pym/portage.py (working copy)
@@ -454,6 +454,57 @@
mygraph.okeys=self.okeys[:]
return mygraph
+def elog_process(cpv, mysettings):
+ mylogfiles = listdir(mysettings[T]+/logging/)
+ # shortcut for packages without any messages
+ if len(mylogfiles) == 0:
+ return
+ # exploit listdir() file order so we process log entries in cronological order
+ mylogfiles.reverse()
+ mylogentries = {}
+ for f in mylogfiles:
+ msgfunction, msgtype = f.split(.)
+ if not msgtype.upper() in mysettings[PORTAGE_ELOG_CLASSES].split() \
+and not msgtype.lower() in mysettings[PORTAGE_ELOG_CLASSES].split():
+ continue
+ if msgfunction not in portage_const.EBUILD_PHASES.split():
+ print !!! can't process invalid log file: %s % f
+ continue
+ if not msgfunction in mylogentries:
+ mylogentries[msgfunction] = []
+ msgcontent = open(mysettings[T]+/logging/+f, r).readlines()
+ mylogentries[msgfunction].append((msgtype, msgcontent))
+
+ # in case the filters matched all messages
+ if len(mylogentries) == 0:
+ return
+
+ # generate a single string with all log messages
+ fulllog =
+ for phase in portage_const.EBUILD_PHASES.split():
+ if not phase in mylogentries:
+ continue
+ for msgtype,msgcontent in mylogentries[phase]:
+ fulllog += %s: %s\n % (msgtype, phase)
+ for line in msgcontent:
+fulllog += line
+ fulllog += \n
+
+ # pass the processing to the individual modules
+ logsystems = mysettings[PORTAGE_ELOG_SYSTEM].split()
+ for s in logsystems:
+ try:
+ # FIXME: ugly ad.hoc import code
+ # TODO: implement a common portage module loader
+ logmodule = __import__(elog_modules.mod_+s)
+ m = getattr(logmodule, mod_+s)
+ m.process(mysettings, cpv, mylogentries, fulllog)
+ except (ImportError, AttributeError), e:
+ print !!! Error while importing logging modules:
+ print e
+ except portage_exception.PortageException, e:
+ print e
+
# valid end of version components; integers specify offset from release version
# pre=prerelease, p=patchlevel (should always be followed by an int), rc=release candidate
# all but _p (where it is required) can be followed by an optional trailing integer
@@ -2505,6 +2556,12 @@
os.chown(mysettings[T],portage_uid,portage_gid)
os.chmod(mysettings[T],02770)
+ logdir = mysettings[T]+/logging
+ if not os.path.exists(logdir):
+ os.makedirs(logdir)
+ os.chown(logdir, portage_uid, portage_gid)
+ os.chmod(logdir, 0770)
+
try: # XXX: negative RESTRICT
if not ((nouserpriv in string.split(mysettings[PORTAGE_RESTRICT])) or \
(userpriv in string.split(mysettings[PORTAGE_RESTRICT]))):
@@ -6679,6 +6736,10 @@
if dircache.has_key(self.dbcatdir):
del dircache[self.dbcatdir]
print ,self.mycpv,merged.
+
+ # Process ebuild logfiles
+ elog_process(self.mycpv, self.settings)
+
return 0
def mergeme(self,srcroot,destroot,outfile,secondhand,stufftomerge,cfgfiledict,thismtime):
Index: pym/portage_const.py
===
--- pym/portage_const.py (revision 2155)
+++ pym/portage_const.py (working copy)
@@ -40,6 +40,8 @@
SANDBOX_PIDS_FILE = /tmp/sandboxpids.tmp
CONFIG_MEMORY_FILE = PRIVATE_PATH + /config
+EBUILD_PHASES = setup unpack compile test install preinst postinst prerm postrm
+
INCREMENTALS=[USE,FEATURES,ACCEPT_KEYWORDS,ACCEPT_LICENSE,CONFIG_PROTECT_MASK,CONFIG_PROTECT,PRELINK_PATH,PRELINK_PATH_MASK]
STICKIES=[KEYWORDS_ACCEPT,USE,CFLAGS,CXXFLAGS,MAKEOPTS,EXTRA_ECONF,EXTRA_EINSTALL,EXTRA_EMAKE]
Index: bin/isolated-functions.sh
===
--- bin/isolated-functions.sh (revision 0)
+++ bin/isolated-functions.sh (revision 0)
@@ -0,0 +1,204 @@
+# Copyright 1999-2004 Gentoo Technologies, Inc.
+# Distributed under the terms of the GNU General Public License v2
+# $Header$
+
+# Internal logging function, don't use this in ebuilds
+elog_base() {
+ local messagetype
+ [ -z ${1} -o -z ${T} -o ! -d ${T}/logging ] return 1
+ case ${1} in
+
[gentoo-portage-dev] [PATCH] elog-modules
This patch depends on elog_base (although it doesn't break anything
without it) and adds the actual logging modules. I've just atatched the
files completely, as a) svn diff doesn't play nice with generating
new-file diffs and b) they are just new files to be dropped in
pym/elog_modules (together with an empty __init__.py).
Marius
--
Public Key at http://www.genone.de/info/gpg-key.pub
In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
import elog_modules.mod_save, portage_exec, portage_exception
def process(mysettings, cpv, logentries, fulltext):
elog_modules.mod_save.process(mysettings, cpv, logentries, fulltext)
if (not PORTAGE_ELOG_COMMAND in mysettings.keys()) \
or len(mysettings[PORTAGE_ELOG_COMMAND]) == 0:
raise portage_exception.MissingParameter(!!! Custom logging requested but PORTAGE_ELOG_COMMAND is not defined)
else:
mylogcmd = mysettings[PORTAGE_ELOG_COMMAND]
mylogcmd.replace(${LOGFILE}, elogfilename)
mylogcmd.replace(${PACKAGE}, cpv)
retval = portage_exec.spawn_bash(mylogcmd)
if retval != 0:
raise portage_exception.PortageException(!!! PORTAGE_ELOG_COMMAND failed with exitcode %d % retval)
return
import smtplib, email.Message, socket, portage_exception
def process(mysettings, cpv, logentries, fulltext):
mymailhost = localhost
mymailport = 25
mymailuser =
mymailpasswd =
myrecipient = [EMAIL PROTECTED]
mysubject = [portage] Ebuild log for %s
# Syntax for PORTAGE_LOG_MAILURI (if defined):
# adress [[user:[EMAIL PROTECTED]:port]]
# where adress: recipient adress
# user: username for smtp auth (defaults to none)
# passwd: password for smtp auth (defaults to none)
# mailserver: smtp server that should be used to deliver the mail (defaults to localhost)
# port: port to use on the given smtp server (defaults to 25, values 10 indicate that starttls should be used on (port-10))
if PORTAGE_ELOG_MAILURI in mysettings.keys():
if in mysettings[PORTAGE_ELOG_MAILURI]:
myrecipient, mymailuri = mysettings[PORTAGE_ELOG_MAILURI].split()
if @ in mymailuri:
myauthdata, myconndata = mymailuri.split(@)
try:
mymailuser,mymailpasswd = myauthdata.split(:)
except ValueError:
print !!! invalid SMTP AUTH configuration, trying unauthenticated ...
else:
myconndata = mymailuri
if : in myconndata:
mymailhost,mymailport = myconndata.split(:)
else:
mymailhost = myconndata
else:
myrecipient = mysettings[PORTAGE_ELOG_MAILURI]
if PORTAGE_ELOG_MAILSUBJECT in mysettings.keys():
mysubject = mysettings[PORTAGE_ELOG_MAILSUBJECT]
try:
mymessage = email.Message.Message()
mymessage.set_unixfrom(portage)
mymessage.set_payload(fulltext)
mymessage[To] = myrecipient
mymessage[Subject] = mysubject % cpv
if int(mymailport) 10:
myconn = smtplib.SMTP(mymailhost, int(mymailport) - 10)
myconn.starttls()
else:
myconn = smtplib.SMTP(mymailhost, mymailport)
if mymailuser != and mymailpasswd != :
myconn.login(mymailuser, mymailpasswd)
myconn.sendmail(portage, myrecipient, mymessage.as_string())
myconn.quit()
except smtplib.SMTPException, e:
raise portage_exception.PortageException(!!! An error occured while trying to send logmail:\n+str(e))
except socket.error, e:
raise portage_exception.PortageException(!!! A network error occured while trying to send logmail:\n+str(e)+\nSure you configured PORTAGE_ELOG_MAILURI correctly?)
return
import os, time
from portage_data import portage_uid, portage_gid
def process(mysettings, cpv, logentries, fulltext):
cpv_path = cpv.replace(/, :)
if mysettings.has_key[PORTAGE_ELOG_SAVEDIR]:
elogdir = mysettings[PORTAGE_ELOG_SAVEDIR]
else:
elogdir = mysettings[PORTAGE_TMPDIR]+/elogs
if not os.path.exists(elogdir):
os.makedirs(elogdir)
os.chown(elogdir, portage_uid, portage_gid)
os.chmod(elogdir, 0770)
elogfilename = elogdir+/+cpv_path+:+time.strftime(%Y%m%d-%H%M%S, time.gmtime(time.time()))+.log
elogfile = open(elogfilename, w)
elogfile.write(fulltext)
elogfile.close()
return
import syslog
from portage_const import EBUILD_PHASES
def process(mysettings, cpv, logentries, fulltext):
syslog.openlog(portage, syslog.LOG_ERR | syslog.LOG_WARNING | syslog.LOG_INFO | syslog.LOG_NOTICE, syslog.LOG_LOCAL5)
for phase in EBUILD_PHASES.split():
if not phase in logentries:
continue
for msgtype,msgcontent in logentries[phase]:
pri = {INFO: syslog.LOG_INFO, WARN: syslog.LOG_WARNING, ERROR: syslog.LOG_ERR, LOG: syslog.LOG_NOTICE}
msgtext = .join(msgcontent)
syslog.syslog(pri[msgtype], %s: %s: %s % (cpv, phase, msgtext))
syslog.closelog()
pgpAMLLKRkhMb.pgp
Description: PGP signature
Re: [gentoo-portage-dev] The road ahead...
Jason Stubbs wrote: After thinking about it, incremental feature creep does seem like the best way to go at this late stage in 2.0's life. The problem is how to guage what is and what is not more trouble than worth. Perhaps adhering to the kernel's rule of Separate each logical change into its own patch would help to ease the possible impact of larger changes? Probably the best solution. Speaking of which, if something does crop up with 2.0.53 while the arch guys are deciding if it's stable, how should that be dealt with in subversion? Continue development under branches/2.0 and, should an issue crop up, `svn cp tags/2.0.53_rc6 tags/2.0.53_rc7` and make any required change directly under the tag? Never commit to a tag, make a branch (branches are cheap in svn), and if the branch is finished make a tag. Btw, anyone object to swap branches/2.0 with trunk (seeing that 2.1 is dead)? Another by the way, `svn -v log ChangeLog` for 2.0.53_rc6. I'm open to anything better if anybody has a good format for autogeneration. The quality of the commit messages themselves isn't really useful though without knowing their context, so this might be a bit of a catch 22.. For 2.0.54, viewsvn should be available so it might be better to just use the tracker bug to manually create a summary of notable changes. Hmm, so you want to change the ChangeLog into release notes? IMO we should have both (a detailed technical ChangeLog and user friendly release notes). Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Custom eclass question
Brian Harring wrote: On Tue, Oct 11, 2005 at 10:37:43AM +0300, Marius Mauch wrote: Brian Harring wrote: On Sun, Oct 09, 2005 at 06:52:24PM -0500, Mikey wrote: http://codeserver.wherever.net/pman/package_ids.php?action=packageid=10105 [snip bits about wget screwing up] Others have already clarified that's it python side rather then bash so eclass is no go, but out of curiousity any got a good reason we can't modify FETCHCOMMAND to include FILE rather then just DISTDIR? With wget, it's just -O. Curl, piping probably (although haven't looked at their opts)... Where do you get the filename from if not SRC_URI? Additional metadata tagged in, or mangling of the syntax in some way. I don't want to imagine how ugly that will get, syntax mangling will probably break things and additional metadata needs a way to properly map entries. Kind of a moot question if we can't extend FETCHCOMMAND/RESUMECOMMAND for it though, hence the question about if any fetchers will seriously choke on this requirement. *shrug*, IIRC we removed prozilla support long ago, so wget (and curl for osx) are currently the only supported fetchers. Anything else falls into user responsibility. Marius -- gentoo-portage-dev@gentoo.org mailing list
Re: [gentoo-portage-dev] Custom eclass question
Mikey wrote: On Sunday 09 October 2005 19:32, Marius Mauch wrote: Well, ebuilds (and therefore eclasses) can't override anything related to the fetch process (other than setting RESTRICT and/or SRC_URI). Your problem has to be fixed server side (assuming you want a proper solution), as portage *needs* the correct filename in SRC_URI for things like md5 verification. So even if $FETCHCOMMAND would save your file with the right name you would be in trouble. I was wondering why pkg_fetch seemed to be the only function that cannot be localized in an ebuild... Why is that? Because fetching isn't done in the bash part, but in the python part of portage, and ebuild can only override things in the bash part ever. Fetching involves a lot more than just calling wget, take a look at fetch() in portage.py if you're interested. Oh, and there are many more things that cannot be overridden, like dep calculation, merging, cleaning, ... Marius -- gentoo-portage-dev@gentoo.org mailing list
