Re: [gentoo-portage-dev] Re: equery refactorization

[Marius Mauch]

On Sun, 7 Dec 2008 13:54:38 +0900
"Douglas Anderson" <[EMAIL PROTECTED]> wrote:

> $ equery list -h
> List all packages matching a query pattern
> Syntax:
>   list  pkgspec
>  is either of:
>   -i, --installed - search installed packages (default)
>   -I, --exclude-installed - do not search installed packages
>   -p, --portage-tree  - also search in portage tree (/usr/portage)
>   -o, --overlay-tree  - also search in overlay tree
> (/usr/portage/local/layman/wschlich-testing
> /usr/portage/local/layman/xwing /usr/portage/local/layman/genscripts
> /usr/portage/local/layman/wschlich-testing
> /usr/portage/local/layman/xwing /usr/portage/local/layman/genscripts
> /usr/local/portage)
>   -f, --full-regex- query is a regular expression
>   -e, --exact-name- list only those packages that exactly
> match -d, --duplicates- list only installed duplicate packages
> 
> That's an awful lot of "searching" there for something that's
> definitely not a search. List is really ambiguous, but whatever.

So replace "search" with "list" or "look" in the help output.

> I understand the point of having a stable interface, but this is
> probably the most widely recommended tool on the forums and #gentoo.
> Stability is not a good enough reason to let it bit rot. Wasn't a more
> unified tool interface also one of the original goals of gentoolkit?

I never said that you should let it bitrot, just that changing the
syntax for IMO questionable reasons shouldn't be done lightly. I was
stictly talking about the user interface, not the API, in case that
wasn't obvious.

Marius

Re: [gentoo-portage-dev] Re: equery refactorization

[Marius Mauch]

On Sun, 7 Dec 2008 12:44:25 +0900
"Douglas Anderson" <[EMAIL PROTECTED]> wrote:

> I also thought about renaming the "list(l)" option as "search",
> because if you look at the help output, almost every module "lists"
> something. equery's "list" is actually a search, I don't see why we
> shouldn't name it that. I think maybe list was used because there were
> already two "s" options, stats and size. Stats is not implemented so
> I'm taking it out of help for now. Size can use the short "z", becaues
> that's quite unique. That would free up "s" for search and it would be
> a whole lot clearer.
> 
> Yes? No?

No. "search" (if used at all) should be reserved for a more
comprehensive search framework (though IMO a separate tool for that is
more appropriate), not just a simple name match. "list" makes sense if
you consider that the pkgspec argument is optional, and one of the main
tasks of it is to simply list the packages in the given repository
(that's why vardb is also the default for it) without further filtering.

Also one of the main goals of equery (according to karltk, the original
author) was to have a stable user interface, compared to the deprecated
qpkg and etcat scripts. And while the equery interface isn't exactly
the best I've seen it has been stable, so you might want to think twice
before renaming options and eventually pissing off users or breaking
third-party scripts.

Marius

[gentoo-portage-dev] Time to say goodbye

[Marius Mauch]

So, time has come for me to realize that my time with Gentoo is over. I
haven't actually been doing much Gentoo work over the last months due
to personal reasons (nothing Gentoo related), and I don't see that
situation changing in the near future. In fact I've already reassigned
or dropped most of my responsibilites in Gentoo a while ago, so there
are just a few pet projects left to give away:
- my gentoo-stats project (in the portage/gentoo-stats svn repository).
I know quite a few people are interested in the idea of collecting
various statistic data from gentoo user systems, and I'd encourage
everyone who wants to implement such a system to at least look at it (I
may have even finished it if I wouldn't have wasted my time focusing on
the wrong problems). There is quite a bit of documentation also that
should help to get you started
- a graphical security update tool (see bug #190397)

So if anyone wants to adopt those, complete or just parts, just take
them. As for Portage, Zac has practically already filled my role.

So I guess that wraps it up. It's been a nice ride most of the time,
but now it's time for me to leave the Gentoo train.

Marius

Re: [gentoo-portage-dev] search functionality in emerge

[Marius Mauch]

On Sun, 23 Nov 2008 21:01:40 -0800 (PST)
devsk <[EMAIL PROTECTED]> wrote:

> > not relying on custom system daemonsrunning in the background.
> 
> Why is a portage daemon such a bad thing? Or hard to do? I would very
> much like a daemon running on my system which I can configure to sync
> the portage tree once a week (or month if I am lazy), give me a
> summary of hot fixes, security fixes in a nice email, push important
> announcements and of course, sync caches on detecting changes (which
> should be trivial with notify daemons all over the place) etc. Why is
> it such a bad thing?

Well, as an opt-in solution it might work (though most of what you
described is IMO just stuff for cron, no need to reinvent the wheel).

What I was saying is that _relying_ on custom system
daemons/filesystems for a _core subsystem_ of portage is the wrong
way, simply because it adds a substantial amount of complexity to the
whole package management architecture. It's one more thing that can
(and will) break, one more layer to take into account for any design
decisions, one more component that has to be secured, one more obstacle
to overcome when you want to analyze/debug things.
And special care must be taken if it requires special kernel support
and/or external packages. Do you want to make inotify support mandatory
to use portage efficiently? (btw, looks like inotify doesn't really
work with NFS mounts, which would already make such a daemon completely
useless for people using a NFS-shared repository)

And finally, if you look at the use cases, a daemon is simply overkill
for most cases, as the vast majority of people only use emerge
--sync (or wrappers) and maybe layman to change the tree, usually once
per day or less often. Do you really want to push another system daemon
on users that isn't of use to them?

> Its crazy to think that security updates need to be pulled in Linux.

That's IMO better be handled via an applet (bug #190397 has some code),
or just check for updates after a sync (as syncing is the only
way for updates to become available at this time). Maybe a message
could be added after sync if there are pending GLSAs, now that the glsa
support code is in portage.

Marius

Re: [gentoo-portage-dev] search functionality in emerge

[Marius Mauch]

On Sun, 23 Nov 2008 07:17:40 -0500
"Emma Strubell" <[EMAIL PROTECTED]> wrote:

> However, I've started looking at the code, and I must admit I'm pretty
> overwhelmed! I don't know where to start. I was wondering if anyone
> on here could give me a quick overview of how the search function
> currently works, an idea as to what could be modified or implemented
> in order to improve the running time of this code, or any tip really
> as to where I should start or what I should start looking at. I'd
> really appreciate any help or advice!!

Well, it depends how much effort you want to put into this. The current
interface doesn't actually provide a "search" interface, but merely
functions to
1) list all package names - dbapi.cp_all()
2) list all package names and versions - dbapi.cpv_all()
3) list all versions for a given package name - dbapi.cp_list()
4) read metadata (like DESCRIPTION) for a given package name and
version - dbapi.aux_get()

One of the main performance problems of --search is that there is no
persistent cache for functions 1, 2 and 3, so if you're "just"
interested in performance aspects you might want to look into that.
The issue with implementing a persistent cache is that you have to
consider both cold and hot filesystem cache cases: Loading an index
file with package names and versions might improve the cold-cache case,
but slow things down when the filesystem cache is populated.
As has been mentioned, keeping the index updated is the other major
issue, especially as it has to be portable and should require little or
no configuration/setup for the user (so no extra daemons or special
filesystems running permanently in the background). The obvious
solution would be to generate the cache after `emerge --sync` (and other
sync implementations) and hope that people don't modify their tree and
search for the changes in between (that's what all the external tools
do). I don't know if there is actually a way to do online updates while
still improving performance and not relying on custom system daemons
running in the background.

As for --searchdesc, one problem is that dbapi.aux_get() can only
operate on a single package-version on each call (though it can read
multiple metadata variables). So for description searches the control
flow is like this (obviously simplified):

result = []
# iterate over all packages
for package in dbapi.cp_all():
# determine the current version of each package, this is 
# another performance issue.
version = get_current_version(package)
# read package description from metadata cache
description = dbapi.aux_get(version, ["DESCRIPTION"])[0]
# check if the description matches
if matches(description, searchkey):
result.append(package)

There you see the three bottlenecks: the lack of a pregenerated package
list, the version lookup for *each* package and the actual metadata
read. I've already talked about the first, so lets look at the other
two. The core problem there is that DESCRIPTION (like all standard
metadata variables) is version specific, so to access it you need to
determine a version to use, even though in almost all cases the
description is the same (or very similar) for all versions. So the
proper solution would be to make the description a property of the
package name instead of the package version, but that's a _huge_ task
you're probably not interested in. What _might_ work here is to add
support for an optional package-name->description cache that can be
generated offline and includes those packages where all versions have
the same description, and fall back to the current method if the
package is not included in the cache. (Don't think about caching the
version lookup, that's system dependent and therefore not suitable for
caching).

Hope it has become clear that while the actual search algorithm might
be simple and not very efficient, the real problem lies in getting the
data to operate on. 

That and the somewhat limited dbapi interface.

Disclaimer: The stuff below involves extending and redesigning some
core portage APIs. This isn't something you can do on a weekend, only
work on this if you want to commit yourself to portage development
for a long time.

The functions listed above are the bare minimum to
perform queries on the package repositories, but they're very
low-level. That means that whenever you want to select packages by
name, description, license, dependencies or other variables you need
quite a bit of custom code, more if you want to combine multiple
searches, and much more if you want to do it efficient and flexible.
See http://dev.gentoo.org/~genone/scripts/metalib.py and
http://dev.gentoo.org/~genone/scripts/metascan for a somewhat flexible,
but very inefficient search tool (might not work anymore due to old
age).

Ideally repository searches could be done without writing any
application code using some kind of query language, similar to how SQL
works for generic database searches (obviously not that 

Re: [gentoo-portage-dev] [RFC] portage-2.1.6 release plans

[Marius Mauch]

On Fri, 07 Nov 2008 15:38:48 -0800
Zac Medico <[EMAIL PROTECTED]> wrote:

> Since portage-2.2 isn't quite ready yet due to ongoing work in
> package sets and preserve-libs, and I don't want ongoing work to
> hold back other features that are stable, I'm planning to split a
> 2.1.6 branch from trunk. This branch will have package sets and
> preserve-libs support disabled.

Disabled as in FEATURES="-preserve-libs" and an almost empty
sets.conf, or disabled as in "remove all traces from the code"?

Marius

Re: [gentoo-portage-dev] [2/4] proto-GLEPS for Tree-signing

[Marius Mauch]

On Tue, 29 Jul 2008 20:51:45 +0100
Mike Auty <[EMAIL PROTECTED]> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Sorry,
>   I lost my notes from when I last looked these over several
> months ago, and only just found them again.  I haven't copied this to
> [EMAIL PROTECTED], so let me know if I should do that.  I just had a quick
> couple of things I was thinking about, and one of them I figured out
> during my re-read, so it's only really the following...
> 
> In this Glep (xx+1), in the section discussing the procedure for
> creating a MetaManifest file, in step 3.3, does that include
> verification of the manifest's signature if it has one?  It would seem
> odd to ignore the signature if it's wrong (I'm not sure about the case
> if a signature isn't present).  I also don't know how this would then
> be handled (a complete abort, or ignoring the latest changeset to that
> ebuild?).

I don't think that verification at this stage would be a good idea. The
only sane way to respond to a failed check would be to either exclude
the whole package from the sync (keeping the state from the last run),
leading to various problems (what if it's a critical bugfix/security
bump, or breaks a the deptree of many packages?), or not record the
Manifest in the Metamanifest, which hasn't any benefits over reyling on
the client doing the verification.

Marius

Re: [gentoo-portage-dev] portage-2.2-rc3 parallel merges quit being parallel

[Marius Mauch]

On Sat, 26 Jul 2008 16:56:20 -0500
Andrew Gaffney <[EMAIL PROTECTED]> wrote:

> Duncan wrote:
> > "--jobs=10 --keep-going --load-average=15"
> 
> For a dual-dual-core setup, a load average of 4.0 is "fully loaded".

Only in ideal cases, when you have long-running processes
hammering the cpu and little or no context switches. This isn't the case
with builds, the actual compile processes that need cpu time usually
terminate very quickly, which increases the load average.
I did some benchmarking a while ago with different combinations of -j
and -l in MAKEOPTS, using the kernel as testcase, and IIRC the fastest
builds were around -l6.0 (on a dual-core system) and high (or
unlimited) values for -j (sidenote here: some ebuilds like openoffice
parse MAKEOPTS to get parameters for their own build systems, but only
recognize/support -j and ignore -l, so one should still be careful with
the value for -j).

Marius

Re: [gentoo-portage-dev] Connecting source and binary packages.

[Marius Mauch]

On Wed, 16 Jul 2008 20:17:37 +0200
Patrick Börjesson <[EMAIL PROTECTED]> wrote:

> On 2008-07-16 14:12, Jason Cipriani uttered these thoughts:
> > I have mozilla-firefox-bin installed, but when I update things that
> > depend on firefox (such as epiphany), they attempt to get
> > mozilla-firefox instead. How do I make emerge realize that I already
> > have the binaries installed and don't need the source distribution?
> 
> man portage, search for package.provided

Assuming that the packages do actually work with the -bin version ...
--
gentoo-portage-dev@lists.gentoo.org mailing list

Re: [gentoo-portage-dev] A seemingly bug found when emerging @preserved-rebuild

[Marius Mauch]

On Wed, 2 Jul 2008 02:41:14 +0800
Zhang Le <[EMAIL PROTECTED]> wrote:

> Is this a bug? or did I miss something here?
> Thanks for your time!

The 'bug' here is that USE=multislot shouldn't exist. People using it
should be able to deal with resulting breakages on their own

Marius
-- 
gentoo-portage-dev@lists.gentoo.org mailing list

Re: [gentoo-portage-dev] Portage persistence structures :: information about ports tree

[Marius Mauch]

On Mon, 09 Jun 2008 17:36:14 -0300
João Macaíba <[EMAIL PROTECTED]> wrote:

> May someone give me some help on this ? How does portage do the
> searchs ? Walk into the ports tree and build some structure or store
> this info on some embedded database like berkeley db or sqlite ?

You're probably looking for the portdbapi class defined in
pym/portage.py (or pym/portage/dbapi/porttree.py in 2.2), in particular
the cp_all(), cpv_all(), cp_list() and aux_get() methods.

Marius
--
gentoo-portage-dev@lists.gentoo.org mailing list

Re: [gentoo-portage-dev] [PATCH] show binhosts as "repository"

[Marius Mauch]

On Mon, 12 May 2008 20:53:35 +0200
Fabian Groffen <[EMAIL PROTECTED]> wrote:

> The following patch shows the url to the binhost in an emerge -av as
> repository name, instead of "unknown".  Unfortunately the patch
> doesn't store the binhost url, such that portage can't show where the
> package comes from when unmerged.

Not sure if this is the right thing to do, or if the reponame in those
cases should be the one from the source repo that was used to generate
those binary packages. Not saying that the binhost name here is
useless, quite the opposite, just a bit concerned about mixing
different things here.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] [PATCH] Repoman subversion support

[Marius Mauch]

On Mon, 12 May 2008 20:01:34 +0200
Fabian Groffen <[EMAIL PROTECTED]> wrote:

> After a short discussion on #-portage, here a feature patch from the
> prefix branch, diffed to the trunk, irrelevant hunks removed.
> 
> Antarus once was working on factoring out the vcs bits of repoman into
> pym/repoman/utilities.py, but he never finished it.  I'm willing to
> give that some more time, provided it'll end up in the trunk.  Sofar,
> here's the svn patch that we tested extensively in Prefix ;)  I still
> believe that it has a significant value for non-Prefix users
> (overlays...), even though the code doesn't deserve any price for its
> beauty.  I'd say it fits nicely with large chunks of Portage :p

Merged in r10325 with some minor changes (removed the 'svn update' bit
until someone remembers why exactly it's there, skip the commit
optimization for svn regarding Manifest regeneration, unify isVcs and
vcs variables)

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] BNF for DEPEND in EAPI0 and EAPI1

[Marius Mauch]

On Sun, 13 Apr 2008 20:04:41 +0200
Björn Michaelsen <[EMAIL PROTECTED]> wrote:

> Hi there,
> 
> I worked through the current PMS and tried to formalize the grammar
> for DEPEND a little better. Maybe someone could have a look at it and
> check it for correctness. It could even become part of the gentoo
> development docs or be an appendix to the PMS. The other grammars
> (SRC_URI etc.) should be fairly easy.
> 
> Any comments?

Offhand the 'package name' part is wrong/incomplete, pretty sure even
PMS says that they can't start/end with underscores or dashes (it's a
real PITA to get it right).
Haven't really read the non-terminals section as it's hard to read in
this form, using EBNF or some markup language might help with that.

Marius
--
gentoo-portage-dev@lists.gentoo.org mailing list

Re: [gentoo-portage-dev] Re: querying whether pkg is installed with version X or upwards ("atom" syntax) (re-post)

[Marius Mauch]

On Wed, 26 Mar 2008 12:07:13 +0200
Amit Dor-Shifer <[EMAIL PROTECTED]> wrote:

> reposting, hoping for a reply this time.
> Thanks,
> Amit
> 
> Amit Dor-Shifer wrote:
> > emerge features syntax for installing a package according to certain
> > bounds. This is referred to in the manpage as "atom" specification.
> >
> > I'm wondering whether equery has the same feature for querying.
> > E.G: I'd like to be able to ask:
> >
> > equery l '>=app-misc/my-pkg-1.1'
> >
> > I'd expect the listing to be empty when my-pkg is not installed, or
> > is installed by of version less than 1.1.

Well, most equery subcommands support the same atom syntax, just 'list'
and 'size' apparently don't implement it, as those support regular
expressions instead.

Marius
-- 
gentoo-portage-dev@lists.gentoo.org mailing list

Re: [gentoo-portage-dev] Re: [gentoo-qa] splitting up package.mask

[Marius Mauch]

On Sat, 15 Mar 2008 01:34:24 -0700
"Alec Warner" <[EMAIL PROTECTED]> wrote:

> On 3/14/08, Mike Frysinger <[EMAIL PROTECTED]> wrote:
> > On Friday 14 March 2008, Alec Warner wrote:
> >  > On 3/14/08, Mike Frysinger <[EMAIL PROTECTED]> wrote:
> >  > > i think the real solution here is allowing masking in a package
> >  >
> >
> > > You want to add a metadata key and cache it you mean?
> >
> >
> > i dont care terribly much about the logistics, just the results.
> > as long as an ebuild can declare itself masked, it sounds good to
> > me.
> >
> >  this doesnt preclude the other ideas as there are often times
> > where you want to have 1 global package mask piece (like large
> > package set bumps ... so X or KDE or GNOME or ...).
> >
> > -mike
> >
> >
> 
> [-gentoo-qa, +gentoo-portage-dev]
> 
> Original thread was splitting up package.mask entries.
> 
> Genone notes the code to do this already is basically in already (we
> just don't invoke it for $PORTDIR/profiles afaik).
> 
> Genone, do we use existing code for package.mask (ie if we switch from
> a file to  a dir will it break existing versions?  I am unsure if we
> used the directory code for $PORTDIR/profiles/*


The handling code for all package.mask files is identical, in
particular they all support recursion (just search for "package.mask"
in portage/__init__.py).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Re: [gentoo-qa] splitting up package.mask

[Marius Mauch]

On Sat, 15 Mar 2008 07:45:19 -0400
Mike Frysinger <[EMAIL PROTECTED]> wrote:

> i dont see it being any different from an ebuild declaring its own
> KEYWORDS. if you want to be lazy, then have a magic KEYWORD: +pmask.

What's wrong with KEYWORDS="" ? (I'm not saying it's the perfect
solution, just looking for specific requirements)

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] please explain use of hooks

[Marius Mauch]

On Sun, 2 Mar 2008 11:18:39 +0100
"Jonas Bernoulli" <[EMAIL PROTECTED]> wrote:

> And if not the handbook this should really go into the portage
> manpage.
> 
> bashrc
> 
> If  needed, this  file can  be  used to  set up  a
> special  environment for  ebuilds,  different from
> the standard root  environment.  The syntax is the
> same as for any other bash script.
> + One thing you can do here is define phase hooks.
> + A phase hook function name begins with a pre_ or post_ prefix to
> indicate that it will be called before or after one of the ebuild
> phases. The prefix is followed by the name of the ebuild function that
> the hook will be associated with. + For example, a hook named
> pre_src_compile will be called before src_compile, and a hook named
> post_src_compile will be called after src_compile.

Care to write a patch? (nobody likes to edit manpages)

> postsync.d should also be documented there.

No, as it isn't used by portage itself.

> All in all now that I have looked at the documentation installed with
> portage I think it is obsolete. Not in the sense that it is outdated
> but everything in it is documented elsewhere already or should be
> moved there.

Well, it's horribly incomplete and a work in progress. Eventually it
will replace the manpages (in the sense that manpages will be generated
from the same source). And authorative documentation (which isn't
exactly the same as a specification) should be kept with the
implementation, and that's not the case with the dev handbook/manual.

> If some developers an the list agree with me and welcome me to do so
> then I could file a bug(s) for those changes.

No.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] state of GPG-signing in portage

[Marius Mauch]

On Sat, 12 Jan 2008 05:49:10 +0100
Marcel Meyer <[EMAIL PROTECTED]> wrote:

> Hi,
> 
> I'm wondering if the GPG-signing feature within portage is already
> useable (if I recall correctly it was startet 2004 or 2005?). If yes,
> how can I use it correctly and where to get the gpg-key "securely"?
> The URLs I found by googling do no longer exist :-(

"signing" (with limited coverage) is usable and used on a wide scale for
years now, but as that only matters to ebuild devs I think you're more
interested in "verification", and that's not usable at all (and before
you ask, there is no ETA about it).

Marius
-- 
gentoo-portage-dev@lists.gentoo.org mailing list

[gentoo-portage-dev] Runtime deps, binary packages and merge order

[Marius Mauch]

Just ran across the following thread in the forums yesterday:
http://forums.gentoo.org/viewtopic-t-626528.html

Which raises an interesting point regarding merge order of runtime
deps. IIRC we currently assume that it's ok to merge runtime deps after
the depending package to resolve dep cycles for example, which is
generally ok, except if a runtime dep is used in pkg_*. For
ebuild-installs that can be worked around easily by using DEPEND (where
order is strictly respected), but for binary packages that obviously
doesn't work. 
This problem probably hasn't been recognized earlier as
it requires several conditions to apply simultaneously (binary merge,
circular rdeps, rdeps used in pkg_*, rdeps not installed
previously)

Assuming I haven't missed anything, I see threee options to deal with
that problem:
a) ignore it, as it only affects a small minority
b) respect merge order for RDEPEND - will cause more unsolvable
depgraphs, though telling people to use PDEPEND more often might reduce
that problem
c) add a new deptype for merge dependencies - looks like overkill to me

Any other other ideas, comments, preferences?

Marius
-- 
[EMAIL PROTECTED] mailing list

[gentoo-portage-dev] New maintainers needed

[Marius Mauch]

For various reasons I can no longer maintain the following packages (in
fact I haven't really maintained them for a while already), so they
need a new maintainer:

* app-editors/zoink - simple gtk based editor, little to no maintenance
required
* dev-util/gambas - Visual Basic look-alike for Linux, fairly active
development (2.0 is about to come out), doesn't work on 64bit archs
(only known to work on x86), has a number of open bugs

Also I'm going to remove myself from the tools-portage team as I
haven't done much there either, which effectively only leaves
Paul (fuzzyray) on it, who is also somewhat inactive recently, so we
could use a few new people there as well.

Finally at the beginning of this year I started to convert packages
from einfo to elog where appropriate, unfortuntely I haven't been able
to complete that job, if anyone is interested in completing it please
let me know (I guess it's currently about 40-60% done).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] use* cleanup

[Marius Mauch]

On Tue, 30 Oct 2007 19:32:52 +0100
"Marijn Schouten (hkBst)" <[EMAIL PROTECTED]> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> The purpose of this patch is to expose a generic function, namely
> _use, which can be used to build your own use* variant if you need
> that. I reimplemented all other current use function using _use (and
> _if) to cut out duplicate and verbose code. Comments expected. I
> didn't test this code.

Please don't post patches inline, there are these things called
attachments that work so much better for such things.

Marius
-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-portage-dev] RFC: replacing "packages"

[Marius Mauch]

On Thu, 25 Oct 2007 22:14:40 -0700
"Alec Warner" <[EMAIL PROTECTED]> wrote:

> > Another issue that isn't directly related, but covered by the
> > proposed solution below, is the so called "implicit system
> > dependency" in ebuilds, which doesn't really exist. It only an
> > assumption by people that the "system" set is satisfied before an
> > ebuild is processed, so its contents don't have to be listed in
> > *DEPEND. If a user breaks that assumption by not regulary ensuring
> > that "system" is satisfied bugs can occur. Another issue is the
> > informal exception when system items are allowed to/must be listed
> > in *DEPEND, IMO that should be formalized.
> 
> I'm going to discuss a bit your implementation below, since you
> address what you are trying to solve here.  My main problem is that
> given a tree (such as gentoo-x86) it may have one or more profiles
> with one or more sub-profiles (children).  These profiles may contain
> a set of packages that make up 'system'.  The ebuilds DEPEND (or
> RDEPEND or whatever) on this set of packages existing.  The problem I
> see with your solution is that it doesn't address different system
> sets in different profiles.  Now in practice this isn't much of a
> problem within gentoo-x86 (most profiles have similar system sets).
> However it places a certain onus on any profile in a given tree, it
> must have a system package 'similar' enough to other sets in
> functioning profiles; otherwise breakage is likely to occur.  How
> would you address this in your system?

How is the problem solved now? Remember that I'm just proposing to
formalize existing assumptions (packages in "system" should state all
their dependencies explicitly). The answer is that there is no real
solution to that "problem", at least not without creating a huge mess by
enumerating all available profiles.
The real question is what "system" is supposed to be. Is it just a more
or less arbitrary list of packages chosen by the profile maintainer, or
does it have a global, predefined purpose, like defining the minimum
requirements for packages in the associated tree? In the first case the
second part of my proposal obviously doesn't work, as you can't make
any valid assumptions about "system". In the second case however the
purpose already mandates that the content of "system" is effectively
identical across different profiles.

Marius
-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-portage-dev] RFC: replacing "packages"

[Marius Mauch]

On Wed, 24 Oct 2007 13:34:44 -0500
Andrew Gaffney <[EMAIL PROTECTED]> wrote:

> For packages that are in the "system" set, wouldn't adding the
> contents of system.{,r}depend to {,R}DEPEND cause problems in dep
> resolution? Would there be a way to prevent the contents of these
> files from being added for packages that are themselves in these
> files?

Did you miss the part about RESTRICT=system-deps?

> How is the "system" set constructed from these files? Would they
> simply be concatenated (minus duplicates) and treated like the old
> "packages" file (minus the magic visibility filtering)?

Maybe, we'd have to see what works best.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

[gentoo-portage-dev] RFC: replacing "packages"

[Marius Mauch]

As package sets are mostly done now, I'm starting to think about
something else. One of my pet peeves with portage is the "packages"
file in profiles, for several reasons:
1) it has two completely independent purposes
2) it implements a redundant visibility filter as package.mask is also
available in profiles
3) the syntax for defining the "system" set plain sucks
4) the name doesn't really say anything about the purpose

Another issue that isn't directly related, but covered by the proposed
solution below, is the so called "implicit system dependency" in
ebuilds, which doesn't really exist. It only an assumption by people
that the "system" set is satisfied before an ebuild is processed, so
its contents don't have to be listed in *DEPEND. If a user breaks that
assumption by not regulary ensuring that "system" is satisfied bugs can
occur. Another issue is the informal exception when system items are
allowed to/must be listed in *DEPEND, IMO that should be formalized.

To solve both issues I propose the following system:
Profiles can contain the files "system.depend" and "system.rdepend"
whose combined contents make up the "system" target. If any of those
two files exists the "packages" file is ignored. The syntax should
either be the same as package.mask or, if desired, also allow complex
atoms (use-conditionals, any-of constructs).
The second part would be that portage implicitly adds the content of
the files to the DEPEND or RDEPEND setting of each ebuild unless it
contains RESTRICT=system-deps. Obviously this would have to be tied to
a new EAPI version. (an undefined cornercase here is if a profile does
not contain the new files, not sure how that should be handled)

Benefits:
- we get rid of all the issues outlined at the top
- should make it easier long-term to setup a no-compile system that only
installs binary packages due to separation of build- and runtime deps
in "system"

Problems:
- obviously it has to be implemented first
- no obvious solution for stacking rules (e.g. child uses system.*, but
parent profile only has "packages")
- some people might rely on the "packages" file

So, comments?

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Re: About "system" and "world"

[Marius Mauch]

On Wed, 24 Oct 2007 00:03:47 +0200
Thomas de Grenier de Latour <[EMAIL PROTECTED]> wrote:

> Bah, you're introducing a new options set, and have (i think) ways to
> emulate all of the old ones with them, so what is stopping you?  You
> would just have to forbid mixes of old style and new style on the same
> command line.

Mainly the work required to implement it ;) And I'd rather finish
dynoparse first so it can be used here.
Oh, and ideally the mentioned values wouldn't be hardcoded lists, but
come from a pluggable resolver system, but that probably counts as
overengineering.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Re: About "system" and "world"

[Marius Mauch]

On Tue, 23 Oct 2007 21:11:48 +0200
Thomas de Grenier de Latour <[EMAIL PROTECTED]> wrote:

> On 2007/10/23, Marius Mauch <[EMAIL PROTECTED]> wrote:
> 
> > On Mon, 22 Oct 2007 21:29:02 -0700
> > Zac Medico <[EMAIL PROTECTED]> wrote:
> > 
> > > Well, you can already use SLOT atoms in your world file if you
> > > don't want the highest available. Packages that pull in >=foo are
> > > a different story though. I suppose we can add something like a
> > > - --upgrade=minimal option that prevents pulling in new slots if
> > > they aren't required.
> > 
> > Don't restrict it to SLOTs though. "minimal" implies that only
> > upgrades required to satisfy the depgraph are performed. 
> 
> Couldn't this "minimal" behavior just be triggered by lack of the
> --upgrade option ("emerge -D ")?

--upgrade != the current --update (at least that's what I assumed)

> Actually, the current --upgrade behavior (with or without --deep)
> is a bit weird imho, i would prefer something like this (with foo 
> being either a set or an atom):

Yes, --update has a history of being a mess of random behavior.
Personally I'd drop it completely, or alias it to --noreplace.

>  * "emerge foo": do the minimum upgrades or new installs required to
> get foo satisfied.  Only its direct deps are checked (and direct
> deps of the unsatisfied ones, etc.), with the "minimal" heuristic
> (upgrade them only if stricly required).
> 
>  * "emerge -u foo": do all required new installs and possible upgrades
> of foo.  Only its direct deps are checked (and direct deps of the
> unsatisfied ones, etc.), with the "minimal" heuristic (upgrade them
> only if strictly needed).
> 
>  * "emerge -D foo": do the minimal upgrades or new installs required
> to get foo satisfied.  Also, check all of its direct and indirect
> deps, with the "minimal" heuristic (upgrade only if strictly needed).
> 
>  * "emerge -uD foo": do all required new installs and possible
> upgrades of foo.  Also, all its direct and indirect deps are checked,
> and upgraded where possible.
 
Maybe --upgrade as name was an unfortunate choice, --resolver might be
better. What I'd like ideally (as a user) is to replace --update,
--deep, --noreplace and --emptytree with the following options:

--resolver={minimal,noslotchange,leastchange,default}
minimal: select the lowest possible version
noslotchange: like default, but try to avoid slotchanges
leastchange: select the version with the smallest version delta
compared to the installed version, otherwise like default
default: current behavior

--rebuild={never,always,changeduse,newuse,changeddeps,selected}
never: never rebuild packages
always: always rebuild packages (like current --emtpytree)
changeduse: rebuild packages if $USE has changed
newuse: rebuild packages if $USE or $IUSE has changed
changeddeps: rebuild packages if any of its direct dependencies were
updated
selected: rebuild the selected arguments (root nodes)

--deplevel=
check n level of dependencies for updates (with -1 == infinite)
of course all dependencies have to be satisfied, this controls only
what deps should be checked for optional updates

current defaults would be --resolver=default --rebuild=selected
--deplevel=0

--update would be --rebuild=never --deplevel=1
--deep would be --deplevel=-1
--emptytree would be --rebuild=always --deplevel=-1

But that's just me dreaming of a better world, where we don't have
to deal with legacies ;)

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] About "system" and "world"

[Marius Mauch]

On Sun, 21 Oct 2007 15:12:31 +0200
Marius Mauch <[EMAIL PROTECTED]> wrote:

> Well, the primary goal is to make all sets behave in a consistent way.
> And some sets have the explicit purpose to rebuild stuff, so making
> sets "selective" by default also has issues.
> The proposed change would also make sets behave in the same way as
> packages which is IMO another benefit.

This actually has more consequences: "selective" is a global setting,
you can't enable it just for some arguments. Therefore if packages and
sets are treated differently it requires that they can't be mixed on
the commandline (and if we'd make the setting configurable for each set
then only one set can be used at any time). And right now I can't think
of another reason why that restriction would be necessary.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Re: About "system" and "world"

[Marius Mauch]

On Mon, 22 Oct 2007 21:29:02 -0700
Zac Medico <[EMAIL PROTECTED]> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Ryan Hill wrote:
> > Zac Medico wrote:
> >> implement greedy atoms for the world set. I've been pondering the
> >> idea of making world non-greedy for slots by default [1], since you
> >> can add slot atoms to the world file to pull in any specific
> >> slot(s) that you want.
> > 
> > That would rock.  Portage always insisting on pulling in the highest
> > SLOT whether needed or not is one of my biggest pet-peeves.
> 
> Well, you can already use SLOT atoms in your world file if you don't
> want the highest available. Packages that pull in >=foo are a
> different story though. I suppose we can add something like a
> - --upgrade=minimal option that prevents pulling in new slots if they
> aren't required.

Don't restrict it to SLOTs though. "minimal" implies that only upgrades
required to satisfy the depgraph are performed. The described
behavior should be another value, e.g. "no-slot-change".

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] localization.py

[Marius Mauch]

On Mon, 22 Oct 2007 04:54:59 +0200
Arfrever Frehtes Taifersar Arahesis <[EMAIL PROTECTED]> wrote:

> 2007-10-21 22:49:10 Marius Mauch napisał(a):
> > On Sun, 21 Oct 2007 12:23:59 -0700
> > Zac Medico <[EMAIL PROTECTED]> wrote:
> > 
> > > -BEGIN PGP SIGNED MESSAGE-
> > > Hash: SHA1
> > > 
> > > Arfrever Frehtes Taifersar Arahesis wrote:
> > > > Hello,
> > > > 
> > > > Does localization.py exist for a reason?
> > > > 
> > > 
> > > Over the years we've had a few people express a desire for
> > > internationalization and localization support in portage (bugs
> > > #13618, #32630, and #191189 and possible others).
> > > 
> > > It seems like a nice idea to me, but I recall Marius being opposed
> > > to the idea due to some potential problems that he was concerned
> > > about. Unfortunately I don't have a link to the previous
> > > discussion but maybe somebody else does.
> > 
> > Probably not the one you meant, but you can find a related
> > discussion on the forums:
> > http://forums.gentoo.org/viewtopic-t-500209.html
> 
> It seems that you just don't like German translation, so you could set
> LC_ALL=C in make.conf.

I don't have an issue with localizations on my systems (I don't
use them anyway), the problem is with other people reporting bugs or
asking for support posting messages we can't understand, the
questionable quality of translations (as we can't really control them)
potentially leading to bogus bug reports and the general added
complexity. It's just my opinion that system tools simply shouldn't be
localized (best example: gcc error messages).

Marius
-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-portage-dev] localization.py

[Marius Mauch]

On Sun, 21 Oct 2007 12:23:59 -0700
Zac Medico <[EMAIL PROTECTED]> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Arfrever Frehtes Taifersar Arahesis wrote:
> > Hello,
> > 
> > Does localization.py exist for a reason?
> > 
> 
> Over the years we've had a few people express a desire for
> internationalization and localization support in portage (bugs
> #13618, #32630, and #191189 and possible others).
> 
> It seems like a nice idea to me, but I recall Marius being opposed
> to the idea due to some potential problems that he was concerned
> about. Unfortunately I don't have a link to the previous discussion
> but maybe somebody else does.

Probably not the one you meant, but you can find a related
discussion on the forums:
http://forums.gentoo.org/viewtopic-t-500209.html
-- 
[EMAIL PROTECTED] mailing list

Re: [gentoo-portage-dev] About "system" and "world"

[Marius Mauch]

On Sun, 21 Oct 2007 05:23:45 -0700
Ned Ludd <[EMAIL PROTECTED]> wrote:

> On Sun, 2007-10-21 at 13:01 +0200, Marius Mauch wrote:
> > So, what do people think about removing (some) of the special
> > treatment for the "system" and "world" targets?
> > Mainly I'm interested in removing the "selective" parameter that's
> > currently enabled for them (so for example without that parameter
> > `emerge world` would default to remerging packages, unless --update
> > or --noreplace are specified). That change has already been
> > requested a few times in the past by users, but OTOH it could
> > probably upset people who don't use --update with world/system.
> 
> What would such a disruptive change really gain us?

The goal is to make package sets behave in a consistent way.

> I personally feel our users need consistency from Gentoo. If they 
> grew up doing 'emerge world' and have come to expect that behavior 
> and all of the sudden we change behavior on them.. Yeah I can see 
> how ppl would get upset.

As do I, which is why I haven't simply changed it.

> Perhaps a less intrusive way would be to introduce another
> flag to get the specified behavior you are after.

Well, the primary goal is to make all sets behave in a consistent way.
And some sets have the explicit purpose to rebuild stuff, so making
sets "selective" by default also has issues.
The proposed change would also make sets behave in the same way as
packages which is IMO another benefit.
But as I said, I can see that it could upset people.

One possible solution that I've thought about would be to remove the
hardcoded "selective" parameter and let the set configuration determine
if a set is selective or not (with missing = "false"), and then enable
it for world and system in our default config, e.g.

[world]
class = portage.sets.files.WorldSet
selective = True

and extend the --rebuild option with new arguments "always" and "never".
Don't really like the additional complexity though (and I haven't
checked how much work it would be).

Marius
-- 
[EMAIL PROTECTED] mailing list

[gentoo-portage-dev] About "system" and "world"

[Marius Mauch]

So, what do people think about removing (some) of the special treatment
for the "system" and "world" targets?
Mainly I'm interested in removing the "selective" parameter that's
currently enabled for them (so for example without that parameter
`emerge world` would default to remerging packages, unless --update or
--noreplace are specified). That change has already been requested a
few times in the past by users, but OTOH it could probably upset people
who don't use --update with world/system.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

[gentoo-portage-dev] Re: [gentoo-dev] [RFC] Properties of package sets

[Marius Mauch]

On Fri, 29 Jun 2007 05:07:28 +0200
Marius Mauch <[EMAIL PROTECTED]> wrote:

> Please reply on gentoo-portage-dev, _not_ on gentoo-dev, thanks.
> 
> One missing feature in portage is the lack of package sets. Before we
> (re)start working on that however I'd like to get some feedback about
> what properties/features people would expect from portage package set
> support.
> Some key questions:
> 
> - should they simply act like aliases for multiple packages? E.g.
> should `emerge -C sets/kde` be equivalent to `emerge -C kdepkg1
> kdepkg2 kdepkg3 ...`? Or does the behavior need to be "smarter" in
> some ways?
> 
> - what kind of atoms should be supported in sets? Simple and versioned
> atoms for sure, but what about complex atoms (use-conditional, any-of,
> blockers)?
> 
> - should sets be supported everywhere, or only in selected use cases?
> (everywhere would include depstrings for example)
> 
> - what use cases are there for package sets? Other than the
> established "system" and "world", and the planned "all" and
> "security" sets.
> 
> - how/where should sets be stored/distributed?

Forgot one question:

- should sets have metadata? (e.g. a description for searching)

There hasn't been much feedback yet, so if you want to add anything now
is your chance, otherwise I'll implement things the way that works
best/is easiest for me, which might be different from what you expect.

Marius

PS: I also accept off-list replies

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] [RFC] Properties of package sets

[Marius Mauch]

On Thu, 28 Jun 2007 21:03:54 -0700
Ned Ludd <[EMAIL PROTECTED]> wrote:

> On Fri, 2007-06-29 at 05:07 +0200, Marius Mauch wrote:
> > Please reply on gentoo-portage-dev, _not_ on gentoo-dev, thanks.
> > 
> > One missing feature in portage is the lack of package sets. Before
> > we (re)start working on that however I'd like to get some feedback
> > about what properties/features people would expect from portage
> > package set support.
> > Some key questions:
> > 
> > - should they simply act like aliases for multiple packages? E.g.
> > should `emerge -C sets/kde` be equivalent to `emerge -C kdepkg1
> > kdepkg2 kdepkg3 ...`? Or does the behavior need to be "smarter" in
> > some ways?
> 
> I like the alias way acting simply as a metapkg.

Well, mind that a very simple alias implementation as outlined above
would for example unmerge kdepkg3 even if it was merged explicitly.

> > - should sets be supported everywhere, or only in selected use
> > cases? (everywhere would include depstrings for example)
> 
> Please NO. emerge.py should know about sets but ebuild.py should be
> oblivious to them. package sets as you are proposing imo should be
> limited to portage only. By putting package sets in ebuild depstrings
> we would be effectively forcing all other pkg managers to support the
> feature. I don't think we should do that.

What about config files (package.*) or other sets?

> > - what use cases are there for package sets? Other than the
> > established "system" and "world", and the planned "all" and
> > "security" sets.
> 
> Assuming you guys run with with the simple alias method I don't see
> how 'security' can fit into this. 

Simple: the list of atoms would be assembled dynamically by checking
the available GLSAs, similar to how the "affected" target works in
glsa-check. But lets keep the backend implementation details out for
now.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

[gentoo-portage-dev] [RFC] Properties of package sets

[Marius Mauch]

Please reply on gentoo-portage-dev, _not_ on gentoo-dev, thanks.

One missing feature in portage is the lack of package sets. Before we
(re)start working on that however I'd like to get some feedback about
what properties/features people would expect from portage package set
support.
Some key questions:

- should they simply act like aliases for multiple packages? E.g.
should `emerge -C sets/kde` be equivalent to `emerge -C kdepkg1 kdepkg2
kdepkg3 ...`? Or does the behavior need to be "smarter" in some ways?

- what kind of atoms should be supported in sets? Simple and versioned
atoms for sure, but what about complex atoms (use-conditional, any-of,
blockers)?

- should sets be supported everywhere, or only in selected use cases?
(everywhere would include depstrings for example)

- what use cases are there for package sets? Other than the established
"system" and "world", and the planned "all" and "security" sets.

- how/where should sets be stored/distributed?

Please reply on gentoo-portage-dev, _not_ on gentoo-dev, thanks.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] [SoC] Idea for emerge

[Marius Mauch]

On Fri, 23 Mar 2007 18:10:13 +0100
Simon Lipp <[EMAIL PROTECTED]> wrote:
> [...]
> $ sudo emerge -vp --interactive-useflags liferea
> [...]
>
> Comments, questions and other are welcome ;)

Long standing policy is that emerge is non-interactive (--ask is just a
convenience feature to avoid duplicate dep resolution), and for things
like this a new tool should be created that can focus on user
interaction.
Besides that, I don't really want to think about the implications of
such a feature on backtracking, esp. regarding issues like bug 1343,
and I'd assume users might get rather confused to answer questions that
are then thrown away later. Another (relatively minor) problem is that
the flags set in such a session would have to be made persistent
somehow, and I wouldn't want emerge to mess with /etc/portage/package.*
files (that's a more generic problem though).
As for listing USE flag descriptions, there are already patches
floating around for that (at least TGL wrote one a while ago), and even
if not it would be hardly worth a complete SoC project by itself.

So, nice try, but no go (IMHO)

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Add a couple new warnings to QA check

[Marius Mauch]

On Mon, 19 Mar 2007 17:24:56 +0200
Petteri Räty <[EMAIL PROTECTED]> wrote:

> Mike Frysinger kirjoitti:
> > On Monday 19 March 2007, Kevin F. Quinn wrote:
> >> To this end it would also be useful if the QA notices were _all_
> >> sent to the elog report; the "Files were installed with user/group
> >> portage" one is, but I don't think any of the others are.
> > 
> > add "qa" to your elog classes and any messages that still arent
> > sent arent properly using the eqawarn func and should be trivial to
> > fix -mike
> 
> Is eqawarn something we can use in eclasses? We have quite a few QA
> checks in the java eclasses that could potentially make use of this
> function. If it's not part of the public API we can of course just if
> it exists and fall back to echo.

Well, it only exists in >=portage-2.1.2 IIRC, so it comes down to how
compatible you want your eclasses to be.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] LC_ALL and friends in make.conf

[Marius Mauch]

On Thu, 8 Mar 2007 20:14:26 +0100
"Kevin F. Quinn" <[EMAIL PROTECTED]> wrote:

> This is an old issue, but I want to suggest a re-visit :)
> 
> As we all know, setting LC_ALL and friends can cause all sorts of
> trouble in package builds.  However, many users really appreciate
> being able to set it so that errors from the compiler etc are in their
> own language.

Actually most people I've talked to prefer to have buildsystem messages
in english even if the rest of their system is localized. Reasons
include:
- many users don't understand the messages anyway, so to them language
is irrelevant
- often the translations are harder to understand than the english
version (IMO)
- when reporting a bug or asking questions on forums/irc/lists/...
they'll usually need the english messages anyway
- when searching for error messages in bugzilla/google/... the english
version will usually get more and/or better results

So going with Mike I'd invert your proposal and have portage (un)set
LC_MESSAGES to get english buildsystem messages, e.g. by adding

export LC_MESSAGES=${PORTAGE_LC_MESSAGES:-en_US}

to ebuild.sh.

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: New preserve-libs feature

[Marius Mauch]

On Fri, 23 Feb 2007 14:22:05 +0100
Carsten Lohrke <[EMAIL PROTECTED]> wrote:

> echo "/path/to/preserved.so" >> /var/lib/portage/preserved_libs
> 
> stores the libraries, and Portage can each time emerge is run look up, if the 
> file lists libraries, check, if those exist, if not remove the lines or 
> otherwise warn the user about the possibly vulnerable libraries and tell him 
> what to do.
> 
> Simple solution at low cost. Fine with this idea?

Yeah, I had planned a registration for those libs anyway to assist
revdep-rebuild, just haven't settled on a format yet (need to track
which package registered the lib).

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] New preserve-libs feature

[Marius Mauch]

On Sat, 17 Feb 2007 14:55:26 +0100
Simon Stelling <[EMAIL PROTECTED]> wrote:

> Marius Mauch wrote:
> > So everyone who has valid objections to the _general idea_ of this
> > implementation (preserving old libraries to avoid some runtime
> > linker errors) speak up now. 
> 
> For how long are these libraries preserved? This might have a security
> impact in cases like the recent openssl-case where you had to upgrade
> to an incompatible ABI because the version using the old one was
> vulnerable. Using preserve-libs it would leave the old lib around,
> making it possible for programs to link against the wrong version and
> ending up being vulnerable. I realize that the feature is meant to
> help the transitional phase until all apps are built against the new
> ABI, but how would you find these vulnerable apps currently?
> revdep-rebuild wouldn't rebuild them since they are still functional.

Currently they are around as long as they are referenced by other
packages or until the package is unmerged. And yes, there should be a
way to tell revdep-rebuild/the user which packages should/need to be
rebuilt, but I haven't made my mind up yet on how to accomplish that
(in fact atm there is no separation between "native" and "imported"
libs in vdb, I'm aware that needs to be added).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] dep resolution weirdness

[Marius Mauch]

On Sat, 17 Feb 2007 13:42:49 +0100
George Shapovalov <[EMAIL PROTECTED]> wrote:

> Ok, found it.
> Thanks Marius! (for the debug hint)
> 
> It was indeed forcing gnat-gcc-4.1.1 by asis-gcc-4.1.1 which has 
> =dev-lang/gnat-4.1.1 (this is an "extension" to compiler and has to
> match it 1 for 1, forgot to bump that one :().
> However, shouldn't --tree have reported the relevant package as
> requiring this downgrade instead of some other, pretty much arbitrary
> (from dependants of gnat-gcc)? 

In an ideal world: yes, but portage is far from being ideal (I won't
pretend to know the internals of the dep resolver, so can't give a good
explanation for this behavior).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

[gentoo-portage-dev] New preserve-libs feature

[Marius Mauch]

If you haven't noticed, I just added a new 'preserve-libs' feature for
bug 62207 that moves shared object that are still used but would be
removed on an update into the new package instance (so on a update from
expat-1 to expat-2 the user would still have libexpat.so.0, owned by
expat-2). The actual match criteria are a bit stricter, but you get the
idea. I think this is an long overdue bugfix, but given past
discussions not everyone has the same opinion, though the last
discussion about it ended without a conclusion (at least none that I
could see).
So everyone who has valid objections to the _general idea_ of this
implementation (preserving old libraries to avoid some runtime linker
errors) speak up now. 

Just keep the following things in mind:
- I'm not claiming that it's a silver bullet to all possible runtime
linker problems, it's supposed to cover some of the common cases (like
the expat problem)
- I'm not claiming that the implementation is perfect yet
- This feature is currently optional, so I'm not forcing it down on
anyone (in fact it's completely hidden for now). Of course if people
start using it ebuild devs will have to deal with it sooner or later,
but lets discuss it here first.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] dep resolution weirdness

[Marius Mauch]

On Sat, 17 Feb 2007 12:30:31 +0100
George Shapovalov <[EMAIL PROTECTED]> wrote:

> Hi guys.
> 
> I am quite confused by the following:
> 
> aldar ~ # emerge -puD world --tree
> These are the packages that would be merged, in reverse order:
> Calculating world dependencies... done!
> [nomerge  ] dev-ada/cbind-6.0
> [nomerge  ]  virtual/gnat-4.1
> [ebuild UD]   dev-lang/gnat-gcc-4.1.1 [4.1.2]
> 
> aldar ~ # emerge -puD cbind --tree
> These are the packages that would be merged, in reverse order:
> Calculating dependencies... done!
> 
> (or, for that matter:
> aldar gnat # e -puD virtual/gnat
> These are the packages that would be merged, in order:
> Calculating dependencies... done!
>
> The relevant dependency lines are here:
> dev-ada/cbind:   DEPEND="virtual/gnat"
> 
> virtual/gnat-4.1: 
> RDEPEND="=dev-lang/gnat-gcc-4.1*"
> DEPEND=""
> 
> (and virtual/gnat-3.4:
> RDEPEND="|| ( =dev-lang/gnat-gcc-3.4*
> =dev-lang/gnat-gpl-3.4* )"
> DEPEND=""
> although it looks like this one is not considered in this particular
> case)
> 
> As you see emerge -uD world wants to downgrade gnat for some reason
> (which is wrong), while emerge -uD cbind (or any other Ada library
> that has similar dependencies) does not (which is right).

Is there maybe a different package in world that depends on
<=gnat-gcc-4.1.1? Anyway, you may want to look at --debug output to get
a clue why it wants to downgrade.

> Oh, and ~dev-lang/gnat-gcc-4.1 as RDEPEND in virtual/gnat does not
> work at all (just, as I understand, it should not)).

It should work, but it's not the same as the dep you stated above:
it would match gnat-gcc-4.1-r1, gnat-gcc-4.1-r2 and so on, but not
gnat-gcc-4.1.1, gnat-gcc-4.1.2, ...

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

[gentoo-portage-dev] Storing origin repository in VDB

[Marius Mauch]

There are several cases where the information from which repository
(portdir, overlays, ...) a package originally came from when it was
installed. Currently that information isn't really available, at most
you can use some heuristics on environment.bz2 or comparing ebuilds
directly, but those methods aren't very reliable (may return both false
positives and false negatives). So storing the origin repo seems like a
good idea, qustion is how:

1) Save (part of) the value of $EBUILD
2) Save the repository name
3) ???

Option 1) has the benefit that it would be useful from the start, but
the drawback that if the path of a repository changes the information
becomes invalid. Option 2) avoids that but would require that all
repositories are named to be useful. Don't know if there is another
option that avoids both problems.
If I'd had to choose I'd probably go for option 2) and encourage people
to name their repositories, but I'm open for suggestions to use option
1) or for a third option.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] [RFC] Depending on "active" version

[Marius Mauch]

On Wed, 31 Jan 2007 17:47:26 +
Stephen Bennett <[EMAIL PROTECTED]> wrote:

> On Tue, 30 Jan 2007 17:06:51 +0100
> Marius Mauch <[EMAIL PROTECTED]> wrote:
> 
> > The idea is to add a special category (let's call it "active" for
> > now) that has the following properties:
> > - this category doesn't exist in portdir or vdb (= no ebuilds)
> > - when portage ($pkgmanager) encounters a "active/foo" atom in a
> > dependency string it executes some special code (e.g.
> > "$PORTDIR/scripts/active-check/foo =active/foo-1") to determine if
> > that atom is satisfied
> 
> Given that in the general case the package manager can't change the
> active provider and will have to bail with an appropriate message that
> the user needs to change it themselves, the obvious solution to this is
> the previously-discussed-somewhere-I-can't-remember ebuild function,
> called on depgraph creation, to check that it will be able to compile
> in the current system environment. It's considerably simpler and more
> generally useful than subverting DEPEND to add weird special-case hacks
> to it.

Yeah, thinking about it I have to agree tat generic dep_check() support (to 
give the thing a name) would be a better solution here.

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] [RFC] Depending on "active" version

[Marius Mauch]

On Tue, 30 Jan 2007 18:04:41 +0200
Petteri Räty <[EMAIL PROTECTED]> wrote:

> Marius Mauch wrote:
> > Sometimes a package has to depend on a specific version of a slotted 
> > package being the "active" one to build correctly, like in the current 
> > "tr1" discussion on -dev [1] or with packages that depend on the running 
> > kernel.
> > 
> > Currently this isn't really possible, however I while ago I got an idea how 
> > to solve this. Keep in mind this is just a rough idea and I'm pretty sure 
> > some people can/will point out why it is a stupid idea, but anyway:
> > 
> > The idea is to add a special category (let's call it "active" for now) that 
> > has the following properties:
> > - this category doesn't exist in portdir or vdb (= no ebuilds)
> > - when portage ($pkgmanager) encounters a "active/foo" atom in a dependency 
> > string it executes some special code (e.g. 
> > "$PORTDIR/scripts/active-check/foo =active/foo-1") to determine if that 
> > atom is satisfied
> > 
> > (and yes, this kinda goes with multi-repo/multi-format support)
> > 
> > Marius
> 
> To work this would need code in the tools like gcc-config. Otherwise
> people could easily break their systems without anything giving them a
> notice.

Not sure I can follow. If we had such a check script for gcc why would it need 
additional code in gcc-config?

Marius

-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] [RFC] Depending on "active" version

[Marius Mauch]

On Tue, 30 Jan 2007 13:01:47 -0500
Alec Warner <[EMAIL PROTECTED]> wrote:

> Marius Mauch wrote:
> > Sometimes a package has to depend on a specific version of a slotted 
> > package being the "active" one to build correctly, like in the current 
> > "tr1" discussion on -dev [1] or with packages that depend on the running 
> > kernel.
> > 
> > Currently this isn't really possible, however I while ago I got an idea how 
> > to solve this. Keep in mind this is just a rough idea and I'm pretty sure 
> > some people can/will point out why it is a stupid idea, but anyway:
> > 
> > The idea is to add a special category (let's call it "active" for now) that 
> > has the following properties:
> > - this category doesn't exist in portdir or vdb (= no ebuilds)
> > - when portage ($pkgmanager) encounters a "active/foo" atom in a dependency 
> > string it executes some special code (e.g. 
> > "$PORTDIR/scripts/active-check/foo =active/foo-1") to determine if that 
> > atom is satisfied
> > 
> > (and yes, this kinda goes with multi-repo/multi-format support)
> > 
> > Marius
> 
> I don't see why how this is any less complicated than just adding more
> functionality to || deps (runtime versus compile time)

Please explain. If a package can only be built if the currently running kernel 
is higher than 2.6.19 for example, how would you possibly express that in 
*DEPEND syntax?

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] [RFC] Depending on "active" version

[Marius Mauch]

On Tue, 30 Jan 2007 08:25:31 -0800
Brian Harring <[EMAIL PROTECTED]> wrote:

> On Tue, Jan 30, 2007 at 05:06:51PM +0100, Marius Mauch wrote:
> > Sometimes a package has to depend on a specific version of a 
> > slotted package being the "active" one to build correctly, like in 
> > the current "tr1" discussion on -dev [1] or with packages that 
> > depend on the running kernel.
> 
> tr1 is partially addressed via addition of a 'binding' modifier for 
> rdeps, to state that ||() deps are locked down after compilation.

And how would that solve the actual issue of expressing "I need /usr/bin/gcc to 
run gcc-4.1 and not gcc-3.4"?
The lockdown of || deps is a completely separate issue, unless I'm missing 
something.

> > The idea is to add a special category (let's call it "active" for 
> > now) that has the following properties:
> > - this category doesn't exist in portdir or vdb (= no ebuilds)
> > - when portage ($pkgmanager) encounters a "active/foo" atom in a 
> > dependency string it executes some special code (e.g. 
> > "$PORTDIR/scripts/active-check/foo =active/foo-1") to determine if 
> > that atom is satisfied
> 
> Non deterministic resolution; previous steps in the graph can cause 
> that value to flip to a different setting by the time the 'dep' is 
> encountered.

Ok, that's a problem, though for the use cases at hand (gcc and kernel) it 
would be mostly irrelevant.

> That's ignoring the kick in the nads usage of this will due to 
> resolution...

Neglectable IMO, it's not such a common use case anyway, and I don't think I 
have to compare it to the current "solution" (die in setup or compile).

> > (and yes, this kinda goes with multi-repo/multi-format support)
> 
> Don't really see how this enables multiple standalone repos in any 
> sane way, so that one requires justification...

Where did I say anything about enabling? It would need more or less a separate 
repository (dbapi) instance, so it would require such support.

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] [RFC] Depending on "active" version

[Marius Mauch]

Sometimes a package has to depend on a specific version of a slotted package 
being the "active" one to build correctly, like in the current "tr1" discussion 
on -dev [1] or with packages that depend on the running kernel.

Currently this isn't really possible, however I while ago I got an idea how to 
solve this. Keep in mind this is just a rough idea and I'm pretty sure some 
people can/will point out why it is a stupid idea, but anyway:

The idea is to add a special category (let's call it "active" for now) that has 
the following properties:
- this category doesn't exist in portdir or vdb (= no ebuilds)
- when portage ($pkgmanager) encounters a "active/foo" atom in a dependency 
string it executes some special code (e.g. "$PORTDIR/scripts/active-check/foo 
=active/foo-1") to determine if that atom is satisfied

(and yes, this kinda goes with multi-repo/multi-format support)

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] [RFC] Mask packages that don't cross-compile

[Marius Mauch]

On Wed, 22 Nov 2006 03:10:45 -0500 (EST)
Daniel Barkalow <[EMAIL PROTECTED]> wrote:

> There are packages (such as perl) which work fine on both x86 and arm, 
> but don't build on x86 cross-compiling for arm. Furthermore, pam-0.78 can 
> be cross-compiled (with a patch available in bug comments), but pam-0.99 
> will require more work to get to cross-compile. It would be useful to be 
> able to have 0.99 use a keyword (or something) such that it is excluded 
> from cross-compiles and these builds get 0.78 instead.
> 
> I've got a working patch that extends the KEYWORDS semantics slightly, 
> such that, if KEYWORDS includes -foo, and ACCEPT_KEYWORDS includes foo but 
> not -foo, the package is masked, even if KEYWORDS also includes bar and 
> ACCEPT_KEYWORDS also includes bar.
> 
> Then you can have the ACCEPT_KEYWORDS value "$ARCH cross" and packages 
> with KEYWORDS including "-cross" will be masked, unless you have "-cross" 
> in packages.keywords.
> 
> This probably horribly mangles the concept of KEYWORDS, but I can't really 
> tell from reading the documentation that it doesn't work this way. (Of 
> course, I know it doesn't from trying things and reading the source, 
> but...) It doesn't specify whether "foo" or "-bar" on a package takes 
> precedence if "foo" and "bar" both apply to the system. (Of course, it 
> does say that keywords are always ARCHes, which would imply that only one 
> could apply.) I think this all works as expected, however, except that a 
> package having "cross" means that it is known to automatically work if 
> you're cross-compiling, rather than simply meaning that it is known 
> to work if you're cross-compiling and it would work if you were building 
> natively on your target.

No. -foo is reserved for incremental negation. Maybe that isn't widely used in 
ACCEPT_KEYWORDS, but it has valid uses there and there can't be repurposed. Is 
there a particular reason why you don't use the normal keyword logic 
(KEYWORDS=cross and setting -cross in p.keywords to mask a package), or simply 
a package.mask entry?

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] [PATCH] --config-root command-line option

[Marius Mauch]

On Thu, 16 Nov 2006 15:43:48 -0800
Zac Medico <[EMAIL PROTECTED]> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Marius Mauch wrote:
> > On Thu, 16 Nov 2006 09:33:05 -0800
> > Zac Medico <[EMAIL PROTECTED]> wrote:
> > 
> >> Sure, that's one way to solve that particular problem.  However, I feel
> >> that command line options are more aesthetically appealing than environment
> >> variables (maybe it's just me).
> >>
> >> Zac
> > 
> > It's just you ;)
> > 
> > Marius
> 
> Do you honestly prefer environment variables over command line options?  I
> prefer command line options because they are part of a clearly defined
> interface.  With environment variables, the interface may not be clearly
> defined and it's easy for variables to accidentally leak in, affecting
> internals in unexpected ways.
> 
> Zac

I prefer to have one way to do things, and the system for variables worked 
quite well so far. Adding CLI overrides on top of that seems not only redundant 
to me but also potentially confusing (which vars have CLI overrides? where in 
the incremental stack does the override fit in?). Add that to the already 
overloaded option system.
Besides, if you need this feature on a somewhat regular base it seems easier to 
me to just export the var once instead of specifying the option on the command 
line all the time (ok, using DEFAULT_OPTS avoids that, but then you have the 
same issues you listed above).
And last but not least a CLI option is bound to emerge, but this feature can 
also be useful for other tools. Checking an env var in the config class would 
enable it implicitly for all users of portage.py, without it everyone would 
have to basically duplicate the relevant code from emerge.
I'm not generally against using CLI options for passing in arguments, but in 
this case I think it's wrong.

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] [PATCH] --config-root command-line option

[Marius Mauch]

On Thu, 16 Nov 2006 09:33:05 -0800
Zac Medico <[EMAIL PROTECTED]> wrote:

> Sure, that's one way to solve that particular problem.  However, I feel
> that command line options are more aesthetically appealing than environment
> variables (maybe it's just me).
> 
> Zac

It's just you ;)

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] GPG support for mod_mail

[Marius Mauch]

On Sat, 28 Oct 2006 15:42:50 +0300
Ali Polatel <[EMAIL PROTECTED]> wrote:

> Hi everyone,
>I wonder if GPG support for mod_mail would be useful.I wrote some
> basic code that works.The user is required to export his public key 
> and put the path to PORTAGE_ELOG_MAILGPG.
>Attached is the patch for mod_mail.py.It won't work with armored
> key files because gpg can't directly encrypt messages with ascii
> armored keys , they need to be --dearmor'ed first.
>Thoughts?

Well, I don't really see a reason why one would want to encrypt those
mails. The only potentially sensitive information (package
name/version) is already contained in the subject and therefore
wouldn't be encrypted. And gpg support can be a bitch to maintain, so
I'd like to see the benefit before adding that feature.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
-- 
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] RFC: new virtual metadata variable to list combined deps

[Marius Mauch]

Ok, I have to admit the subject may be a bit confusing, so let me get
the motivation first:
Often people want to know the dependencies of a package. Not much of a
problem when they know how to use auxget or the portageq metadata
commands. However these all work on the specific DEPEND, RDEPEND and
PDEPEND vars directly, so to get all dependencies of a package one has
to read all three variables. And as in general some entries exist in
both DEPEND and RDEPEND one gets even more info than wanted. At least
this me this manual parsing gets annoying over time.
So in my auxget script[1] I added support for a new virtual metadata
variable '*DEPEND' that basically reads and concatenates the three dep
variables, then eliminates duplicates (of course only on the top level,
complex atoms are treated as a single unit). This allows a quick check
what dependencies a package has ignoring if it's buildtime, runtime or
postbuild (or however you want translate the P in PDEPEND).

So now I was wondering a) if I'm the only one who finds this
feature useful and b) if adding it at the dbapi level (in dbapi.aux_get)
would be considered a good idea, so it could be used by other tools?

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Max parallelization setting

[Marius Mauch]

On Tue, 10 Oct 2006 18:16:16 -0700
Brian Harring <[EMAIL PROTECTED]> wrote:

> Say I've got a 32x system; I screw up and have -j32 in my makeopts, 
> and PARALLELIZATION=32.

So you screwed up ... You can do that already ;)

> Now either the pkg manager needs to understand MAKEOPTS (beyond just 
> adding a simple string to it), and check for -j*, or it's going to 
> wind up allowing 32 seperate build jobs, each with -j32.

Is PARALLELIZATIOn the big global var or the low level var for the number of 
parallel merges? I didn't mean to merge those two.

> Personally, I *really* would love to see that loadavg (that would be 
> one nifty screenshot).

Depends on the value of -l* ;)

> Allowing MAKEOPTS to override the alloted build slices  the manager 
> hands it totally defeats the purpose of moving the parallelization 
> factor into a seperate var; the intention is to give the manager a 
> max, and allow it to allocate as it needs depending on the # of build 
> jobs that can be ran at that point. 

You misunderstood: I meant that one could either define the parallelization on 
the super-high level and hope that the pkgmanager does the best with it or 
define the different settings manually. It's up to the user which one he wants.

> Response of course is "don't have -j* in your MAKEOPTS"; well dar, 
> thus why allow it? ;)

No, reponse is "use -l* in MAKEOPTS" ;)

Just in case your wondering about the reasons why I want to keep the 
alternative of low-level vars:
a) not all build systems are equal (efficiency, stability, flexibility), so one 
might want to have different settings for them
b) generally only src_compile (the actual build) is cpu-bound, the other phases 
are generally IO-bound so it makes sense to have separate vars for them in some 
cases
c) debugging/analyzing/testing

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Max parallelization setting

[Marius Mauch]

On Tue, 10 Oct 2006 00:04:57 -0700
Brian Harring <[EMAIL PROTECTED]> wrote:

> I might be daft (likely), but why not just introduce a var indicating 
> max parallelization instead?  Tweak portage to push that setting into 
> MAKEOPTS="${MAKEOPTS+${MAKEOPTS} } -j${PARALLELIZATION}".
> 
> Might sound daft, but -j is hardcoded parallelization; if you're 
> trying to run 3 build processes, the original var of -j isn't all
> that useful, nor will the hardcoded -j# var set for 3 package build 
> processes be useful if you're doing a single build.
> 
> Depending on number of seperate package build processes possible, the 
> # of build processes allocated per build process needs to vary 
> (essentially).

Seems useful as *alternative* to the low level vars, but shouldn't
replace them (so if the low level vars are set they override the global
setting). So the order in your MAKEOPTS assignment above would have to
be reversed (assuming sequential option parsing).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] globally disable use expand

[Marius Mauch]

On Sat, 30 Sep 2006 23:25:55 +0200
Michiel de Bruijne <[EMAIL PROTECTED]> wrote:

> Hi guys,
> 
> If I'm not mistaken, behaviour in portage changed recently. USE="-*"
> in make.conf disabled use expand as well. Now use expand stays
> enabled.

USE="-*" never disabled USE_EXPAND by default (such behavior would
have led to interesting problems).

> What is the proper way to disable it globally? I couldn't
> find anything in the documentation. Thanks!

If your really want to do that you can set USE_EXPAND="-*" in make.conf.

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] Project overlay

[Marius Mauch]

We now have one of these new fancy project overlays on
http://overlays.gentoo.org/proj/portage
Currently only I have access and the only available package there is
gentoo-stats, if anyone else (blubb?) wants to put something there
please notify me.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] [PATCH] use.force and package.use.force (bug #142853)

[Marius Mauch]

On Mon, 7 Aug 2006 14:14:11 -0700
Brian Harring <[EMAIL PROTECTED]> wrote:

> On Mon, Aug 07, 2006 at 01:13:34PM -0700, Zac Medico wrote:
> > Brian Harring wrote:
> > > Semantics of USE=-gtk not working on a package that has gtk
> > > forced doesn't sound all that nice btw;
> > 
> > Which is why the flag shouldn't be forced unless it's almost 
> > certain that the flag shouldn't be disabled.  The gtk flag might
> > not ever fall into that category, but something like cxx might
> > (nocxx inverted).
> 
> Flag shouldn't be forced, period imo.

What about masked flags, should those exist? Because use.force and
use.mask are pretty much the same: accounting for configurations
(=profiles) where support for a flag is not available/not optional
(IIRC one example was USE=hardened on hardened profiles, solar may
correct me here).
So if you want to deny one, you should deny both.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Atom matching behavior

[Marius Mauch]

On Mon, 31 Jul 2006 15:48:23 -0700
Brian Harring <[EMAIL PROTECTED]> wrote:

> > but I'd suspect that many
> > people share my original assumption and expect it to only match full
> > version components
> 
> Hear a bit of screaming from it once every 4-6 months; personally, I 
> interpret that as devs know which to use usually- additionally, once 
> the (bluntly) hissy fit from the dev subsides, and they're reminded 
> "yes it's annoying, but if you want it changed take it to dev to get 
> consensus" folks promptly forget about it.
> 
> Either they're silently working around it, or it's not that much of
> an issue (I suspect the latter, but am neutral towards the change).

Well, practically it's not a big issue as in 99% of all cases it works
like people expect.

> Should be discussed on -dev, not here imo; they're the ones affected 
> by it (it's essentially their standard after all).  Changing it?  
> Sure, but it's a required eapi bump; portage chokes on .* now.
> 
> I'd also not bump eapi just for one change; there is a boatload of 
> other stuff that's waiting for an apt time to be pushed out together.

Sure going to sned this to -dev, just usually going for input from a
smaller audience first.
Haven't yet thought about details in case of changing it, first gonna
decide what the "correct" behavior is.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

[gentoo-portage-dev] Atom matching behavior

[Marius Mauch]

Was just brought to my attention that the =* operator doesn't work as I
thought, as for example =foo-1.2* matches foo-1.20 as well as foo-1.2.3.
This wouldn't be a bug problem if it could be used as a general glob
operator like with =foo-1.2.*, but it's use is strictly limited to the
above version (can only be used when a version component separator may
appear), so atm there is no facility to reliably lock an atom at a
specific version component when you have to account for multi-digit
components.
Now the question is if we want this glob-style behavior or not. From
the code comments it seems to be intentional, but I'd suspect that many
people share my original assumption and expect it to only match full
version components (as that is the much more common use case). Doesn't
help that the atom description in ebuild(5) doesn't specify the
behavior for this case either, 

"*  means  match  any version of the package so long as the specified
base is matched"

can be read both ways.

Opinions?

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Re: Portage phase hooks patch

[Marius Mauch]

On Mon, 24 Jul 2006 16:41:26 -0400
Mike Kelly <[EMAIL PROTECTED]> wrote:

> On Mon, 24 Jul 2006 21:22:20 +0200
> Marius Mauch <[EMAIL PROTECTED]> wrote:
> 
> > > The problem with having these in the tree is that they should be
> > > able to take advantage of the unique features of each package
> > > manager. Otherwise, they really are just eclasses.
> > 
> > "unique features" such as? What exact benefits does this hooks
> > system give you over using an eclass?
> 
> The biggest one is the ability to clean up after myself if the emerge
> fails (if the fail hooks idea I mentioned in my previous mail makes
> sense, at least).
> 
> Basically, what I need to do is:
> 
>  - Run a script to intelligently add / manage package manager-added
>users. At the moment, it would seem this should just be run before
>or during the pkg_setup phase, but this may well be sub-optimal,
>particularly in the case if binary packages or non-standard ROOTs.

can be done within an eclass.

>  - Properly let that set of scripts know when an added user or group
> is safe to remove. This code is intimately tied with the
>implementation-specific details of the above scripts, which may
>change from version to version, hence my trepidation about adding
>this code directly into portage itself.

can be done with an eclass too. Also that eclass can depend on your
scripts or whatever package it needs to do this, seems better than to
add dependencies in every package using this functionality.
Also should be able to have a generic interface for this.

>  - Have the ability to clean up from myself if a build goes awry, or
>is aborted by the operator. As far as I know, this particular
>requirement would exclude the use of eclasses, but I could well be
>wrong.

Don't see how the proposed hook system would allow that either without
additional mods in abort_handler.

>  - Have this user addition handled in an intelligent way when we're
>only building/installing a binary package. I'm not really sure on
>what would be a logical way to do this.

Yeah, that one is tricky.

> So, if the answer to the above is to not use some sort of hooks, then
> I would appreciate pointers on where to look to achieve all of the
> above with portage.

Well, I'm not saying to not use hooks, just trying to find the (IMHO)
best solution to the problem (and being a bit resistant against an
generic unprotected hook system).
From my POV this project should work like you provide a service (your
management scripts) and portage/paludis/whatever use this service by
calling the scripts. This would mean you wouldn't really have to care
about the glue code but just provide an interface, and the pkgmanager
has to conform to that interface.
Right now it sounds like you want to work on the glue code, might be
better to define the interface first.

Btw, you haven't really answered my question, what benefits do you/we
get by having implementation specific hooks instead of using an
eclass/repo level hook system (that "unique features" bit)?

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Re: Portage phase hooks patch

[Marius Mauch]

On Mon, 24 Jul 2006 13:52:23 -0400
Mike Kelly <[EMAIL PROTECTED]> wrote:

> On Sat, 22 Jul 2006 22:39:28 -0700
> Brian Harring <[EMAIL PROTECTED]> wrote:
> > > I still think that those hooks should go into the tree rather than
> > > being installed by packages.
> >
> > They're called eclasses ;)

Good argument ;)

> The problem with having these in the tree is that they should be able
> to take advantage of the unique features of each package manager.
> Otherwise, they really are just eclasses.

"unique features" such as? What exact benefits does this hooks system
give you over using an eclass?

> > It also has the drawback that via transferring control over to the 
> > tree, the pkg manager cannot do proper cleanup.
> > 
> > What you say?
> > 
> > What happens if $PKGMANGER adds user blah for building xyz, xyz
> > fails? Proper course of action would be to punt this unused user if
> > it's refcount is just xyz, no?
> >   
> > Transfer control of fundamental features like that to delegation 
> > hooks, you either have to add more hooks so the delegation target
> > can cleanup after itself, or you plain don't do cleanup.

If you were replying to me I don't see how this is affected by the
location of the hooks but the general concept. You have the same problem
independent where they would live.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] [RFC] Naming Conventions

[Marius Mauch]

On Sat, 22 Jul 2006 16:25:01 -0700
Zac Medico <[EMAIL PROTECTED]> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Chris White wrote:
> > 1) Create aliases to the new functions, then at some
> > yet-to-be-determined point, kill the aliases and bomb on the scripts
> > (this suffers from procrastination).
> > 
> > 2) Make an official release with the new function names and no
> > aliases, as well as the soon to come docs.  I sort of like this
> > method because those with official portage tools can adjust their
> > scripts, and simply alter the depend atoms for >= (new API
> > versions) and <= (old versions), effectively forcing/preventing
> > upgrades.
> 
> I vote for #1 because it's smoother and easier (which is good for me
> especially because I do releases).  The disruptive change proposed in
> #2 seems like it would cause unnecessary problems with no practical
> advantage over #1.  

combination of both.

Phase one: change function names and add aliases
Phase two: make the aliases generate warnings
Phase three: drop the aliases

Maybe combine phases one and two.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Portage phase hooks patch

[Marius Mauch]

On Sat, 22 Jul 2006 16:52:38 -0700
Zac Medico <[EMAIL PROTECTED]> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
> 
> Mike Kelly wrote:
>
> > In my case, I feel this functionality would be very useful as it
> > allows for me to integrate my GLEP 27 implementation into portage
> > without portage needing to worry about my implementation specifics,
> > which may well change in later versions. I am sure there are other
> > practical ways in which these hooks could be used.
> > 
> > Patch at:
> > http://soc.gentoo.org/viewcvs.py/*checkout*/glep0027/patches/portage-hooks.patch
> > 
> > See bug# 141215.
> > 
> 
> I like the idea.  My main concern is that, like /etc/portage/bashrc,
> this creates lots of potential for foreign code to interfere with the
> internal workings of the ebuild environment.  At a minimum, I think
> there should be something in the `emerge --info` output that
> indicates whether or not any phase hooks exist.
> 
> Traditionally, configurable files that affect portage behavior go
> mostly in /etc/portage.  I see that your intention is to
> make /var/libexec/portage/hooks/ a location for third-party packages
> to install hooks, so it makes sense to keep those separate from hooks
> that the user might install.  I know that portage-utils currently
> installs a post-sync hook in /etc/portage/postsync.d/q-reinitialize,
> so there is some inconsistency there.  We need to develop a
> consistent policy for appropriate locations of files like these.

I still think that those hooks should go into the tree rather than
being installed by packages. That's mainly so they get increased
visibility and gives us (us being Gentoo, not just Portage) more
access and control over them, like we have with eclasses.

This also moves responsibility from hook authors to pkgmanager authors
(the package mamanger has to support the hook format in the tree, not
the hook has to support (all of) the specific package manager hook
formats). I know Mike has taken care of Paludis support already, but
look at a larger scale:
a) n hook authors supporting m package manager interfaces: O(n*m)
b) n hook authors and m package managers supoprting one repository
interface: O(n+m)
Also with a) you have to play what I call the "catch up game", e.g. a
new package manager gets out and all hooks have to account for a new
interface (even if it's just a new path).
Of course a) has the "drawback" that it requires a solid spec of the
interface, but that's something we want anyway.

However, *if* they are going to be installed by packages they should go
either into /etc/portage/foobar or /usr/lib/portage/foobar (like any
3rd party elog or cache modules), foobar being something we'd have to
decide on.
The suggested /var/libexec/portage has several issues:
- /var/libexec is AFAIK not defined by FHS
- executable data doesn't belong in /var unless it's application data
which isn't the case here
- libexec generally sucks (even in /usr), especially as we already
use /usr/lib for what generally goes there (support scripts)

Independent on the location I'd like to see the ability to blacklist
individual hooks for testing, troubleshooting, debugging and probably a
few other reasons. And no, this should not be done with $FEATURES ;)

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] disable checking sums of downloaded sources

[Marius Mauch]

[EMAIL PROTECTED] schrieb:

Hi,

Is it any way to disable checking digests after download sources via
`ebuild program.ebuild fetch`?

Any INTERNAL flag or VARIABLE ?


No.

Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] 'emerge' request: make --tree not imply --pretend

[Marius Mauch]

On Sat, 24 Jun 2006 18:02:19 -0700
"Dan Corson" <[EMAIL PROTECTED]> wrote:

> I am sorry, you are indeed correct.  What I should have requested is:
> a way to have the package list output in non-interactive,
> package-installing (that is, not --pretend) execution.  It may be
> that --pretend is a little over-loaded currently.  (I was confused,
> at any rate, as you can see.) Thanks, --Dan

I think this is stuff for a local wrapper script and not yet another
emerge option that is almost identical to --ask in this case (from a
user POV).
Also don't see much use for this feature, the overhead for calling
emerge twice isn't that big, especially when talking about automated
stuff (lie your case seems to be).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] Manifest verification

[Marius Mauch]

On Sun, 18 Jun 2006 04:43:28 -0700 (PDT)
Andrei Slavoiu <[EMAIL PROTECTED]> wrote:

> Hello, I want to raise a issue with the way that
> package integrity verification is done by portage. For
> an example of such an issue see bug 136742.
> The idea is that portage now checks the Manifest as a
> whole, not allowing a version of the package to be
> emerged if any of the files mentioned by the Manifest
> fail the verification. This makes unstable ebuilds
> wich change often and have a tendency to break from
> time to time prevent the emerge of a stable version of
> the same package.
> Do you think there is any way to fix this in portage?

This isn't exactly new behavior with FEATURES=strict (which is the
default setting).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] GWN -portage-2.1 announcement.

[Marius Mauch]

On Mon, 12 Jun 2006 18:20:16 -0700
Brian <[EMAIL PROTECTED]> wrote:

> I also just did a man portage and the man page did not appear to be
> updated for this change.

I guess quite a few docs aren't up to date in 2.1. Anyone wanna work on
that?

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] GWN -portage-2.1 announcement.

[Marius Mauch]

On Mon, 12 Jun 2006 17:18:44 -0700
Brian <[EMAIL PROTECTED]> wrote:

> On Mon, 2006-12-06 at 18:58 -0500, Andrew Gaffney wrote:
> > Brian wrote:
> > > Did I understand it wrong or did they get this mixed up in the GWN
> > > announcement:
> > > 
> > > /etc/portage/package.unmask/kde, /etc/portage/package.unmask/xorg
> > > 
> > > Shouldn't it be:
> > > 
> > > /etc/portage/kde/package.*, /etc/portage/xorg/package.*
> > 
> > You understood it wrong. /etc/portage/package.unmask/foo is correct.
> > 
> > -- 
> > Andrew Gaffney  
> 
> Thanks.  I'm glad now I didn't have the time to add that to porthole
> yet.  Might have been a quick fix anyway though.

Haven't looked at porthole code, but if you use portage_util.grab* the
fix is trivial (just add a recursive=1 parameter).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] GWN -portage-2.1 announcement.

[Marius Mauch]

On Mon, 12 Jun 2006 16:47:19 -0700
Brian <[EMAIL PROTECTED]> wrote:

> Did I understand it wrong or did they get this mixed up in the GWN
> announcement:
> 
> /etc/portage/package.unmask/kde, /etc/portage/package.unmask/xorg
> 
> Shouldn't it be:
> 
> /etc/portage/kde/package.*, /etc/portage/xorg/package.*

Nope.
The code only looks at /etc/portage/package.foo, and if it's a
directory it recusively adds all the files in there.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] feature suggestions

[Marius Mauch]

On Fri, 05 May 2006 11:05:47 +0200
Hanno Fietz <[EMAIL PROTECTED]> wrote:

> Here's a suggestion for two portage features that I would appreciate.
> Please tell me if this is the wrong list.
> 
> I would like to have a system where ebuilds are classified such that
> when updating a package, I can tell whether this is a security patch
> or critical bugfix or new feature release or whatever. Thus, I could
> set up automatic updates that check only for critical patches and do
> not install every tiny new fancy thing.

Short version: needs modular dep resolver first (or some new fancy
type of visiblity filter), no chance otherwise. And even then I don't
think it's possible to note these things in ebuilds as the ebuild
doesn't know the upgrade path.

> Also, a system would be nice, where authors of a package or the ebuild
> maintainers could supply a message to the user along with the ebuild,
> for example to warn them of massive numbers of reverse dependencies
> that are going to be broken by the update (-> libexpat! I would have
> chosen a different time for the update if I had known what I had to
> update after that).

elog and/or glep 42.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] default postsync

[Marius Mauch]

Zac Medico schrieb:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Ned Ludd wrote:

How do you think we should handle it?
Should we install a post_sync in a postinst phase outside of portage's 
handling if and only if not post_sync already exists?

Should we change it to handled a postsync.d by default?
Should we do both? I'm open as heck but would like to start to 
finalize then document it's behavior. I feel it could be one of the 
next untapped really useful features of portage. glsa-checking, news 
handling, search db updating, and stuff etc..



Given the existing support for /etc/portage/bin/post_sync, the user has the 
freedom to do anything they want.  If a program needs to make use of the 
post_sync trigger, it's documentation can simply state that a certain command 
needs the be run and the user can add that command to their post_sync script.  
Is that not easy and flexible enough?


Well, it requires user intervention, pretty much the same argument as 
for any other foo.d change in any package (I don't have a personal 
preference here, though using bashrc for this definitely isn't a good idea).


Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Perl, sort, and locale

[Marius Mauch]

[EMAIL PROTECTED] schrieb:

I doubt this is the right place to bring this up, but maybe some one
can tell me where to go :-)


Yeah, this is definitely the wrong place, not really sure where the 
appropriate place for this is though. Probably a perl or coreutils 
related list/forum might help you.



I have discovered an inconsistency in how perl and sort handle locale.
Here are two commands you can run in a shell ...

(echo '/'; echo '?') | sort

(echo '/'; echo '?') | perl -e '@x = <>; print $_ foreach sort @x'

With LANG=en_US.UTF-8, sort says ? comes before /, perl says the
opposite.  Setting LC_COLLATE=C switches the sort behavior.  I have no
idea what else changes or where else perl and sort disagree, or what
other programs do.

Seems like something is wrong, but I don't know what.  This happens on
other versions of Linux.  It is not a gentoo specific problem.


--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Stablizing portage 2.1

[Marius Mauch]

Alec Warner schrieb:

Why Branch at 2.1_pre9?
Manifest2 is already in the tree and needs refinement.  Branching at
pre7 is also a canidate, but i would rather press for keeping manifest2
in the tree and fixing up it's code instead.


Why not pre10?


TimeLine: If all goes well, we can do an rc sometime this week:
May 3rd : RC1
May 6th : RC2
May 9th : RC3
May 12th : RC4
May 15th : RC5
May 18th : RC6
May 21st : RC7
May 24th : RC8
May 27th : RC9 ( if needed )
May 30th : RC10 ( if needed )
June 5th : ~arch sys-apps/portage-2.1
July 6th : sys-apps/portage-2.1


What's the point of planning a dozen rc versions ahead of time? Make a 
rc1, see how it goes and release another rc version only if necessary 
(for bugfixes). Also I absolutely *hate* date based roadmaps, just make 
it "stable 2.1 when no new rc has been made for n weeks".



Problems:  We may miss the timeline and thats ok.  Releng wants a
working portage, not a bugging POS portage-2.1 that wasn't ready for
release.  This timeline is relatively tight and I think it's a nice goal
to set, it's not imperative that we reach it.

Comments, Questions, opinions?


Date based roadmaps suck. They add a lot of pressure for no benefit 
(other than a rough target date, and for that you don't need a roadmap).


Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] glsa implemented as a special set

[Marius Mauch]

On Mon, 24 Apr 2006 07:04:13 -0700
Brian <[EMAIL PROTECTED]> wrote:

> On Mon, 2006-24-04 at 14:20 +0200, Marius Mauch wrote:
> > On Sun, 23 Apr 2006 08:55:58 -0700
> > Brian <[EMAIL PROTECTED]> wrote:
> > 
> > > I was thinking that /etc/portage/sets/glsa could be a symlink to
> > > set list in the current metadata/glsa directory of the portage
> > > tree.  That file should be relatively easy to auto-generate from
> > > the existing glsa*.xml files there already.  Perhaps a
> > > FEATURES="GLSA_SET" would generate that file on completion of an
> > > "emerge --sync"  I could also then put a GLSA field into
> > > porthole's package Summary view as well as a GLSA notebook
> > > page(s) to display the appropriate glsa?.xml file(s).
> > 
> > Too complicated. First you currently need gentoolkit for glsa.py,
> > and portage shouldn't depend on gentoolkit.
> 
> I did not mean portage should and I din't want to depend on gentoolkit
> either.

Not sure I understand your idea then, I was under the impression that
`FEATURE=GLSA_SET emerge --sync` would create that file, is that not
what you meant?

> >  Also you can't store
> > system-specific files in the tree.
> 
> Yeah, that was a bit of a thought evolution while I was typing. I
> realized after I hit "send".  At first I thought it could be included
> in the sync.  Then thought it's only a duplication of the data already
> there, so why not generate it (save bandwidth), since the data will
> only change at sync time, just do it once then.

Ehm, you couldn't include it in the sync as it's system specific. It
would have to be generated locally, or you have to treat it special
again (only update packages that are installed, don't install new
packages).

> >  And finally using an intermediate
> > file creates some additional issues (check for IO/FS problems,
> > checking permissions, etc).
> > Any reason you need a real file for this instead of just generating
> > the list on the fly?
> 
> I thought a smaller stripped down glsa.py module could generate the
> file at completion of the sync.  Then no special code is needed
> internal in porthole beyond checking for set inclusion, atom
> matching, just a glsa_flag=True to ignore members that are not
> already installed.
> 
> Once portage was able to handle sets, it would almost automatically be
> able to handle a glsa set as well.  The only difference is not
> installing a set member that is not already installed.  

*Shrug*, generating the list dynamically shouldn't take more than 10 or
20 lines using glsa.py, basically it's

pkg_list = []
glsa_list = [Glsa(x) for x in get_glsa_list(glsadir, glsaconfig)]
for x in glsa_list:
if x.isVulnerable():
pkg_list.extend(x.getMergelist())

plus some error handling. Add some dep_getkey() calls if you don't want
the glsa resolver logic of minimal intrusion (might be problematic
though).
If you need a file interface wrap the list in a StringIO instance.
It's really better to keep interdependencies to a minimum here, and
when portage gets set support it will generate the glsa update list
dynamically anyway, so portage wouldn't benefit from a file at all.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] glsa implemented as a special set

[Marius Mauch]

On Sun, 23 Apr 2006 08:55:58 -0700
Brian <[EMAIL PROTECTED]> wrote:

> I was thinking that /etc/portage/sets/glsa could be a symlink to set
> list in the current metadata/glsa directory of the portage tree.  That
> file should be relatively easy to auto-generate from the existing
> glsa*.xml files there already.  Perhaps a FEATURES="GLSA_SET" would
> generate that file on completion of an "emerge --sync"  I could also
> then put a GLSA field into porthole's package Summary view as well as
> a GLSA notebook page(s) to display the appropriate glsa?.xml file(s).

Too complicated. First you currently need gentoolkit for glsa.py, and
portage shouldn't depend on gentoolkit. Also you can't store
system-specific files in the tree. And finally using an intermediate
file creates some additional issues (check for IO/FS problems, checking
permissions, etc).
Any reason you need a real file for this instead of just generating the
list on the fly? Just check glsa-check how to do that, it's not much
more than a wrapper for glsa.py.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] 2.1 release candidate soon?

[Marius Mauch]

On Thu, 06 Apr 2006 19:11:49 -0700
Zac Medico <[EMAIL PROTECTED]> wrote:

> The manifest code doesn't have very many use cases so I'd expect that
> we would have hit most major problems by now (even with a small
> sample).  Any necessary changes are likely to be small patches.  As
> an alternative, we can cut the 2.1 branch at the point before
> manifest2 was merged (2.1_pre7, essentially).

Releasing 2.1 without manifest2 is a no go, it would significantly
delay the deployment and transition. I'm not requesting to delay 2.1
for another few months, just one more pre release so people get a
chance to test it for one or two weeks.

> > The remaining feature I'd like to get into 2.1 is the
> > tree-format-check issue, but that could probably be slipped in in
> > the rc phase (don't really like that idea, but it's an option).
> 
> I don't want to rush the development of new features such as
> manifest2 or the tree-format-check.  We have a 2.1 branch that, in
> it's current state (2.1_pre7-r4, for example), provides significant
> benefits over the 2.0.x branch.  By delaying 2.1's release for the
> addition of _new_ features, we run the risk of the release being
> delayed indefinitely by "just one more feature" syndrome.
> Personally, I'd rather have shorter release periods so that "just one
> more feature" syndrome becomes less of an issue.

Ehm, this is not "just one more feature", both manifest2 and
the tree-format-check are things to improve forward compability (or for
the latter even enable forward compability at all), so delaying them
will hinder future development, not only for us.
Also this isn't exactly news to you all as I sent my intentions already
a while ago, and last I asked you all agreed with them, so is there any
reason to rush this now?

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] 2.1 release candidate soon?

[Marius Mauch]

On Thu, 06 Apr 2006 13:13:34 -0700
Zac Medico <[EMAIL PROTECTED]> wrote:

> So, I'd like to create 2.1 branch that is closed to mostly anything
> except regression fixes and release it as portage-2.1_rc1 this
> Saturday.  We can continue to do ~arch releases of the development
> branch (2.2 or whatever it becomes) every 2 weeks.  Does now seem
> like a good time for this transition?  Your feedback would be
> appreciated.

I'd like to hold off with the rc status until the manifest2 code got
some public testing first in another pre release. I don't feel
comfortable labeling 2.1 as "basically release ready" when only a dozen
people have tested something that critical.
The remaining feature I'd like to get into 2.1 is the tree-format-check
issue, but that could probably be slipped in in the rc phase (don't
really like that idea, but it's an option).

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Repoman Help Extraction

[Marius Mauch]

On Mon, 03 Apr 2006 21:13:16 -0400
Alec Warner <[EMAIL PROTECTED]> wrote:

> Two patches attached, one basically shows the repoman help,
> modes/options, and QA related text removed; relevant references
> replaced by repohelp. or repohelp..keys().
> 
> Second path shows basically that stuff being placed in a second file,
> with function calls replacing global code.
> 
> IIRC I think the qahelp() function call is missing from repoman in
> these patches, but I'm too lazy atm to modify them ;)
> 
> Mostly looking at yay or nay for the idea ;)

Please not another foohelp.py file. Better spend the time on a real
improvement that incorporates manpage/help generation from a common
source (talking about the warning help messages here, not necessarily
the option descriptions)

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] tree dependency check

[Marius Mauch]

On Thu, 30 Mar 2006 08:30:17 +0900
Jason Stubbs <[EMAIL PROTECTED]> wrote:

> On Thursday 30 March 2006 01:21, Marius Mauch wrote:
> > Marius Mauch schrieb:
> > > So after manifest2 is in, I'll revive the other issue that IMO is
> > > a requirement for 2.1: enforcing dependencies needed to use the
> > > tree (see old threads or glep44 for reasoning).
> 
> Can you summarise the reasoning again please?

a) avoid massive breakage when certain new features are introduced
(past examples being cascading profiles or new-style virtuals)
b) similar to a) allow people to use new features without having to
wait for a year or two

> > - at sync time first check the tree format of the remote tree, if
> > we can't handle it the sync is aborted
> 
> While I haven't checked over the code at all, the point above above
> sounds problematic. What happens if the a user installs the latest
> with this patch and then doesn't sync for three years? The tree may
> then require a later portage but the user can't sync to be able to
> emerge a later portage.

!!! This upgrade might fail, in that case please read the
!!! following documents for help:
!!! http://www.gentoo.org/proj/en/portage/doc/common-problems.xml
!!!
http://www.gentoo.org/proj/en/portage/doc/manually-fixing-portage.xml

These docs will then tell the user about the following options
a) use the secret override
b) manually update the tree with wget+tar
c) perform an manual update (as in wget portage-version.tar.bz2, ...)

That should be an extreme exception though, there will be some rules
regarding format bumps (like "only use formats where that haven been
supported by stable portage for n months").

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] tree dependency check

[Marius Mauch]

On Wed, 29 Mar 2006 14:42:25 -0800
"Brian Harring" <[EMAIL PROTECTED]> wrote:

> 2) code isn't root aware.

If root == $ROOT, then I don't see how it would affect this. Even with
$ROOT != / we're using the tools in / for most things, $ROOT is just
for deployment.
If root means something else here please clarify.

> 3) no snapshot integration.

Wanted to wait for feedback first before adjusting all tools.

> 4) code invalidly assumes that all later version of the pulled atom
> from vdb will work.

That assumption is only a fallback if not atom mapping for a format can 
be found. You have a better fallback strategy?

> 6) This breaks _all_ syncing for users who have overlays but lack the
> format versions file.  That's a massive no go, you don't break
> compatibility introducing compatibility checks (nor do you piss off
> several thousand overlay using users for questionable gain).

Reread the code, missing format_version raises a warning, nothing else
(for now).

> 7) even cooler, say you're running max visible portage, and using an
> overlay that lacks a format_version file.  With the vdb portage
> lookup, it'll tell the user that they need a version later then the
> max version.  Nice way to get people to test package masked portage's,
> but it still is wrong.

see above.

> 8) (minor) output of todo is going to be fugly if anyone uses actual
> atom constructs, boolean ORs fex (the print implicitly assumes it's
> just a list of atoms without any boolean constructs).

Yeah, if anyone want to take a shot to convert it to feed the list into
the depresolver feel free.

> 9) the attempted check to see if a pkg is in the passed in myfiles
> won't work if myfiles holds atoms; eg,
> myfiles, pkg = [">=sys-apps/portage-2.0.54"], "sys-apps/portage"
> assert pkg not in myfiles

reread the code, we're only comparing the keys (still not perfect, but
covers your case).

> 2) It's overengineered.  There is _no_ reason to hit up a webserver
> just to get atoms; that data can be bundled in the tree in a seperate
> file.  As is, this breaks users who sync without a connection without
> any gain.  Realistically, I'd be surprised if any alt package managers
> go this route (I know I won't be hitting a webserver up for pkgcore).

a) it's a fallback if the system and tree don't have a depmap
b) it doesn't break
c) it doesn't matter what other package managers will do

> 3) What the portage project thinks a repo tree needs does not map to
> what my tree may need.  Clarifying, format 1 specifies portage xyz and
> bash-3 (ebuilds in the tree use bash regex).  My personal tree needs
> portage xyz (manifest/layout changes), but requires just bash-2.  With
> the central db approach, portage will assume my tree is valid via the
> version #, and if the number differs, it'll assume that I require
> bash3 when in reality, my tree is bash-2 and up.  This points to why
> the format -> depends mapping should be bundled with the tree.

See above.

> 4) portage-2.0 portage-2.1 are bad names for tree formats.  Back to
> the EAPI int discussion, but I really don't see any gain in string
> names, regardless the 2.0/2.1 sucks as a descriptive name (subjective
> I realize) :)

Well, in the end you need a format->feature map, and these names simply
have that mapping implicit in a way.

> Requiring folks to jam a file with a random string in it in
> each of their overlays also sucks, ways to make that less sucky would
> be advisable.

s/Requiring/Recommending/
Ideally we could detect if an overlay is local or from a third party,
and only perform checks for the latter, but that's not doable atm.
Zac: The error-on-missing was your idea, feel free to defend it.

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] tree dependency check

[Marius Mauch]

On Wed, 29 Mar 2006 14:42:25 -0800
"Brian Harring" <[EMAIL PROTECTED]> wrote:

> 2) code isn't root aware.

If root == $ROOT, then I don't see how it would affect this. Even with
$ROOT != / we're using the tools in / for most things, $ROOT is just
for deployment.
If root means something else here please clarify.

> 3) no snapshot integration.

Wanted to wait for feedback first before adjusting all tools.

> 4) code invalidly assumes that all later version of the pulled atom
> from vdb will work.

That assumption is only a fallback if not atom mapping for a format can 
be found. You have a better fallback strategy?

> 6) This breaks _all_ syncing for users who have overlays but lack the
> format versions file.  That's a massive no go, you don't break
> compatibility introducing compatibility checks (nor do you piss off
> several thousand overlay using users for questionable gain).

Reread the code, missing format_version raises a warning, nothing else
(for now).

> 7) even cooler, say you're running max visible portage, and using an
> overlay that lacks a format_version file.  With the vdb portage
> lookup, it'll tell the user that they need a version later then the
> max version.  Nice way to get people to test package masked portage's,
> but it still is wrong.

see above.

> 8) (minor) output of todo is going to be fugly if anyone uses actual
> atom constructs, boolean ORs fex (the print implicitly assumes it's
> just a list of atoms without any boolean constructs).

Yeah, if anyone want to take a shot to convert it to feed the list into
the depresolver feel free.

> 9) the attempted check to see if a pkg is in the passed in myfiles
> won't work if myfiles holds atoms; eg,
> myfiles, pkg = [">=sys-apps/portage-2.0.54"], "sys-apps/portage"
> assert pkg not in myfiles

reread the code, we're only comparing the keys (still not perfect, but
covers your case).

> 2) It's overengineered.  There is _no_ reason to hit up a webserver
> just to get atoms; that data can be bundled in the tree in a seperate
> file.  As is, this breaks users who sync without a connection without
> any gain.  Realistically, I'd be surprised if any alt package managers
> go this route (I know I won't be hitting a webserver up for pkgcore).

a) it's a fallback if the system and tree don't have a depmap
b) it doesn't break
c) it doesn't matter what other package managers will do

> 3) What the portage project thinks a repo tree needs does not map to
> what my tree may need.  Clarifying, format 1 specifies portage xyz and
> bash-3 (ebuilds in the tree use bash regex).  My personal tree needs
> portage xyz (manifest/layout changes), but requires just bash-2.  With
> the central db approach, portage will assume my tree is valid via the
> version #, and if the number differs, it'll assume that I require
> bash3 when in reality, my tree is bash-2 and up.  This points to why
> the format -> depends mapping should be bundled with the tree.

See above.

> 4) portage-2.0 portage-2.1 are bad names for tree formats.  Back to
> the EAPI int discussion, but I really don't see any gain in string
> names, regardless the 2.0/2.1 sucks as a descriptive name (subjective
> I realize) :)

Well, in the end you need a format->feature map, and these names simply
have that mapping implicit in a way.

> Requiring folks to jam a file with a random string in it in
> each of their overlays also sucks, ways to make that less sucky would
> be advisable.

s/Requiring/Recommending/
Ideally we could detect if an overlay is local or from a third party,
and only perform checks for the latter, but that's not doable atm.
Zac: The error-on-missing was your idea, feel free to defend it.

Marius
-- 
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] tree dependency check

[Marius Mauch]

Marius Mauch schrieb:
So after manifest2 is in, I'll revive the other issue that IMO is a 
requirement for 2.1: enforcing dependencies needed to use the tree (see 
old threads or glep44 for reasoning). A patch for that is available at 
dev.gentoo.org/~genone/patches/treedeps.diff. Unless somebody objects 
I'll add that somewhere next week.


Ok, from a discussion with Zac and a few others it seems that we should 
add a layer of indirection, so instead of specifying atoms the tree has 
a freeform version identifier, and portage can use it to look up the 
required atoms by using a (remote) mapping file.
This new approach is implemented in 
dev.gentoo.org/~genone/pacthes/format-check.diff (it's basically a 
combination of strategies D and E in d.g.o/~genone/docs/treedeps.txt)


This new patch also has a few other improvements:
- checks overlays too
- at sync time first check the tree format of the remote tree, if we 
can't handle it the sync is aborted
- if no format specification is found warn the user about it (with the 
intention to fail in a future version)


New intended merge date is this weekend or beginning of next week, so if 
you have any objections, make them now.


Marius
--
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] tree dependency check

[Marius Mauch]

So after manifest2 is in, I'll revive the other issue that IMO is a 
requirement for 2.1: enforcing dependencies needed to use the tree (see 
old threads or glep44 for reasoning). A patch for that is available at 
dev.gentoo.org/~genone/patches/treedeps.diff. Unless somebody objects 
I'll add that somewhere next week.


Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Re: User created package lists

[Marius Mauch]

On Wed, 22 Mar 2006 23:23:25 -0800
Brian <[EMAIL PROTECTED]> wrote:


Will user created lists be located in the /etc/portage directory along
with the other user configs?  If so will the format be similar to the
package.* files?  For user package lists I imagine there could be the
need to control version ranges, so the standard atoms should somehow
apply.


Assuming this is directed at me:
Set support will be more generic than just static package lists (think
`emerge system`, `emerge security` or `emerge category`), they will be
defined by python modules feeding the root nodes into the dep resolver.

Static package lists would just be one such module, and you
could have multiple different modules for different formats, so don't
worry too much about being compatible to portage (I haven't thought
much about syntax yet, though there will definitely be a
"world"-module), if your implementation won't match the "default" list
module provided by portage it will be easy to provide your own module
(assuming your format can be translated to simple atoms).

Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] User created package lists

[Marius Mauch]

Brian schrieb:

Is there any plans for officially supported user lists?  eg:


Yeah, set support is planned whenever the dep resolver is split out and 
emerge is more or less rewritten, the current codebase is just too nasty 
to add it (IMHO).


Marius
--
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] make.conf.* madness

[Marius Mauch]

Don't think I have to reiterate the whole story of having arch specific
make.conf files and why that sucks, but just had an idea how to solve
the problem other than removing them completely:
Just move the arch specific crap (CHOST and CFLAGS) to a separate file,
and reintegrate it at install time. So make.conf.x86 in svn would only
contain CHOST/CFLAGS and related comments while make.conf (which
currently is only a fallback) contains the portage relevant parts, and
the ebuild just concatenates them in src_compile to create a single
arch specific make.conf.example.
First I also considerd using the "source" instruction instead of
reintegrating the files, but that doesn't really work for an example
file that's mainly intended to be read by the user.

Of course removing the arch specific files completely and only have
the fallback make.conf is still an option, but would need a discussion
with releng and the docs team probably.

Any objections against doing this for the next releases?

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] [PATCH] Manifest2 reloaded

[Marius Mauch]

Marius Mauch schrieb:

The first should be delayed until there is some consensus how the gpg
stuff should work in the future, the others I don't see the use for.
Also I only checked portage.py for changes, so emerge/repoman/... might
still have to be fixed.
Last but not least: I did some basic testing with this and the
important stuff seems to work, but I'm quite sure the code still has a
lot of bugs/issues, and this being a core functionality it needs a
*lot* of testing, so I'd really appreciate if you could all give it a
spin (but do not commit anything to the tree without manually checking
it first).


Does the lack of feedback (only got a reaction from Brian so far) mean 
that noone tried it or that it doesn't have any issues?


Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Hello! && portage UI

[Marius Mauch]

Brian wrote:

On Tue, 2006-14-03 at 13:14 +0200, tvali wrote:


Ok, i was, yes, speaking about kde.

I will check out this Porthole :) I was actually thinking more about c
++, but nothing against python -- i was quite a fan of python when i
first found it.



I believe Kuroo is in C, maybe c++


For the record: Kuroo is known to be conceptually broken (uses the cache 
directly), but the author is looking to fix that in a future version.
Generally writing portage utils in languages other than python is 
difficult as you can't use the primary API directly.


Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Few things, which imho would make portage better

[Marius Mauch]

tvali wrote:

I did think about some priorities too, so that it could be perfect for me.

It should be possible to add package with a priority. I will give you an 
use case and explanation how i would use portage.


Heh, make the dep resolver even more complex ;)
Also don't really see a need for such a feature, pretty much no benefit 
with a lot of additional complexity.


Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Few things, which imho would make portage better

[Marius Mauch]

tvali wrote:
I think that it would make things more clear and simple if portage did 
support the following (maybe some are already included, but not found by 
me):


* Add package to "world" checking all dependencies, but not emerging


Not sure what exactly that is supposed to mean, but I guess it's already 
doable with --noreplace and --onlydeps.



* Remove package from "world" without unmerging


$EDITOR /var/lib/portage/world
or
sed -i -e '/\//d' /var/lib/portage/world

Maybe worth integrating when generic set support is available.

* Save all important messages into file instead of beeping (i may be 
away from home); optionally replay them at end of update


see http://forums.gentoo.org/viewtopic-t-131795.html

* When updating or installing new package, if something goes wrong, 
still emerge other packages, which havent failed packages as 
dependencies -- in my case, for example, when i do --update, for 
example, then the fact that update of first package fails does not 
definitely mean that i dont want to update others


This has been rejected in the past as it isn't always a good idea as you 
might want to change your plans if a package fails. Maybe as an optional 
feature, but even then it's rather low priority.


Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Few things, which imho would make portage better

[Marius Mauch]

tvali wrote:

Ok, i send a lot of them, but hopefully they're interesting :)

I did research a bit about adding SQL support to portage -- as much as
i see, mysql is smallest sql server, which could be emerged with
python module.

In beginning, i think that SQL database structure should be created,
which supports basic fields from ebuilds (DESCRIPTION, HOMEPAGE,
SRC_URI, LICENSE, SLOT, KEYWORDS, IUSE, DEPEND, RDEPEND).

Then emerge --createsql command should be set up, which adds data from
/usr/portage/ filetree into this sql database.


You're talking about the cache, take a look at the cache subsystem and 
write a mysql module for it. This will never become a default though (we 
would get killed if portage starts to depend on mysql).



I have some questions now:
* Where i could find basic ebuild specification?


Developer handbook, ebuild(5), bash(1)


* Where i could find basic portage tree datastructure specification,
if there is any?


Not really.


* Does portage have some code documentation for faster learning?
(or should i just read the code?)


code documentation ... what's that? ;)

Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Enabling RSYNC_OPTIONS to be set in make.conf

[Marius Mauch]

Ned Ludd wrote:

On Fri, 2006-03-10 at 14:38 -0600, Robert Larson wrote:


Hello list,

I am needing to allow for further configuration of rsync for syncing portage.  
The addition of RSYNC_OPTIONS in make.conf could alleviate this need for me, 
but what do you guys think?


[snip]

I think genone has a more complete patch for this already. If he has not
committed it already I'm sure he will soon.


First I have to "convince" my server to not lock up anymore every few 
hours :(


Marius
--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] ExtractKernelVersion

[Marius Mauch]

On Tue, 07 Mar 2006 11:16:24 -0500
Alec Warner <[EMAIL PROTECTED]> wrote:

> The kernel ( along with QA ) has reported that the portage generated
> $KV is sometimes quite horribly wrong.  They would like us to retire
> the kernel generated $KV, ebuilds that need $KV should inherit
> linux-info, unless they are themselves a kernel-module, in which case
> they should inherit linux-mod.
> 
> This also consolidates all the kernel version parsing nastiness into 1
> place ( the eclass ) which means less work for us!
> 
> The QA team will run testing against the tree to locate incorrect uses
> of $KV, granted the KV value is probably wrong right now for some
> users, we still need to migrate them over.
> 
> We may need to retain the $KV variable for EAPI 0 portage, but I think
> that could actually depend on how bad the migration really is.

Hmm, I thought we removed that crap long ago already...
Let it die I say.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.


signature.asc
Description: PGP signature

Re: [gentoo-portage-dev] [rfc] variable naming for marking binaries as QA ignorable

[Marius Mauch]

On Sun, 5 Mar 2006 20:46:25 -0500
Mike Frysinger <[EMAIL PROTECTED]> wrote:

> On Sunday 05 March 2006 19:48, Kevin F. Quinn (Gentoo) wrote:
> > Ned Ludd <[EMAIL PROTECTED]> wrote:
> > > On Fri, 2006-03-03 at 23:32 -0500, Mike Frysinger wrote:
> > > > so we've found some cases where a package installs objects that
> > > > either need to be ignored by some of the scanelf checks ...
> > > >
> > > > ...
> > > >
> > > > what this e-mail is about is naming convention ... i'm thinking
> > > > that an ebuild sets up a variable with a list of relative paths
> > > > to $D of files that should be skipped for various checks ... so
> > > > with slmodem, we'd have like: QA_EXEC_STACK="usr/sbin/slmodemd
> > > > usr/sbin/slmodem_test"
> > > >
> > > > if, in the future, we need to add an ignore list for TEXTRELs,
> > > > we'd use QA_TEXTRELS=""
> > >
> > > This becomes tricky when looking at tests across all CHOSTs.
> > > What holds true for one arch defiantly is not the case for others.
> >
> > This could be done via the profiles, perhaps - package.qa, something
> > like package.mask/use/keywords:
> 
> i hate such things ... imo this information should stay in the ebuild
> and nowhere else ...
> 
> be trivial to expand the support like:
> QA_TEXTRELS="..."   # for all arches
> QA_TEXTRELS_arch="..."   # for just one arch
> 
> so in the case of slmodem:
> QA_EXEC_STACK="usr/sbin/slmodemd"
> in the case of some other package that only has issues on x86:
> QA_EXEC_STACK_x86="some/foo"
> 
> this thread was about the naming convention :P
> does QA_EXEC_STACK and QA_TEXTRELS work for people ?

Personally I'd call it QA_IGNORE_*, but that's just me.

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
-- 
gentoo-portage-dev@gentoo.org mailing list

[gentoo-portage-dev] [PATCH] Manifest2 reloaded

[Marius Mauch]

So while on my way to FOSDEM I decided to do something useful with the
time and wrote a new manifest2 implementation. This has nothing to do
with the original prototype I posted a while ago, it's been written
completely from scratch.
Basically all functionality (creation, parsing, validation) is
encapsulated in the new portage_manifest.Manifest class, including
compability code to read/write old style digests.
The changes to portage.py only change the digest*() functions to use
this new class instead of handling the task themselves (exception:
digestCheckFiles() which apparently was only used internally by other
digest* functions), they should more or less behave like with the old
code. Any new code however should use the Manifest() class directly
however.
While this patch implements the basic functionality some extra stuff
that was in the old code isn't included yet:
- gpg verification
- FEATURES=autoaddcvs
- FEATURES=cvs (probably obsolete anyway)
- emerge --digest / FEATURES=digest (may or may not work)

The first should be delayed until there is some consensus how the gpg
stuff should work in the future, the others I don't see the use for.
Also I only checked portage.py for changes, so emerge/repoman/... might
still have to be fixed.
Last but not least: I did some basic testing with this and the
important stuff seems to work, but I'm quite sure the code still has a
lot of bugs/issues, and this being a core functionality it needs a
*lot* of testing, so I'd really appreciate if you could all give it a
spin (but do not commit anything to the tree without manually checking
it first).

Marius

-- 
Public Key at http://www.genone.de/info/gpg-key.pub

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
diff -ru --exclude=CVS --exclude=.svn -N pym/portage.py.org pym/portage.py
--- pym/portage.py.org	2006-03-04 02:25:20.957635000 +
+++ pym/portage.py	2006-03-04 03:12:19.545785750 +
@@ -90,6 +90,7 @@
 
 	from portage_data import ostype, lchown, userland, secpass, uid, wheelgid, \
 	 portage_uid, portage_gid
+	from portage_manifest import Manifest
 
 	import portage_util
 	from portage_util import atomic_ofstream, dump_traceback, getconfig, grabdict, \
@@ -2049,181 +2050,67 @@
 			return 0
 	return 1
 
-
-def digestCreate(myfiles,basedir,oldDigest={}):
-	"""Takes a list of files and the directory they are in and returns the
-	dict of dict[filename][CHECKSUM_KEY] = hash
-	returns None on error."""
-	mydigests={}
-	for x in myfiles:
-		print "<<<",x
-		myfile=os.path.normpath(basedir+"///"+x)
-		if os.path.exists(myfile):
-			if not os.access(myfile, os.R_OK):
-print "!!! Given file does not appear to be readable. Does it exist?"
-print "!!! File:",myfile
-return None
-			mydigests[x] = portage_checksum.perform_multiple_checksums(myfile, hashes=portage_const.MANIFEST1_HASH_FUNCTIONS)
-			mysize   = os.stat(myfile)[stat.ST_SIZE]
-		else:
-			if x in oldDigest:
-# DeepCopy because we might not have a unique reference.
-mydigests[x] = copy.deepcopy(oldDigest[x])
-mysize   = copy.deepcopy(oldDigest[x]["size"])
-			else:
-print "!!! We have a source URI, but no file..."
-print "!!! File:",myfile
-return None
-
-		if mydigests[x].has_key("size") and (mydigests[x]["size"] != mysize):
-			raise portage_exception.DigestException, "Size mismatch during checksums"
-		mydigests[x]["size"] = copy.deepcopy(mysize)
-	return mydigests
-
-def digestCreateLines(filelist, mydict):
-	mylines = []
-	mydigests = copy.deepcopy(mydict)
-	for myarchive in filelist:
-		mysize = mydigests[myarchive]["size"]
-		if len(mydigests[myarchive]) == 0:
-			raise portage_exception.DigestException, "No generate digest for '%(file)s'" % {"file":myarchive}
-		for sumName in mydigests[myarchive].keys():
-			if sumName not in portage_checksum.get_valid_checksum_keys():
-continue
-			mysum = mydigests[myarchive][sumName]
-
-			myline  = sumName[:]
-			myline += " "+mysum
-			myline += " "+myarchive
-			myline += " "+str(mysize)
-			mylines.append(myline)
-	return mylines
-
-def digestgen(myarchives,mysettings,overwrite=1,manifestonly=0):
+def digestgen(myarchives,mysettings,db=None,overwrite=1,manifestonly=0):
 	"""generates digest file if missing.  Assumes all files are available.	If
-	overwrite=0, the digest will only be created if it doesn't already exist."""
-
-	# archive files
-	basedir=mysettings["DISTDIR"]+"/"
-	digestfn=mysettings["FILESDIR"]+"/digest-"+mysettings["PF"]
-
-	# portage files -- p(ortagefiles)basedir
-	pbasedir=mysettings["O"]+"/"
-	manifestfn=pbasedir+"Manifest"
-
-	if not manifestonly:
-		if not os.path.isdir(mysettings["FILESDIR"]):
-			os.makedirs(mysettings["FILESDIR"])
-		mycvstree=cvstree.getentries(pbasedir, recursive=1)
-
-		if ("cvs" in features) and os.path.exists(pbasedir+"/CVS"):
-			if not cvstree.isadded(mycvstree,"files"):
-if "autoaddcvs" in features:
-		

Re: [gentoo-portage-dev] Questions regarding the new portage API (savior branch)

[Marius Mauch]

Brian Harring wrote:
On 2/28/06, *Michael Schilling* <[EMAIL PROTECTED] 
> wrote:


- Is one of these svn-web-repository up to date?
  * http://sources.gentoo.org/viewcvs.py/portage/main/branches/savior/
  * http://mzz.mine.nu/bzr/savior-svn/portage/



I switched over to bzr about 2 months back; svn doesn't allow for 
offline committing, nor does gentoo's vcs allow for anon*... bzr 
natively allows for those capabilities, so that's what I'm using. :)


http://gentooexperimental.org/~ferringb/bzr/saviour
Is where I'll be updating the code  for at least the near future.



Does that mean we should drop the SVN branch?

Marius

--
gentoo-portage-dev@gentoo.org mailing list

Re: [gentoo-portage-dev] Switching LOCALE during build?

[Marius Mauch]

Patrick Lauer wrote:

Hi all,

this is an idea we had at FOSDEM to make bugreports easier to read:

During build portage should change the locale to en_us so that any error
messages are easy to read even if the user has set it differently. This
shouldn't affect users much and could get rid of the bugreports where it
is hard to understand what went wrong because everything is in french.

Portage changes should be limited to setting LC_ALL in all subshells it
spawns.

Does this cause any unexpected problems? (Programs changing their
language during build would be one possible side-effect). Is it
needed/useful?

Thanks for any feedback,


http://bugs.gentoo.org/show_bug.cgi?id=57973
--
gentoo-portage-dev@gentoo.org mailing list