Re: [gentoo-portage-dev] [PATCH] Manifest._apply_max_mtime: account for removals and renames (bug 567920)

[Michał Górny]

On Tue, 15 Dec 2015 08:50:10 -0800
Zac Medico  wrote:

> On 12/15/2015 12:43 AM, Michał Górny wrote:
> > On Tue, 15 Dec 2015 00:10:03 -0800
> > Zac Medico  wrote:
> >   
> >> Include directory mtimes in the max mtime calculation, in order
> >> to account for removals and renames.  
> > 
> > Mildly irrelevant but I think this reached the point when we want to
> > make this optional, and off by default. In other words, we want
> > 'repoman manifest' run explicitly to work the old way.
> >   
> 
> I suppose we could support a "stable-mtime = true" attribute that would
> be set in layout.conf and/or repos.conf.

No. This is about tool behavior. Tool behavior should be set
in the tool, not in the object the tool is working on. It should be
a command-line option to repoman or egencache.

-- 
Best regards,
Michał Górny



pgpyM19BNOzGj.pgp
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] Manifest._apply_max_mtime: account for removals and renames (bug 567920)

[Zac Medico]

On 12/15/2015 12:43 AM, Michał Górny wrote:
> On Tue, 15 Dec 2015 00:10:03 -0800
> Zac Medico  wrote:
> 
>> Include directory mtimes in the max mtime calculation, in order
>> to account for removals and renames.
> 
> Mildly irrelevant but I think this reached the point when we want to
> make this optional, and off by default. In other words, we want
> 'repoman manifest' run explicitly to work the old way.
> 

I suppose we could support a "stable-mtime = true" attribute that would
be set in layout.conf and/or repos.conf.

Theoretically, the difference in behavior is negligible except where
comparisons by tools like rsync are concerned. I have to admit that I
was a little bit nervous that something like bug 567920 might come up.
Now that directory mtimes are accounted for, I'm practically certain
that any relevant changes will cause the Manifest mtime to increase.
-- 
Thanks,
Zac

Re: [gentoo-dev] Re: converting copyright/license information in OpenRC

[William Hubbs]

On Tue, Dec 15, 2015 at 02:24:22PM -0500, Mike Frysinger wrote:
> On 15 Dec 2015 09:31, William Hubbs wrote:
> > On Tue, Dec 15, 2015 at 12:05:07AM -0500, Mike Frysinger wrote:
> > > On 11 Dec 2015 14:16, Patrick McLean wrote:
> > > > On Fri, 11 Dec 2015 15:37:48 -0600 William Hubbs wrote:
> > > > > On Fri, Dec 11, 2015 at 09:04:47PM +0100, Ulrich Mueller wrote:
> > > > > > > On Fri, 11 Dec 2015, William Hubbs wrote:  
> > > > > >   
> > > > > Well, the OpenRC project is currently inconsistent about this, so the
> > > > > intention is to make it consistent.
> > > > > 
> > > > >  The .c/.h files have file-scope licenses, but that isn't true for
> > > > >  everything in the project.
> > > > > 
> > > > > I am willing to make the effort to do this, I was just wondering if
> > > > > there are any legal pitfalls I need to worry about.
> > > > > 
> > > > > My theory is I can probably use git to find out who all of the authors
> > > > > are, and generate an Authors list from that information and from
> > > > > looking at copyright notices.
> > > > 
> > > > One concern about this is the possibility of copied code. If OpenRC
> > > > ever copied code from other BSD licensed projects, then dropping the
> > > > notice from the top of the file would be a violation of the upstream
> > > > license.
> > > 
> > > OpenRC isn't purely Gentoo copyright, so it's already a violation.
> > > the majority of entries belong to Roy.
> > 
> > I have no idea what you mean by "it's already a violation", and I'm not
> > sure what Gentoo Copyright has to do with it.
> 
> your description sounds like you want to run:
>   s/Copyright .*/Copyright OpenRC Authors/
> 
> i'm saying you can't do that
 
 That's the first part, the second part is to have an Authors file at
 the top level that lists all of the authors and refer to that in the
 copyright statement, see the license branch of the main github repo and
 let me know if this is legal. The site I linked seems to think so
 because I'm not deleting copyright attributions, just moving them around.

> > Altering Copyright statements to try to claim Gentoo copyright would
> > definitely be a violation, but that's not what I'm wanting to do.
> 
> adding multiple entries isn't a problem and in fact could/should be done
> in most openrc files at this time

Multiple entries are what I want to get away from; it is a nightmare to
maintain, and the vcs shows far better than you or I ever could
which code belongs to who (see git blame). That is what the site I
linked talks about.

William



signature.asc
Description: Digital signature

Re: [gentoo-dev] Use GLEP27!

[Alec Warner]

On Tue, Dec 15, 2015 at 6:19 AM, Mike Frysinger  wrote:

> On 15 Dec 2015 07:28, Ulrich Mueller wrote:
> > > On Tue, 15 Dec 2015, Mike Frysinger wrote:
> >
> > > On 14 Dec 2015 21:22, Ulrich Mueller wrote:
> > >> The spec seems incomplete. I cannot find a description of the user and
> > >> group files' format. (But in fact, there is a standard format which
> > >> suggests itself, namely that of the passwd(5) and group(5) files.)
> >
> > > i recall going with xml at the time, but i can't find reference to it.
> >
> > So the package manager would have to parse XML? We have tried to avoid
> > that, so far.
>
> not entirely accurate considering we have metadata.xml
>
> > >> Also having whole directory trees seems wasteful and doesn't fit so
> > >> well into the existing design of profiles. It might be simpler to put
> > >> "user" (or "passwd") and "group" files directly in the profile.
> > >> (If directories are really needed, we could use the scheme foreseen
> > >> in [1] for package.* and use.* files.)
> >
> > > we implemented this GLEP in Chromium OS and have been using it for a
> while:
> > >
> https://chromium.googlesource.com/chromiumos/overlays/eclass-overlay/+/master
> >
> > > having a directory of files is way more user friendly imo and allows
> for
> > > a format that is easier to read.  /etc/passwd and /etc/group format are
> > > not that easy to scan and aren't portable.
> >
> > As we most likely will introduce profile file directories in EAPI 7
> > (see bug 282296), I'd rather use the same scheme for the user and
> > group files, but not something different.
>
> a flat text file akin to /etc/passwd is not readable.  xml is readable.
>

u trollin' bro?


>
> a markdown like format would work -- easy to parse by machines & humans
> and is a single stackable file.
> user:ntp
> uid:203
> gid:203
> user:man
> uid:13
> gid:13
> (using : as delimiter since that's what *NIX uses in /etc/passwd)
>
> the main one would grow probably to about 2000+ lines (~400 users in the
> tree and each entry takes ~4 lines assuming we enforce eliding of defaults)
> which isn't that terrible.  CrOS has a python script already for validating
> the db consistency.
>
> > >> Also a mechanism how a subprofile could undefine a user or group
> > >> defined in its parent seems to be missing.
> >
> > > what exactly do you mean by that ?  you want to make it so attempts to
> > > use the account yield an undefined error ?  or you want to have it so
> > > a child can revert back to an earlier definition ?
> >
> > The former.
>
> we already handle this in CrOS by explicitly including a flag in the file:
> defunct:true
>
> thus the PM need not care about the status when locating files or otherwise
> include any special handling.  if the last file you hit is marked as
> defunct,
> then enew{user,group} will throw an error when they're called.
> -mike
>

Re: [gentoo-dev] Re: converting copyright/license information in OpenRC

[Mike Frysinger]

On 15 Dec 2015 09:31, William Hubbs wrote:
> On Tue, Dec 15, 2015 at 12:05:07AM -0500, Mike Frysinger wrote:
> > On 11 Dec 2015 14:16, Patrick McLean wrote:
> > > On Fri, 11 Dec 2015 15:37:48 -0600 William Hubbs wrote:
> > > > On Fri, Dec 11, 2015 at 09:04:47PM +0100, Ulrich Mueller wrote:
> > > > > > On Fri, 11 Dec 2015, William Hubbs wrote:  
> > > > >   
> > > > Well, the OpenRC project is currently inconsistent about this, so the
> > > > intention is to make it consistent.
> > > > 
> > > >  The .c/.h files have file-scope licenses, but that isn't true for
> > > >  everything in the project.
> > > > 
> > > > I am willing to make the effort to do this, I was just wondering if
> > > > there are any legal pitfalls I need to worry about.
> > > > 
> > > > My theory is I can probably use git to find out who all of the authors
> > > > are, and generate an Authors list from that information and from
> > > > looking at copyright notices.
> > > 
> > > One concern about this is the possibility of copied code. If OpenRC
> > > ever copied code from other BSD licensed projects, then dropping the
> > > notice from the top of the file would be a violation of the upstream
> > > license.
> > 
> > OpenRC isn't purely Gentoo copyright, so it's already a violation.
> > the majority of entries belong to Roy.
> 
> I have no idea what you mean by "it's already a violation", and I'm not
> sure what Gentoo Copyright has to do with it.

your description sounds like you want to run:
  s/Copyright .*/Copyright OpenRC Authors/

i'm saying you can't do that

> Altering Copyright statements to try to claim Gentoo copyright would
> definitely be a violation, but that's not what I'm wanting to do.

adding multiple entries isn't a problem and in fact could/should be done
in most openrc files at this time
-mike


signature.asc
Description: Digital signature

Re: [gentoo-dev] Re: converting copyright/license information in OpenRC

[Mike Frysinger]

On 15 Dec 2015 13:37, William Hubbs wrote:
> On Tue, Dec 15, 2015 at 02:24:22PM -0500, Mike Frysinger wrote:
> > your description sounds like you want to run:
> >   s/Copyright .*/Copyright OpenRC Authors/
> > 
> > i'm saying you can't do that
>  
>  That's the first part, the second part is to have an Authors file at
>  the top level that lists all of the authors and refer to that in the
>  copyright statement, see the license branch of the main github repo and
>  let me know if this is legal. The site I linked seems to think so
>  because I'm not deleting copyright attributions, just moving them around.

i don't think it's kosher.  just reach out to Roy and see if he's OK
with it and that'll avoid any ambiguity.
-mike


signature.asc
Description: Digital signature

Re: [gentoo-dev] Use GLEP27!

[Mike Frysinger]

On 15 Dec 2015 10:35, Alec Warner wrote:
> On Tue, Dec 15, 2015 at 6:19 AM, Mike Frysinger wrote:
> > a flat text file akin to /etc/passwd is not readable.  xml is readable.
> 
> u trollin' bro?

in this domain, it's harder to screw up xml and not notice (either human or
machine) whereas it's pretty easy to add/omit an extra : and not notice as
the file will be valid from a parsing pov.

quick, off the top of your head, which of these are correct ?
and tell me which field exactly each one is supposed to go to ?
and then reasonably expect new comers to read this and not choke ?
lp:x:4:7::/var/spool/lpd:/bin/false
lp:x:4:7:/var/spool/lpd:/bin/false
lp:x:4:7:/var/spool/lpd::/bin/false
lp:x:4:7::lpd:/bin/false
lp:x:4:7:lpd:/bin/false
lp:x:4:7:lpd::/bin/false

i consider this format simply a different form of hazing except we all lose ;)
-mike


signature.asc
Description: Digital signature

Re: [gentoo-dev] Use GLEP27!

[Mike Frysinger]

On 15 Dec 2015 10:24, Anthony G. Basile wrote:
> i looked through portage code and we do have xml parsing sprinkled
> throughout, mostly in repoman for obvious reasons.  why are we trying to
> avoid xml?  to be honest i don't have strong feelings about either the
> flat file (a la /etc/passwd) or xml.
> 
> i'm interested in this because the hardened-sources kernel does make use
> of some special uid/gids.  gengor pointed me to glep 27 and suggested
> that i implement it but i wasn't that interested.  still, it would clean
> up the hardened.

i don't think XML adds that much over RST, and are trivial to parse, both
on an ad-hoc basis, as well as python has modules to enable it.  i think
most people would agree RST or flat text files are less verbose than XML
w/out really losing that much (if anything).  at the time i was thinking
XML purely because we were much more of an XML shop at the time (we had
guidexml and metadata.xml), but with the rise of wiki/github/glep and the
fall of guidexml, the project has moved on.
-mike


signature.asc
Description: Digital signature

[gentoo-dev] New Project: C++

[Sergey Popov]

Hi. I would like to announce creation of a new project - Gentoo C++.

Well, it's not really new, it made mostly because of recent events
around GLEP 67. We have cpp herd in Gentoo for ages, so it would be nice
to not let it bitrot when herds will be deprecated.

I took the liberty to create project page on wiki[1] and add current
herd maintainers as project members. If someone was added by mistake,
feel free to leave a project.

Currently we have no lead and project page is mostly stub. Feel free to
join the project and/or improve project page.

[1] - https://wiki.gentoo.org/wiki/Project:C++

-- 
Best regards, Sergey Popov
Gentoo developer
Gentoo Desktop Effects project lead
Gentoo Quality Assurance project lead



signature.asc
Description: OpenPGP digital signature

Re: [gentoo-portage-dev] [PATCH] Add .git dir to excluded dirs in default PORTAGE_RSYNC_OPTS

[Brian Dolbec]

On Mon, 14 Dec 2015 08:54:27 -0800
Brian Dolbec  wrote:

> On Mon, 14 Dec 2015 10:52:50 -0500
> NP-Hardass  wrote:
> 
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA256
> > 
> > My apologies for the ambiguity.  I didn't realize there was a bug on
> > bugzilla regarding the original incident.
> > https://bugs.gentoo.org/show_bug.cgi?id=567074
> > The .git directory was pushed out over rsync, portage refused to
> > sync via rsync once it had a .git dir, and so rsync was broken for
> > users for a period of time.  So this is effectively a user-side
> > remeditation to reinforce the already taken server-side remediation.
> > 
> > On Mon, 14 Dec 2015 14:15:08 +0100
> > Alexander Berntsen  wrote:
> >   
> > > -BEGIN PGP SIGNED MESSAGE-
> > > Hash: SHA512
> > > 
> > > On 14/12/15 04:08, NP-Hardass wrote:
> > > > like those that we recently experienced.  
> > > I don't know what this is in reference too, so it should be
> > > explained better.
> > > 
> > > I have no real objects on the patch, but defer the ACK to Brian.
> > > Could you have a look, please?
> > > - -- 
> > > Alexander
> > > berna...@gentoo.org
> > > https://secure.plaimi.net/~alexander  
> >   
> 
> Yeah, this is fine, it was handled user side, but it is good to make
> this default.  I'll merge it in a bit.
> 

Applied pand pushed to master.

NP-Hardass and @all.  Please don't clearsign patches.  It was a bitch
to manually edit the mbox file to remove the signature cruft preventing
git from applying it correctly.  I imagine a detached sig would be fine
(untested).

-- 
Brian Dolbec 

[gentoo-portage-dev] [PATCH] Manifest._apply_max_mtime: account for removals and renames (bug 567920)

[Zac Medico]

Include directory mtimes in the max mtime calculation, in order
to account for removals and renames.

Fixes: 6dacd0ed9f6d ("Manifest.write: stable/predictable Manifest mtime for 
rsync (bug 557962)")
X-Gentoo-Bug: 567920
X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=567920
---
 pym/portage/manifest.py | 40 +---
 1 file changed, 29 insertions(+), 11 deletions(-)

diff --git a/pym/portage/manifest.py b/pym/portage/manifest.py
index f5cf0f5..451f869 100644
--- a/pym/portage/manifest.py
+++ b/pym/portage/manifest.py
@@ -282,7 +282,8 @@ class Manifest(object):
try:
myentries = list(self._createManifestEntries())
update_manifest = True
-   existing_st = None
+   preserved_stats = {}
+   preserved_stats[self.pkgdir.rstrip(os.sep)] = 
os.stat(self.pkgdir)
if myentries and not force:
try:
f = 
io.open(_unicode_encode(self.getFullname(),
@@ -290,7 +291,7 @@ class Manifest(object):
mode='r', 
encoding=_encodings['repo.content'],
errors='replace')
oldentries = 
list(self._parseManifestLines(f))
-   existing_st = os.fstat(f.fileno())
+   preserved_stats[self.getFullname()] = 
os.fstat(f.fileno())
f.close()
if len(oldentries) == len(myentries):
update_manifest = False
@@ -312,7 +313,7 @@ class Manifest(object):
# non-empty for all currently known use 
cases.
write_atomic(self.getFullname(), 
"".join("%s\n" %
_unicode(myentry) for myentry 
in myentries))
-   self._apply_max_mtime(existing_st, 
myentries)
+   self._apply_max_mtime(preserved_stats, 
myentries)
rval = True
else:
# With thin manifest, there's no need 
to have
@@ -332,17 +333,21 @@ class Manifest(object):
raise
return rval
 
-   def _apply_max_mtime(self, existing_st, entries):
+   def _apply_max_mtime(self, preserved_stats, entries):
"""
Set the Manifest mtime to the max mtime of all relevant files
-   (the existing Manifest mtime is included in order to account for
-   eclass modifications that change DIST entries). This results in 
a
+   and directories. Directory mtimes account for file renames and
+   removals. The existing Manifest mtime accounts for eclass
+   modifications that change DIST entries. This results in a
stable/predictable mtime, which is useful when converting thin
manifests to thick manifests for distribution via rsync. For
portability, the mtime is set with 1 second resolution.
 
@param existing_st: stat result for existing Manifest
@type existing_st: posix.stat_result
+   @param preserved_stats: maps paths to preserved stat results
+   that should be used instead place of os.stat() calls
+   @type preserved_stats: dict
@param entries: list of current Manifest2Entry instances
@type entries: list
"""
@@ -350,18 +355,31 @@ class Manifest(object):
# it always rounds down. Note that stat_result.st_mtime will 
round
# up from 0.9 to 1.0 when precision is lost during 
conversion
# from nanosecond resolution to float.
-   max_mtime = None if existing_st is None else 
existing_st[stat.ST_MTIME]
+   max_mtime = None
+   _update_max = (lambda st: max_mtime if max_mtime is not None
+   and max_mtime > st[stat.ST_MTIME] else 
st[stat.ST_MTIME])
+   _stat = (lambda path: preserved_stats[path] if path in 
preserved_stats
+   else os.stat(path))
+
+   for stat_result in preserved_stats.values():
+   max_mtime = _update_max(stat_result)
+
+   dirs = set()
for entry in entries:
if entry.type == 'DIST':
continue
abs_path = (os.path.join(self.pkgdir, 'files', 
entry.name) if
entry.type == 'AUX' else 
os.path.join(self.pkgdir, entry.name))
- 

Re: [gentoo-dev] Use GLEP27!

[Mike Frysinger]

On 15 Dec 2015 22:35, Ulrich Mueller wrote:
> Whatever the format will be, the more important question is where this
> would be implemented:
> 
> - In the package manager, with user and group definition in profiles.
>   This seems to be what GLEP 27 suggests, and as far as I can see, it
>   would require an EAPI bump. Certainly doable, but last time we
>   bumped profiles to a new EAPI we had a rather long transition
>   period.
> 
> - In user.eclass, which could be extended to use the EUSERS and
>   EGROUPS variables defined in ebuilds. The problem is, where would
>   one store the user and group definitions then? Profiles cannot be
>   accessed from an eclass.

long term, i think profiles are better to hold the db as it provides for
clean stacking and is trivial for site-specific extension/control, as well
as image builders via something like catalyst.

short/mid term, i was thinking of writing a new package that holds the db
and tools to query/manage it.  user.eclass would DEPEND on it and ask it
for details, perhaps even doing the actual fs updates (the bash code here
is not pretty wrt locks and python would be much nicer).  that tool could
even search additional site paths (like /usr/local) to locate overrides.

the API to ebuilds/eclasses would be unchanged.  in CrOS, we only look at
the first argument (the user/group name) and load all other details from
the db.  we could seamlessly migrate over existing ebuilds by opting in to
this simpler form.

maybe the short/mid term solution is enough to not get into profile mess
even if i think it's the correct data storage location.
-mike


signature.asc
Description: Digital signature

Re: [gentoo-dev] Use GLEP27!

[Ulrich Mueller]

> On Tue, 15 Dec 2015, Mike Frysinger wrote:

> On 15 Dec 2015 15:56, Ulrich Mueller wrote:
>> ESR's case study about the password file format seems to disagree:
>> http://www.catb.org/esr/writings/taoup/html/ch05s01.html#id2901332

> because you cited it, i read it anyways.  that document is about how text
> formats should be preferred over binary formats because they do not require
> custom tools to modify/update, and because it's easier for binary formats
> to screw themselves over from a portability/extensible pov.  it does not
> champion the passwd format all by itself, and even says that it's a bit
> rigid, and you should consider tagged formats if you want something more.
> which we do.

> see also the example i posted to Alec as why the format is hostile to devs
> whereas my simple RST proposal has none of these issues.

Whatever the format will be, the more important question is where this
would be implemented:

- In the package manager, with user and group definition in profiles.
  This seems to be what GLEP 27 suggests, and as far as I can see, it
  would require an EAPI bump. Certainly doable, but last time we
  bumped profiles to a new EAPI we had a rather long transition
  period.

- In user.eclass, which could be extended to use the EUSERS and
  EGROUPS variables defined in ebuilds. The problem is, where would
  one store the user and group definitions then? Profiles cannot be
  accessed from an eclass.

Ulrich


pgpygJDBAnbgK.pgp
Description: PGP signature

[gentoo-dev] Last rites: dev-lisp/cl-clx

[Chema Alonso]

# Jos?? Mar??a Alonso Josa  (15 Dec 2015)
# Refers to same package as dev-lisp/clx
# Masked for removal in 30 days, bug 568188
dev-lisp/cl-clx

[gentoo-portage-dev] [PATCH] depgraph._resolve: consider unresolved @system atoms fatal (bug 568354)

[Zac Medico]

Since @system atoms are presumably chosen by someone who knows what
they are doing (usually a gentoo developer), it makes sense to consider
resolution failures for these atoms as fatal errors.

X-Gentoo-Bug: 568354
X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=568354
---
 pym/_emerge/depgraph.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pym/_emerge/depgraph.py b/pym/_emerge/depgraph.py
index fd2c771..d971749 100644
--- a/pym/_emerge/depgraph.py
+++ b/pym/_emerge/depgraph.py
@@ -4015,7 +4015,7 @@ class depgraph(object):
continue
 
if not (isinstance(arg, SetArg) 
and \
-   arg.name in 
("selected", "system", "world")):
+   arg.name in 
("selected", "world")):

self._dynamic_config._unsatisfied_deps_for_display.append(
((myroot, 
atom), {"myparent" : arg}))
return 0, myfavorites
-- 
2.4.10

Re: [gentoo-dev] rfc: OpenRC public API definition

[Mike Frysinger]

On 03 Dec 2015 11:36, William Hubbs wrote:
> 1) All definitions in rc.h, even though they are not formally documented
> in man pages.

this.  if it's not meant to be public, it should have been in librc.h or
a similar internal header.

> I'm bringing this up, because I am looking at redesigning one of the
> undocumented functions soon, and the redesign will definitely break
> things if people are using the function outside of OpenRC.

bump the SONAME and forget about it.  as mentioned in the other thread,
there's not really any consumers today.  plus, i'm pretty sure we've
broken things a bit in the past but no one noticed/complained.
-mike


signature.asc
Description: Digital signature

Re: [gentoo-dev] Use GLEP27!

[Mike Frysinger]

On 15 Dec 2015 15:56, Ulrich Mueller wrote:
> > On Tue, 15 Dec 2015, Mike Frysinger wrote:
> 
> > a flat text file akin to /etc/passwd is not readable.  xml is readable.
> 
> ESR's case study about the password file format seems to disagree:
> http://www.catb.org/esr/writings/taoup/html/ch05s01.html#id2901332

because you cited it, i read it anyways.  that document is about how text
formats should be preferred over binary formats because they do not require
custom tools to modify/update, and because it's easier for binary formats
to screw themselves over from a portability/extensible pov.  it does not
champion the passwd format all by itself, and even says that it's a bit
rigid, and you should consider tagged formats if you want something more.
which we do.

see also the example i posted to Alec as why the format is hostile to devs
whereas my simple RST proposal has none of these issues.

we also know the format works because it's been in use by CrOS for two
years now, including:
(1) profile stacking/overrides
(2) marking users as "dead"
(3) tools to verify/check consistency at rest
(4) parallel installs
(5) correct handling of ROOT and SYSROOT
although 4/5 were written only with Linux/flat files in mind, so they do
not support other OS's or account formats (e.g. ldap/etc...).  i imagine
we'll need to merge with the existing user.eclass logic rather than drop
in replace.
-mike


signature.asc
Description: Digital signature

[gentoo-portage-dev] [PATCH] egencache: add --stable-mtime option

[Zac Medico]

Disable Manifest "stable mtime" behavior by default, and add
a corresponding egencache option.

Suggested-by: Michał Górny 
---
 bin/egencache  |  6 +-
 man/egencache.1|  3 +++
 pym/portage/manifest.py| 14 +-
 .../package/ebuild/_parallel_manifest/ManifestProcess.py   |  6 --
 .../package/ebuild/_parallel_manifest/ManifestScheduler.py |  7 +--
 .../package/ebuild/_parallel_manifest/ManifestTask.py  |  8 +---
 6 files changed, 31 insertions(+), 13 deletions(-)

diff --git a/bin/egencache b/bin/egencache
index 7e3387e..07665e8 100755
--- a/bin/egencache
+++ b/bin/egencache
@@ -120,6 +120,9 @@ def parse_args(args):
choices=('y', 'n'),
metavar="",
help="manually override layout.conf sign-manifests setting")
+   common.add_argument("--stable-mtime",
+   action="store_true",
+   help="apply stable mtime to generated manifests (for rsync)")
common.add_argument("--strict-manifests",
choices=('y', 'n'),
metavar="",
@@ -1151,7 +1154,8 @@ def egencache_main(args):
force_sign_key=force_sign_key,
max_jobs=options.jobs,
max_load=options.load_average,
-   event_loop=event_loop)
+   event_loop=event_loop,
+   manifest_kwargs=dict(stable_mtime=options.stable_mtime))
 
signum = run_main_scheduler(scheduler)
if signum is not None:
diff --git a/man/egencache.1 b/man/egencache.1
index 7fd17c2..081e8c1 100644
--- a/man/egencache.1
+++ b/man/egencache.1
@@ -100,6 +100,9 @@ Manually override layout.conf sign-manifests setting.
 .BR "\-\-strict\-manifests< y | n >"
 Manually override "strict" FEATURES setting.
 .TP
+.BR "\-\-stable\-mtime"
+Apply stable mtime to generated manifests (for rsync).
+.TP
 .BR "\-\-thin\-manifests< y | n >"
 Manually override layout.conf thin-manifests setting.
 .TP
diff --git a/pym/portage/manifest.py b/pym/portage/manifest.py
index 818515f..0724272 100644
--- a/pym/portage/manifest.py
+++ b/pym/portage/manifest.py
@@ -128,7 +128,7 @@ class Manifest(object):
def __init__(self, pkgdir, distdir=None, fetchlist_dict=None,
manifest1_compat=DeprecationWarning, from_scratch=False, 
thin=False,
allow_missing=False, allow_create=True, hashes=None,
-   find_invalid_path_char=None):
+   find_invalid_path_char=None, stable_mtime=False):
""" Create new Manifest instance for package in pkgdir.
Do not parse Manifest file if from_scratch == True (only 
for internal use)
The fetchlist_dict parameter is required only for 
generation of
@@ -145,6 +145,7 @@ class Manifest(object):
find_invalid_path_char = _find_invalid_path_char
self._find_invalid_path_char = find_invalid_path_char
self.pkgdir = _unicode_decode(pkgdir).rstrip(os.sep) + os.sep
+   self.stable_mtime = stable_mtime
self.fhashdict = {}
self.hashes = set()
 
@@ -283,7 +284,6 @@ class Manifest(object):
myentries = list(self._createManifestEntries())
update_manifest = True
preserved_stats = {}
-   preserved_stats[self.pkgdir.rstrip(os.sep)] = 
os.stat(self.pkgdir)
if myentries and not force:
try:
f = 
io.open(_unicode_encode(self.getFullname(),
@@ -291,7 +291,9 @@ class Manifest(object):
mode='r', 
encoding=_encodings['repo.content'],
errors='replace')
oldentries = 
list(self._parseManifestLines(f))
-   preserved_stats[self.getFullname()] = 
os.fstat(f.fileno())
+   if self.stable_mtime:
+   
preserved_stats[self.getFullname()] = os.fstat(f.fileno())
+   
preserved_stats[self.pkgdir.rstrip(os.sep)] = os.stat(self.pkgdir)
f.close()
if len(oldentries) == len(myentries):
update_manifest = False
@@ -313,7 +315,8 @@ class Manifest(object):
# non-empty for all currently known use 
cases.
write_atomic(self.getFullname(), 
"".join("%s\n" %
_unicode(myentry) for 

Re: [gentoo-dev] Use GLEP27!

[Mike Frysinger]

On 15 Dec 2015 20:23, Anthony G. Basile wrote:
> On 12/15/15 4:55 PM, Mike Frysinger wrote:
> > On 15 Dec 2015 22:35, Ulrich Mueller wrote:
> >> Whatever the format will be, the more important question is where this
> >> would be implemented:
> >>
> >> - In the package manager, with user and group definition in profiles.
> >>   This seems to be what GLEP 27 suggests, and as far as I can see, it
> >>   would require an EAPI bump. Certainly doable, but last time we
> >>   bumped profiles to a new EAPI we had a rather long transition
> >>   period.
> >>
> >> - In user.eclass, which could be extended to use the EUSERS and
> >>   EGROUPS variables defined in ebuilds. The problem is, where would
> >>   one store the user and group definitions then? Profiles cannot be
> >>   accessed from an eclass.
> >
> > long term, i think profiles are better to hold the db as it provides for
> > clean stacking and is trivial for site-specific extension/control, as well
> > as image builders via something like catalyst.
> >
> > short/mid term, i was thinking of writing a new package that holds the db
> > and tools to query/manage it.  user.eclass would DEPEND on it and ask it
> > for details, perhaps even doing the actual fs updates (the bash code here
> > is not pretty wrt locks and python would be much nicer).  that tool could
> > even search additional site paths (like /usr/local) to locate overrides.
> 
> how do we get our own uid/gid's in there for our packages?  just open a
> bug against the new package?

i would open the git repo to all devs and just let anyone push directly
and roll releases.  maybe just push a tag and that's what the ebuild would
hit.  no need to be gate keepers here since we don't today w/ebuilds and
calls to enew{user,group}.

the question is how to handle the tools.  i'm thinking we should have two
distinct packages so the db could be completely overridden by overlays.

the tools could be in the same repo or a sep one.  i don't feel strongly
either way as we'd just mark base-system@ as the maintaining herd.
-mike


signature.asc
Description: Digital signature

Re: [gentoo-dev] Use GLEP27!

[Mike Frysinger]

On 15 Dec 2015 07:28, Ulrich Mueller wrote:
> > On Tue, 15 Dec 2015, Mike Frysinger wrote:
> 
> > On 14 Dec 2015 21:22, Ulrich Mueller wrote:
> >> The spec seems incomplete. I cannot find a description of the user and
> >> group files' format. (But in fact, there is a standard format which
> >> suggests itself, namely that of the passwd(5) and group(5) files.)
> 
> > i recall going with xml at the time, but i can't find reference to it.
> 
> So the package manager would have to parse XML? We have tried to avoid
> that, so far.

not entirely accurate considering we have metadata.xml

> >> Also having whole directory trees seems wasteful and doesn't fit so
> >> well into the existing design of profiles. It might be simpler to put
> >> "user" (or "passwd") and "group" files directly in the profile.
> >> (If directories are really needed, we could use the scheme foreseen
> >> in [1] for package.* and use.* files.)
> 
> > we implemented this GLEP in Chromium OS and have been using it for a while:
> > https://chromium.googlesource.com/chromiumos/overlays/eclass-overlay/+/master
> 
> > having a directory of files is way more user friendly imo and allows for
> > a format that is easier to read.  /etc/passwd and /etc/group format are
> > not that easy to scan and aren't portable.
> 
> As we most likely will introduce profile file directories in EAPI 7
> (see bug 282296), I'd rather use the same scheme for the user and
> group files, but not something different.

a flat text file akin to /etc/passwd is not readable.  xml is readable.

a markdown like format would work -- easy to parse by machines & humans
and is a single stackable file.
user:ntp
uid:203
gid:203
user:man
uid:13
gid:13
(using : as delimiter since that's what *NIX uses in /etc/passwd)

the main one would grow probably to about 2000+ lines (~400 users in the
tree and each entry takes ~4 lines assuming we enforce eliding of defaults)
which isn't that terrible.  CrOS has a python script already for validating
the db consistency.

> >> Also a mechanism how a subprofile could undefine a user or group
> >> defined in its parent seems to be missing.
> 
> > what exactly do you mean by that ?  you want to make it so attempts to
> > use the account yield an undefined error ?  or you want to have it so
> > a child can revert back to an earlier definition ?
> 
> The former.

we already handle this in CrOS by explicitly including a flag in the file:
defunct:true

thus the PM need not care about the status when locating files or otherwise
include any special handling.  if the last file you hit is marked as defunct,
then enew{user,group} will throw an error when they're called.
-mike


signature.asc
Description: Digital signature

Re: [gentoo-dev] Use GLEP27!

[Michał Górny]

On Tue, 15 Dec 2015 15:56:34 +0100
Ulrich Mueller  wrote:

> > On Tue, 15 Dec 2015, Mike Frysinger wrote:  
> 
> > a flat text file akin to /etc/passwd is not readable.  xml is readable.  
> 
> ESR's case study about the password file format seems to disagree:
> http://www.catb.org/esr/writings/taoup/html/ch05s01.html#id2901332
> 
> I think the name:password:uid:gid:gecos:directory:shell format is
> readable well enough for human eyes. Certainly it is machine readable;
> even with standard tools like fgetpwent(3) (or its equivalent in other
> programming languages).

Alternatively we could go for whitespace-separated, like fstab. Column
aligning even more.

-- 
Best regards,
Michał Górny



pgpnCL9K9stQI.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Use GLEP27!

[Michał Górny]

On Tue, 15 Dec 2015 09:19:36 -0500
Mike Frysinger  wrote:

> On 15 Dec 2015 07:28, Ulrich Mueller wrote:
> > > On Tue, 15 Dec 2015, Mike Frysinger wrote:  
> >   
> > > On 14 Dec 2015 21:22, Ulrich Mueller wrote:  
> > >> The spec seems incomplete. I cannot find a description of the user and
> > >> group files' format. (But in fact, there is a standard format which
> > >> suggests itself, namely that of the passwd(5) and group(5) files.)  
> >   
> > > i recall going with xml at the time, but i can't find reference to it.  
> > 
> > So the package manager would have to parse XML? We have tried to avoid
> > that, so far.  
> 
> not entirely accurate considering we have metadata.xml

Not entirely accurate considering the contents of metadata.xml are not
necessary to build and install packages.

-- 
Best regards,
Michał Górny



pgpCdVq8eOJrq.pgp
Description: OpenPGP digital signature

Re: [gentoo-dev] Use GLEP27!

[Ulrich Mueller]

> On Tue, 15 Dec 2015, Mike Frysinger wrote:

> a flat text file akin to /etc/passwd is not readable.  xml is readable.

ESR's case study about the password file format seems to disagree:
http://www.catb.org/esr/writings/taoup/html/ch05s01.html#id2901332

I think the name:password:uid:gid:gecos:directory:shell format is
readable well enough for human eyes. Certainly it is machine readable;
even with standard tools like fgetpwent(3) (or its equivalent in other
programming languages).

> a markdown like format would work -- easy to parse by machines & humans
> and is a single stackable file.

Reinventing the wheel?

> user:ntp
> uid:203
> gid:203
> user:man
> uid:13
> gid:13
> (using : as delimiter since that's what *NIX uses in /etc/passwd)

> the main one would grow probably to about 2000+ lines (~400 users in
> the tree and each entry takes ~4 lines assuming we enforce eliding
> of defaults) which isn't that terrible.

To me this looks like a lot of added redundancy for little (if any)
benefit.

Ulrich


pgpa9ze0yoR2W.pgp
Description: PGP signature

Re: [gentoo-dev] Use GLEP27!

[Anthony G. Basile]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/15/15 4:55 PM, Mike Frysinger wrote:
> On 15 Dec 2015 22:35, Ulrich Mueller wrote:
>> Whatever the format will be, the more important question is where this
>> would be implemented:
>>
>> - In the package manager, with user and group definition in profiles.
>>   This seems to be what GLEP 27 suggests, and as far as I can see, it
>>   would require an EAPI bump. Certainly doable, but last time we
>>   bumped profiles to a new EAPI we had a rather long transition
>>   period.
>>
>> - In user.eclass, which could be extended to use the EUSERS and
>>   EGROUPS variables defined in ebuilds. The problem is, where would
>>   one store the user and group definitions then? Profiles cannot be
>>   accessed from an eclass.
>
> long term, i think profiles are better to hold the db as it provides for
> clean stacking and is trivial for site-specific extension/control, as well
> as image builders via something like catalyst.
>
> short/mid term, i was thinking of writing a new package that holds the db
> and tools to query/manage it.  user.eclass would DEPEND on it and ask it
> for details, perhaps even doing the actual fs updates (the bash code here
> is not pretty wrt locks and python would be much nicer).  that tool could
> even search additional site paths (like /usr/local) to locate overrides.

how do we get our own uid/gid's in there for our packages?  just open a
bug against the new package?

>
>
> the API to ebuilds/eclasses would be unchanged.  in CrOS, we only look at
> the first argument (the user/group name) and load all other details from
> the db.  we could seamlessly migrate over existing ebuilds by opting in to
> this simpler form.
>
> maybe the short/mid term solution is enough to not get into profile mess
> even if i think it's the correct data storage location.
> -mike


- -- 
Anthony G. Basile, Ph.D.
Gentoo Linux Developer [Hardened]
E-Mail: bluen...@gentoo.org
GnuPG FP  : 1FED FAD9 D82C 52A5 3BAB  DC79 9384 FA6E F52D 4BBA
GnuPG ID  : F52D4BBA
-BEGIN PGP SIGNATURE-
Comment: GPGTools - https://gpgtools.org

iEYEARECAAYFAlZwvQcACgkQl5yvQNBFVTVEaACfT1nMMFXsPyqM0u4rGDHJP29/
pFkAn0XOcHTmVAAp9K9opvWT9isuMOxp
=xPUD
-END PGP SIGNATURE-

[gentoo-portage-dev] Re: [PATCH] depgraph._resolve: consider unresolved @system atoms fatal (bug 568354)

[Zac Medico]

On 12/15/2015 12:46 PM, Zac Medico wrote:
> Since @system atoms are presumably chosen by someone who knows what
> they are doing (usually a gentoo developer), it makes sense to consider
> resolution failures for these atoms as fatal errors.
> 
> X-Gentoo-Bug: 568354
> X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=568354

Hmm, maybe this should suggest config changes, which could even be
written with --autounmask-write. That would be much friendlier.
-- 
Thanks,
Zac

[gentoo-dev] Re: rfc: openrc c api (librc) usage

[Duncan]

Mike Frysinger posted on Tue, 15 Dec 2015 00:00:51 -0500 as excerpted:

> On 10 Dec 2015 10:32, William Hubbs wrote:
>> I want to start a discussion about the usage of OpenRC's C api as
>> defined in rc.h.
> 
> very few projects ever picked up the API/libraries.  probably for the
> best.
> 
>> I have no idea which projectss out there are using it, or which
>> functions they are using.
> 
> look at the RDEPEND in ebuilds.  if a package uses the libs, it should
> be listed there.

Even with openrc still in @system?  Doesn't look like that bug has been 
fixed yet.[1]

> if you don't find any of those (few) packages actually
> using the API, then just make the change and wait for someone to
> complain.

With openrc still in @system, that's probably the way it'll need to be 
handled, tho ideally everything that needs it will be RDEPENDing on it by 
now in preparation for removing openrc from @system, so nothing will 
break.

---
[1] I've had my entire @system negated in /etc/portage/profiles/packages 
since long before I switched to systemd, and after symlinking
/etc/init.d/functions.sh -> /lib/gentoo/functions.sh, I unmerged openrc.  
So to see if it was in the normal @system I had to manually check if 
openrc was still starred in a packages file any of the broadly sourced 
profiles and it is, in base, with the only negations in the prefix and 
freebsd profiles.

Obviously nothing I run breaks without any of the other files openrc 
provided, but that doesn't mean nothing else in the tree does.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: [gentoo-dev] Use GLEP27!

[Mike Frysinger]

On 15 Dec 2015 16:00, Michał Górny wrote:
> On Tue, 15 Dec 2015 15:56:34 +0100 Ulrich Mueller wrote:
> > > On Tue, 15 Dec 2015, Mike Frysinger wrote:  
> > 
> > > a flat text file akin to /etc/passwd is not readable.  xml is readable.  
> > 
> > ESR's case study about the password file format seems to disagree:
> > http://www.catb.org/esr/writings/taoup/html/ch05s01.html#id2901332
> > 
> > I think the name:password:uid:gid:gecos:directory:shell format is
> > readable well enough for human eyes. Certainly it is machine readable;
> > even with standard tools like fgetpwent(3) (or its equivalent in other
> > programming languages).
> 
> Alternatively we could go for whitespace-separated, like fstab. Column
> aligning even more.

we cannot because whitespace is valid, and we use it already in fields
-mike


signature.asc
Description: Digital signature

Re: [gentoo-dev] Use GLEP27!

[Mike Frysinger]

On 15 Dec 2015 15:33, Michał Górny wrote:
> On Tue, 15 Dec 2015 09:19:36 -0500 Mike Frysinger wrote:
> > On 15 Dec 2015 07:28, Ulrich Mueller wrote:
> > > > On Tue, 15 Dec 2015, Mike Frysinger wrote:  
> > >   
> > > > On 14 Dec 2015 21:22, Ulrich Mueller wrote:  
> > > >> The spec seems incomplete. I cannot find a description of the user and
> > > >> group files' format. (But in fact, there is a standard format which
> > > >> suggests itself, namely that of the passwd(5) and group(5) files.)  
> > >   
> > > > i recall going with xml at the time, but i can't find reference to it.  
> > > 
> > > So the package manager would have to parse XML? We have tried to avoid
> > > that, so far.  
> > 
> > not entirely accurate considering we have metadata.xml
> 
> Not entirely accurate considering the contents of metadata.xml are not
> necessary to build and install packages.

read the context.  your statement is fairly obvious.
-mike


signature.asc
Description: Digital signature

Re: [gentoo-dev] Re: converting copyright/license information in OpenRC

[William Hubbs]

On Tue, Dec 15, 2015 at 12:05:07AM -0500, Mike Frysinger wrote:
> On 11 Dec 2015 14:16, Patrick McLean wrote:
> > On Fri, 11 Dec 2015 15:37:48 -0600 William Hubbs wrote:
> > > On Fri, Dec 11, 2015 at 09:04:47PM +0100, Ulrich Mueller wrote:
> > > > > On Fri, 11 Dec 2015, William Hubbs wrote:  
> > > >   
> > > Well, the OpenRC project is currently inconsistent about this, so the
> > > intention is to make it consistent.
> > > 
> > >  The .c/.h files have file-scope licenses, but that isn't true for
> > >  everything in the project.
> > > 
> > > I am willing to make the effort to do this, I was just wondering if
> > > there are any legal pitfalls I need to worry about.
> > > 
> > > My theory is I can probably use git to find out who all of the authors
> > > are, and generate an Authors list from that information and from
> > > looking at copyright notices.
> > 
> > One concern about this is the possibility of copied code. If OpenRC
> > ever copied code from other BSD licensed projects, then dropping the
> > notice from the top of the file would be a violation of the upstream
> > license.
> 
> OpenRC isn't purely Gentoo copyright, so it's already a violation.
> the majority of entries belong to Roy.

I have no idea what you mean by "it's already a violation", and I'm not
sure what Gentoo Copyright has to do with it.

Altering Copyright statements to try to claim Gentoo copyright would
definitely be a violation, but that's not what I'm wanting to do.

My goal is to centralize the Copyright and license information we
already have, as much as possible [1]. This site seems to imply that
moving Copyright/Author information around is legal.

William

[1] https://softwarefreedom.org/resources/2012/ManagingCopyrightInformation.html


signature.asc
Description: Digital signature

Re: [gentoo-dev] Use GLEP27!

[Mike Frysinger]

On 15 Dec 2015 15:56, Ulrich Mueller wrote:
> > On Tue, 15 Dec 2015, Mike Frysinger wrote:
> 
> > a flat text file akin to /etc/passwd is not readable.  xml is readable.
> 
> ESR's case study about the password file format seems to disagree:
> http://www.catb.org/esr/writings/taoup/html/ch05s01.html#id2901332

i don't generally find anything ESR has to say useful, and his code even
less so

> I think the name:password:uid:gid:gecos:directory:shell format is
> readable well enough for human eyes.

having looked at way too many of these, i disagree.  it's harder (imo) to
scan due to the lack of alignment, it's more of a pita to deal with defaults,
and it doesn't permit for additional metadata like i already quoted (the
defunct field).

> Certainly it is machine readable;
> even with standard tools like fgetpwent(3) (or its equivalent in other
> programming languages).

again, not portable.  the format proposed is also trivial to parse by
python and awk.
-mike


signature.asc
Description: Digital signature

[gentoo-portage-dev] [PATCH] repoman: use metadata.dtd from rsync tree if available (bug 567746)

[Zac Medico]

Search for metadata.dtd in current repository and masters, and if that
fails then fetch is as usual.

X-Gentoo-Bug: 567746
X-Gentoo-Bug-url: https://bugs.gentoo.org/show_bug.cgi?id=567746
---
 pym/repoman/_xml.py   |  9 +
 pym/repoman/checks/ebuilds/pkgmetadata.py |  6 --
 pym/repoman/scanner.py| 10 +-
 3 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/pym/repoman/_xml.py b/pym/repoman/_xml.py
index 0acda28..4ca6a0a 100644
--- a/pym/repoman/_xml.py
+++ b/pym/repoman/_xml.py
@@ -51,11 +51,12 @@ class 
_MetadataTreeBuilder(xml.etree.ElementTree.TreeBuilder):
 
 class XmlLint(object):
 
-   def __init__(self, options, repoman_settings):
-   self.metadata_dtd = os.path.join(repoman_settings["DISTDIR"], 
'metadata.dtd')
+   def __init__(self, options, repoman_settings, metadata_dtd=None):
+   self.metadata_dtd = (metadata_dtd or
+   os.path.join(repoman_settings["DISTDIR"], 
'metadata.dtd'))
self.options = options
self.repoman_settings = repoman_settings
-   self._is_capable = False
+   self._is_capable = metadata_dtd is not None
self.binary = None
self._check_capable()
 
@@ -65,7 +66,7 @@ class XmlLint(object):
self.binary = find_binary('xmllint')
if not self.binary:
print(red("!!! xmllint not found. Can't check 
metadata.xml.\n"))
-   else:
+   elif not self._is_capable:
if not fetch_metadata_dtd(self.metadata_dtd, 
self.repoman_settings):
sys.exit(1)
# this can be problematic if xmllint changes their 
output
diff --git a/pym/repoman/checks/ebuilds/pkgmetadata.py 
b/pym/repoman/checks/ebuilds/pkgmetadata.py
index f22ef19..74fec69 100644
--- a/pym/repoman/checks/ebuilds/pkgmetadata.py
+++ b/pym/repoman/checks/ebuilds/pkgmetadata.py
@@ -40,18 +40,20 @@ from repoman._xml import _XMLParser, _MetadataTreeBuilder, 
XmlLint
 class PkgMetadata(object):
'''Package metadata.xml checks'''
 
-   def __init__(self, options, qatracker, repoman_settings):
+   def __init__(self, options, qatracker, repoman_settings, 
metadata_dtd=None):
'''PkgMetadata init function
 
@param options: ArgumentParser.parse_known_args(argv[1:]) 
options
@param qatracker: QATracker instance
@param repoman_settings: settings instance
+   @param metadata_dtd: path of metadata.dtd
'''
self.options = options
self.qatracker = qatracker
self.repoman_settings = repoman_settings
self.musedict = {}
-   self.xmllint = XmlLint(self.options, self.repoman_settings)
+   self.xmllint = XmlLint(self.options, self.repoman_settings,
+   metadata_dtd=metadata_dtd)
 
def check(self, xpkg, checkdir, checkdirlist, repolevel):
'''Performs the checks on the metadata.xml for the package
diff --git a/pym/repoman/scanner.py b/pym/repoman/scanner.py
index 9e5a313..9a87f65 100644
--- a/pym/repoman/scanner.py
+++ b/pym/repoman/scanner.py
@@ -82,6 +82,13 @@ class Scanner(object):
portage.util.stack_lists([self.categories], 
incremental=1))
self.categories = self.repo_settings.repoman_settings.categories
 
+   metadata_dtd = None
+   for path in 
reversed(self.repo_settings.repo_config.eclass_db.porttrees):
+   path = os.path.join(path, 'metadata/dtd/metadata.dtd')
+   if os.path.exists(path):
+   metadata_dtd = path
+   break
+
self.portdb = repo_settings.portdb
self.portdb.settings = self.repo_settings.repoman_settings
# We really only need to cache the metadata that's necessary 
for visibility
@@ -201,7 +208,8 @@ class Scanner(object):
self.status_check = VCSStatus(self.vcs_settings, self.qatracker)
self.fetchcheck = FetchChecks(
self.qatracker, self.repo_settings, self.portdb, 
self.vcs_settings)
-   self.pkgmeta = PkgMetadata(self.options, self.qatracker, 
self.repo_settings.repoman_settings)
+   self.pkgmeta = PkgMetadata(self.options, self.qatracker,
+   self.repo_settings.repoman_settings, 
metadata_dtd=metadata_dtd)
self.thirdparty = 
ThirdPartyMirrors(self.repo_settings.repoman_settings, self.qatracker)
self.use_flag_checks = USEFlagChecks(self.qatracker, uselist)
self.keywordcheck = KeywordChecks(self.qatracker, self.options)
-- 
2.4.10