Re: [gentoo-portage-dev] Questions regarding the new portage API (savior branch)

2006-03-03 Thread Donnie Berkholz
Brian Harring wrote:
 If y'all want to mirror it, might I suggest poking marienz for his 
 tailorization knowledge?  Afaik, he had a bzr-svn push working, or at 
 least has investigated it.

From what I've heard, tailor has absolutely no knowledge of branches. So
if you use branches, might want to tread carefully with tailor.


Description: OpenPGP digital signature

[gentoo-portage-dev] [PATCH] Manifest2 reloaded

2006-03-03 Thread Marius Mauch
So while on my way to FOSDEM I decided to do something useful with the
time and wrote a new manifest2 implementation. This has nothing to do
with the original prototype I posted a while ago, it's been written
completely from scratch.
Basically all functionality (creation, parsing, validation) is
encapsulated in the new portage_manifest.Manifest class, including
compability code to read/write old style digests.
The changes to only change the digest*() functions to use
this new class instead of handling the task themselves (exception:
digestCheckFiles() which apparently was only used internally by other
digest* functions), they should more or less behave like with the old
code. Any new code however should use the Manifest() class directly
While this patch implements the basic functionality some extra stuff
that was in the old code isn't included yet:
- gpg verification
- FEATURES=autoaddcvs
- FEATURES=cvs (probably obsolete anyway)
- emerge --digest / FEATURES=digest (may or may not work)

The first should be delayed until there is some consensus how the gpg
stuff should work in the future, the others I don't see the use for.
Also I only checked for changes, so emerge/repoman/... might
still have to be fixed.
Last but not least: I did some basic testing with this and the
important stuff seems to work, but I'm quite sure the code still has a
lot of bugs/issues, and this being a core functionality it needs a
*lot* of testing, so I'd really appreciate if you could all give it a
spin (but do not commit anything to the tree without manually checking
it first).


Public Key at

In the beginning, there was nothing. And God said, 'Let there be
Light.' And there was still nothing, but you could see a bit better.
diff -ru --exclude=CVS --exclude=.svn -N pym/ pym/
--- pym/	2006-03-04 02:25:20.957635000 +
+++ pym/	2006-03-04 03:12:19.545785750 +
@@ -90,6 +90,7 @@
 	from portage_data import ostype, lchown, userland, secpass, uid, wheelgid, \
 	 portage_uid, portage_gid
+	from portage_manifest import Manifest
 	import portage_util
 	from portage_util import atomic_ofstream, dump_traceback, getconfig, grabdict, \
@@ -2049,181 +2050,67 @@
 			return 0
 	return 1
-def digestCreate(myfiles,basedir,oldDigest={}):
-	Takes a list of files and the directory they are in and returns the
-	dict of dict[filename][CHECKSUM_KEY] = hash
-	returns None on error.
-	mydigests={}
-	for x in myfiles:
-		print ,x
-		myfile=os.path.normpath(basedir+///+x)
-		if os.path.exists(myfile):
-			if not os.access(myfile, os.R_OK):
-print !!! Given file does not appear to be readable. Does it exist?
-print !!! File:,myfile
-return None
-			mydigests[x] = portage_checksum.perform_multiple_checksums(myfile, hashes=portage_const.MANIFEST1_HASH_FUNCTIONS)
-			mysize   = os.stat(myfile)[stat.ST_SIZE]
-		else:
-			if x in oldDigest:
-# DeepCopy because we might not have a unique reference.
-mydigests[x] = copy.deepcopy(oldDigest[x])
-mysize   = copy.deepcopy(oldDigest[x][size])
-			else:
-print !!! We have a source URI, but no file...
-print !!! File:,myfile
-return None
-		if mydigests[x].has_key(size) and (mydigests[x][size] != mysize):
-			raise portage_exception.DigestException, Size mismatch during checksums
-		mydigests[x][size] = copy.deepcopy(mysize)
-	return mydigests
-def digestCreateLines(filelist, mydict):
-	mylines = []
-	mydigests = copy.deepcopy(mydict)
-	for myarchive in filelist:
-		mysize = mydigests[myarchive][size]
-		if len(mydigests[myarchive]) == 0:
-			raise portage_exception.DigestException, No generate digest for '%(file)s' % {file:myarchive}
-		for sumName in mydigests[myarchive].keys():
-			if sumName not in portage_checksum.get_valid_checksum_keys():
-			mysum = mydigests[myarchive][sumName]
-			myline  = sumName[:]
-			myline +=  +mysum
-			myline +=  +myarchive
-			myline +=  +str(mysize)
-			mylines.append(myline)
-	return mylines
-def digestgen(myarchives,mysettings,overwrite=1,manifestonly=0):
+def digestgen(myarchives,mysettings,db=None,overwrite=1,manifestonly=0):
 	generates digest file if missing.  Assumes all files are available.	If
-	overwrite=0, the digest will only be created if it doesn't already exist.
-	# archive files
-	basedir=mysettings[DISTDIR]+/
-	digestfn=mysettings[FILESDIR]+/digest-+mysettings[PF]
-	# portage files -- p(ortagefiles)basedir
-	pbasedir=mysettings[O]+/
-	manifestfn=pbasedir+Manifest
-	if not manifestonly:
-		if not os.path.isdir(mysettings[FILESDIR]):
-			os.makedirs(mysettings[FILESDIR])
-		mycvstree=cvstree.getentries(pbasedir, recursive=1)
-		if (cvs in features) and os.path.exists(pbasedir+/CVS):
-			if not cvstree.isadded(mycvstree,files):
-if autoaddcvs in features:
-	print  Auto-adding files/ dir to CVS...
-	spawn(cd +pbasedir+; cvs 

[gentoo-portage-dev] [rfc] variable naming for marking binaries as QA ignorable

2006-03-03 Thread Mike Frysinger
so we've found some cases where a package installs objects that either need to 
be ignored by some of the scanelf checks ...

first off, we have kernel binary objects that a package installs (the h*modem 
packages do this), so they should not be subjected to the exec stack scans

next up is the slmodem package ... this puppy is partly binary only and we 
have no access to the source code and upstream is dead ... one of the 
pre-built binary objects has an exec stack enabled via gcc (meaning it's a 
legit use of exec stack) ... so we need to skip that

what this e-mail is about is naming convention ... i'm thinking that an ebuild 
sets up a variable with a list of relative paths to $D of files that should 
be skipped for various checks ... so with slmodem, we'd have like:
QA_EXEC_STACK=usr/sbin/slmodemd usr/sbin/slmodem_test

if, in the future, we need to add an ignore list for TEXTRELs, we'd use 
-- mailing list