Re: [gentoo-dev] RFC: put new additions along with removals in GWN
On Sat, 28 Oct 2006 17:09:10 -0400 Alec Warner <[EMAIL PROTECTED]> wrote: > It doesn't cover updates/. I don't think that corner case is easily > covered. Ehm, if you have a list of CP entries that were added/removed and access to the update files it shouldn't be all that complicated. Just made up pseudocode: for x in $raw_added_cps; do grep "^move [^ ]* $x\$" $current_update_file || $real_added_cps="$real_added_cps $x" done Hardest part is probably to select $current_update_file correctly. Marius -- Public Key at http://www.genone.de/info/gpg-key.pub In the beginning, there was nothing. And God said, 'Let there be Light.' And there was still nothing, but you could see a bit better. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
Yuri Vasilevski wrote: Hi, On Sat, 28 Oct 2006 19:58:09 +0100 Ciaran McCreesh <[EMAIL PROTECTED]> wrote: On Sat, 28 Oct 2006 20:42:14 +0200 Paul de Vrieze <[EMAIL PROTECTED]> wrote: | > Someone needs to implement the foo; however. | | (Not on cvs, but on a normal tree, but maybe works on cvs. There is a | sanity check by checking that a dir contains at least 1 ebuild) If the foo were that simple, someone would have done it long ago. It needs to take into account updates/. I have sent my script directly to wolf31o2 some days ago thinking that the exact technical details can considered spam for the majority subscribed on the list, but for completeness I'm attaching in in this e-mail. It works comparing two different portage snapshots that it can also download. Best, Yuri. I was planning to have this cron'd on lark (cvs.gentoo.org) and run against the cvs history file. #!/usr/bin/env python # Authored by Alec Warner <[EMAIL PROTECTED]> # Released under the GPL Version 2 # Copyright Gentoo Foundation 2006 # Changelog: Initial release 2006/10/27 doc = """ # Purpose: This script analyzes the cvs history file in an attempt to locate package # additions and removals. It takes 3 arguments; two of which are optional. It needs # the path to the history file to read. If a start_date is not provided it will read # the entire file and match any addition/removal. If you provide a start date it will # only match thins that are after that start_date. If you provide an end date you can # find matches over date ranges. If an end date is not provided it defaults to now() """ import sys, os, re, time, datetime new_package = re.compile("^A(.*)\|.*gentoo-x86\/(.*)\/(.*)\|.*\|ChangeLog$") removed_package = re.compile("^R(.*)\|.*gentoo-x86\/(.*)\/(.*)\|.*\|ChangeLog$") class record(object): def __init__(self, who, date, cp): """ Who is a string date is a unix timestamp cp is a category/package """ self.who = who self.date = datetime.datetime.fromtimestamp( date ) self.package = cp def __str__( self ): return "Package %s was removed by %s on %s" % (self.package, self.who, self.date) def main(): if (len(sys.argv) < 2): usage() sys.exit(1) args = sys.argv[1:] history_file = args[0] if len(args) >= 2: start_date = args[1] start_date = time.strptime( start_date, "%d/%m/%Y") start_date = time.mktime( start_date ) else: start_date = 0 # no real start date then. if len(args) >= 3: end_date = args[2] end_date = time.strptime( end_date, "%d/%m/%Y") end_date = time.mktime( end_date ) else: end_date = time.time() try: lines = open( history_file ).readlines() except (IOError, OSError), e: print "Failed to open History file!" raise e removals = [] adds = [] for line in lines: match = new_package.match( line ) if match: t = match.groups()[0] split = t.split("|") t = split[0] who = split[1] try: t = int(t, 16) except e: print "Failed to convert hex timestamp to integer" raise e if t < end_date and t > start_date: rec = record( who, t, match.groups()[1] + "/" + match.groups()[2] ) adds.append( rec ) continue else: continue # date out of range match = removed_package.match( line ) if match: t = match.groups()[0] split = t.split("|") t = split[0] who = split[1] try: t = int(t, 16) except e: print "Failed to convert hex timestamp to integer" raise e if t < end_date and t > start_date: rec = record( who, t, match.groups()[1] + "/" + match.groups()[2] ) removals.append( rec ) continue else: continue # date out of range print "Removed Packages:" for pkg in removals: print pkg print "Added Packages:" for pkg in adds: print pkg print print "Done." def usage(): print sys.argv[0] + " [end date]" print "start date should be in dd/mm/ format" print "end date is not required and defaults to 'now'" print doc if __name__ == "__main__": main()
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
Hi, On Sat, 28 Oct 2006 19:58:09 +0100 Ciaran McCreesh <[EMAIL PROTECTED]> wrote: > On Sat, 28 Oct 2006 20:42:14 +0200 Paul de Vrieze <[EMAIL PROTECTED]> > wrote: > | > Someone needs to implement the foo; however. > | > | (Not on cvs, but on a normal tree, but maybe works on cvs. There is > a | sanity check by checking that a dir contains at least 1 ebuild) > > If the foo were that simple, someone would have done it long ago. It > needs to take into account updates/. I have sent my script directly to wolf31o2 some days ago thinking that the exact technical details can considered spam for the majority subscribed on the list, but for completeness I'm attaching in in this e-mail. It works comparing two different portage snapshots that it can also download. Best, Yuri. gen_newpkg_table.sh Description: application/shellscript
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
Paul de Vrieze wrote: On Thursday 26 October 2006 00:57, Alec Warner wrote: Caleb Cushing wrote: reporting additions of new programs aren't feasible? or are you referring to version updates and package bumps and such Reporting removals will be done by treecleaners once I have it implemented. Reporting additions may require some cvs foo on lark; such as new directories in ${PORTDIR}/$CAT/ Someone needs to implement the foo; however. (Not on cvs, but on a normal tree, but maybe works on cvs. There is a sanity check by checking that a dir contains at least 1 ebuild) = start = OLDPKGS=/var/cache/gwn/oldpkgs NEWPKGS=`mktemp -t` find /usr/portage -mindepth 2 -maxdepth 2 |while read in do EB=`echo $in/*.ebuild` if [ -n "$EB" ]; then echo $in fi done >$NEWPKGS #These are new packages cat $OLDPKGS $OLDPKGS $NEWPKGS|sort |uniq -u #Old packages can be retrieved almost similarly: #cat $OLDPKGS $NEWPKGS $NEWPKGS|sort |uniq -u mv $NEWPKGS $OLDPKGS = end = Paul I've got some foo that can run on lark and figure out new and removed packages; as well as do it given date ranges. It uses the cvs history file. My foo basically looks at new and removed Changelogs in gentoo-x86. It excludes profiles/ (there are Changelogs in there). I'll post it a bit later; I have a few bits of it to fix. It doesn't cover updates/. I don't think that corner case is easily covered. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
On Sat, 28 Oct 2006 20:42:14 +0200 Paul de Vrieze <[EMAIL PROTECTED]> wrote: | > Someone needs to implement the foo; however. | | (Not on cvs, but on a normal tree, but maybe works on cvs. There is a | sanity check by checking that a dir contains at least 1 ebuild) If the foo were that simple, someone would have done it long ago. It needs to take into account updates/. -- Ciaran McCreesh Mail: ciaranm at ciaranm.org Web : http://ciaranm.org/ as-needed is broken : http://ciaranm.org/show_post.pl?post_id=13 signature.asc Description: PGP signature
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
On Thursday 26 October 2006 00:57, Alec Warner wrote: > Caleb Cushing wrote: > > reporting additions of new programs aren't feasible? or are you > > referring to version updates and package bumps and such > > Reporting removals will be done by treecleaners once I have it implemented. > > Reporting additions may require some cvs foo on lark; such as new > directories in ${PORTDIR}/$CAT/ > > Someone needs to implement the foo; however. (Not on cvs, but on a normal tree, but maybe works on cvs. There is a sanity check by checking that a dir contains at least 1 ebuild) = start = OLDPKGS=/var/cache/gwn/oldpkgs NEWPKGS=`mktemp -t` find /usr/portage -mindepth 2 -maxdepth 2 |while read in do EB=`echo $in/*.ebuild` if [ -n "$EB" ]; then echo $in fi done >$NEWPKGS #These are new packages cat $OLDPKGS $OLDPKGS $NEWPKGS|sort |uniq -u #Old packages can be retrieved almost similarly: #cat $OLDPKGS $NEWPKGS $NEWPKGS|sort |uniq -u mv $NEWPKGS $OLDPKGS = end = Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgp2RvKjuIfSk.pgp Description: PGP signature
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
Caleb Cushing wrote: reporting additions of new programs aren't feasible? or are you referring to version updates and package bumps and such Reporting removals will be done by treecleaners once I have it implemented. Reporting additions may require some cvs foo on lark; such as new directories in ${PORTDIR}/$CAT/ Someone needs to implement the foo; however. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
On Wed, 2006-10-25 at 16:25 -0400, Caleb Cushing wrote: > reporting additions of new programs aren't feasible? or are you > referring to version updates and package bumps and such None of it is feasible if I'm left to do it by hand. I have much better things to do (like actually add new packages) than try to keep track of what everyone else is doing. The GWN takes a significant amount of time as it is to put together. Any process that isn't automated is pretty much out of the question due to time constraints. -- Chris Gianelloni Release Engineering Strategic Lead Alpha/AMD64/x86 Architecture Teams Games Developer/Council Member/Foundation Trustee Gentoo Foundation signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
reporting additions of new programs aren't feasible? or are you referring to version updates and package bumps and such -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
On Wed, 2006-10-25 at 14:56 -0500, Yuri Vasilevski wrote: > If you think this is a good idea, I'd be glad to write some scripts for > this. If you write all the code, I'll run it, but I'm not taking my time to track down all of the additions. The current removals is done by hand by some volunteers for the GWN. There's nothing automatic about it. Someone else already suggested this to us, but I explained to him that unless someone had some way to automate it that it simply wasn't feasible to do by hand. -- Chris Gianelloni Release Engineering Strategic Lead Alpha/AMD64/x86 Architecture Teams Games Developer/Council Member/Foundation Trustee Gentoo Foundation signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] RFC: put new additions along with removals in GWN
On Wednesday 25 October 2006 21:56, Yuri Vasilevski wrote: > After seeing "Upcoming package removals", for couple of weeks now, in > GWN I'm beginning to think that I would like to see also a list of new > packages added to portage next to the list of packages to be removed. http://packages.gentoo.org/ -- Diego "Flameeyes" Pettenò - http://farragut.flameeyes.is-a-geek.org/ Gentoo/Alt lead, Gentoo/FreeBSD, Video, Sound, ALSA, PAM, KDE, CJK, Ruby ... pgpW9VJdUOkQ7.pgp Description: PGP signature
[gentoo-dev] RFC: put new additions along with removals in GWN
Hello, After seeing "Upcoming package removals", for couple of weeks now, in GWN I'm beginning to think that I would like to see also a list of new packages added to portage next to the list of packages to be removed. This will give a better view on the actual state of portage tree, as well as, will help people (like me) to find potentially useful programs that we didn't knew existed (as I'll certainly will check the packages that sound interesting to me from a quick look on that list). I think it'll be quite easy to implement by doing some CVS magic, or better yet a diff of the two portage snapshots corresponding to the days of this and the previous GWNs. If you think this is a good idea, I'd be glad to write some scripts for this. Best, Yuri. -- gentoo-dev@gentoo.org mailing list