Re: [gentoo-dev] [RFC] Treewide metadata.xml
Hello Danny Let's keep the metadata.xml in each package's directory in _CVS only_. Don't propagate them via rsync. Instead, use a script to compile all metadata.xml files into one central (XML) file. (This would probably need slight changes to the DTD). This file would then be placed into gentoo-portage/metadata/ and Portage,devs and users could easily parse it. Sounds good for me, it would also save some inodes and blocks on the hard drive. Implementing the compilation should be easy using XSLT. Greets, Michael -- Gentoo Linux Developer using m0n0wall | http://hansmi.ch/ pgpvoeFS99o2z.pgp Description: PGP signature
Re: [gentoo-dev] [RFC] Treewide metadata.xml
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Paul, Paul de Vrieze schrieb: The script is kindoff slow because of all the correction stuff, but it is workable. You can find it (and the accompanying xslt script) in: http://dev.gentoo.org/~pauldv/pkgList.tar.bz2 H, can you chmod a+r it please ? ;-) Danny - -- Danny van Dyk [EMAIL PROTECTED] Gentoo/AMD64 Project, Gentoo Scientific Project -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFClwGvaVNL8NrtU6IRAsfNAJ9GecWUtYgqEXn1k9KXZpny65f94wCdEDTO ywUZI7isC7/Az+lxADJVreA= =v0/i -END PGP SIGNATURE- -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] [RFC] Treewide metadata.xml
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Brian What's the gain, aside from implication of collapsing it into a single file? Honestly my only use for metadata.xml is looking up who I get to poke about fixing broken ebuilds... The gain is: ... that you portage people could use it for emerge -s instead of using a DESCRIPTION-cache. ... we don't need to find the metadata.xml file before parsing it. ... reducing the number of files in the (public) tree. Danny - -- Danny van Dyk [EMAIL PROTECTED] Gentoo/AMD64 Project, Gentoo Scientific Project -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) iD8DBQFClwjYaVNL8NrtU6IRAqK6AKCQ8VejfXaqU6y6swWXviE0wWfz+gCePlZc Ck/GoaeNxOlhE54dJ/slZQI= =Ooia -END PGP SIGNATURE- -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] [RFC] Treewide metadata.xml
On Fri, 27 May 2005 13:47:37 +0200 Danny van Dyk [EMAIL PROTECTED] wrote: | What's the gain, aside from implication of collapsing it into a | single file? Honestly my only use for metadata.xml is looking up | who I get to poke about fixing broken ebuilds... | The gain is: | ... that you portage people could use it for emerge -s instead of | using | a DESCRIPTION-cache. Eww! emerge having to parse XML? Bad! Bad! Bad! If you want a search tool, install one. -- Ciaran McCreesh : Gentoo Developer (Vim, Shell tools, Fluxbox, Cron) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm pgpMjWdZnM8tw.pgp Description: PGP signature
Re: [gentoo-dev] [RFC] Treewide metadata.xml
On Friday 27 May 2005 13:17, Danny van Dyk wrote: Hi Paul, Paul de Vrieze schrieb: The script is kindoff slow because of all the correction stuff, but it is workable. You can find it (and the accompanying xslt script) in: http://dev.gentoo.org/~pauldv/pkgList.tar.bz2 H, can you chmod a+r it please ? ;-) Done! Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgpaLp9MqOG0n.pgp Description: PGP signature
Re: [gentoo-dev] [RFC] Treewide metadata.xml
On Friday 27 May 2005 13:01, Simon Stelling wrote: Sounds good, if your script validates the per-package metadata.xml before transform it to the global one. It'd really suck if a single missing '' could screw the whole tree's metadata. This shouldn't be a problem, especially if you transform the information with XSLT. I definitively like the idea, it should speed up emerge -s enormously and save quite some inodes. A reduction of 8% of files sounds really good. My script does that (it was written before repoman and the server did xml parsing). Check it out ;-) Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgptDFbjAMhYK.pgp Description: PGP signature
Re: [gentoo-dev] [RFC] Treewide metadata.xml
On Fri, May 27, 2005 at 01:47:37PM +0200, Danny van Dyk wrote: Hi Brian What's the gain, aside from implication of collapsing it into a single file? Honestly my only use for metadata.xml is looking up who I get to poke about fixing broken ebuilds... The gain is: ... that you portage people could use it for emerge -s instead of using a DESCRIPTION-cache. 'you portage people' ? :) ... we don't need to find the metadata.xml file before parsing it. Portage's emerge -s doesn't use metadata.xml. Guessing you meant emerge -S (--searchDesc), but that too, doesn't use metadata.xml. So, a few implications in what you mean/are after then. 1) This global description cache would have to be duplicated, and recreated on cvs-rsync runs. Why? Unless you're padding extra bytes in the description cache, updates _will_ kill performance. Personally, I'm not much for it because there is a minimal window for cvs-rsync infra-side to get it's thing done, and this will jack up the runtime. 2) You're still doing entry by entry. Y'all are assuming having this data shoved into one file is going to make it quicker for reads (in reality, you're still reading 19000+ records, just your solution is out of a single file). This may be quicker due to syscall overhead, but I posit the drawbacks aren't worth it. 3) This complicates the hell out of cache updates, and still suffers the same issues eix/esearch suffer- namely that it's not sensitive to cache updates. If we make it sensitive to cache updates, you're looking at regen runtimes going through the roof (see #1 comment on updates). This is regardless of if it's a duplication approach or description is stored in it's own db outside of the normal flat_list cache files. 4) This proposal breaks the cache up into seperate chunks. That's the cache backends decision frankly, and _cannot_ be imposed onto the cache backend implementation from above. I moved eclass data into the cache backend in cvs head explicitly for the purpose of allowing the cache to be effectively standalone, and able to be bound to a remote tree. You force this change from above, it breaks the cache design (pure and simple), and ultimately isn't what you're after (see below). Frankly, any comments that this is going to make things faster are ignoring the existing code. Why is emerge -S so damned slow? Better question, why is it that a mysql cache backend _still_ is so damned slow on emerge -S? That should be hella fast compared to opening 19000 files, right? Because the current stable cache design allows *only* for individual record lookups. In other words, even with an rdbms implementation, it goes record by record. What is needed is a way to hand off to the cache hey you, give me all cpv's that have metadata that matches this criteria. Move the lookup/searching into the cache backend, which is already built into the cache refactoring I wrote for cvs head. If you want to collapse all of the description data into some faster lookup, fine, do so _strictly_ within that cache backend, and modify that class so that it has an appropriate get_matches lookup that's able to do a specific metadata lookup faster. People are free to disgaree mind you, but this talk of speed gains frankly seems to be missing the boat on how our cache actually works, let alone the issues with it. Collapsing all metadata down into a single file, yeah that would be nifty from the standpoint of less files/wasted space on fs's. Centralized DESCRIPTION cache implemented in xml? Eh... ~brian pgpROvbIkKbMs.pgp Description: PGP signature
Re: [gentoo-dev] baselayout-1.11.12-r2 request for testers
On Thu, 2005-05-26 at 12:08 -0400, Mike Frysinger wrote:
On Thursday 26 May 2005 11:12 am, Chris Gianelloni wrote:
On Thu, 2005-05-26 at 10:50 -0400, Mike Frysinger wrote:
On Thursday 26 May 2005 10:04 am, Chris Gianelloni wrote:
I'm asking because I will need to modify the livecd-tools scripts to
take into account a way to disable DHCP when either nodhcp or
nodetect are passed to the release media.
speaking of livecd updates, someone pointed out on a bug that we shouldnt
need to check CDBOOT anymore in the volume addon code
(lvm/lvm2/evms/raid/etc) ... the livecd should set RC_VOLUME_ORDER to
... maybe we can do this in the ebuild ?
What ebuild? baselayout?
yes, when USE=livecd
Where would the livecd set RC_VOLUME_ORDER?
it would reset $S/etc/conf.d/rc
Why should it?
instead of having to do [[ -z ${CDBOOT} ]] all the time ... the lvm/emvs/raid
have been removed from baselayout and placed into sep addon modules ... but
only modules listed in RC_VOLUME_ORDER will be automagically sourced by
baselayout at boot
so instead of checking $CDBOOT in all the modules, we could just have the
livecd say 'dont check volume addon modules'
Where would the livecd say this?
Also, what if we *want* the livecd to check the volume addon modules?
--
Chris Gianelloni
Release Engineering - Strategic Lead/QA Manager
Games - Developer
Gentoo Linux
signature.asc
Description: This is a digitally signed message part
Re: [gentoo-dev] virtual/eject ?
On Fri, 2005-05-27 at 02:02 +0200, Diego 'Flameeyes' Petten wrote: On Monday 23 May 2005 18:12, Diego 'Flameeyes' Petten wrote: This can't be done everytime, and hoping for, in future, having an universal eject (libcdio should allow that) I wanted to ask if is possible to add a virtual/eject provided by sys-apps/eject and sys-apps/eject-bsd so that they can be made dependencies as needed. News about this? We really need a solution to this. I'm just curious, but why exactly is a virtual needed? Why can't it be something like: !blah? ( sys-apps/eject ) blah? (sys-apps/eject-bsd ) A virtual is usually reserved for when it affects a large number of packages. If it affects one or two, then simply using some fun *DEPEND syntax usually covers the issue. -- Chris Gianelloni Release Engineering - Strategic Lead/QA Manager Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] baselayout-1.11.12-r2 request for testers
On Thu, 2005-05-26 at 22:15 -0500, Daniel Goller wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Mike Frysinger wrote: | On Thursday 26 May 2005 10:04 am, Chris Gianelloni wrote: | |I'm asking because I will need to modify the livecd-tools scripts to |take into account a way to disable DHCP when either nodhcp or |nodetect are passed to the release media. | | | speaking of livecd updates, someone pointed out on a bug that we shouldnt need | to check CDBOOT anymore in the volume addon code (lvm/lvm2/evms/raid/etc) ... | the livecd should set RC_VOLUME_ORDER to ... maybe we can do this in the | ebuild ? | -mike you mean so i will not have to use 'gentoo nolvm2' on the next livecd on my system? That had nothing to do with the init scripts, so these changes would no affect that in any way. All of the no* commands affect things that are specific to the livecd. -- Chris Gianelloni Release Engineering - Strategic Lead/QA Manager Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] New developer: Stuart Longland (Redhatter)
On Friday 27 May 2005 16:40, Tom Martin wrote: Stuart's beginnings with Linux come from his experiences with FreeBSD. Ehi that can be useful ;) Want to help with portage on FreeBSD? :P -- Diego Flameeyes Petten Gentoo Developer (Gentoo/FreeBSD, Video, Gentoo/AMD64) http://dev.gentoo.org/~flameeyes/ pgp7M40tupIQ2.pgp Description: PGP signature
Re: [gentoo-dev] virtual/eject ?
On Fri, 2005-05-27 at 15:45 +0200, Diego 'Flameeyes' Petten wrote: On Friday 27 May 2005 15:44, Chris Gianelloni wrote: I'm just curious, but why exactly is a virtual needed? Well there are nine packages (and probably there can be others) which needs eject, and the eject commands are going to be quite a few if there isn't an universal eject as every os has its own way to eject a CD-Rom. So we'll have eject for linux, eject-bsd for freebsd (and maybe openbsd, don't know), eject-osx (and maybe eject-darwin) and eventual eject-solaris if there will ever be a solaris port. Every kernel has its own syscalls, and using conditional deps for every kernel on more than a couple of packages will surely be something annoying for maintainers. OK... You sold me. So create a virtual/eject... =] -- Chris Gianelloni Release Engineering - Strategic Lead/QA Manager Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] baselayout-1.11.12-r2 request for testers
On Friday 27 May 2005 09:40 am, Chris Gianelloni wrote: Where would the livecd say this? all the code i referenced was from inside baselayout ... and the livecd wouldnt do it, when you emerge baselayout with USE=livecd, the config file would be setup Also, what if we *want* the livecd to check the volume addon modules? Not My Problem ? the volume code has *always* been disabled in baselayout for livecds, nothing is changing ... plus, the startup code is for people who have configured crap already ... you dont exactly have config files on a livecd for your volumes :p -mike -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] [RFC] Treewide metadata.xml
Brian Harring wrote: I definitively like the idea, it should speed up emerge -s enormously Unlikely... stable portage knows of metadata.xml *explicitly* in two places, repoman's commit code, and digest checking, neither of which come into play for an emerge -s. You'll remove one entry from the listdir returns for a package directory, per package directory, bout it. You're right, i mixed up description with longdescribtion. -- blubb Gentoo/AMD64 Developer http://www.blubb.li/ -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.11.12-r2 request for testers
On Fri, 2005-05-27 at 11:13 -0400, Mike Frysinger wrote: On Friday 27 May 2005 09:40 am, Chris Gianelloni wrote: Where would the livecd say this? all the code i referenced was from inside baselayout ... and the livecd wouldnt do it, when you emerge baselayout with USE=livecd, the config file would be setup OK... except we don't remerge baselayout anymore... it could be done, of course... Also, what if we *want* the livecd to check the volume addon modules? Not My Problem ? the volume code has *always* been disabled in baselayout for livecds, nothing is changing ... plus, the startup code is for people who have configured crap already ... you dont exactly have config files on a livecd for your volumes :p Makes sense to me... I guess this doesn't have anything to do with the volume scanning done on the livecd... I really don't understand what any of the lvm/evms junk does anyway... and I'm sure you understand it much better than I do, so I'll take your word for it... and if something breaks, I know where to send the bugs... *grin* -- Chris Gianelloni Release Engineering - Strategic Lead/QA Manager Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
[gentoo-dev] Re: New developer: Stuart Longland (Redhatter)
_ Stuart Longland (a.k.a Redhatter)
/\ ______ __| |__ __ __ Gentoo Linux/MIPS Cobalt and Docs
- ( ) \ / \ ; \(__ __)/ \ / \Developer
\// O _| / /\ \ | | | /\ | /\ |
/ / \ /__| / \ \ | | | \/ | \/ |
(___/ \/|_; |_| \_/ \__/ \__/ http://dev.gentoo.org/~redhatter
That's one cool sig! First time I've seen it.
--
Duncan - List replies preferred. No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master. Richard Stallman in
http://www.linuxdevcenter.com/pub/a/linux/2004/12/22/rms_interview.html
--
gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] baselayout-1.11.12-r2 request for testers
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Chris Gianelloni wrote: | On Thu, 2005-05-26 at 22:15 -0500, Daniel Goller wrote: | |-BEGIN PGP SIGNED MESSAGE- |Hash: SHA1 | |Mike Frysinger wrote: || On Thursday 26 May 2005 10:04 am, Chris Gianelloni wrote: || ||I'm asking because I will need to modify the livecd-tools scripts to ||take into account a way to disable DHCP when either nodhcp or ||nodetect are passed to the release media. || || || speaking of livecd updates, someone pointed out on a bug that we |shouldnt need || to check CDBOOT anymore in the volume addon code |(lvm/lvm2/evms/raid/etc) ... || the livecd should set RC_VOLUME_ORDER to ... maybe we can do this |in the || ebuild ? || -mike |you mean so i will not have to use 'gentoo nolvm2' on the next livecd on |my system? | | | That had nothing to do with the init scripts, so these changes would no | affect that in any way. All of the no* commands affect things that are | specific to the livecd. | then i'll have to make sure to test as many of the next livecd release candidates to see if it still requires me to specify that or not -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFCmJZSUpKYMelfdYERAidpAJ4iy8x1IFFijFBSWnVAz0g7r90lmQCfch5C 8Xbt4qqCbmMBs2UOKefq3+o= =6mRj -END PGP SIGNATURE- -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Re: New developer: Stuart Longland (Redhatter)
Duncan wrote:
_ Stuart Longland (a.k.a Redhatter)
/\ ______ __| |__ __ __ Gentoo Linux/MIPS Cobalt and Docs
- ( ) \ / \ ; \(__ __)/ \ / \Developer
\// O _| / /\ \ | | | /\ | /\ |
/ / \ /__| / \ \ | | | \/ | \/ |
(___/ \/|_; |_| \_/ \__/ \__/ http://dev.gentoo.org/~redhatter
That's one cool sig! First time I've seen it.
Thanks...
Probably because it's the first time I've used it. :-P
Yes... I spent about 15 mins in gVim comming up with one for Gentoo --
I didn't see it approrpiate advertising Atomic Linux whilst using my
Gentoo address. :-D
--
_ Stuart Longland (a.k.a Redhatter)
/\ ______ __| |__ __ __ Gentoo Linux/MIPS Cobalt and Docs
- ( ) \ / \ ; \(__ __)/ \ / \Developer
\// O _| / /\ \ | | | /\ | /\ |
/ / \ /__| / \ \ | | | \/ | \/ |
(___/ \/|_; |_| \_/ \__/ \__/ http://dev.gentoo.org/~redhatter
signature.asc
Description: OpenPGP digital signature
Re: [gentoo-dev] [RFC] Treewide metadata.xml
I like the xml generation pass but i have a few questions: does the xml-publish transaction lend a greater window of opportunity for mirror rsync de-sync? does it make sense for digest info be sucked up into the xml pass, to be superceded by the legacy files where present ala ebuild x-y/z.ebuild digest should the xml be digested? On May 27, 2005, at 8:51 AM, Simon Stelling wrote: Brian Harring wrote: I definitively like the idea, it should speed up emerge -s enormously Unlikely... stable portage knows of metadata.xml *explicitly* in two places, repoman's commit code, and digest checking, neither of which -- gentoo-dev@gentoo.org mailing list
