Re: [gentoo-portage-dev] Performance tuning and parallelisation
On 26/08/2021 17:38, Marco Sirabella wrote: > - > > Hi Ed, > > I’ve taken a dabble at trying to track down portage’s bottlenecks (and have > stopped for the time > being at solving them :/ ) > > Can anyone give me a leg up on how I could benchmark this further and > look for the hotspot? > Perhaps someone understand the architecture of this point more intimately > and could point at > whether there are opportunities to do some of the processing on mass, > rather than per file? > > From my notes at the timem, it looks like yappi > <https://pypi.org/project/yappi/> worked a bit > better than python’s built in cProfile for me because it properly dove into > async calls. I used > snakeviz <https://jiffyclub.github.io/snakeviz/> for visualizing the profile > results. > > I was taking a look at depclean, but I found similarly that a lot of > duplicate process was being > done due to encapsulated abstractions not being able to communicate that the > same thing was being > done multiple times eg removing each package processes a massive json > structure for each package > removed, although I opted to work on the more-understandable unicode > conversions. > > My stalled progress can be found here: #700 > <https://github.com/gentoo/portage/pull/700>. Lost the > drive to continue for now unfortunately :< > > Good luck! Looking forward to your optimizations > > – Marco Sirabella > Hi All, thanks for the replies. Wow, Marco, that patch touches a lot of stuff...! OK, I will start by trying to get the profilers going and work from there... (Alec, to avoid replying separately: Thanks for your notes. Yes, I am not clear which of the install/merge phases specifically is the culprit, but it feels like something in that area is "slow", especially when run under qemu user mode. I think using qmerge won't work for my build script, but great idea to use it for benchmarking to narrow things down - thanks) Thanks all Ed W
[gentoo-portage-dev] Performance tuning and parallelisation
Hi All Consider this a tentative first email to test the water, but I have started to look at performance of particularly the install phase of the emerge utility and I could use some guidance on where to go next Firstly, to define the "problem": I have found gentoo to be a great base for building custom distributions and I use it to build a small embedded distro which runs on a couple of different architectures. (Essentially just a "ROOT=/something emerge $some_packages"). However, I use some packaging around binpackages to avoid uncessary rebuilds, and this highlights that "building" a complete install using only binary packages rarely gets over a load of 1. Can we do better than this? Seems to be highly serialised on the install phase of copying the files to the disk? (Note I use parallel build and parallel-install flags, plus --jobs=N. If there is code to compile then load will shoot up, but simply installing binpackages struggles to get the load over about 0.7-1.1, so presumably single threaded in all parts?) Now, this is particularly noticeable where I cheated to build my arm install and just used qemu user-mode on an amd64 host (rather than using cross-compile). Here it's very noticeable that the install/merge phase of the build is consuming much/most of the install time. eg, random example (under qemu user mode) # time ROOT=/tmp/timetest emerge -1k --nodeps openssl >>> Emerging binary (1 of 1) dev-libs/openssl-1.1.1k-r1::gentoo for >>> /tmp/timetest/ ... real 0m30.145s user 0m29.066s sys 0m1.685s Running the same on the native host is about 5-6sec, (and I find this ratio fairly consistent for qemu usermode, about 5-6x slower than native) If I pick another package with fewer files, then I will see this 5-6 secs drop, suggesting (without offering proof) that the bulk of the time here is some "per file" processing. Note this machine is a 12 core AMD ryzen 3900x with SSDs that bench around the 4GB/s+. So really 5-6 seconds to install a few files is relatively "slow". Random benchmark on this machine might be that I can backup 4.5GB of chroot with tar+zstd in about 4 seconds. So the question is: I assume that further parallelisation of the install phase will be difficult, therefore the low hanging fruit here seems to be the install/merge phase and why there seems to be quite a bit of CPU "per file installed"? Can anyone give me a leg up on how I could benchmark this further and look for the hotspot? Perhaps someone understand the architecture of this point more intimately and could point at whether there are opportunities to do some of the processing on mass, rather than per file? I'm not really a python guru, but interested to poke further to see where the time is going. Many thanks Ed W
Re: [gentoo-dev] rfc: bash arrays in OpenRC
On 12/07/2011 16:54, Mike Frysinger wrote: baselayout-1 vs openrc isnt the issue. it's bash array support vs flat strings. the former is supported in baselayout-1, and should be supported in openrc. William is talking about dropping support from openrc and i've laid out the way to get from here to there. -mike The original vision for OpenRC was that it should be posix compliant. Please don't revert to using bash features because it creates a dependency on bash that then makes it hard to support small platforms such as busybox At present OpenRC works very nicely on a minimal (embedded) system using uclibc/busybox in a couple of MB. Its also decently fast and oldnet actually seems to work quite nicely for a headless, but heavily dynamic environment like a router with USB things getting plugged/unplugged Thanks for listening... Ed W
Re: [gentoo-dev] rfc: bash arrays in OpenRC
On 13/07/2011 19:19, Mike Frysinger wrote: On Wednesday, July 13, 2011 13:36:29 Ed W wrote: On 12/07/2011 16:54, Mike Frysinger wrote: baselayout-1 vs openrc isnt the issue. it's bash array support vs flat strings. the former is supported in baselayout-1, and should be supported in openrc. William is talking about dropping support from openrc and i've laid out the way to get from here to there. The original vision for OpenRC was that it should be posix compliant. Please don't revert to using bash features because it creates a dependency on bash that then makes it hard to support small platforms such as busybox please read the thread (again?). no one is talking about requiring bash. -mike Fair enough. I misunderstood we should support bash arrays as meaning bash will become required... Openrc works very nicely - many thanks to all who built and maintain it! Ed W
Re: [gentoo-dev] avoiding urgent stabilizations
and build time dependencies separately? eg you might need gzip to build something, but not to run it? So far I think you could easily convert your briegel recipes to a simple ebuild script and run them side by side? It's actually such an easy thing to do, to build a new stage2/3/4 that it's a shame folks don't do it more! Good luck Ed W
Re: [gentoo-dev] avoiding urgent stabilizations
a bit I think you will find some recipes for very minimal VMs using this method where the base VM is a very minimal install... Is there some way to drop at least parts of the standard system set, so eg. portage, python, gcc, etc, etc get unmerged by --depclean if nobody else (in world set) doesn't explicitly require them ? You are almost thinking about it all wrong. (There is no spoon...) This is gentoo, so at this more advanced level, stop thinking about standard system set and instead free your mind to start with nothing. Go read the old bootstrap from stage 1 instructions, plus the TinyGentoo pages and you can quickly see that Catalyst builds your working installation by starting from a working installation, creating an empty directory, adding some minimal packages to that directory and building up from there. So absolutely nothing stops you from just starting with an empty directory and just emerging a few basic packages into it (couple MB) and then chrooting into it and having some fun... There is *no* minimal package set, you can install whatever you want (as long as it boots). Largely the portage dependency tracker will help you pull in the minimal needed dependencies, but beware that system packages arent generally explicitly tracked so you may stumble across some deps when you are going really basic and omiting standard system packages (just use common sense: it should be fairly obvious if an application requires a compiler and you didn't install one then you have a conflict of interest...) Have another look at gentoo! I definitely believe that it's flexibility to build you highly customised packages, plus strong templating of those packages, plus decent ability to distribute binaries of the end result is a very strong combo! Better binary support is really the only thing missing here, but it's pretty adequate as it stands! Good luck Ed W
Re: [gentoo-dev] avoiding urgent stabilizations
On 21/02/2011 00:11, Enrico Weigelt wrote: * Markos Chandrashwoar...@gentoo.org schrieb: My suggestion, as I said to fosdem, is to freeze, or take a snapshot if you like, of the current tree, stabilize what you need to stabilize, test the whole tree ( at least compile wise ) for a couple of weeks and then replace the existing stable tree. hmm, would it make sense to add a new masking for the testing tree, so users could decide which stability grade vs they wish ? or perhaps use overlays for that ? For example, I'd like to have the critical packages (everything that's needed to bootup and do basic remote maintenance) from the new frozen-stable tree, but other things should stay as they are. Perhaps this is an argument for a git based portage tree? Master can stay as the current status quo and anyone who wants to can maintain a branch or fork which points to a slightly different subset of the tree? I doubt we actually have the capacity to make this work, but it would at least in theory be cool to have a (weekly/monthly) branch which gets cut, run through a tinderbox in various forms and then pushed? Or if someone wants to maintain a redhat style antique set of packages where the tree is largely held back to 2005 state with only bug fixes and essential packages bumped? Just thinking... Ed W
Re: [gentoo-dev] avoiding urgent stabilizations
Hi I'm starting to put together a portage/stable server configuration for a large number of gentoo VM's that will eventually be hosted on a VMware ESX 4.1U1 cluster - with the goal of limiting major changes to once/year and otherwise only applying security/minimum necessary updates. I doubt it will be easy but I'm doing my best at it :) This sounds very interesting. I haven't yet plugged through your blog, but just to chime in: I maintain a, likely much smaller, number of VMs using linux vservers. The approach here is to almost cut each machine down to a chroot that runs only one (or thereabouts) interesting service. To do this I have found customised portage profiles to be the under-plugged secret since they allow you to basically push a set of packages which should be installed and control per type of vm use flags and package keywords (eg I have www-nginx, www-apache, mail, proxy, mysql, ftp, etc profiles). Additionally I have a small overlay of local ebuilds that sit in the same tree Up until now I haven't really made any effort to sync this whole tree across multiple physical machines and it's a bit of an ad-hoc process. Using something like git would probably be perfect The still missing step is configuration management across the machine types, eg I want to upgrade all my Apache-WWW class machines and merge in all changes in /etc in a certain way... At the moment I just run dispatch-conf across all machines, but it can be quite boring merging 20 instances of sshd.conf... Seems like Puppet/Chef could be a solution here, but the step up and investment to make it work seems pretty large? It does appear like managing large numbers of virtual machines is one are that gentoo could score very well? Interested to see any chatter on how others solve this problem, or any general advocacy? Probably we should start a new thread though... Regards Ed W
Re: [gentoo-dev] [Gentoo Phoenix] an official Gentoo wiki
Hi Show me a wiki that makes it easy to create tables, for example, compare RadeonProgram from the x.org wiki: http://www.x.org/wiki/RadeonProgram?action=edit ||-2 style=text-align: center; background-color: #66 '''Native''' ||style=text-align: center; background-color: #66 '''R100''' ||style=text-align: center; background-color: #66 '''R200''' ||style=text-align: center; background-color: #66 '''R300''' ||style=text-align: center; background-color: #66 '''R400''' ||style=text-align: center; background-color: #66 '''RS690''' ||style=text-align: center; background-color: #66 '''R500''' ||style=text-align: center; background-color: #66 '''R600''' ||style=text-align: center; background-color: #66 '''R700''' || . . . that's one line of cells. One. Ugly. Compare it to: http://www.gentoo.org/doc/en/xml-guide.xml#doc_chap5_pre1 table tr thFoo/th thBar/th /tr tr tiThis is an example for indentation/ti timore stuff/ti /tr /table Which is easier to read and instantly comprehend? Yes, but the wiki layout is badly written, you should be comparing it to: || '''Native''' || '''R100''' || '''R200''' || '''R300''' || '''R400''' || '''RS690''' || '''R500''' || '''R600''' || '''R700''' || I think this reads ok? In fact with a bit of thought from some premade styles even the ''' bit should go? By moving to a wiki, you'll lose a huge percentage of what GuideXML can do, in exchange for quicker and easier editing and creation of docs, though neither of these have been qualified. I think this summarises the basic tradeoff - you trade editing speed for simplicity of syntax and readability. Clearly as your example shows it's possible to write complicated looking stuff in any syntax though, but in general wiki's win where the content is most important and styling is done separately using CSS (a bit like guideXML really) As some others on this list have mentioned, wiki syntax is downright ugly and simply not as consistent or readable as plain ol' XML or HTML. I think this is a point of contention. Certainly it was a design goal for the wiki syntax to be simple and easily readable, but one man's simple is another mans XML... From what I've seen, the biggest objection to GuideXML is folks don't want to take the time to learn a few tags. Well, you'll have to learn tags and syntax for either system, so pick your poison. I've yet to see a wiki that even has as much sense as HTML, which is pretty low on the totem pole of consistency. Actually I think GuideXML is excellent - if there is a wiki style engine which allows you to post in GuideXML then we should do it? I think it's not an objection to the GuideXML which is the problem, but creating a system which can be edited quickly and easily in a granular fashion. Eg imagine all the guideXML docs being in a git repo with open access to pull/push changes - you could build a web engine around that which rebuilds the web pages interactively as people push edits and this would be cool! In the meantime wiki's are just trying to solve the same goal of easy edits with small granularity of edits However, I love the idea of a wiki based around git using GuideXML! (probably it kind of works like this right now - I think it's the access control which is the secret sauce...) I ain't out to stop ya'll from using a wiki. I do agree that they have some advantages. However, I will point out how limited wikis are. They're not a magic bullet that will solve all our problems. Definitely. Good luck Ed W
Re: [gentoo-dev] [Gentoo Phoenix] an official Gentoo wiki
On 05/04/2010 03:43, Ben de Groot wrote: On 5 April 2010 03:13, Joshua Saddlernightmo...@gentoo.org wrote: Let the renderer take care of the final rendering, as really, tags and markup are all arbitrary. What should matter is how it appears in your webbrowser, since that'll vary from the source view anyways. So why are you such a staunch defender of GuideXML then? If markup is arbitrary really, then why not allow people to use what is convenient? I do think arguing about the syntax is the wrong target (as I think you agree above). The magic of a wiki is: - Focus on the text and not on the formatting - Goal of simplicity to bang in a bunch of content without needing to worry about formatting - Granularity of edits, eg edit a single word and not get overwritten by another change which edits a different single word - Web based editing from any machine without installing stuff - Extremely low barriers to contributing I think these goals could be satisfied by a decent system around GuideXML as much as from an arbitrary Wiki engine? The real magic is in getting lots of users to start contributing and that largely comes from having very few barriers to contributing. If you remember the original Wikipedia it involved requiring to pass some tests to become a contributor and it was basically a closed editor system. It failed dismally... The revamped wikipedia allowed anyone to edit and whilst we can debate the merits of the final product, it's certainly been popular. So I claim that low barriers to entry and ease of editing is the real target - the markup is important, but definitely secondary to the engine itself Good luck Ed W
Re: [gentoo-dev] RFC: News item for removal of 2008.0 and old hardened profiles
On 05/03/2010 18:54, Mike Frysinger wrote: On Thursday 04 March 2010 11:08:52 Samuli Suominen wrote: Attached you can find the news item for up coming profile cleanup. do profiles really need to be culled this often ? we used to let the tail run longer and no one complained. it's easier to upgrade an old system when the current profile is sane than having to create one from scratch because the new profile uses features not actively available. i.e. let's set a limit of like 3 years on profiles -mike I think I have mostly upgraded my machines, but I completely agree - I sometimes let some old virtual machines sit unbooted for a year and then suddenly want to use them and bring them up to date and occasionally this can be a right old pain in the derrier... Surely 4 months is too short a warning for profiles which can easily be simply left to rot instead?
[gentoo-dev] OpenRC + Bridge + Tap not working as expected?
Hi, I am using openrc 0.4.2 and trying to setup a bridge for use with openvpn. Starting the bridge (/etc/init.d/br0 start) doesn't seem to be correctly bringing up the dependent interfaces (should it?) When I try and bring up my bridge I get only this: # /etc/init.d/br0 start * Bringing up interface br0 * Destroying bridge br0... * Removing port eth0... [ ok ] [ ok ] * Creating bridge br0... * Adding ports to br0 * eth0... [ ok ] * tap0...SIOCGIFFLAGS: No such device SIOCGIFFLAGS: No such device interface tap0 does not exist! SIOCGIFFLAGS: No such device [ !! ] * ERROR: net.br0 failed to start then I check: #ifconfig eth0 Link encap:Ethernet HWaddr 00:1b:21:19:09:8d inet6 addr: fe80::21b:21ff:fe19:98d/64 Scope:Link UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1 RX packets:9717 errors:4 dropped:0 overruns:0 frame:2 TX packets:5923 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1326386 (1.2 MiB) TX bytes:1079556 (1.0 MiB) Base address:0xa000 Memory:e902-e904 loLink encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:4243 errors:0 dropped:0 overruns:0 frame:0 TX packets:4243 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:570436 (557.0 KiB) TX bytes:570436 (557.0 KiB) (Curious eth0 is up... Lets try and stop it) #/etc/init.d/net.eth0 stop * WARNING: net.eth0 is already stopped If I simply start the tap0 and eth0 interfaces deliberately first then everything runs fine. My understanding though is that the bridge should bring up dependent interfaces? Config in /etc/conf.d/net config_eth0=( null ) tuntap_tap0=tap config_tap0=( 0.0.0.0 promisc ) bridge_br0=eth0 tap0 config_br0=( 192.168.105.4/24 ) routes_br0=( default via 192.168.105.254 ) # Make the bridge depend on tap0 / eth0 RC_NEED_br0=net.eth0 net.tap0 Any suggestions on how to fix? Thanks Ed W
[gentoo-dev] OT: OpenRC - thanks Roy
Hi, bit off topic, but there hasn't been much discussion of the OpenRC progress recently - I just wanted to say a bit thanks to Roy for his work on this - it's been working well on my vservers for a good few months now, but I hadn't realised just how fast it was until I converted my mythtv box to OpenRC... At first I thought my splash screen was busted, but no it booted the whole machine and had the window manager running in about the time is used to take the splash screen to start to spin... Absolutely stunning! Thanks Roy and hope we can look forward to this becoming stable in the near future? Ed W -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Baselayout-2 progress?
Hmm, all interesting stuff You mention in the notes also that openrc has some kind of keepalive function which can restart crashing services. Can point me towards how that works (assuming it needs some kind of config?) I haven't had any time yet to try this on a test machine, but interested to give it a whirl on my embedded (busybox+uclibc) target... Cheers Ed W -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Monthly Gentoo Council Reminder for March
At one time there were some apps which reported back usage from people's systems and showed package versions in use? Now, whilst this in itself is not an indication of package quality or bug-freeness. Perhaps it would be an interesting datastream to assist in deciding whether to mark a package stable or not? An incremental improvement to such a plan might be to consider how to split the data into high quality devs and testers running stuff stuff, keen users and dev boxes (which might be crashing and are of low quality). Sure it's a fairly low quality data source, but it might give a bit of confidence to take a punt unmasking a package if you can see that others are using it actively? Just my 2p Ed W -- gentoo-dev@lists.gentoo.org mailing list
[gentoo-dev] Baselayout-2 progress?
Is it dead..? Is anyone still working on it? I have had a lot of success using it for linux vservers and in an embedded build. Would really hate to see it stall though...? What are the big picture items still missing? Seems that it's close to becoming a stable upgrade? I have filed a few minor bugs against it (some more to come) - is there anything I can do to help progress development further? Cheers Ed W -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Baselayout-2 progress?
Hi baselayout-2 was renamed to openrc when Roy left Gentoo as an official dev. Answering my own question (for the record). I found some explanation here: http://lycos.dropcode.net/gregarius/author.php?author=Roy_Marples__uberlord_ Does Roy hang out here? Roy: Is this intended to be a baselayout replacement? How likely is this to be on-track to become a gentoo official baselayout? Do you (try to) support busybox and vserver environments? Don't know. Yes. Very. Yes Yes. Excellent - this is exciting to hear On the other hand since there still isn't a masked ebuild in portage (and I seem some notes on my on Roy's site) then I have to assume that in fact we are still a good way away from calling it a replacement and starting to push it out to users? Would it not make sense to start to snapshot some builds and push openrc out for testing? (Seems like a gentoo job rather than an upstream is the reason I ask here?) Cheers Ed W -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Baselayout-2 progress?
Alon Bar-Lev wrote: Check out OpenRC it is baselayout successor and works great! Funnily enough I came across this earlier today for different reasons. However, I hadn't realised that it was a full baselayout competitor? Does Roy hang out here? Roy: Is this intended to be a baselayout replacement? How likely is this to be on-track to become a gentoo official baselayout? Do you (try to) support busybox and vserver environments? Cheers Ed W -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] baselayout-2 LVM
Hi lvm2: 09 May 2007; Doug Goldstein [EMAIL PROTECTED] +files/lvm.rc, lvm2-2.02.17.ebuild: added baselayout-2 compatible init script from bug #175983 For lvm2 this was added without increasing the ebuild revision, but later there were some updates of lvm2, so all users of ~arch that are up to date should have this. OK, masked was the wrong word. Perhaps I should have said unstable or keyworded. Anyway, yes, installing that newer lvm2 fixed the problems. Probably worth a note somewhere in the upgrade docs anyway Cheers Ed W -- [EMAIL PROTECTED] mailing list
[gentoo-dev] baselayout-2 LVM
Is it *supposed* to NOT startup automatically out of the box?
Looks like there are some emails on this list about there being only
stubs to support LVM, etc. I found some startup functions in
rcscripts/addons, but nothing to actually call them at startup? There
is some stuff in conf.d/volumes which suggests that there is support for
changing the scan order, but again, couldn't spot anything which used it?
I simply copied /etc/init.d/device-mapper and tweaked it as shown below
and now LVM starts up at boot for me (note that I am using it on top of
md raid1 in case that's relevant). Perhaps this could be added to the
new baselayout unless there is a better solution?
Cheers
Ed W
#!/sbin/runscript
# Copyright 1999-2007 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: Exp $
depend() {
if [ -e /lib/librc.so ]; then
# on baselayout-1 this causes
# a dependency loop with checkroot (before *)
before checkroot
fi
}
start() {
if [ ! -e /lib/librc.so ]; then
eerror The ${SVCNAME} init script is written for
baselayout-2
eerror Please do not use it with baselayout-1
return 1
fi
. /lib/rcscripts/addons/lvm-start.sh
}
stop() {
if [ ! -e /lib/librc.so ]; then
eerror The ${SVCNAME} init script is written for
baselayout-2
eerror Please do not use it with baselayout-1
return 1
fi
. /lib/rcscripts/addons/lvm-stop.sh
}
--
[EMAIL PROTECTED] mailing list
Re: [gentoo-dev] baselayout-2 LVM
You will basically need to remerge sys-fs/lvm2 and sys-fs/device-mapper and Darn, sorry for the noise Didn't think to check the masked packages - however, there it is clear as day in the changelog... Thanks Ed W -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] baselayout-2 stablisation plans
Regarding init scripts. Init scripts should now be strictly bourne or POSIX shell. ie, no bashisms. bash init scripts will work, but ONLY if /bin/sh is bash. Shells as /bin/sh that I've tested and found to be working are bash dash busybox zsh FreeBSD sh Also, as there's no bashisms, that also means no bash arrays. I presume that this is hitting the /etc/init.d/util-vserver startup scripts /usr/lib/util-vserver/functions: line 778: `pkgmgmt.guessStyle': not a valid identifier Where to log a bug? What's the simplest way to temporarily workaround this? Cheers Ed W -- [EMAIL PROTECTED] mailing list
Re: [gentoo-dev] [Summary] tentative x86 arch team glep
| Is there any possibility that easier low quality contribution makes | the high quality contributions easier? Only to the extent that they get me to write better documentation :) | Look at wikipedia - it's amazing that such high quality work (in | general) can come from lightly peer review material with low barriers | to entry. | | Clearly not an appropriate model here, but I can't help wondering if | there is not another way... Well... Sometimes maintainer-wanted ebuilds are worked upon by multiple people. It happens, but not very often. I was pondering this last night. Whilst there is clearly no substitute for a high quality standard for x86, etc, it seems to me that we are missing a trick with all the maintainer wanted ebuilds which tend to be scattered around the web. It seems to me that perhaps it would be useful to have a centralised development area where stuff can gestate before making it into the testing pool that we have today. It could be argued that this exists and is called bugzilla, but I wonder if we can do better? What about adding another layer (or two) to the flags so that development ebuilds can be developed centrally to gentoo and hence available in portage, but lowering the barrier to entry. At the simplest this could be used to allow a non core developer to bump an ebuild to a new version in response to some release. It goes into the highly unstable section which shouldn't be seen by any normal person, yet at the same time makes it available to the kiddies who like to test the latest and greatest. Now, the follow on to this idea is as follows: It seems to me to be a little arbitrary when something goes stable and I find myself with a number of ~ flags set on an otherwise fairly stable system (I dare say you have a different opinion, but remember I am looking from the outside in). Now, at one point in the past there was a gentoo package which phoned home and reported which version of every package you were using. Could these statistics not be used to help direct development time to the most useful areas? I'm thinking along the lines of noticing that 90% of the userbase is running a version of xmltv which is 5 versions newer than the stable one, hence it's probably fairly stable, and in need up being marked as stable... These statistics could also be used as a first line quality check for any ebuilds in the proposed development ebuild area. So for example, if there is a hard-core of users using my pmwiki ebuild (which is currently marked as maintainer wanted), then this is a clue that it must be fairly stable and popular and worth including (since it will probably require minimal effort). It seems like this would go some way towards easing the easy development bits and giving everyone more time to work on the important stuff, whilst also making use of the distributed testing effort of some of the more adventurous users... Workable? Ed W -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] [Summary] tentative x86 arch team glep
Ciaran McCreesh wrote: On Tue, 06 Sep 2005 18:03:37 +0100 Ed W [EMAIL PROTECTED] wrote: | As an outsider reading that summary the message *I* read is that | there is some strain over fitting the development model into | stable, ~, and package.mask. I think I see people basically | saying that they have differing views over what qualifies for each | level? The system basically works. The problems are: * It's not always used correctly. * It's not entirely understood by some users. * Some users think it should be easier to unmask a group of related packages. Might there be an option 4 which is that a slightly different system might stop everyone bitching over the current one and hence avoid wasting some time? Nope, no idea what that would be, but the thought does occur when you see some time being wasted on trivial issues... | Also, as someone who has submitted a few patches and some ebuilds and | then seen nothing happen to them and my offers to act as maintainer | have gone unresponded I also wonder if there is some way to make | better use of casual contributors like me? (I'm not bitter, it's just | that I feel I could contribute more, but don't know how to?) The problem is... Getting someone ready to be able to commit to the main tree is expensive in terms of existing developer time. The solution isn't lowering the standards for commit access, because that just leads to even more wasted developer time. There's the two tier system that gets proposed every now and again, but that would a) require svn rather than cvs and b) require that certain people who currently have main tree access be moved to branch access only. A bigger tree is all well and good, but the tree we have right now is only half maintained... Is there any possibility that easier low quality contribution makes the high quality contributions easier? Look at wikipedia - it's amazing that such high quality work (in general) can come from lightly peer review material with low barriers to entry. Clearly not an appropriate model here, but I can't help wondering if there is not another way... I did read the FAQ here and I take your point though: http://dev.gentoo.org/~ciaranm/docs/mw-faq/maintainer.txt Thanks for listening Ed W -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] [Summary] tentative x86 arch team glep
As an outsider reading that summary the message *I* read is that there is some strain over fitting the development model into stable, ~, and package.mask. I think I see people basically saying that they have differing views over what qualifies for each level? Perhaps part of the solution is to review the current list of levels of stability? Debian for example use several levels with something like: stable, unstable, testing, development (or whatever they are called). Perhaps something more like that would be useful for gentoo? I do know as a user it can be quite frustrating trying to find the ebuild for a package and having to dig around bugs.gentoo, and some other website, then patch up a dodgy ebuild found on some website, etc, etc. Perhaps it would be more useful to have development quality ebuilds straight from portage (labelled as dangerous and unstable of course) and then I could more easily file back patches to fix problems that I find, and development would be more centralised...? Also, as someone who has submitted a few patches and some ebuilds and then seen nothing happen to them and my offers to act as maintainer have gone unresponded I also wonder if there is some way to make better use of casual contributors like me? (I'm not bitter, it's just that I feel I could contribute more, but don't know how to?) Good luck. I'm a big gentoo fan. I hope this extends gentoos lead even further! All the best Ed W -- gentoo-dev@gentoo.org mailing list
[gentoo-dev] Where to post bug report/questions about webapp-config?
Hi, (I previously sent this message directly to Stuart and got no reply - is there a better mailing list to discuss this stuff?) I am writing an ebuild for sql-ledger an accounting package for small businesses. I am hitting a problem with symlinks not being copied across to the virtual directory Consider the original installed dir: tb htdocs # ls -al /usr/share/webapps/sql-ledger/2.4.14/htdocs/ total 115 drwxr-xr-x 10 root root 1000 Jul 30 00:18 . drwxr-xr-x 7 root root 328 Jul 30 00:18 .. drwxr-xr-x 2 root root 624 Jul 30 00:18 SL -rw-r--r-- 2 root root 7 Jul 30 00:18 VERSION lrwxrwxrwx 1 root root 8 Jul 30 00:18 admin.pl - login.pl -rwxr-xr-x 2 root root 4195 Jul 30 00:18 am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 ap.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 ar.pl - am.pl drwxr-xr-x 5 root root 120 Jul 30 00:18 bin lrwxrwxrwx 1 root root 5 Jul 30 00:18 bp.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 ca.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 cp.pl - am.pl drwxr-xr-x 2 root root80 Jul 30 00:18 css lrwxrwxrwx 1 root root 5 Jul 30 00:18 ct.pl - am.pl -rw-r--r-- 2 root root 3638 Jul 30 00:18 favicon.ico lrwxrwxrwx 1 root root 5 Jul 30 00:18 gl.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 hr.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 ic.pl - am.pl -rw-r--r-- 2 root root 177 Jul 30 00:18 index.html lrwxrwxrwx 1 root root 5 Jul 30 00:18 ir.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 is.pl - am.pl drwxr-xr-x 42 root root 1008 Jul 30 00:18 locale -rwxr-xr-x 2 root root 3055 Jul 30 00:18 login.pl -rw-r--r-- 2 root root 12774 Jul 30 00:18 menu.ini lrwxrwxrwx 1 root root 5 Jul 30 00:18 menu.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 oe.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 pe.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 ps.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 rc.pl - am.pl lrwxrwxrwx 1 root root 5 Jul 30 00:18 rp.pl - am.pl -rwxr-xr-x 2 root root 10020 Jul 30 00:18 setup.pl drwxr-xr-x 2 root root72 Jul 30 00:18 spool drwxr-xr-x 2 root root 3872 Jul 30 00:18 sql -rw-r--r-- 1 root root 1034 Jul 30 00:18 sql-ledger.conf -rw-r--r-- 2 root root 1034 Jul 30 00:18 sql-ledger.conf.default -rw-r--r-- 2 root root 28551 Jul 30 00:18 sql-ledger.gif -rw-r--r-- 2 root root 3594 Jul 30 00:18 sql-ledger.png drwxr-x--x 2 root root 17040 Jul 30 00:18 templates drwxr-xr-x 2 root root 112 Jul 30 00:18 users Now, see the results of using webapp-config -I: tb htdocs # ls -al /var/www/localhost/htdocs/sql-ledger/ total 259 drwxr-xr-x 10 root root 640 Jul 30 00:20 . drwxr-xr-x 5 root root 1408 Jul 30 00:19 .. -rw--- 1 root root 327 Jul 30 00:20 .webapp -rw-r--r-- 1 root root 139604 Jul 30 00:20 .webapp-sql-ledger-2.4.14 drwxr-xr-x 2 root root 624 Jul 30 00:19 SL -rw-r--r-- 2 root root7 Jul 30 00:18 VERSION -rwxr-xr-x 2 root root 4195 Jul 30 00:18 am.pl drwxr-xr-x 5 root root 120 Jul 30 00:19 bin drwxrwxr-x 2 apache apache 80 Jul 30 00:19 css -rw-r--r-- 2 root root 3638 Jul 30 00:18 favicon.ico -rw-r--r-- 2 root root 177 Jul 30 00:18 index.html drwxr-xr-x 42 root root 1008 Jul 30 00:19 locale -rwxr-xr-x 2 root root 3055 Jul 30 00:18 login.pl -rw-r--r-- 2 root root12774 Jul 30 00:18 menu.ini -rwxr-xr-x 2 root root10020 Jul 30 00:18 setup.pl drwxrwxr-x 2 apache apache 72 Jul 30 00:19 spool drwxr-xr-x 2 root root 3872 Jul 30 00:19 sql -rw-r--r-- 1 root root 1034 Jul 30 00:19 sql-ledger.conf -rw-r--r-- 2 root root 1034 Jul 30 00:18 sql-ledger.conf.default -rw-r--r-- 2 root root28551 Jul 30 00:18 sql-ledger.gif -rw-r--r-- 2 root root 3594 Jul 30 00:18 sql-ledger.png drwxrwxr-x 2 apache apache 17040 Jul 30 00:20 templates drwxrwxr-x 2 apache apache112 Jul 30 00:19 users Is this a known issue? Is there a workaround? Where is the correct place to discuss this...? Thanks Ed W -- gentoo-dev@gentoo.org mailing list
