Re: [gentoo-dev] Official overlay support
On Friday 24 March 2006 20:18, Chris Gianelloni wrote: I really can't think of much besides kernel + toolchain that can have such devastating effects to the rest of the tree. The only other massive breakages would be via eclasses, which was my main target. glibc is a good candidate. And portage a second one. Does anyone have any ideas how we could resonably reduce problems reported from things such as toolchain breakages in an overlay, yet still not punish the people running the overlay by disallowing it? I surely wouldn't want to limit the toolchain maintainers from being able to enjoy the use of an overlay if they wished it. Perhaps we could ask people who run overlays with dangerous ebuilds, to have these ebuilds protected by some environment variables. (The var must be set for the ebuild to work.) Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgpRW5v2dGLh5.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Friday 24 March 2006 16:41, Andres Loeh wrote: At the moment, Haskell is only a herd and a team, not a project. But this is certainly something that can be addressed, should it be necessary to change that. In my regards you are a project, just not one that has a project page. Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgp7qUmmyC6B9.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Sat, Mar 25, 2006 at 08:37:24PM +0100, Paul de Vrieze wrote: On Friday 24 March 2006 20:18, Chris Gianelloni wrote: I really can't think of much besides kernel + toolchain that can have such devastating effects to the rest of the tree. The only other massive breakages would be via eclasses, which was my main target. glibc is a good candidate. And portage a second one. *libc in general. binutils coreutils (Screwing up this is really fun, sort/xargs/tail etc.) And a general class, the reason I've had stuff in my own overlays: - Trying to develop clean/safe automated upgrade paths for complex packages. Early versions of these tend to do nasty things to data (openldap was esp. painful). Does anyone have any ideas how we could resonably reduce problems reported from things such as toolchain breakages in an overlay, yet still not punish the people running the overlay by disallowing it? I surely wouldn't want to limit the toolchain maintainers from being able to enjoy the use of an overlay if they wished it. Perhaps we could ask people who run overlays with dangerous ebuilds, to have these ebuilds protected by some environment variables. (The var must be set for the ebuild to work.) Only if portage can check the variable before starting to compile any packages. -- Robin Hugh Johnson E-Mail : [EMAIL PROTECTED] GnuPG FP : 11AC BA4F 4778 E3F6 E4ED F38E B27B 944E 3488 4E85 pgpXPDpRpLVa5.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Friday 24 March 2006 10:41, Andres Loeh wrote: At the moment, Haskell is only a herd and a team, not a project. Then stand up my brother! perl wasn't so different, so we snuck in a project page. no one said anything. i'm going with we're a project to govern the direction and maintenance of the herd until i hear otherwise ;) if 'they' (the forces of evil in the universe, the consortium, what have you) turn to ban us as projects, we could always come together as the 'languages-you-use-and-abuse project'. might be fun that way. we could have pizza socials and such. ~mcummings pgpwE2PrpPbiF.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Thursday 23 March 2006 22:32, Donnie Berkholz wrote: Paul de Vrieze wrote: I can only assume that other developers have similar overlays too. These overlays form actually a wealth of resources that are hidden away. If there were a semi-public overlay system in which developers could keep their overlays, this might help in getting this out to the public. Heh, such a thing sort of exists in a non-standardized form at dev.gentoo.org/~developername/overlay/ -- seems to be the way most people go. Would be easy though if it were a subversion repository. That way one does not need to copy it around all the time. Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgpHSr3xIgfrs.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
Hi Andres, On 3/23/06, Andres Loeh [EMAIL PROTECTED] wrote: dcoutts has described the current practice we use in the Haskell team, but that doesn't necessarily mean that it's the only practice that would work for us. I can imagine that if we can come up with reasonable policies for o.g.o, we can switch to a slightly different (i.e., more public) scheme ... I want to thank both you and Duncan for your explaination of how the Haskell overlay works. I would love to get your overlay onto overlays.g.o, without disrupting the working practices that you've found successful. Which means we need to take another look at the vision of all overlays being publically readable, because having a non-public overlay seems to be a key part of what you're already doing. I really don't want o.g.o to carry 'secret' overlays. But maybe there's a middle ground that fits with what you need? If the overlay's changelog is included on o.g.o's front-page, and the wiki / GuideXML site is publically readable, but we just disallow anonymous access to the overlay itself (only if requested, this wouldn't be the default setup) ... how would that work for you? Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
It is a Gentoo problem, because Gentoo gets innundated with bogus bug reports when users screw up their systems in weird ways and don't realise the cause. Convince me that this is something more than just a power play, and I'll work with you. But that's the hurdle you'll need to overcome first. Second hurdle is that you need to convince me that you get what the overlays are there for. If you can't, then I can have no confidence that any policies you bring forward are appropriate for the work we're trying to enable. Thrid hurdle is that you need to convince me that you're capable of treating the overlays differently to how the main tree is treated. If you can't, then I'll feel that you hoodwinked me at the second hurdle. I'm sure you've got a lot to offer, to help make the overlays a success. But your agenda has to be appropriate - otherwise you'll just do more damage that good. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Ciaran McCreesh wrote: On Thu, 23 Mar 2006 19:57:07 +0100 Jakub Moc [EMAIL PROTECTED] wrote: | Sounds like a perfect way to break lots and lots of systems. Those | policies are mostly there for good reason. | | You want to apply policies on overlays? Well no - sorry, overlays are | none of QA's or any other policy business. They are overlays, not | official tree. If user installs ebuilds from overlay and breaks his | system, then well - not a Gentoo problem. Why should any policies | apply? It is a Gentoo problem, because Gentoo gets innundated with bogus bug reports when users screw up their systems in weird ways and don't realise the cause. We get innundated with tons of bogus bug reports every day, overlays or not - see the number of invalid/duplicate bugs flowing every days. We got a couple of bugs in last two a three days basically stating ZOMG, glibc downgrade broke my system, t3h Gentoo bug!!11!! - so what? They get marked as invalid, live goes on. This argument really doesn't stand. As this should be a separate thread, just one reason or example - I'm really uncomfortable e.g. w/ QA intervening in overlays stuff, considering the current way QA is being done in Gentoo... Current non-interactivity policy has clearly influenced multiple ebuilds in portage to the extent that forced me to read the ebuilds very carefully multiple times to be sure what the outcome will be with my particular USE flags. This is a bad thing (TM) for me and I clearly oppose to having such stuff forced in overlays. I could see a place for QA volunteers in this overlay stuff, but that would require a fundamentally different approach from what QA takes now. (If you wish to discuss this further, move it to a separate thread, please). Common sense always applies, but generally - overlays are not a place for bureaucracy... -- Best regards, Jakub Moc mailto:[EMAIL PROTECTED] GPG signature: http://subkeys.pgp.net:11371/pks/lookup?op=getsearch=0xCEBA3D9E Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95 B30F 8717 D5FD CEBA 3D9E ... still no signature ;) signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
Hi Stuart. dcoutts has described the current practice we use in the Haskell team, but that doesn't necessarily mean that it's the only practice that would work for us. I can imagine that if we can come up with reasonable policies for o.g.o, we can switch to a slightly different (i.e., more public) scheme ... I want to thank both you and Duncan for your explaination of how the Haskell overlay works. I would love to get your overlay onto overlays.g.o, without disrupting the working practices that you've found successful. Which means we need to take another look at the vision of all overlays being publically readable, because having a non-public overlay seems to be a key part of what you're already doing. Not really. I discussed this with Duncan and we're perfectly ok with a publically readable repo. In fact, our overlay is currently publically readable. It's just not very well-known or advertised. If we change the situation, we'll have to write up a few guidelines specific to our repository. Here's a list of things that I think are essential or highly helpful to our working process: * We should be allowed to continue using darcs for our version management. If that's not possible on Gentoo infra, we should be allowed to host on another machine and just have a pointer or ChangeLog on o.g.o. * It should be possible for us to assign commit permissions to any people we think are qualified without any formalities and delay. If the overlay's changelog is included on o.g.o's front-page, and the wiki / GuideXML site is publically readable, but we just disallow anonymous access to the overlay itself (only if requested, this wouldn't be the default setup) ... how would that work for you? It would work, of course, and it would help prevent certain complaints, but it's not absolutely necessary. If on request is chosen, it's also important that read access can be given by us without any delay, i.e., without going through any formal process. Cheers, Andres pgp1gbtnP4HWd.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Fri, 2006-03-24 at 08:59 +, Stuart Herbert wrote: It is a Gentoo problem, because Gentoo gets innundated with bogus bug reports when users screw up their systems in weird ways and don't realise the cause. Convince me that this is something more than just a power play, and I'll work with you. But that's the hurdle you'll need to overcome first. Perhaps because he's not the only one saying it? Really, people, can we leave our personal bullshit out of a technical discussion *just once*? Second hurdle is that you need to convince me that you get what the overlays are there for. If you can't, then I can have no confidence that any policies you bring forward are appropriate for the work we're trying to enable. They are there to speed development and to allow users to contribute more directly. They should not be readable by the public, otherwise we run into the problem of users that don't know what they are doing and followed some half-way written guide on the forums using ebuilds from these overlays, breaking their systems, and filing bugs. This is *not* acceptable. I see no problems with allowing users to gain read and even write access to overlays, but it must be done with a certain level of caution of the main tree, or you'll have a very hard time getting support from the developer community at large. Thrid hurdle is that you need to convince me that you're capable of treating the overlays differently to how the main tree is treated. If you can't, then I'll feel that you hoodwinked me at the second hurdle. I'm sure you've got a lot to offer, to help make the overlays a success. But your agenda has to be appropriate - otherwise you'll just do more damage that good. Again, try to keep this technical discussion technical and leave your personal biases out of it. -- Chris Gianelloni Release Engineering - Strategic Lead x86 Architecture Team Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Official overlay support
On Fri, 2006-03-24 at 10:16 +0100, Jakub Moc wrote: As this should be a separate thread, just one reason or example - I'm really uncomfortable e.g. w/ QA intervening in overlays stuff, considering the current way QA is being done in Gentoo... Current non-interactivity policy has clearly influenced multiple ebuilds in portage to the extent that forced me to read the ebuilds very carefully multiple times to be sure what the outcome will be with my particular USE flags. This is a bad thing (TM) for me and I clearly oppose to having such stuff forced in overlays. I could see a place for QA volunteers in this overlay stuff, but that would require a fundamentally different approach from what QA takes now. (If you wish to discuss this further, move it to a separate thread, please). Except nobody says that QA needs to intervene in an overlay. My request is *simple*. If the overlay is accessible to the general public, then it *cannot* break other packages in the tree via its use. If the overlay is private (meaning it has a restricted access list, developers and users are both welcome) then it can break whatever policy that it wants. Common sense always applies, but generally - overlays are not a place for bureaucracy... Nor are they a place to allow a free-for-all where people can commit anything that can cause any amount of damage to the tree, while still being official and hosted on Gentoo infrastructure. -- Chris Gianelloni Release Engineering - Strategic Lead x86 Architecture Team Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Official overlay support
On Fri, 2006-03-24 at 12:46 +0100, Andres Loeh wrote: If the overlay's changelog is included on o.g.o's front-page, and the wiki / GuideXML site is publically readable, but we just disallow anonymous access to the overlay itself (only if requested, this wouldn't be the default setup) ... how would that work for you? It would work, of course, and it would help prevent certain complaints, but it's not absolutely necessary. If on request is chosen, it's also important that read access can be given by us without any delay, i.e., without going through any formal process. See, I have no problem with this. The operation of the overlay *should* lie completely with the overlay owners. You *should* be able to add *anyone* that you feel is worth adding to read *or* write access, with no further process. As I've said, my only request is a single policy that before an overlay can become publicly readable on overlays.gentoo.org (which is Gentoo infrastructure) that it does not break packages in the main tree that are not in the overlay. If this single policy were in place, then I would fully support overlays.gentoo.org being created. My main point is I don't want one of my tree packages to break because some ricer told some n00b to use some crazy ebuild from some random overlay that isn't really fit for the general masses. If we take at least *some* measures to prevent this, then I'm OK with it. Allowing a free-for-all in the overlays is not acceptable. -- Chris Gianelloni Release Engineering - Strategic Lead x86 Architecture Team Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Official overlay support
On Friday 24 March 2006 14:55, Chris Gianelloni wrote: My main point is I don't want one of my tree packages to break because some ricer told some n00b to use some crazy ebuild from some random overlay that isn't really fit for the general masses. If we take at least *some* measures to prevent this, then I'm OK with it. Allowing a free-for-all in the overlays is not acceptable. I think that this is a reasonable requirement. Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgp83vZXed3YX.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
Chris Gianelloni wrote: [Fri Mar 24 2006, 08:55:30AM EST] As I've said, my only request is a single policy that before an overlay can become publicly readable on overlays.gentoo.org (which is Gentoo infrastructure) that it does not break packages in the main tree that are not in the overlay. This makes sense, but what's the content of such a policy? Take your gcc-5.1.99 example: say it breaks lots of packages in portage. Is it not allowed to be in a publicly accessible overlay in that case? If that's not the policy, then what policy allows gcc-5.1.99 but still covers the ground you want covered? Aron pgpWrcVTWdrd5.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
Hi Chris, On 3/24/06, Chris Gianelloni [EMAIL PROTECTED] wrote: As I've said, my only request is a single policy that before an overlay can become publicly readable on overlays.gentoo.org (which is Gentoo infrastructure) that it does not break packages in the main tree that are not in the overlay. If this single policy were in place, then I would fully support overlays.gentoo.org being created. I accept the principle. I've just one question about the practice. Overlays will (probably more often than not) contain packages that have problems. For example, I'm thinking of when we developed the new PHP packages, the packages delivered PHP in a new way, with different USE flags. We deliberately added a blocker, so that dev-lang/php and dev-php/php (the version from the tree) couldn't be installed at the same time. This meant that, until the webapps had their DEPs updated, if you had PHP from the overlay installed, it would block webapps from the tree from installing (because they depended on dev-php/php). Do you consider a practice like that to violate the policy you seek? Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Stuart, I like the idea of overlays but your email here is completely bogus. Ciaran just explained why overlays are a Gentoo problem, rebutting Jakub's assertion that there's no need for policies. I don't see any agenda here, so either you're pulling in external context, or you're reading into it. Aron Stuart Herbert wrote: [Fri Mar 24 2006, 03:59:54AM EST] It is a Gentoo problem, because Gentoo gets innundated with bogus bug reports when users screw up their systems in weird ways and don't realise the cause. Convince me that this is something more than just a power play, and I'll work with you. But that's the hurdle you'll need to overcome first. Second hurdle is that you need to convince me that you get what the overlays are there for. If you can't, then I can have no confidence that any policies you bring forward are appropriate for the work we're trying to enable. Thrid hurdle is that you need to convince me that you're capable of treating the overlays differently to how the main tree is treated. If you can't, then I'll feel that you hoodwinked me at the second hurdle. I'm sure you've got a lot to offer, to help make the overlays a success. But your agenda has to be appropriate - otherwise you'll just do more damage that good. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Hi Andres, On 3/24/06, Andres Loeh [EMAIL PROTECTED] wrote: Here's a list of things that I think are essential or highly helpful to our working process: * We should be allowed to continue using darcs for our version management. If that's not possible on Gentoo infra, we should be allowed to host on another machine and just have a pointer or ChangeLog on o.g.o. Unless I've missed it, no-one else has spoken up and suggested what the second VCS should be. I'll have a play with darcs as soon as I can, and talk with infra about whether we can support it or not. Is it okay if I come and pick your brains to help with this? * It should be possible for us to assign commit permissions to any people we think are qualified without any formalities and delay. Absolutely. This is one of the corner-stones of the project. It would work, of course, and it would help prevent certain complaints, but it's not absolutely necessary. If on request is chosen, it's also important that read access can be given by us without any delay, i.e., without going through any formal process. The only formallity is that the request will need to come from the project lead listed on your project's page under www.g.o/proj/lang/project/. I need to talk to infra first, but in principle I don't see a problem with project leads being allowed to delegate that power. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Chris Gianelloni wrote: On Fri, 2006-03-24 at 08:59 +, Stuart Herbert wrote: It is a Gentoo problem, because Gentoo gets innundated with bogus bug reports when users screw up their systems in weird ways and don't realise the cause. Convince me that this is something more than just a power play, and I'll work with you. But that's the hurdle you'll need to overcome first. Perhaps because he's not the only one saying it? Really, people, can we leave our personal bullshit out of a technical discussion *just once*? Second hurdle is that you need to convince me that you get what the overlays are there for. If you can't, then I can have no confidence that any policies you bring forward are appropriate for the work we're trying to enable. They are there to speed development and to allow users to contribute more directly. They should not be readable by the public, otherwise we run into the problem of users that don't know what they are doing and followed some half-way written guide on the forums using ebuilds from these overlays, breaking their systems, and filing bugs. This is *not* acceptable. I see no problems with allowing users to gain read and even write access to overlays, but it must be done with a certain level of caution of the main tree, or you'll have a very hard time getting support from the developer community at large. +1 At least in my mind the overlays should be developmental overlays; not for public comsumption. This doesn't mean don't tell anyone about it so that no one shows up. It means interested users will probably inquire about helping out, etc...and can be granted access fairly easily. -Alec Warner -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
If the overlay's changelog is included on o.g.o's front-page, and the wiki / GuideXML site is publically readable, but we just disallow anonymous access to the overlay itself (only if requested, this wouldn't be the default setup) ... how would that work for you? It would work, of course, and it would help prevent certain complaints, but it's not absolutely necessary. If on request is chosen, it's also important that read access can be given by us without any delay, i.e., without going through any formal process. See, I have no problem with this. The operation of the overlay *should* lie completely with the overlay owners. You *should* be able to add *anyone* that you feel is worth adding to read *or* write access, with no further process. Ok. As I've said, my only request is a single policy that before an overlay can become publicly readable on overlays.gentoo.org (which is Gentoo infrastructure) that it does not break packages in the main tree that are not in the overlay. If this single policy were in place, then I would fully support overlays.gentoo.org being created. I see your point. Then I'd suggest to have o.g.o host two classes of overlays: (A) publically readable ones that fulfill basic policies and don't break the main tree, and (B) others where read and write access can be granted on request by the overlay owners, but isn't available automatically. Every overlay would belong to class B at first -- in fact, o.g.o could go online only supporting B. We can take our time and work out goog guidelines for class A repos, and then gradually change some of the overlays to class A status. My main point is I don't want one of my tree packages to break because some ricer told some n00b to use some crazy ebuild from some random overlay that isn't really fit for the general masses. If we take at least *some* measures to prevent this, then I'm OK with it. Allowing a free-for-all in the overlays is not acceptable. Yes, as I said, I understand that. Cheers, Andres pgpTGFDDdtMXQ.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Fri, 24 Mar 2006 10:16:15 +0100 Jakub Moc [EMAIL PROTECTED] wrote: | We get innundated with tons of bogus bug reports every day, overlays | or not - see the number of invalid/duplicate bugs flowing every days. | We got a couple of bugs in last two a three days basically stating | ZOMG, glibc downgrade broke my system, t3h Gentoo bug!!11!! - so | what? They get marked as invalid, live goes on. This argument really | doesn't stand. They get marked as invalid after how long? There're some really subtle ways in which libraries can screw things up. I've dealt with far too many bug reports where it took a heck of a lot of debugging before it became clear that the cause was some dodgy external stuff. And that was with me understanding the packages in question -- there's no way bug wranglers could've figured it out. | As this should be a separate thread, just one reason or example - I'm | really uncomfortable e.g. w/ QA intervening in overlays stuff, | considering the current way QA is being done in Gentoo... I'm really uncomfortable with QA intervening anywhere. It would be far nicer if the appropriate developers ensured that they weren't breaking anything. -- Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-dev] Official overlay support
On 3/24/06, Andres Loeh [EMAIL PROTECTED] wrote: Here's a list of things that I think are essential or highly helpful to our working process: * We should be allowed to continue using darcs for our version management. If that's not possible on Gentoo infra, we should be allowed to host on another machine and just have a pointer or ChangeLog on o.g.o. Unless I've missed it, no-one else has spoken up and suggested what the second VCS should be. I'll have a play with darcs as soon as I can, and talk with infra about whether we can support it or not. Is it okay if I come and pick your brains to help with this? Sure. Best place to find any of us is #gentoo-haskell ... * It should be possible for us to assign commit permissions to any people we think are qualified without any formalities and delay. Absolutely. This is one of the corner-stones of the project. Great. It would work, of course, and it would help prevent certain complaints, but it's not absolutely necessary. If on request is chosen, it's also important that read access can be given by us without any delay, i.e., without going through any formal process. The only formallity is that the request will need to come from the project lead listed on your project's page under www.g.o/proj/lang/project/. I need to talk to infra first, but in principle I don't see a problem with project leads being allowed to delegate that power. At the moment, Haskell is only a herd and a team, not a project. But this is certainly something that can be addressed, should it be necessary to change that. Cheers, Andres pgp3mMReSqoe7.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
Chris Gianelloni wrote: On Fri, 2006-03-24 at 12:46 +0100, Andres Loeh wrote: As I've said, my only request is a single policy that before an overlay can become publicly readable on overlays.gentoo.org (which is Gentoo infrastructure) that it does not break packages in the main tree that are not in the overlay. If this single policy were in place, then I would fully support overlays.gentoo.org being created. How are you going to enforce that policy? Heh, you can't do that preemptively even in the main tree. :) Every day there are bugs about package A upgrade breaking packages B, C and D. I can also create an overlay with two completely innocent ebuilds, get it opened and then keep filing it w/ ricer food like patched-to-hell toolchain stuff? How are you going to verify that ebuild X in overlay doesn't break anything in the tree? And that its next revision won't do it. You volunteer for such job? Great. Or not? Well, then there's no point in suggesting a policy that can just never work... So - what you are telling us is that you *assume* that people are going to publish ebuilds *known* to break in-portage stuff on overlays.g.o.? Weird assumption really... :/ -- Best regards, Jakub Moc mailto:[EMAIL PROTECTED] GPG signature: http://subkeys.pgp.net:11371/pks/lookup?op=getsearch=0xCEBA3D9E Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95 B30F 8717 D5FD CEBA 3D9E ... still no signature ;) signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
On 3/24/06, Ciaran McCreesh [EMAIL PROTECTED] wrote: I'm really uncomfortable with QA intervening anywhere. It would be far nicer if the appropriate developers ensured that they weren't breaking anything. +1 -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Ciaran McCreesh wrote: On Fri, 24 Mar 2006 10:16:15 +0100 Jakub Moc [EMAIL PROTECTED] wrote: | We get innundated with tons of bogus bug reports every day, overlays | or not - see the number of invalid/duplicate bugs flowing every days. | We got a couple of bugs in last two a three days basically stating | ZOMG, glibc downgrade broke my system, t3h Gentoo bug!!11!! - so | what? They get marked as invalid, live goes on. This argument really | doesn't stand. They get marked as invalid after how long? There're some really subtle ways in which libraries can screw things up. I've dealt with far too many bug reports where it took a heck of a lot of debugging before it became clear that the cause was some dodgy external stuff. And that was with me understanding the packages in question -- there's no way bug wranglers could've figured it out. Yeah, and the point is? It happens every day, there are already tons of third-party overlays used by Gentoo users, but once this thread about official overlays started, you came here to tell us wow, this all will cause terrible borkage and flood developers w/ tons of stupid invalid bugs, we need policies? I really don't see how overlays run mostly by Gentoo devs would cause any more borkage than totally uncontrolled third-party overlays run by whomever creates and publishes them, sorry. -- jakub signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
On 3/24/06, Alec Warner [EMAIL PROTECTED] wrote: At least in my mind the overlays should be developmental overlays; not for public comsumption. This doesn't mean don't tell anyone about it so that no one shows up. It means interested users will probably inquire about helping out, etc...and can be granted access fairly easily. That undermines the idea of using the overlays to attract current non-contributors. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Dňa Fri, 24 Mar 2006 17:15:37 +0100 Jakub Moc [EMAIL PROTECTED] napísal: Yeah, and the point is? It happens every day, there are already tons of third-party overlays used by Gentoo users, but once this thread about official overlays started, you came here to tell us wow, this all will cause terrible borkage and flood developers w/ tons of stupid invalid bugs, we need policies? I really don't see how overlays run mostly by Gentoo devs would cause any more borkage than totally uncontrolled third-party overlays run by whomever creates and publishes them, sorry. One of the reasons might be that while many users have enough sense not to report bugs too eagerly when using third-party overlays of varying quality, they would do so if they were using an official overlay - which, with your no-policy approach, would undoubtedly crawl with bugs. -- Andrej signature.asc Description: PGP signature
Re: [gentoo-dev] Official overlay support
On 3/24/06, Chris Gianelloni [EMAIL PROTECTED] wrote: Again, try to keep this technical discussion technical and leave your personal biases out of it. It's not meant as a personal critisism of Ciaran. Ciaran's being very helpful in this thread. It just happens that it was his post that created the opportunity for me to make that comment. I stand by the critera I stated. They're exactly what you are doing, and exactly what Ciaran is doing - failure avoidance. I am *not* going to keep this just on a technical level. (I will, however, keep this discussion professional). The overlays are a social tool just as much as a technical one. The social aspect keeps getting ignored in this thread, or dismissed as unimportant. That frustrates me, I don't mind admitting. Best regards, Stu -- -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On Fri, 2006-03-24 at 09:47 -0500, Aron Griffis wrote: Chris Gianelloni wrote: [Fri Mar 24 2006, 08:55:30AM EST] As I've said, my only request is a single policy that before an overlay can become publicly readable on overlays.gentoo.org (which is Gentoo infrastructure) that it does not break packages in the main tree that are not in the overlay. This makes sense, but what's the content of such a policy? Take your gcc-5.1.99 example: say it breaks lots of packages in portage. Is it not allowed to be in a publicly accessible overlay in that case? If that's not the policy, then what policy allows gcc-5.1.99 but still covers the ground you want covered? I see your point here and honestly do not have an answer. I don't want to limit what people can do in the overlays so much as reduce the collateral damage that can be done. Honestly stuff like the toolchain is a bit of an exception only because that information all shows up on an emerge --info already. I really can't think of much besides kernel + toolchain that can have such devastating effects to the rest of the tree. The only other massive breakages would be via eclasses, which was my main target. Does anyone have any ideas how we could resonably reduce problems reported from things such as toolchain breakages in an overlay, yet still not punish the people running the overlay by disallowing it? I surely wouldn't want to limit the toolchain maintainers from being able to enjoy the use of an overlay if they wished it. -- Chris Gianelloni Release Engineering - Strategic Lead x86 Architecture Team Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Official overlay support
Hi Stuart, I'd like to comment on some of your plans for overlays.g.o. Am Mittwoch, 22. März 2006 23:03 schrieb Stuart Herbert: It's probably the right time for me to flesh out what I've been planning for overlays.g.o. The vision I have for overlays.g.o is one official home for all Gentoo overlays run by projects and by individual Gentoo devs. I see the Also for Arch/Herd Testers? homepage itself running Planet (just like planet.g.o), using the RSS feeds from the overlays to display the changelogs from all the overlays. The links down the left hand side of the page go to the homepage for each of the overlays. The homepages are separate wiki instances. Well, there is a couple of Gentoo devs who are not particularly comfortable with wikis (including myself). Why change things the way they are currently? I'd suggest to use one repository per project and to add a 'website' or 'site' directory. In this case we could use the good ol' GuideXML/SCM combination. http://overlays.g.o/proj/project-name/ for overlays run by herds listed in herds.xml http://overlays.g.o/svn/project-name/ for the SVN repos and Like above: use http://o.g.o/proj/project-name/ for the information content and http://o.g.o/proj/project-name/svn/ for the overlay. http://overlays.g.o/dev/developer/ for overlays run by individual developers http://overlays.g.o/svn/developer/ for the SVN repos same as above :-) There's a practical limit to the amount of software we can support on overlays.g.o. The less we install, the less the overhead of administrating this system for infra and the overlays admin team (I'm looking for responsible volunteers to join that team, if you're interested). Another reason for dropping the wiki I'd like to offer two wiki engines and two version control systems on overlays.g.o. I believe that gives us enough choice without us loading the box with too much software for us to keep on top of. In case we use wiki, why _two_ wiki engines? To answer Daniel's question about official ... the overlays hosted on overlays.g.o would be official. The overlays project will be accountable for overlays.g.o overall. It would make sense for the overlays project to be a sub-project of infra. To ensure officialness and (what I personally care more about) accountability, project overlays will be created for projects that meet the description of a project in the metastructure [1]. The overlays team will have to be strict on this, to ensure officialness. The overlay must be requested by one of the leads of the project. The lead(s) would be jointly accountable for the overlay and all its contents. Leads will be able to ask for commit / wiki edit access for non-devs. Please consider also to let QA and Security have commit access to all overlays (If they're so inclined). Developer overlays would only be created for active Gentoo developers, and they would be accountable for its contents. Non-developers will not be given write access to developer overlays. By default - working on the same principle of trust that governs all developers w.r.t. the Portage tree - all developers will be able to commit to all overlays. If we can't trust you to respect other people's overlays, then we can't trust you with commit access to the Portage tree, and you're not fit to be a Gentoo dev in the first place I would say this should be clarified some more. Surely anybody with an access to an overlay can commit, but the projects should be the keepers of the keys. Overlays are not the tree, they are probably very experimental. I could imagine that i wouldn't want anyone to modify say an experimental gcc ebuild from toolchain's overlay for example. Stuart: Overall, i like your proposal. I would suggest to turn it into a GLEP and I'm willing to help you with this. Danny -- Danny van Dyk [EMAIL PROTECTED] Gentoo/AMD64 Project, Gentoo Scientific Project -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Hi Danny, On 3/23/06, Danny van Dyk [EMAIL PROTECTED] wrote: Hi Stuart, I'd like to comment on some of your plans for overlays.g.o. :) The vision I have for overlays.g.o is one official home for all Gentoo overlays run by projects and by individual Gentoo devs. I see the Also for Arch/Herd Testers? Sure. Well, there is a couple of Gentoo devs who are not particularly comfortable with wikis (including myself). Why change things the way they are currently? Because previous threads here on -dev asking for a wiki prove that not everyone is comfortable / happy with how things currently are ... myself included. Wikis are a much lower barrier to entry than GuideXML ... and they have advantages over GuideXML. There's no reason why you can't create GuideXML and store / publish it via the overlay, even if there is a wiki. We do that with the PHP overlay. I'd suggest to use one repository per project and to add a 'website' or 'site' directory. In this case we could use the good ol' GuideXML/SCM combination. That's easy enough. We can establish a 'known location' in the VCS tree where GuideXML will be stored, and run a simple cron script to extract it and put it into the website directory for public viewing. Or you could just publish it on www.g.o/proj/en/project/ instead :) Like above: use http://o.g.o/proj/project-name/ for the information content and http://o.g.o/proj/project-name/svn/ for the overlay. Could do. I always preferred separate paths to ensure no clash with any other content under /proj/project-name/. Another reason for dropping the wiki No. We can make the wiki optional, but not offering a wiki at all makes it impossible for existing successful, externally hosted overlays to move to overlays.g.o. In case we use wiki, why _two_ wiki engines? Because different people have different preferences, and I don't believe in 'one size fits all'. Adding a little bit of flexibility in the right places will make o.g.o more successful. Please consider also to let QA and Security have commit access to all overlays (If they're so inclined). That's already covered under the principle that QA and Security are staffed by devs. I would say this should be clarified some more. Surely anybody with an access to an overlay can commit, but the projects should be the keepers of the keys. Overlays are not the tree, they are probably very experimental. I could imagine that i wouldn't want anyone to modify say an experimental gcc ebuild from toolchain's overlay for example. I want to operate the overlays on the same principles that we use to manage the Portage tree. We tell developers that they have to respect other people's packages, and can't go around changing what they feel like. The same should hold true for the overlays. Stuart: Overall, i like your proposal. I would suggest to turn it into a GLEP and I'm willing to help you with this. I already have Infra's agreement and active support to deliver this (I can't thank Lance and Kurt enough for their help to date on this). It's a new service - one that no-one is required to use (although I ferverently hope that it proves very popular). I don't believe that it needs a GLEP. What it does need is a successful overlay project team, to administer the service and help it evolve over the coming years. Best regards, Stu -- -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On Wed, 2006-03-22 at 22:03 +, Stuart Herbert wrote: I'd like to offer two wiki engines and two version control systems on overlays.g.o. I believe that gives us enough choice without us loading the box with too much software for us to keep on top of. One thing that was never planned was any form of shell access to this box, except for the team creating/destroying overlays. It looks like this will be necessary to support a distributed vcs. I'll talk to infra and see what we could do about providing some form of ssh access to help us support a distributed vcs. Trac and SVN would be my first choice. MoinMoin would be my recommendation for the second wiki engine. What should the second version control system be? I don't use them, I have no experience with them, and so I have no preference of what this should be. To answer Daniel's question about official ... the overlays hosted on overlays.g.o would be official. The overlays project will be accountable for overlays.g.o overall. It would make sense for the overlays project to be a sub-project of infra. To ensure officialness and (what I personally care more about) accountability, project overlays will be created for projects that meet the description of a project in the metastructure [1]. The overlays team will have to be strict on this, to ensure officialness. The overlay must be requested by one of the leads of the project. The lead(s) would be jointly accountable for the overlay and all its contents. Leads will be able to ask for commit / wiki edit access for non-devs. Developer overlays would only be created for active Gentoo developers, and they would be accountable for its contents. Non-developers will not be given write access to developer overlays. By default - working on the same principle of trust that governs all developers w.r.t. the Portage tree - all developers will be able to commit to all overlays. If we can't trust you to respect other people's overlays, then we can't trust you with commit access to the Portage tree, and you're not fit to be a Gentoo dev in the first place :P The only restriction will be that you'll need to ask the overlay project team to setup your access the very first time. Anyone wanting a secret overlay needs to make their own hosting arrangements. To answer Daniel's other question, about bugs.g.o ... trac on overlays.g.o will have its bug tracking system disabled. We already have one bug tracking system - bugs.g.o - and that's sufficient. [1] http://www.gentoo.org/proj/en/glep/glep-0039.html Hi Stuart I agree with the wiki because it seems to be an easy way to users and developers comunicate together and work. Like i said a few months ago the documentation won't give any problems to GDP since GDP provides high level docs. The wiki will also help our projects since it can be added TODO's, roadmaps and all that stuff. About the overlay the best way imo is provide a unique overlay with external contribs maintained by users and devs (of course commit rights only for devs.). -- Luis Medinas [EMAIL PROTECTED] http://dev.gentoo.org/~metalgod Gentoo Linux Developer: AMD64,Printing,Media-Optical,Sound -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Stuart Herbert wrote: Developer overlays would only be created for active Gentoo developers, and they would be accountable for its contents. Non-developers will not be given write access to developer overlays. This removes much of the motivation for merging overlays to o.g.o, at least some of the ones I am aware of. Donnie signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
On 3/23/06, Donnie Berkholz [EMAIL PROTECTED] wrote: Stuart Herbert wrote: Developer overlays would only be created for active Gentoo developers, and they would be accountable for its contents. Non-developers will not be given write access to developer overlays. This removes much of the motivation for merging overlays to o.g.o, at least some of the ones I am aware of. Then I'll re-consider. I was hoping that developers would setup projects if there are multiple contributors (project overlays will allow write access to non-devs). Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On 23/03/06, Stuart Herbert [EMAIL PROTECTED] wrote: The vision I have for overlays.g.o is one official home for all Gentoo overlays run by projects and by individual Gentoo devs. I see the Also for Arch/Herd Testers? The discussion seems to have moved from the original how can we become more open to enable our users to contribute? to how to provide testing overlays for existing gentoo devs. I think that the use of overlays is more a symptom of a problem with portage. Overlay problems: They remove ebuilds from the tree Users have to add yet another overlay / retrieve the ebuilds somehow Conflicts between overlays Increases time to moving packages into the tree Overlay becomes a developers personal space making it difficult to contribute if package is neglected (though that is also a problem now...) Lose repository metadata moving a package from overlay to tree Reduced responsibility for package QA (I expect we don't care about overlays to become a standard response on bugs.g.o) And what do we gain: Eases testing - can push in alpha quality ebuilds Developers feel safer because they can't break the tree Surely the solution is to provide that safety net within the tree? Rather than pushing changes into a live tree, push them into a testing tree, then after they pass repoman/QA checks, and a build check, apply the changes to the live tree, otherwise email a rejection. And allow developers to add their own testing ebuilds to the tree (for a start, they will be more widely tested). The current system of overlay usage is very annoying for users, particularly when bugs are hanging around with packages in the tree, and after filing bug reports the user is told that the bug is already fixed in the overlay. Or when new packages are added to overlays instead of the tree. How are users expected to find them? Another thing that needs fixing is the massive number of packages that aren't really maintained. Either the maintainer doesn't respond to bugs, or the package is maintained by a herd and so no one feels it's actually their responsibility to deal with the boring bugs, and when some developer outside of the herd comes across it, they feel like they can't fix the bug without stepping on someone's toes. What's worse is that in a lot of these cases there will be a user on bugs.g.o posting fixes and new ebuilds, and yet they never make it into the tree. A system where developer ebuilds are kept in the tree, and users have a way to automatically contribute ebuilds, either human reviewed, or in some reputation based system, would be very useful. Users also need feedback - how many times does a user submit an ebuild via bugzilla to be told that it doesn't meet QA standards? Why isn't there a system in place to run repoman/QA/build checks on user ebuilds/patches to make sure they are fixed *before* being submitted for inclusion into the tree? And if this could be linked to a bug reporting system where people report/fix individual ebuilds or packages, and I can just type 'gbugs -l pkgname' and get a list of problems and fixes that other people have proposed, even better. I'm not sure whether the answer is more openness of the existing system, some custom submission flow system, or a distributed SCM, but I do think there's a lot that could be changed to make things better. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Hi Luis, On 3/23/06, Luis Medinas [EMAIL PROTECTED] wrote: I agree with the wiki because it seems to be an easy way to users and developers comunicate together and work. Like i said a few months ago the documentation won't give any problems to GDP since GDP provides high level docs. The wiki will also help our projects since it can be added TODO's, roadmaps and all that stuff. About the overlay the best way imo is provide a unique overlay with external contribs maintained by users and devs (of course commit rights only for devs.). Every team will have their own preference on how to get the most out of an overlay. No-one's required to use an overlay, and no-one's required to give out access to non-devs if they don't want to. It'll be okay for different teams to have different practices for their overlays. I believe the overlay team's job is to provide the capability, clearly communicate what isn't allowed, provide technical documentation and some examples of practices already in use ... and then get out of your way. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Stuart Herbert wrote: On 3/23/06, Donnie Berkholz [EMAIL PROTECTED] wrote: Stuart Herbert wrote: Developer overlays would only be created for active Gentoo developers, and they would be accountable for its contents. Non-developers will not be given write access to developer overlays. This removes much of the motivation for merging overlays to o.g.o, at least some of the ones I am aware of. Then I'll re-consider. I was hoping that developers would setup projects if there are multiple contributors (project overlays will allow write access to non-devs). I don't think I'm understanding your intent here, because I've read things two different ways. My main goal is to allow easy contribution by non-devs, via allowing them to commit directly to some overlay. How is that possible in your vision? Thanks, Donnie signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
Hi Chris, On 3/23/06, Chris Bainbridge [EMAIL PROTECTED] wrote: I think that the use of overlays is more a symptom of a problem with portage. Overlay problems: [snip] Developers are already using overlays, and some teams (including ones I've been involved in) are actively and successfully using them to help with recruitment and to provide a way to access ebuilds that would otherwise still be rotting in Bugzilla. I'm not saying overlays are for everyone. It's one approach that some groups like. You don't have to use an overlay yourself for your work if it's an approach that doesn't work for you. Overlays are not about to become a mandatory way of working around here. Surely the solution is to provide that safety net within the tree? I cannot imagine a day when non-devs are given commit access to the Portage tree. As long as that limitation remains in place, if we're going to reach out beyond our developer community, we have to reach out beyond the Portage tree too. The current system of overlay usage is very annoying for users, particularly when bugs are hanging around with packages in the tree, and after filing bug reports the user is told that the bug is already fixed in the overlay. Or when new packages are added to overlays instead of the tree. How are users expected to find them? Users have pre-conceived ideas about the contents of the Portage tree. I've seen first-hand how badly users react when a hard-masked package in the tree is withdrawn because it was an experimental approach that ultimately failed. Users have unrealistic expectations about the tree. If developers are telling users in Bugzilla that bugs are fixed in the overlay, it's the responsibility of the developers to tell the users where to go to get those fixes. I'd have thought that was basic common sense. Establishing overlays.g.o as the usual place to go will help with this, as will promoting very helpful tools such as layman. Another thing that needs fixing is the massive number of packages that aren't really maintained. That's a very valid concern. Overlays can help here, as explained by the Haskell team, because they make it possible for more people to contribute. They're more than a technical solution ... they're a social tool to build a more sustainable team around. What's worse is that in a lot of these cases there will be a user on bugs.g.o posting fixes and new ebuilds, and yet they never make it into the tree. I'm finding that overlays address this specific scenario very successfully. I talk to those users and find out if they're willing to contribute to an overlay. More often than not, I find that the fixes and new ebuilds are coming from a personal overlay that the user is maintaining anyway. Being able to commit directly to a shared overlay means that they can get more involved ... and it gives me a good level of interaction to help decide whether the person is worth recruiting as a full-blown dev or not. A system where developer ebuilds are kept in the tree, and users have a way to automatically contribute ebuilds, either human reviewed, or in some reputation based system, would be very useful. The overlays project doesn't prevent this system being developed or introduced at all. We're not looking to corner the market at all. We're only providing a resource which will be useful to some teams and developers. Users also need feedback [snip] You have good ideas. What are you doing to make them happen? I'm not sure whether the answer is more openness of the existing system, some custom submission flow system, or a distributed SCM, but I do think there's a lot that could be changed to make things better. I don't think that there is any one approach that will work for all devs, all non-devs, all the time. What I'm doing here is resourcing one specific approach, and working with Infra and User Relations to deliver something that provides one bridge between the developer and non-dev community. We will need other bridges to be built too. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Hi Donnie, On 3/23/06, Donnie Berkholz [EMAIL PROTECTED] wrote: I don't think I'm understanding your intent here, because I've read things two different ways. My main goal is to allow easy contribution by non-devs, via allowing them to commit directly to some overlay. How is that possible in your vision? That's possible because a) non-devs can be given commit access to the overlay's svn/a.n.other repository b) non-devs can be given create/edit access to the overlay's wiki / GuideXSL I do this already for the overlays I host on svn.gnqs.org. I can't migrate those overlays to overlays.g.o if we don't grant non-dev commit access! The confusion is probably because, in the original vision statement, I said that these things would only happen for overlays setup by, and for, official projects. I wanted a disctinction between who could commit to overlays run by projects, and who could commit to overlays run by individual developers. However, if the concensus is that we want non-devs to be able to commit to individual developer overlays too, we'll make that possible. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Stuart Herbert wrote: The confusion is probably because, in the original vision statement, I said that these things would only happen for overlays setup by, and for, official projects. I wanted a disctinction between who could commit to overlays run by projects, and who could commit to overlays run by individual developers. However, if the concensus is that we want non-devs to be able to commit to individual developer overlays too, we'll make that possible. Ah, now it's clear. I didn't understand that distinction earlier, and I could deal with this proposal. But it seems rather artificial to me, and I suspect some devs might enjoy contributions to their non-topical overlays. Thanks, Donnie signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
But it seems rather artificial to me, and I suspect some devs might enjoy contributions to their non-topical overlays. It *is* artificial; that's fair critisism. I have a personal bias towards projects. I'll withdraw the distinction. So, to be clear: the owners of an overlay (the leads for a project, the developer for a developer overlay) can request that a non-dev be given commit rights to the overlay's VCS and / or wiki. The owners of an overlay are responsible for everything that happens to an overlay - including the contributions of non-developers. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On 23/03/06, Stuart Herbert [EMAIL PROTECTED] wrote: Developers are already using overlays, and some teams (including ones I've been involved in) are actively and successfully using them to help with recruitment and to provide a way to access ebuilds that would otherwise still be rotting in Bugzilla. Developers are using overlays, however, the majority of users aren't. If the usage of overlays is to increase, then remote overlay support should be added to emerge. Additionally, in order for users to be able to contribute to the overlays, it would help if they had anonymous read access. Surely the solution is to provide that safety net within the tree? I cannot imagine a day when non-devs are given commit access to the Portage tree. As long as that limitation remains in place, if we're going to reach out beyond our developer community, we have to reach out beyond the Portage tree too. The safety net was intended for developers. Packages often get broken in unexpected ways - something depends on it, a patch is conditional on some USE flag or arch etc. It would be useful to get an email 5 minutes after a commit saying you broke something, rather than a bug report being filed a week later. The current system of overlay usage is very annoying for users, particularly when bugs are hanging around with packages in the tree, and after filing bug reports the user is told that the bug is already fixed in the overlay. Or when new packages are added to overlays instead of the tree. How are users expected to find them? Users have pre-conceived ideas about the contents of the Portage tree. I've seen first-hand how badly users react when a hard-masked package in the tree is withdrawn because it was an experimental approach that ultimately failed. Users have unrealistic expectations about the tree. I don't think it is unrealistic to expect that if a user puts a lot of work into an ebuild, and it works, then it should somehow end up in the tree. Unfortunately the options at the moment are to either reject the ebuild, leave it to rot in bugzilla, or recruit the user as a developer. Rejecting isn't very nice, the amount of effort and barriers to become a dev are too great, so we end up with good ebuilds not being added. Adding the ebuilds to overlays is one option, but then other users will be expected to find an overlay with their package in, and then add it to make.conf. As the number of overlays increases, the amount of effort in synchronising dependencies and dealing with other problems between them will increase. Maybe in the real world managing the relationships between overlays won't be as big a problem as it appears it could be. [snip] You have good ideas. What are you doing to make them happen? Not much - time is a great constraint, and writing emails takes much less time than writing code... -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On Wed, 2006-03-22 at 22:03 +, Stuart Herbert wrote: To answer Daniel's other question, about bugs.g.o ... trac on overlays.g.o will have its bug tracking system disabled. We already have one bug tracking system - bugs.g.o - and that's sufficient. Umm... no? If some random developer goes out there and creates his own fork of catalyst in his overlay, I sure don't want to receive a *single* bug on it. Ever. If you're already using Trac, you should keep the bug tracking enabled, so the bugs stay with the overlay. Once something moves into the official tree, then it can use bugs.gentoo.org for its bug tracking. This means developers that don't wish to participate in the overlays are not forced to waste their time troubleshooting problems in these overlays and can focus on our *core* product, the portage tree. I have no problem with someone, for example, making their own fork of catalyst for testing new stuff and then, after extensive testing, submitting it upstream to the official repository, but forcing me to waste my time trying to figure out that some random developer out there has made a fork that does something different from the official version when a user has a bug is completely counter-productive. -- Chris Gianelloni Release Engineering - Strategic Lead x86 Architecture Team Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Official overlay support
Hi Chris, On 3/23/06, Chris Gianelloni [EMAIL PROTECTED] wrote: If some random developer goes out there and creates his own fork of catalyst in his overlay, I sure don't want to receive a *single* bug on it. Ever. Your nightmare scenario seems unavoidable. Enabling per-overlay bug tracking doesn't stop users posting bugs in bugzilla. It just causes confusion for users, because they're not sure where to go. Normally, it's not a problem - because the overlay contributors are normally the owners of the real package. A hostile fork of Catalyst is very much a special case. What we could do is say that overlays are for package trees only; ie they are not general-purpose repositories for holding source trees. That would ensure that your nightmare scenario is even less likely to happen, and that if it does, it's through no fault of the overlays project :) Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On Thu, 2006-03-23 at 10:09 +, Chris Bainbridge wrote: Reduced responsibility for package QA (I expect we don't care about overlays to become a standard response on bugs.g.o) You will *definitely* get this from developers that won't be using the overlays. Let's just say you decide to use a toolchain overlay and it exposes some problem in random app foo because you're using gcc 5.1.99 and we only have 4.0 in the tree. You file a bug against package foo without a patch. I'm the maintainer. You've now made me spend my time supporting something that isn't even in the tree, and could be an artifact of the overlay itself and something that will *never* end up in the tree. Why should I do this? What we have done here is actually *reduced* the amount of productive work that I can do by forcing me to deal with these overlays, even if I choose not to participate. Surely the solution is to provide that safety net within the tree? Rather than pushing changes into a live tree, push them into a testing tree, then after they pass repoman/QA checks, and a build check, apply the changes to the live tree, otherwise email a rejection. And allow developers to add their own testing ebuilds to the tree (for a start, they will be more widely tested). While this would be great, there is one major obstacle that people miss. Horsepower. Let's say I add a new openoffice ebuild to the tree for a security bug. We now have to wait how many hours for it to hit the tree? What if the KDE team is committing a new KDE version at the same time? I'm sure you can guess how this could bring all of our development to a complete and grinding halt. I wouldn't mind seeing an actual unstable designation added to KEYWORDS. The basic premise would be like package.mask packages where things can be done *within the tree* but still has the same air of this might be totally busted at some point as overlays. Users would be very unlikely to run unstable on their machines. Heck, we could even make it impossible to actually use unstable globally if we wanted to. The whole point is that I completely agree with you, a better solution would be to try to work within the tree to resolve this problem, rather than moving it outside the tree and into hundreds of individual overlays, which could be conflicting with each other. The current system of overlay usage is very annoying for users, particularly when bugs are hanging around with packages in the tree, and after filing bug reports the user is told that the bug is already fixed in the overlay. Or when new packages are added to overlays instead of the tree. How are users expected to find them? They should not have to. It's as simple as that. Bugs in the tree should be fixed in the tree. New packages should be added to the tree. If it isn't a candidate for ~arch, then add it under package.mask instead. That is why we have package.mask in the first place. Another thing that needs fixing is the massive number of packages that aren't really maintained. Either the maintainer doesn't respond to bugs, or the package is maintained by a herd and so no one feels it's actually their responsibility to deal with the boring bugs, and when some developer outside of the herd comes across it, they feel like they can't fix the bug without stepping on someone's toes. What's worse is that in a lot of these cases there will be a user on bugs.g.o posting fixes and new ebuilds, and yet they never make it into the tree. This is really both a political/social problem and one of manpower. There's a lot more users out there than there are developers. There are many developers out there who aren't quite so territorial. The main thing is us being civil with each other. There are many times where a developer wants to make a fix or change to a game. If they ask us, we let them. It's that simple. Heck, we usually ask them if they want to maintain it. Also, a herd is a group of packages, not a group of developers. A group of developers could share responsibility in looking out for a herd (or many) but they are not a herd themselves. A system where developer ebuilds are kept in the tree, and users have a way to automatically contribute ebuilds, either human reviewed, or in some reputation based system, would be very useful. Users also need feedback - how many times does a user submit an ebuild via bugzilla to be told that it doesn't meet QA standards? Why isn't there a system in place to run repoman/QA/build checks on user ebuilds/patches to make sure they are fixed *before* being submitted for inclusion into the tree? And if this could be linked to a bug reporting system where people report/fix individual ebuilds or packages, and I can just type 'gbugs -l pkgname' and get a list of problems and fixes that other people have proposed, even better. Ebuilds that are human reviewed are exactly what we have now. The process isn't automated, but it cannot be automated if you
Re: [gentoo-dev] Official overlay support
I personally think this is a bad idea. I can understand and support links to different overlay repositories, however I do not think that gentoo should host or support overlays on its own infrastructure. For one thing supporting overlays on our infrastructure looks like we are supporting broken ebuilds. This will also lead to more confusion with users who find these official overlays and then the overlays conflict with each other and cause problems that leads to comments like well gentoo should just know how to fix it and make it all work. I also think that this overlay structure will not provide incentives for people to commit to the main tree. They will get their ebuild in an overlay and its hosted on gentoo and distributed to the mirrors. At that point its easy for them to continue to use the overlay. Over time the overlay will diverge more and more from other overlays and even the main tree. If this still goes forward I think we should look at the debian layout where there is the core product and then the security branches etc. Personally I think this is going to cause more bug reports and less updates to the main tree. I also agree that a hostile fork is unlikely, however it is more possible with the overlay layout as anyone can get an ebuild mirrored on our infrastructure at this point. Another thing to concider is how would people be able to contribute to the overlays? Is there a review process? Is there a checkin process? If no then anyone can post a malicious ebuild that would be a security nightmare. I think this security viewpoint alone is enough to re-evaluate this plan of action. Eric On 14:41 Thu 23 Mar , Stuart Herbert wrote: Hi Chris, On 3/23/06, Chris Gianelloni [EMAIL PROTECTED] wrote: If some random developer goes out there and creates his own fork of catalyst in his overlay, I sure don't want to receive a *single* bug on it. Ever. Your nightmare scenario seems unavoidable. Enabling per-overlay bug tracking doesn't stop users posting bugs in bugzilla. It just causes confusion for users, because they're not sure where to go. Normally, it's not a problem - because the overlay contributors are normally the owners of the real package. A hostile fork of Catalyst is very much a special case. What we could do is say that overlays are for package trees only; ie they are not general-purpose repositories for holding source trees. That would ensure that your nightmare scenario is even less likely to happen, and that if it does, it's through no fault of the overlays project :) Best regards, Stu -- gentoo-dev@gentoo.org mailing list pgpTLZmlX4YYR.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
Chris Gianelloni wrote: On Wed, 2006-03-22 at 22:03 +, Stuart Herbert wrote: To answer Daniel's other question, about bugs.g.o ... trac on overlays.g.o will have its bug tracking system disabled. We already have one bug tracking system - bugs.g.o - and that's sufficient. Umm... no? If some random developer goes out there and creates his own fork of catalyst in his overlay, I sure don't want to receive a *single* bug on it. Ever. If you're already using Trac, you should keep the bug tracking enabled, so the bugs stay with the overlay. Once something moves into the official tree, then it can use bugs.gentoo.org for its bug tracking. This means developers that don't wish to participate in the overlays are not forced to waste their time troubleshooting problems in these overlays and can focus on our *core* product, the portage tree. Well, I don't care much, as long as: - there's a separate Overlays product in bugzilla for this - each such overlay has its own component under Overlay product with default assignees set up (no, I won't check out all those overlays to find out the maintainer, also, almost none of them uses metadata.xml) - users are *vigorously* :P instructed to file the bugs under that product/component and/or (?) mark them with something like [overlay-xxx] so that I don't have to ponder who's maintaining that thing. If they don't, the bugs end up as invalid b/c the ebuild is not in portage. Easy enough. While keeping those bugs in trac bug trackers seemed as a good idea to me originally, most users are simply unable to do that anyway. We tried w/ php overlay, didn't work much. -- Best regards, Jakub Moc mailto:[EMAIL PROTECTED] GPG signature: http://subkeys.pgp.net:11371/pks/lookup?op=getsearch=0xCEBA3D9E Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95 B30F 8717 D5FD CEBA 3D9E ... still no signature ;) signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
On Thu, 2006-03-23 at 14:41 +, Stuart Herbert wrote: Hi Chris, On 3/23/06, Chris Gianelloni [EMAIL PROTECTED] wrote: If some random developer goes out there and creates his own fork of catalyst in his overlay, I sure don't want to receive a *single* bug on it. Ever. Your nightmare scenario seems unavoidable. Enabling per-overlay bug tracking doesn't stop users posting bugs in bugzilla. It just causes confusion for users, because they're not sure where to go. Normally, it's not a problem - because the overlay contributors are normally the owners of the real package. No, it does not stop them, but it sure will curb the number of users posting their bugs to the wrong place. Remember that only more advanced users are the ones using overlays. We won't have Joe Sixpack using an overlay. Instead it'll be Bob Developer-to-be. How is that confusing? I went to http://overlays.gentoo.org/catalyst-ng and saw the overlay. I also saw the link the the bug tracker. A hostile fork of Catalyst is very much a special case. No. It isn't. Look in many developer overlays and you'll see packages that they have made that work how *they* want them to, even if it is *very* different from what is in the tree. This is the case for packages that are not maintained by them, too. Any ebuild that is done by someone that isn't the maintainer is a fork. There's nothing hostile about it. What we could do is say that overlays are for package trees only; ie they are not general-purpose repositories for holding source trees. That would ensure that your nightmare scenario is even less likely to happen, and that if it does, it's through no fault of the overlays project :) It has nothing to do with a source tree. I could store the source tree, and tarball, anywhere. The ebuilds to use these tarballs would be just as dangerous. I see no problem with overlays in concept, such as the php overlay that is very successful. The main reason that it is successful is because the same people that maintain php maintain the overlay. Yes, there are other contributors, but the maintainers of the overlay are still the developers. I see no problem with providing these sorts of overlays to bridge the gap between contributing users and developers. I *do* see a problem with simply allowing random overlays from any developer for anything. -- Chris Gianelloni Release Engineering - Strategic Lead x86 Architecture Team Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Official overlay support
On 3/23/06, Chris Gianelloni [EMAIL PROTECTED] wrote: I see no problem with providing these sorts of overlays to bridge the gap between contributing users and developers. I *do* see a problem with simply allowing random overlays from any developer for anything. That's a reasonable point, and it's an experience I've not personally been on the receiving end of. But the flip side is reasonable too. Our governing metastructure is explicitly clear that: a) Projects are not mandatory, and b) That competing projects are allowed I don't see how we can limit overlays to just packages owned by the developers who own the overlay without violating those two policies. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On 23/03/06, Chris Gianelloni [EMAIL PROTECTED] wrote: On Thu, 2006-03-23 at 14:41 +, Stuart Herbert wrote: Your nightmare scenario seems unavoidable. Enabling per-overlay bug tracking doesn't stop users posting bugs in bugzilla. It just causes confusion for users, because they're not sure where to go. Normally, it's not a problem - because the overlay contributors are normally the owners of the real package. No, it does not stop them, but it sure will curb the number of users posting their bugs to the wrong place. Remember that only more advanced users are the ones using overlays. We won't have Joe Sixpack using an overlay. Instead it'll be Bob Developer-to-be. If the software a user wants is in an overlay, then the user will be forced to install the overlay. Another thing that some people may not have considered - with many developers using various permutations of overlays, how can you guarantee that what is being checked into the main tree will build for a normal user? In order to test that, a developer would have to disable all overlays, unemerge everything provided by the overlays, and then build and test with a plain non-overlay gentoo. That's a lot of work; I doubt most developers are doing it. There is a discussion on the forums at the moment along a similar topic http://forums.gentoo.org/viewtopic-t-443469.html - the vote seems to indicate 58% of users are not really happy with the way the portage tree is handled. -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Chris Bainbridge wrote: Another thing that some people may not have considered - with many developers using various permutations of overlays, how can you guarantee that what is being checked into the main tree will build for a normal user? In order to test that, a developer would have to disable all overlays, unemerge everything provided by the overlays, and then build and test with a plain non-overlay gentoo. That's a lot of work; I doubt most developers are doing it. You are wrong in my case. I have a vanilla root, which used to be just a chroot, but now i'm trying out vmware, where I test all the changes I make. I think many other devs test in a similar manner (on another box, chroot, ...). /Ehmsen -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Chris Gianelloni wrote: I wouldn't mind seeing an actual unstable designation added to KEYWORDS. The basic premise would be like package.mask packages where things can be done *within the tree* but still has the same air of this might be totally busted at some point as overlays. Users would be very unlikely to run unstable on their machines. Heck, we could even make it impossible to actually use unstable globally if we wanted to. Sounds like package.mask / -* right now. Thanks, Donnie signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
On 3/23/06, Chris Gianelloni [EMAIL PROTECTED] wrote: Think about it this way, what if we had two competing products in the tree that do the same thing, with the same file names? We would add a blocker, no? So what mechanism is there to ensure that there's no blocking issues between an official in-tree project, and these external overlays that are not in the tree? With the tree, we have a well-defined policy on this. What policy would we use for the overlays? What about if we just skip your policies and let the overlays be a free place where people can handle issues how they think it is right for the specific case and not how $super_dev said somewhere. That is what overlays are about, not? - Stefan -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On Thu, 23 Mar 2006 19:31:24 +0100 Stefan Schweizer [EMAIL PROTECTED] wrote: | What about if we just skip your policies and let the overlays be a | free place where people can handle issues how they think it is right | for the specific case and not how $super_dev said somewhere. That is | what overlays are about, not? Sounds like a perfect way to break lots and lots of systems. Those policies are mostly there for good reason. -- Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-dev] Official overlay support
Ciaran McCreesh wrote: On Thu, 23 Mar 2006 19:31:24 +0100 Stefan Schweizer [EMAIL PROTECTED] wrote: | What about if we just skip your policies and let the overlays be a | free place where people can handle issues how they think it is right | for the specific case and not how $super_dev said somewhere. That is | what overlays are about, not? Sounds like a perfect way to break lots and lots of systems. Those policies are mostly there for good reason. You want to apply policies on overlays? Well no - sorry, overlays are none of QA's or any other policy business. They are overlays, not official tree. If user installs ebuilds from overlay and breaks his system, then well - not a Gentoo problem. Why should any policies apply? -- Best regards, Jakub Moc mailto:[EMAIL PROTECTED] GPG signature: http://subkeys.pgp.net:11371/pks/lookup?op=getsearch=0xCEBA3D9E Primary key fingerprint: D2D7 933C 9BA1 C95B 2C95 B30F 8717 D5FD CEBA 3D9E ... still no signature ;) signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
On Thu, 2006-03-23 at 19:31 +0100, Stefan Schweizer wrote: On 3/23/06, Chris Gianelloni [EMAIL PROTECTED] wrote: Think about it this way, what if we had two competing products in the tree that do the same thing, with the same file names? We would add a blocker, no? So what mechanism is there to ensure that there's no blocking issues between an official in-tree project, and these external overlays that are not in the tree? With the tree, we have a well-defined policy on this. What policy would we use for the overlays? What about if we just skip your policies and let the overlays be a free place where people can handle issues how they think it is right for the specific case and not how $super_dev said somewhere. That is what overlays are about, not? I'm fine with that, so long as we keep them *far* from *any* Gentoo infrastructure. Once it hits *.gentoo.org then it needs to follow some basic policy. Otherwise, it is allowing anyone to completely bypass any policies we have and allows anyone to cause any kind of breakages that they want, with exactly 0 repercussions. I'm sorry, but I am not OK with just standing by and watching us give complete access to do anything with no accountability. If you are, perhaps you really need to rethink your commitment to our users and your fellow developers. -- Chris Gianelloni Release Engineering - Strategic Lead x86 Architecture Team Games - Developer Gentoo Linux signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Official overlay support
On Thursday 23 March 2006 13:57, Jakub Moc wrote:
Ciaran McCreesh wrote:
On Thu, 23 Mar 2006 19:31:24 +0100 Stefan Schweizer
[EMAIL PROTECTED] wrote:
| What about if we just skip your policies and let the overlays be a
| free place where people can handle issues how they think it is right
| for the specific case and not how $super_dev said somewhere. That is
| what overlays are about, not?
Sounds like a perfect way to break lots and lots of systems. Those
policies are mostly there for good reason.
You want to apply policies on overlays? Well no - sorry, overlays are
none of QA's or any other policy business. They are overlays, not
official tree. If user installs ebuilds from overlay and breaks his
system, then well - not a Gentoo problem. Why should any policies apply?
That is not what Stuart said, he indicated that overlays would be treated as
supported systems including the use of our bugzilla system to track defects.
If that is the case it crosses the line into the land of the official in
which case policys start to be applied. While I agree that it would be very
useful to have any overlays centrally located I do find the term Official
overlay to be insanely oxymoronic in your use of the term overlay, which I
gather to be a set of ebuilds not bound by the QA and/or security
requirements of the portage tree.
If I understand correctly what you are pushing then is a Official overlay
respository for Unofficial overlays
You can't have it both ways, either they are wholey Unofficial and do not get
tracked in bugzilla at all (something which would have to be made VERY clear
to our users, e.g. a you use it you get to keep the pieces policy, and the
developer or team in question is the *only* point of contact for fixing
things) -or- it is an Official overlay with official support which means it
needs to abide by the rules...
--
Daniel Ostrow
Gentoo Foundation Board of Trustees
Gentoo/{PPC,PPC64,DevRel}
[EMAIL PROTECTED]
pgpQyEPKB2Xy4.pgp
Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Thu, 2006-03-23 at 13:55 -0500, Chris Gianelloni wrote: I'm sorry, but I am not OK with just standing by and watching us give complete access to do anything with no accountability. If you are, perhaps you really need to rethink your commitment to our users and your fellow developers. The way the Haskell team manages this is that we don't tell our end users about our testing overlay. So we don't get bug reports from them. We have three outside contributers with write access to the overlay repo. They make changes in consultation with the team. So we're not giving complete access without accountability. We have a couple other users who use the overlay but they know what they're doing. We don't make the overlay that easy to use on purpose because we don't want inexperienced users using it. So apart from not advertising it, we don't keep digests in the repo. I think the point is that these overlays should be a useful way of getting contributers more closely involved. However we should not encourage end users to use these overlays without thinking. For example using more than one at once seems like a really bad idea. Perhaps if we make them sufficiently hard to use then end users will not use them and we'll just get the contributers we want. -- Duncan Coutts : Gentoo Developer (Haskell herd team lead) email : dcoutts at gentoo dot org -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On 3/23/06, Daniel Ostrow [EMAIL PROTECTED] wrote: You can't have it both ways, either they are wholey Unofficial and do not get tracked in bugzilla at all (something which would have to be made VERY clear to our users, e.g. a you use it you get to keep the pieces policy, and the developer or team in question is the *only* point of contact for fixing things) -or- it is an Official overlay with official support which means it needs to abide by the rules... I think we need both: An aggregation of unofficial and clearly stated unofficial overlays which are currently in place. And an official gentoo user and developer overlay, where ebuilds have to conform to policy. The difference between official and unofficial: bugs can be on bugs.gentoo.org or not. Then both goals would be met: Access to a gentoo-overlay for non-developers and an aggregation of all gentoo overlays out there. - Stefan -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On Thu, 2006-03-23 at 18:55 +, Chris Bainbridge wrote: On 23/03/06, Chris Gianelloni [EMAIL PROTECTED] wrote: On Thu, 2006-03-23 at 16:40 +, Chris Bainbridge wrote: If the software a user wants is in an overlay, then the user will be forced to install the overlay. It shouldn't be in the overlay, is I think the point many are trying to make. If the software is good enough for any of our users, it should be good enough for the tree. I agree. I would ask, what are the advantages of overlays that developers find so compelling that they use them rather than the portage tree? Would it not be a better idea to find a way to bring those advantages to the tree, rather the proliferation of overlays we are seeing? The advantages we see are: We use it as a staging area for our herd's ebuilds. We can start with an untested ebuild and between several team members and outside testers we can iteratively test and refine the ebuild. This relies on a low latency between committing changes and other devs and outside testers receiving those changes. We have a lag of several seconds rather than 30 minutes for the anoncvs. It means we get much higher QA before ebuilds actually end up in portage because by the time they get there they have been reviewed and tested by other team members and outside helpers (often including testing on several arches). If we did this in the cvs tree we'd need to keep the packages masked all the time we were improving them (overhead). We'd need changelog entries for every change (overhead). We wouldn't be able to share the development and testing with our outside helpers (due to anoncvs lag). And of course we wouldn't be able to grant out outside helpers write access. So the lower latency helps to run an AT-style system and the write access allows for a safe intermediate stage in the recruitment process between AT and dev status. -- Duncan Coutts : Gentoo Developer (Haskell herd team lead) email : dcoutts at gentoo dot org -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
On 3/23/06, Daniel Ostrow [EMAIL PROTECTED] wrote: That is not what Stuart said, he indicated that overlays would be treated as supported systems including the use of our bugzilla system to track defects. If that is the case it crosses the line into the land of the official in which case policys start to be applied. We do need to apply policies to the overlays. We can't have an absolute free-for-all. Projects and individual developers need to be responsible for their overlays - just like they are responsible for the packages they add to the tree. But we are not going to treat overlays like they're the Portage tree. That would undermine the whole point of having the overlays in the first place, and would bring zero benefit to our users and our developers. They are different, and they require a different approach. Best regards, Stu -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
Duncan Coutts wrote: The way the Haskell team manages this is that we don't tell our end users about our testing overlay. So we don't get bug reports from them. We have three outside contributers with write access to the overlay repo. They make changes in consultation with the team. So we're not giving complete access without accountability. Hmmm, that kinda defeats the whole point of having an overlay at all, IMHO. Of course you can have an overlay strictly for internal development between a couple of people, but that's not quite what this debate is about I'd say. I find overlays really useful for testing stuff before it gets into portage - that's completely pointless if the overlay is secret. Also, you are able to find potential future devs among the overlay users, people who actually submit fixes etc. How are you going to get that if noone knows about the overlay? We have a couple other users who use the overlay but they know what they're doing. We don't make the overlay that easy to use on purpose because we don't want inexperienced users using it. So apart from not advertising it, we don't keep digests in the repo. Oh well, seems like you have a very specific use for this, probably not what most users are interested in. I think the point is that these overlays should be a useful way of getting contributers more closely involved. However we should not encourage end users to use these overlays without thinking. For example using more than one at once seems like a really bad idea. Perhaps if we make them sufficiently hard to use then end users will not use them and we'll just get the contributers we want. As said above, how are you going to get new contributors without people that are actually using/testing that stuff? -- jakub signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
As said above, how are you going to get new contributors without people that are actually using/testing that stuff? We find the via Bugzilla and/or irc and point them at the overlay. That way, we more or less know who's using the overlay and make sure they are briefed a bit before they start using it. dcoutts has described the current practice we use in the Haskell team, but that doesn't necessarily mean that it's the only practice that would work for us. I can imagine that if we can come up with reasonable policies for o.g.o, we can switch to a slightly different (i.e., more public) scheme ... Cheers, Andres pgpfWwY8sU09S.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Thursday 23 March 2006 15:54, Eric Edgar wrote: I personally think this is a bad idea. I can understand and support links to different overlay repositories, however I do not think that gentoo should host or support overlays on its own infrastructure. For one thing supporting overlays on our infrastructure looks like we are supporting broken ebuilds. This will also lead to more confusion with users who find these official overlays and then the overlays conflict with each other and cause problems that leads to comments like well gentoo should just know how to fix it and make it all work. I also think that this overlay structure will not provide incentives for people to commit to the main tree. They will get their ebuild in an overlay and its hosted on gentoo and distributed to the mirrors. At that point its easy for them to continue to use the overlay. Over time the overlay will diverge more and more from other overlays and even the main tree. I think that overlays are too useful to not provide. Let me give an example. Currently my office workstation is hosting an overlay for ebuilds for vmware-server. This overlay came about because I wanted to keep it in an overlay/svn repos for myself. This is much preferable than downloading 10+ files from a bugzilla bugreport. As I thought others might appreciate it, I posted the link at the bug. Another dev requested that the contributor (trainee dev actually) get access. I saw no problem with that, so agreed. This setup works quite satisfactory. As the repos is special purpose, I have no doubt that the ebuilds will finally end up in the tree. Currently the software and ebuilds are only beta though so would have to be hard masked. From this experience I agree with Stuart that it can create a good bridge between gentoo and trusted users. Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgpAijzGAYiph.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Thursday 23 March 2006 16:31, Chris Gianelloni wrote: No. It isn't. Look in many developer overlays and you'll see packages that they have made that work how *they* want them to, even if it is *very* different from what is in the tree. This is the case for packages that are not maintained by them, too. Any ebuild that is done by someone that isn't the maintainer is a fork. There's nothing hostile about it. Probably this is the reason that my personal overlay is not public. For example it contains a gtk+ version with the save dialog fixed. I see no problem with overlays in concept, such as the php overlay that is very successful. The main reason that it is successful is because the same people that maintain php maintain the overlay. Yes, there are other contributors, but the maintainers of the overlay are still the developers. I see no problem with providing these sorts of overlays to bridge the gap between contributing users and developers. I *do* see a problem with simply allowing random overlays from any developer for anything. On the other hand, my personal overlay contains various fixes to packages I use, but who are from other developers. As I'm not the maintainer I don't always bother to post bugs, and find it wrong to fix it without telling the maintainer. Other ebuilds concern packages that are unmaintained or not stable enough. It also contains my own kde meta packages that select only that part of kde that I want. I can only assume that other developers have similar overlays too. These overlays form actually a wealth of resources that are hidden away. If there were a semi-public overlay system in which developers could keep their overlays, this might help in getting this out to the public. Paul -- Paul de Vrieze Gentoo Developer Mail: [EMAIL PROTECTED] Homepage: http://www.devrieze.net pgpwEz4cQuAjN.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
Paul de Vrieze wrote: I can only assume that other developers have similar overlays too. These overlays form actually a wealth of resources that are hidden away. If there were a semi-public overlay system in which developers could keep their overlays, this might help in getting this out to the public. Heh, such a thing sort of exists in a non-standardized form at dev.gentoo.org/~developername/overlay/ -- seems to be the way most people go. Donnie signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
Ciaran McCreesh wrote: [Wed Mar 22 2006, 12:33:10PM EST] On Wed, 22 Mar 2006 09:03:38 -0800 Donnie Berkholz [EMAIL PROTECTED] wrote: | This definitely sounds like a fun idea. It would be even cooler if we | were using a distributed SCM on both ends that allowed for easy | merging. Word of warning to anyone planning to implement one of these that includes rsync with cache as a frontend: you will be giving root access to your box to any user with commit access. Could you give some explanation or reference for this? Thanks, Aron pgpgv5X6M9Xir.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
Chris Gianelloni wrote: [Thu Mar 23 2006, 09:41:25AM EST] On Thu, 2006-03-23 at 10:09 +, Chris Bainbridge wrote: Reduced responsibility for package QA (I expect we don't care about overlays to become a standard response on bugs.g.o) You will *definitely* get this from developers that won't be using the overlays. Let's just say you decide to use a toolchain overlay and it exposes some problem in random app foo because you're using gcc 5.1.99 and we only have 4.0 in the tree. You file a bug against package foo without a patch. I'm the maintainer. You've now made me spend my time supporting something that isn't even in the tree, and could be an artifact of the overlay itself and something that will *never* end up in the tree. Why should I do this? What we have done here is actually *reduced* the amount of productive work that I can do by forcing me to deal with these overlays, even if I choose not to participate. Some of this could be mitigated with some additional or modified tools. For example, emerge --info could be augmented to take a package argument and list the installed dependency tree for that package. The list could also include *where* the package and deps came from, PORTDIR or an overlay. The result would be required information in a bug report, similar to the existing emerge --info requirement. So if I were submitting a report about keychain, I would be required to include the result of emerge --info keychain It becomes a lot easier for devs to determine that a problem might be due to an overlay, then take whatever action they prefer based on that information. For some devs, the fact that gcc-5.1.99 breaks their package might be a welcome early warning. Another possible enhancement would be to include a checkbox in the bug report to indicate whether overlays are in use. Hopefully checked by the reporter, but alternatively auto-detected by emerge --info in comment #1, or checked by our ever-vigilant wranglers. This would make winnowing of overlay-caused bugs easier. Just some thoughts... Aron pgpwrrFEvKPXO.pgp Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Thu, 23 Mar 2006 19:57:07 +0100 Jakub Moc [EMAIL PROTECTED] wrote: | Sounds like a perfect way to break lots and lots of systems. Those | policies are mostly there for good reason. | | You want to apply policies on overlays? Well no - sorry, overlays are | none of QA's or any other policy business. They are overlays, not | official tree. If user installs ebuilds from overlay and breaks his | system, then well - not a Gentoo problem. Why should any policies | apply? It is a Gentoo problem, because Gentoo gets innundated with bogus bug reports when users screw up their systems in weird ways and don't realise the cause. -- Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-dev] Official overlay support
Stuart Herbert wrote: I've been very happy with using svn+trac overlays to bridge this gap. They provide a sandbox for contributions to be shared and evaluated. They provide a place where I've been able to give commit access to non-devs, so that they can learn the ropes w/out threatening the Portage tree proper. They provide a place where people who just want to write docs for a single package can contribute. Overlays create a sense of participation that's lacking with Bugzilla patch submissions. Backed up with regular communication (I recommend not recruiting anyone who won't spend time in the IRC channels, but that's a personal preference), they help us get things done quicker. The downside with overlays at the moment is that they're scattered around the net, and if you don't know where to look they can be very hard to find. I've been talking with infra about providing overlays.g.o as a central hosting service for herd and individual developer overlays. Infra have been very supportive of the idea. I just need to free up some time to get the service launched. This definitely sounds like a fun idea. It would be even cooler if we were using a distributed SCM on both ends that allowed for easy merging. Donnie signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] Official overlay support
On Wednesday 22 March 2006 12:03, Donnie Berkholz wrote:
Stuart Herbert wrote:
I've been very happy with using svn+trac overlays to bridge this gap.
They provide a sandbox for contributions to be shared and evaluated.
They provide a place where I've been able to give commit access to
non-devs, so that they can learn the ropes w/out threatening the
Portage tree proper. They provide a place where people who just want
to write docs for a single package can contribute.
Overlays create a sense of participation that's lacking with Bugzilla
patch submissions. Backed up with regular communication (I recommend
not recruiting anyone who won't spend time in the IRC channels, but
that's a personal preference), they help us get things done quicker.
The downside with overlays at the moment is that they're scattered
around the net, and if you don't know where to look they can be very
hard to find. I've been talking with infra about providing
overlays.g.o as a central hosting service for herd and individual
developer overlays. Infra have been very supportive of the idea. I
just need to free up some time to get the service launched.
This definitely sounds like a fun idea. It would be even cooler if we
were using a distributed SCM on both ends that allowed for easy merging.
I agree, I'd love to see something like this, that way I could have my xfce
stuff someplace more public then my devspacethe only thing that would
have to be clear is how official the overlays actually were, e.g. how prone
the team looking after the overlay would be to accepting bugs via the usual
b.g.o channels etc.
--
Daniel Ostrow
Gentoo Foundation Board of Trustees
Gentoo/{PPC,PPC64,DevRel}
[EMAIL PROTECTED]
pgpwacZblRAun.pgp
Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Wed, 22 Mar 2006 09:03:38 -0800 Donnie Berkholz [EMAIL PROTECTED] wrote: | This definitely sounds like a fun idea. It would be even cooler if we | were using a distributed SCM on both ends that allowed for easy | merging. Word of warning to anyone planning to implement one of these that includes rsync with cache as a frontend: you will be giving root access to your box to any user with commit access. -- Ciaran McCreesh : Gentoo Developer (Wearer of the shiny hat) Mail: ciaranm at gentoo.org Web : http://dev.gentoo.org/~ciaranm signature.asc Description: PGP signature
Re: [gentoo-dev] Official overlay support
On Wed, 2006-03-22 at 09:03 -0800, Donnie Berkholz wrote: Stuart Herbert wrote: I've been very happy with using svn+trac overlays to bridge this gap. They provide a sandbox for contributions to be shared and evaluated. They provide a place where I've been able to give commit access to non-devs, so that they can learn the ropes w/out threatening the Portage tree proper. They provide a place where people who just want to write docs for a single package can contribute. Overlays create a sense of participation that's lacking with Bugzilla patch submissions. Backed up with regular communication (I recommend not recruiting anyone who won't spend time in the IRC channels, but that's a personal preference), they help us get things done quicker. The downside with overlays at the moment is that they're scattered around the net, and if you don't know where to look they can be very hard to find. I've been talking with infra about providing overlays.g.o as a central hosting service for herd and individual developer overlays. Infra have been very supportive of the idea. I just need to free up some time to get the service launched. This definitely sounds like a fun idea. It would be even cooler if we were using a distributed SCM on both ends that allowed for easy merging. We do this within the Haskell herd and with a small handful of outside contributers. We use darcs for our distributed SCM which makes the merging trivial. If works like so: We keep our testing ebuilds in a shared overlay managed with darcs. The Haskell herd members have write access. Trusted external contributers have write access to the overlay too (mostly people who are in the middle of the recruitment process). The existing devs are of course responsible for actually committing anything to portage cvs. Other contributers have read only access but they can darcs send us patches. These do not automatically get applied (as with our trusted contributers) but get emailed to us and any Haskell herd dev can review and apply patches sent in this manner quite easily. We have found that this system works rather well. It allows our testers and helpers to participate in writing ebuilds which has made the recruitment process smoother. It provides an intermediate phase in the recruitment process where they can participate in the herd's work without any danger of messing up portage cvs. Bugzilla is ok for submitting whole new ebuilds but it's got far too large an overhead for one line patches. Using darcs gives us a very low overhead. We also use the shared overlay as a testing zone for our herd's ebuilds. Our modus-operandi is to commit changes in ebuilds to the overlay, get peer review from other herd members and then commit to portage when we are satisfied. Of course this also makes it easy for our testers to keep up with the latest versions of ebuilds. With the combination of darcs and irc, we can get very quick turnaround on our testers finding bugs to fixing them and getting those changes back to our testers. -- Duncan Coutts : Gentoo Developer (Haskell herd team lead) email : dcoutts at gentoo dot org -- gentoo-dev@gentoo.org mailing list
Re: [gentoo-dev] Official overlay support
This definitely sounds like a fun idea. It would be even cooler if we were using a distributed SCM on both ends that allowed for easy merging. Donnie It's probably the right time for me to flesh out what I've been planning for overlays.g.o. The vision I have for overlays.g.o is one official home for all Gentoo overlays run by projects and by individual Gentoo devs. I see the homepage itself running Planet (just like planet.g.o), using the RSS feeds from the overlays to display the changelogs from all the overlays. The links down the left hand side of the page go to the homepage for each of the overlays. The homepages are separate wiki instances. http://overlays.g.o/proj/project-name/ for overlays run by herds listed in herds.xml http://overlays.g.o/svn/project-name/ for the SVN repos and http://overlays.g.o/dev/developer/ for overlays run by individual developers http://overlays.g.o/svn/developer/ for the SVN repos There's a practical limit to the amount of software we can support on overlays.g.o. The less we install, the less the overhead of administrating this system for infra and the overlays admin team (I'm looking for responsible volunteers to join that team, if you're interested). I'd like to offer two wiki engines and two version control systems on overlays.g.o. I believe that gives us enough choice without us loading the box with too much software for us to keep on top of. One thing that was never planned was any form of shell access to this box, except for the team creating/destroying overlays. It looks like this will be necessary to support a distributed vcs. I'll talk to infra and see what we could do about providing some form of ssh access to help us support a distributed vcs. Trac and SVN would be my first choice. MoinMoin would be my recommendation for the second wiki engine. What should the second version control system be? I don't use them, I have no experience with them, and so I have no preference of what this should be. To answer Daniel's question about official ... the overlays hosted on overlays.g.o would be official. The overlays project will be accountable for overlays.g.o overall. It would make sense for the overlays project to be a sub-project of infra. To ensure officialness and (what I personally care more about) accountability, project overlays will be created for projects that meet the description of a project in the metastructure [1]. The overlays team will have to be strict on this, to ensure officialness. The overlay must be requested by one of the leads of the project. The lead(s) would be jointly accountable for the overlay and all its contents. Leads will be able to ask for commit / wiki edit access for non-devs. Developer overlays would only be created for active Gentoo developers, and they would be accountable for its contents. Non-developers will not be given write access to developer overlays. By default - working on the same principle of trust that governs all developers w.r.t. the Portage tree - all developers will be able to commit to all overlays. If we can't trust you to respect other people's overlays, then we can't trust you with commit access to the Portage tree, and you're not fit to be a Gentoo dev in the first place :P The only restriction will be that you'll need to ask the overlay project team to setup your access the very first time. Anyone wanting a secret overlay needs to make their own hosting arrangements. To answer Daniel's other question, about bugs.g.o ... trac on overlays.g.o will have its bug tracking system disabled. We already have one bug tracking system - bugs.g.o - and that's sufficient. [1] http://www.gentoo.org/proj/en/glep/glep-0039.html Best regards, Stu -- gentoo-dev@gentoo.org mailing list
