Re: Discussion of guidelines for additional version control mechanisms (fwd)
> Certainly -- the intent that I expressed in my original e-mail was to fish > for people's thoughts on the issue, which would then be codified in some > form. I'm interested in hearing a little more back on how people feel > about the notion of how larger projects should coordinate work given the > assumption that "locks" are advisory, but plan to write up a strawman > sometime in the next week or so. Probably the process forward there will > be to run the first pass by the core team for some immediate feedback > regarding the strength of the recommendations, and whether it meets our > goals concerning addressing the problems that have been discussed. Then > I'll spit a copy out for more general comments. Sounds like a great plan to me. Alas I think you'll get more feedback once you have the strawman proposal, people just seem to work that way. Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Thu, 28 Feb 2002, George V. Neville-Neil wrote: > > What I mean by "imposing structure" is: > > > > - Identify patterns of development and structure that seem to have evolved > > naturally as part of the maturing of the FreeBSD Project > > - Determine which patterns tend to result in the most productive and > > parallel development efforts, not to mention which are the most popular > > - Selectively reinforce those patterns to improve the ability of > > developers to rely on the patterns > > This sounds fine to me but we're going to have to write it down > somewhere and then discuss it. This tossing ideas in the ether is not > going to work because to refer back to earlier points we have to either > have huge emails with everyone's comments in them, or constantly be > reading the archives. > > So, please, if you think this is important write down your > findings/current beliefs somewhere (I think TWiki or something like it > is perfect for this) and let everyone comment and modify them until we > have an understanding. Then lets figure out what structure we need and > write that down as well. Think of it as a FreeBSD constitution if you > must, only much more lightweight. > > You're obviously taking the lead here and that's fine, every project > (and this discussion is really a project) needs a leader. Certainly -- the intent that I expressed in my original e-mail was to fish for people's thoughts on the issue, which would then be codified in some form. I'm interested in hearing a little more back on how people feel about the notion of how larger projects should coordinate work given the assumption that "locks" are advisory, but plan to write up a strawman sometime in the next week or so. Probably the process forward there will be to run the first pass by the core team for some immediate feedback regarding the strength of the recommendations, and whether it meets our goals concerning addressing the problems that have been discussed. Then I'll spit a copy out for more general comments. > Yes, you read me correctly. My mantra is, "Write it down." Something I've had in mind all along. BTW, your writings are much appreciated -- I hope to make the developer summit notes available in the next couple of days (I'm currently at an Apple-hosted security conference through Friday, when I'll get a chance to sit down with it over the weekend). Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
> What I mean by "imposing structure" is: > > - Identify patterns of development and structure that seem to have evolved > naturally as part of the maturing of the FreeBSD Project > - Determine which patterns tend to result in the most productive and > parallel development efforts, not to mention which are the most popular > - Selectively reinforce those patterns to improve the ability of > developers to rely on the patterns This sounds fine to me but we're going to have to write it down somewhere and then discuss it. This tossing ideas in the ether is not going to work because to refer back to earlier points we have to either have huge emails with everyone's comments in them, or constantly be reading the archives. So, please, if you think this is important write down your findings/current beliefs somewhere (I think TWiki or something like it is perfect for this) and let everyone comment and modify them until we have an understanding. Then lets figure out what structure we need and write that down as well. Think of it as a FreeBSD constitution if you must, only much more lightweight. You're obviously taking the lead here and that's fine, every project (and this discussion is really a project) needs a leader. > I think we're in the same boat here. You believe that process can help > streamline the development process, and be used to help avoid the > disagreements we've run into recently (assuming I read you correctly :-). > I think so too. I agree we can't bring too much process--that won't work > for a large number of reasons. But a little bit of process can go a long > way. In particular, I'm looking for a little bit of process that will > help address the perceived problems of the current situation. > Yes, you read me correctly. My mantra is, "Write it down." :-) Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Wed, 27 Feb 2002, George V. Neville-Neil wrote: > I think we need to avoid the concept of "imposing some modicum of > structure." If we create structure it is because we need it. Just like > software. There was a good comment recently about "software gets > created to scratch an itch." I'd say that structure gets created > because you're tired of losing fingers and toes to the person next to > you wielding the axe. It's great that your friends are helping you > clear the forest but you'd all like to be able to walk and pick up your > cup of coffee at the end of the day as well. What I mean by "imposing structure" is: - Identify patterns of development and structure that seem to have evolved naturally as part of the maturing of the FreeBSD Project - Determine which patterns tend to result in the most productive and parallel development efforts, not to mention which are the most popular - Selectively reinforce those patterns to improve the ability of developers to rely on the patterns An example of this might be investment of time in a rote API change: such changes are time-consuming requiring a lot of brute force source code modification. There's a strong motiviation to avoid redundant work here, but frequently the work is necessary, so it's also desirable to know that it's going to happen. A normal model here might be to look for a developer who's willing to do it, and wait for them to come back in a week or two when it's done. An unproductive tendancy is for four developers to go off, and do the work over two weeks, and all discover they've done it at the end. This can be avoided by making an attempt to coordinate. On a small scale and for individual cases, the "problem" here can be lived with. When it's in the context of a larger piece of work, it can be highly counter-productive. One reinforcement that could be made is to encourage developers to indicate what works in progress they have going, and to encourage other developers interested in a change to give that list a skim to see if someone else has already started on it, and if someone has but appears to be timing out, to encourage them to talk to the developer who has claimed it (in some sense) to find out what is going on. > We are always going to have "first past the post" problems. If someone > comes along and has rewritten a whole subsystem, and testing and > performance measurement show that it's an order of magnitude better > (faster, smaller, etc.) than what we have we'd be idiots not to take it, > right? Obviously. But the contentious cases often aren't the ones where that's the case. > The question is "What processes do we need to put in place to make a > project of this size and dynamisticity work?" I put forward a few of > them. I suggest we start somewhere (including airing gripes people have > with the current system) and write down (i.e. build a web page, use > TWiki) what we're going to do about it. > > I hate to make this analogy but we need a constitution or something like > it. Not so grandiose of course, but a written set of rules and are easy > to interpret that can take care of the 80% case. The 20% we'll always > get to argue over but I'd rather not argue over the 100%. I think we're in the same boat here. You believe that process can help streamline the development process, and be used to help avoid the disagreements we've run into recently (assuming I read you correctly :-). I think so too. I agree we can't bring too much process--that won't work for a large number of reasons. But a little bit of process can go a long way. In particular, I'm looking for a little bit of process that will help address the perceived problems of the current situation. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
> One of the disagreements that seems to be evolving is whether or not the > project formally supports a task-oriented structure. A couple of people > have asserted that people might claim tasks (such as myself) and by virtue > of claiming the task, be provided with some notion of ownership that is > supported in a more formal sense. Others have pointed out that in a > volunteer environment, people simply do what they want to regardless of > any task ownership, and would prefer a first-past-the-post model to a task > ownership model. My assumption had been that disagreement existed to some > extent based on the nature and strength of ownership, but it seems that > I've made a fundamental assumption there that not everyone agrees with. > > My feeling has always been that imposing some modicrum of structure is > important: to avoid people stepping on toes, people can announce what > they're working on, and expect that others might avoid replicating the > work, or at least be communicated with before it happens. The rationale > for this lies both in efficiency (non-replication of work), and to avoid > toe-stepping, since there's a natural notion of ownership over work done, > and a desire to not see it discarded. Perhaps this can't be supported in > our environment. I think we need to avoid the concept of "imposing some modicum of structure." If we create structure it is because we need it. Just like software. There was a good comment recently about "software gets created to scratch an itch." I'd say that structure gets created because you're tired of losing fingers and toes to the person next to you wielding the axe. It's great that your friends are helping you clear the forest but you'd all like to be able to walk and pick up your cup of coffee at the end of the day as well. We are always going to have "first past the post" problems. If someone comes along and has rewritten a whole subsystem, and testing and performance measurement show that it's an order of magnitude better (faster, smaller, etc.) than what we have we'd be idiots not to take it, right? The question is "What processes do we need to put in place to make a project of this size and dynamisticity work?" I put forward a few of them. I suggest we start somewhere (including airing gripes people have with the current system) and write down (i.e. build a web page, use TWiki) what we're going to do about it. I hate to make this analogy but we need a constitution or something like it. Not so grandiose of course, but a written set of rules and are easy to interpret that can take care of the 80% case. The 20% we'll always get to argue over but I'd rather not argue over the 100%. Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
+---[ George V. Neville-Neil ]-- | > In addition to process it might be attitude. | > | | So, how do we get our attitudes adjusted before hitting a wall, | as many companies I've worked for did? Alcohol and a cam-corder d8) -- Totally Holistic Enterprises Internet| | Andrew Milton The Internet (Aust) Pty Ltd | | ACN: 082 081 472 ABN: 83 082 081 472 | M:+61 416 022 411 | Carpe Daemon PO Box 837 Indooroopilly QLD 4068|[EMAIL PROTECTED]| To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
> In addition to process it might be attitude. > It might be a stretch but they are kind of the same. Good processes come from good attitude. It is extraordinarily hard for most people (especially smart people) to be introspective on such questions when they think they already have the answer. Sometimes people think that because something works for them it will work in a group. But we know that's not always the case. So, how do we get our attitudes adjusted before hitting a wall, as many companies I've worked for did? It comes back to agreeing on a process by which we work. We have one now, it may not all be written down, but we do have it. Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
> There are only only 8 core team members, unless you mean something > different by "core" here than [EMAIL PROTECTED] I guess I was going based on the meeting I attended back at BSD Con. > Otherwise, I tend to agree with your basic premise that it is process > and not tools at the heart of this problem. Of that I am glad. Later, George -- George V. Neville-Neil [EMAIL PROTECTED] NIC:GN82 "Those who would trade liberty for temporary security deserve neither" - Benjamin Franklin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
: My feeling has always been that imposing some modicrum of structure is : important: to avoid people stepping on toes, people can announce what : they're working on, and expect that others might avoid replicating the : work, or at least be communicated with before it happens. The rationale : for this lies both in efficiency (non-replication of work), and to avoid : toe-stepping, since there's a natural notion of ownership over work done, : and a desire to not see it discarded. Perhaps this can't be supported in : our environment. In the past, we haven't been strictly a "first one past the post" project. We have rejected things as not being ready for inclusion in FreeBSD all the time. Jon Chen's Cardbus work was the second or third effort that was presented to FreeBSD. It was the first one "acceptable" to those folks that took a look at it. Sure, it had its problems, but it was something that could be worked with and people generally agreed that it was the direction we wanted to go. The first one that was presented worked as well, but there were some issues with the direction that it took, so we didn't integrate it. We had the first CardBus implementation almost 9 months or a year before the second one. Of course, looking at the past to get precedent can be dicy, since we've done many things right as a project and many things wrong. :-) Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
One of the disagreements that seems to be evolving is whether or not the project formally supports a task-oriented structure. A couple of people have asserted that people might claim tasks (such as myself) and by virtue of claiming the task, be provided with some notion of ownership that is supported in a more formal sense. Others have pointed out that in a volunteer environment, people simply do what they want to regardless of any task ownership, and would prefer a first-past-the-post model to a task ownership model. My assumption had been that disagreement existed to some extent based on the nature and strength of ownership, but it seems that I've made a fundamental assumption there that not everyone agrees with. My feeling has always been that imposing some modicrum of structure is important: to avoid people stepping on toes, people can announce what they're working on, and expect that others might avoid replicating the work, or at least be communicated with before it happens. The rationale for this lies both in efficiency (non-replication of work), and to avoid toe-stepping, since there's a natural notion of ownership over work done, and a desire to not see it discarded. Perhaps this can't be supported in our environment. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
In message <[EMAIL PROTECTED]>, "M. Warner Losh" writes: >In message: <[EMAIL PROTECTED]> >"George V. Neville-Neil" <[EMAIL PROTECTED]> writes: >: The problem here is process. The FreeBSD project now has more than >: 12 core members and more than 12 committers. With any number larger >: than 12 it is VERY HARD to reach consensus on anything. Without a >: process by which we agree to reach consensus it is impossible. > >There are only only 8 core team members, unless you mean something >different by "core" here than [EMAIL PROTECTED] > >Otherwise, I tend to agree with your basic premise that it is process >and not tools at the heart of this problem. In addition to process it might be attitude. Core@ was not elected to be ornamental. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control
My general opinion is that a developer should not claim ownership of anything, it should simply be apparent from the traffic the developer posts to the public lists, discussion, and his commits. This implies that the developer is only actively working on one thing at a time, at least in regards to non-trivial projects, which further implies that the work can be committed in smaller chunks (or A smaller chunk) verses otherwise. While this ideal cannot always be met I believe it is a good basis for people working on a large project without formal management (i.e. open source). In the relatively rare case where a large rip-up must be done all at once an exception is made. For the FreeBSD project such an exception would be something like CAM or KSEs, but virtually nothing else. As an example of this I will note some non-trivial things that I have done using this model: * Implementation of idle processes. Anyone remember how long that took me? It turned out to be a good basis for further work now didn't it? * Pushing Giant through (most of) the syscall code and into the syscalls. * The critical_*() patch is an excellent example of this. The engineering cycle was 3 days (non-inclusive of all the crap that is preventing me from comitting it), and it is rock solid. * The rewrite of the swap system (In two pieces: added radix tree bitmap infrastructure, then switched out the swapper). I think my engineering cycle on this was 1.5 weeks. DG might remember better. So as you can see, from my viewpoint something like UCRED is just not that big a deal. With the infrastructure incrementally comitted and in place the final UCRED pushdown is something one could write, test, and commmit, from scratch, in just a few days. That is far more efficient then trying to keep it in a private tree for months on end, having to constantly sync it with code and algorithmic changes occuring in the rest of the tree. The same can be said for many of the other subsystems sitting in P4, like preemption. Experimental code has it's uses, but when I've gleened the information I need from one of my experiments I typically scrap the source entirely so it doesn't get in the way of other work. Later I rewrite the feature from scratch when the infrastructure has developed enough to support it. It may seem inefficient but the reality is that it speeds up my overall engineering and design cycles and, at least for me, the end result is pretty damn good code. Because I typically focus on one thing at a time, 3 days to get something simple like critical_*() done often seems like an eternity to me. I can't just switch my focus to something else, that isn't how I work. I can do a few things in parallel, like help find bugs in this or that, but real engineering cycles I do one at a time. Personally speaking if I do something complex and instrumenting it is straightforward, I always go ahead and instrument it because it makes debugging by others easy. That is why I have been and want to instrument Giant in the places where Giant is otherwise being removed, and why for example I had a sysctl to allow the critical_*() code behavior to change on the fly for testing purposes. The thing about instrumentation is that it's easy to put in if you integrate it right off the bat, and utterly trivial to rip out months or years down the line when you don't need it any more. I don't understand why people complain about 'putting in instrumentation that we'll just have to rip out later'. That kind of attitude is tantamount to saying 'I'm not going to bother to make this code debuggable because I've found all the bugs in it'. Yah, right. From my point of view instrumentation reduces the overall time required to add and stabilize a new feature, whereas someone saving source lines by not instrumenting his code is simply setting himself up for a long, buggy engineering cycle down the line (and not being a good neighbor to his peers much either). There is a damn good reason why I am rabid about instrumenting a complex piece of code. I don't care *how* long a developer has tested a big piece of code, it is simply naive to believe that anything short of exposure to the entire development community will exercise the code enough to really give it a good test. In that respect I have a strong dislike for the idea of sub-groups of developers testing a non-experimental feature (i.e. intended for commit) in a side-tree. I do not feel that it adds anything to the project and, in fact, I believe it is a detriment to the project. -Matt To Unsubscribe: send
Re: Discussion of guidelines for additional version control mechanisms (fwd)
Hi Folks, Before I address Robert's questions directly I'd like to make a few comments. Now, I'm not a committer, I'm a newbie as far as working on FreeBSD itself goes, but I've been a faithful consumer of this stuff since early NetBSD days and was runing Free around 2.2.7. I am also a software engineer with a lot of experience in dealing with OS development and various development models. I spent several years of my time at Wind River being called into meetings on just these subjects, so I do have a bit of a clue and I'm sure that many people out there have similar information but I'd like to put it down in green and black (well whatever colors you've picked...) The problem is NOT (as David Greenman has pointed out) a particular tool. I've used CVS and Clear Case. I have not used Perforce but I know some folks who worked on it and I have a clue as to how it works as well. I find that I like CVS's simplicity but I understand why people could want to use Clear Case as well and I have heard good things about Perforce. In this case I do NOT have a bias about the tools. The problem here is process. The FreeBSD project now has more than 12 core members and more than 12 committers. With any number larger than 12 it is VERY HARD to reach consensus on anything. Without a process by which we agree to reach consensus it is impossible. I read with some amusement the discussion of locks that Robert's email touched off. That discussion showed the group's confusion on this point. Optimally the "tree" would only be locked for such reasons as labelling it, or during a code freeze. Locking sections of it so someone can work on it does not make sense in a distributed project. We'll spend all our time in email doing human locking protocols and will probably lead to more code forking. This is bad. What would a possible solution look like? 1) We need to decide as a group how the framework that supports the OS is broken up and how it is glued together. This is the biggest technical problem, we need to make it so that pervasive changes are not necessary and so that APIs between chunks are either versioned or as static as possible. 2) Once we have areas an area lead will volunteer (probably for a period of time not exceeding a year and not less than 3 months) for each section and be vetted by the core team. 3) The area lead's job is to coordinate all work to be integrated into his/her section on some sort of schedule. We have a 3 times per year release on -STABLE that's a good driver IMHO. We need to do something like that with -CURRENT and perhaps a snapshot schedule should be set up. 4) All locking and release policies come from the above rules as well as the core team and area leads. The problem we are trying to tackle is partly human. How do we coordinate work in a volunteer project and produce a high quality product? It can either be done with a small team, or with fascist techniques (though this does not work well with volunteers) or by lots of cooperation. Now, to answer Robert's questions. > Question 1: How should the presence of on-going work in an external > repository be announced to the broader community? > > Suggestion: e-mail to -arch, -current, or a related mailing list > Email is fine. A pointer to a web page with design docs etc. should also be part of that. > Question 2: How should the status of on-going work be announced to the > broader community? > > Suggestion: Bi-monthly developer status report > Suggestion: Status web page for the project Collapse the two, put that status report on the web (in TWiki?) > Suggestion: Regular status reports on work to relevant mailing lists > Just have a cron job send out the web links in email. > Question 3: How should the results of the on-going work be made available > to the broader community? > > Suggestion: cvsup10 export of the Perforce tree > Suggestion: patchsets sent to appropriate mailing lists with status > Suggestion: patchsets generated automatically and posted to the mailing > list Patches can really suck. You want one main line of development and only have a few small branches for projects. > Question 4: How agressively should on-going work be pushed back into the > base tree? (Note that the answer here depends a great deal on the nature > of the work: both its rationale, its goals, etc). In particular note that > currently Perforce is used to hold experimental changes, as well as ones > destined for eventual production use. > > Suggestion: For work requiring large source tree sweeps, API changes, etc, > only when the work is ready to commit. API changes are most dangerous. The API change should be announced well in advance of committing it so that people who depend on the API can get a clue before they get screwed. > Suggestion: For more minor work, P4 might be used only for brief > collaboration, or to ass
Re: Discussion of guidelines for additional version control
At 7:27 PM -0800 2/26/02, Julian Elischer wrote: >On Tue, 26 Feb 2002, Garance A Drosihn wrote: > >> That would be me... >> > > I meant "lock" in the sense of expecting no one to make any > > major changes in the same area of code. I seem to remember > > you asking for such a "lock" (to use the term loosely) in July, > > and the KSE work going in around August or so. > >I never asked for that.. Oops, my mistake then. I probably shouldn't have driven to Boston on so little sleep that weekend... Still, it was a great example, even though it was imaginary! :-) -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms(fwd)
On Tue, 26 Feb 2002, Robert Watson wrote: > > On Tue, 26 Feb 2002, Julian Elischer wrote: > > > > On the other hand, you could easily argue that the expectations might be > > > much lower for smaller pieces of work. For example, the move to td_ucred > > > required a substantial amount of infrastructure, but the patches > > > themselves are relatively sane and small. Once the pieces are all in > > > place (which they mostly are), knowing that someone has a lock on it is > > > probably worth a "are you still working on this" followed by a wait of a > > > week or two to see if it turns up before forging ahead. > > > > Bad choice.. > > p->p_ucred => td->td_ucred in istself requires almost no > > infrastracture change, the bulk of the commit is trivial, (AND STILL NOT > > COMMITTED) and waiting for weeks on such a trivial thing before being able > > to proceed on some interlocking piece is foolish. > > I was referring to its dependency on KSE. > > > > (1) The timeout begins when contention occurs, of the lock has been > > > declared. This means that if you seriously intend to do some work, > > > you can say "I'm going to do the work", but you don't risk losing the > > > lock until someone comes to you and says "Hey did you ever...". > > > > Locks shouldn't be unilateral and they shouldn't last more that the > > amount of time that it takes to import a change and get it stable. i.e. > > maybe a week or so at most. Someone used KSE-II as an example.. They > > said I had a 2 month lock (??) I don't know where they got that idea > > from.. I had a vague concensus that maybe things may be broken for a DAY > > OR TWO. while the commit was happenning. I the end it was about right. > > So it sounds like we disagree on what a lock is. My interpretation of the > word lock was that it refered to ownership of a task. For example, we > gave you ownership of the general task of pushing KSE forwards. This > means that if someone turns up and says "I have SAs, I did it entirely > differently, I'm going to commit it now" we might say "Hold just a second > there". We'd probably say that even if they said "I have SAs, I did them > entirely the same way, I'm going to commit it now". ] Sore point: phk: "I've got DEVFS, I did it differently it's much better than tho old fully working one. I'm going to check it in now" core: "OK" phk: Ok I deleted parts of the old devfs so that it's now totally uselss. [2 years pass, system has no useable devfs..] phk: ok here's my devfs.. of course it doesn't work as well as the old one but it's much better. [checks in non working code] oops: spends 3 months getting bugs ironed out. Completed devfs still doesn't do (by design) as much as the old one.. > > The model I'm thinking of is really about someone saying "I'm going to go > work on . Please don't work on without talking to me first, > because it would be a shame to waste all my time, or all your time, by not > coordinating." I consider this to be more of a lock based on intention > than a "hard lock". It's about avoiding duplicate work, trodden on toes, > etc. It might be akin to claiming a task from a public task list. > > The lock you're describing appears to be more of a source tree lock: no > one touch this section of the source tree, because someone is doing work > relating to it in another tree. For example, someone might ask that no > one touch the USB code for a few days while a change is phased in. Or > that people expect the tree to break for a few days as a large API change > is phased in and the final integration tested. This is not the kind of > lock I'm talking about. But you see I am doing KSE despite the fact that other people are changing and improving code that is central to KSE all the time. When it happens I integrate the change, modify it to handle threads and move on. Even John does this but it's OTHER PEOPLE who keep leaping up and demanding that things not be committed becasue it may give john a headache.. well, his "lock" is to long (and anyhow it's not a lock. It's just what he's working on. it can't be allowed to screw over everybody. > > Robert N M Watson FreeBSD Core Team, TrustedBSD Project > [EMAIL PROTECTED] NAI Labs, Safeport Network Services > > > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control
mechanisms (fwd) In-Reply-To:Message-ID: <[EMAIL PROTECTED]> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Tue, 26 Feb 2002, Garance A Drosihn wrote: > At 6:55 PM -0800 2/26/02, Julian Elischer wrote: > > > (1) The timeout begins when contention occurs, of the lock has been > >> declared. This means that if you seriously intend to do some work, > >> you can say "I'm going to do the work", but you don't risk losing the > >> lock until someone comes to you and says "Hey did you ever...". > > > >Locks shouldn't be unilateral and they shouldn't last more that the > >amount of time that it takes to import a change and get it stable. > >i.e. maybe a week or so at most. Someone used KSE-II as an example.. > >They said I had a 2 month lock (??) I don't know where they got that > >idea from.. I had a vague concensus that maybe things may be broken > >for a DAY OR TWO. while the commit was happenning. I the end it was > >about right. > > That would be me... > > I meant "lock" in the sense of expecting no one to make any major > changes in the same area of code. I seem to remember you asking for > such a "lock" (to use the term loosely) in July, and the KSE work > going in around August or so. My memory is admittedly vague. My > point was you explicitly asked for permission to go ahead with that > work, even though (at the time) we were shooting for a release date > in November. > > I don't mean the period of time that -current was actually unstable, > but the amount of time that other developers were asked to stay away > from a major section of code. I never asked for that.. > > -- > Garance Alistair Drosehn= [EMAIL PROTECTED] > Senior Systems Programmer or [EMAIL PROTECTED] > Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Tue, 26 Feb 2002, Garance A Drosihn wrote: > That would be me... > > I meant "lock" in the sense of expecting no one to make any major > changes in the same area of code. I seem to remember you asking for > such a "lock" (to use the term loosely) in July, and the KSE work going > in around August or so. My memory is admittedly vague. My point was > you explicitly asked for permission to go ahead with that work, even > though (at the time) we were shooting for a release date in November. > > I don't mean the period of time that -current was actually unstable, but > the amount of time that other developers were asked to stay away from a > major section of code. So it seems to me we have at least three types of locks now, actually: (1) Locks protecting specific bits of code in the tree; for example, during a merge effort that requires several commits over a few days to a week. This should be very short. (2) Locks that protect a subsystem from substantial changes, such as a request to not change vm_mmap.c for a week or two during a rewrite: small changes might be fine, but larger ones would substantially hamper the work. These should also be short. (3) Locks that protect a particular task on a particular piece of code. These simply assert "Don't do this particular activity, becasue I'm doing it". My feeling is that these should be permitted to be long, and are in fact desirable. They wouldn't protect against changes in the subsystem, just identical changes in the subsystem, subject to common sense. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
>>Some things are too impractically large to do incrementally and are an >>all-or-nothing thing. I recall seeing your early VM commits which were huge, >>you had been working on for months, and were not incremental things. > > Actually, most VM system work that was done was developed over a period of >a few weeks at most, and in most cases were developed and tested in less than >a week. John Dyson had some stuff that brewed for a month or so and in fact >that caused some problems when I wanted to work in a similar area. In those >cases, John D and I collaborated very closely on the VM work, and often >emailed each other patches to integrate into each other's trees. ...but the >important point is that I don't believe that we ever told people not to work >on the VM system because it might conflict with our work. We told people not >to work on it because it was too delicate and too easily broken. :-) Oh, and one more thing... It was ALWAYS forefront in our minds to get any work that we had done committed as soon as possible, specifically to avoid having to deal with conflicts that other people may make, to avoid bit-rot in our working kernel trees, and to get the changes out to the masses for maximal testing. I continue to feel strongly that this is the only way to be successful in developing for FreeBSD. Now, I'm talking about changes to existing files in FreeBSD of course. People that bring whole new arch ports to the kernel have a different problem, but in those cases the overlap with existing files in FreeBSD is very minimal, so longer delays to commit would not normally affect anyone else. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
>>Anyway, my point is that the Perforce repo itself isn't the problem. The >> problem is that people are maintaining private patch sets for long periods >> and making claims to the areas that their patches cover. Step-wise evolution >> is the only way to go in this distributed development model and in order to >> acheive this, private development trees need to be minimized as much as >> possible. > >Some things are too impractically large to do incrementally and are an >all-or-nothing thing. I recall seeing your early VM commits which were huge, >you had been working on for months, and were not incremental things. Actually, most VM system work that was done was developed over a period of a few weeks at most, and in most cases were developed and tested in less than a week. John Dyson had some stuff that brewed for a month or so and in fact that caused some problems when I wanted to work in a similar area. In those cases, John D and I collaborated very closely on the VM work, and often emailed each other patches to integrate into each other's trees. ...but the important point is that I don't believe that we ever told people not to work on the VM system because it might conflict with our work. We told people not to work on it because it was too delicate and too easily broken. :-) -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Tue, 26 Feb 2002, Julian Elischer wrote: > > On the other hand, you could easily argue that the expectations might be > > much lower for smaller pieces of work. For example, the move to td_ucred > > required a substantial amount of infrastructure, but the patches > > themselves are relatively sane and small. Once the pieces are all in > > place (which they mostly are), knowing that someone has a lock on it is > > probably worth a "are you still working on this" followed by a wait of a > > week or two to see if it turns up before forging ahead. > > Bad choice.. > p->p_ucred => td->td_ucred in istself requires almost no > infrastracture change, the bulk of the commit is trivial, (AND STILL NOT > COMMITTED) and waiting for weeks on such a trivial thing before being able > to proceed on some interlocking piece is foolish. I was referring to its dependency on KSE. > > (1) The timeout begins when contention occurs, of the lock has been > > declared. This means that if you seriously intend to do some work, > > you can say "I'm going to do the work", but you don't risk losing the > > lock until someone comes to you and says "Hey did you ever...". > > Locks shouldn't be unilateral and they shouldn't last more that the > amount of time that it takes to import a change and get it stable. i.e. > maybe a week or so at most. Someone used KSE-II as an example.. They > said I had a 2 month lock (??) I don't know where they got that idea > from.. I had a vague concensus that maybe things may be broken for a DAY > OR TWO. while the commit was happenning. I the end it was about right. So it sounds like we disagree on what a lock is. My interpretation of the word lock was that it refered to ownership of a task. For example, we gave you ownership of the general task of pushing KSE forwards. This means that if someone turns up and says "I have SAs, I did it entirely differently, I'm going to commit it now" we might say "Hold just a second there". We'd probably say that even if they said "I have SAs, I did them entirely the same way, I'm going to commit it now". The model I'm thinking of is really about someone saying "I'm going to go work on . Please don't work on without talking to me first, because it would be a shame to waste all my time, or all your time, by not coordinating." I consider this to be more of a lock based on intention than a "hard lock". It's about avoiding duplicate work, trodden on toes, etc. It might be akin to claiming a task from a public task list. The lock you're describing appears to be more of a source tree lock: no one touch this section of the source tree, because someone is doing work relating to it in another tree. For example, someone might ask that no one touch the USB code for a few days while a change is phased in. Or that people expect the tree to break for a few days as a large API change is phased in and the final integration tested. This is not the kind of lock I'm talking about. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
David Greenman wrote: > >In the past week, a number of comments have been made both for and against > >additional version control mechanisms being used to supplement the FreeBSD > >Project official CVS server. Proponents of additional mechanisms, such as > >It's my view that work that happens outside of our official CVS repo is > private work no matter what source control system is used (or if one is used > at all). It is always a problem for one developer to say "I've got changes to > , so don't touch that part of the system." This might be justified > for a very short period (measured in a few days at most), but anything longer > term will almost certainly put off other developers that may wish to work > in the same or related area. >This isn't a new problem, either. We've had similar turf conflicts before > that very much resembled the one at hand. So...What's that phrase? - "Either > take a dump or get off the pot". Something like that. Developers need to > develop in ways that their work gets out as soon as possible so that 1) Other > developers can review the work as it happens, make comments - perhaps > influencing the design at an early stage when it really needs to be, and > 2) So that you don't end up with some buggy mega-commit that has so much > stuff wound up in it that it's nearly impossible to isolate the bugs. >Anyway, my point is that the Perforce repo itself isn't the problem. The > problem is that people are maintaining private patch sets for long periods > and making claims to the areas that their patches cover. Step-wise evolution > is the only way to go in this distributed development model and in order to > acheive this, private development trees need to be minimized as much as > possible. Some things are too impractically large to do incrementally and are an all-or-nothing thing. I recall seeing your early VM commits which were huge, you had been working on for months, and were not incremental things. I'm not taking sides on what has been done offline in the current fights with this statement, but I do take issue with statements that everything can and should be done incrementally. Take the sparc64 stuff. For quite some time, they had hacks in the generic PCI code that was deemed unacceptable to the rest of the platforms. The trick to things working smoothly is to not overuse stuff out of the cvs tree. Cheers, -Peter -- Peter Wemm - [EMAIL PROTECTED]; [EMAIL PROTECTED]; [EMAIL PROTECTED] "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms(fwd)
On Tue, 26 Feb 2002, Robert Watson wrote: > > On Tue, 26 Feb 2002, Garance A Drosihn wrote: > > > On the other hand, you could easily argue that the expectations might be > much lower for smaller pieces of work. For example, the move to td_ucred > required a substantial amount of infrastructure, but the patches > themselves are relatively sane and small. Once the pieces are all in > place (which they mostly are), knowing that someone has a lock on it is > probably worth a "are you still working on this" followed by a wait of a > week or two to see if it turns up before forging ahead. Bad choice.. p->p_ucred => td->td_ucred in istself requires almost no infrastracture change, the bulk of the commit is trivial, (AND STILL NOT COMMITTED) and waiting for weeks on such a trivial thing before being able to proceed on some interlocking piece is foolish. Further more as td_ucred can be used almost everywhere p_ucred is used, and p_ucred is not going away, it could have been done piecemeal without any danger.. I could change one occurance of p_ucred to be td->ucred pehour over a 2 month period whenever I had a few spare cycles and the tree would be none the worse for it. > > (1) The timeout begins when contention occurs, of the lock has been > declared. This means that if you seriously intend to do some work, > you can say "I'm going to do the work", but you don't risk losing the > lock until someone comes to you and says "Hey did you ever...". Locks shouldn't be unilateral and they shouldn't last more that the amount of time that it takes to import a change and get it stable. i.e. maybe a week or so at most. Someone used KSE-II as an example.. They said I had a 2 month lock (??) I don't know where they got that idea from.. I had a vague concensus that maybe things may be broken for a DAY OR TWO. while the commit was happenning. I the end it was about right. > > (2) The strength (timeout) of the lock depends on the level of investment > by the person holding the lock. That means if it's a trivial patch, > the time to break the lock is short, perhaps nil, but if it's a large > piece of work, there's the opportunity to say "Yes, it's a work in > progress, is that OK?", "Give me a week to finish it up", or "It's > going to take me a long time, why don't you take over", or in the > worst case "I'm doing it" followed by a timeout. > > (3) Common sense applies. > > It also suggests that there be a way to register intent and interest > relating to a topic. For example, I maintain a web page expressing intent > and on-going work regarding the TrustedBSD Project. It's linked from the > FreeBSD projects page. I've posted about it on lists. That suggests that > I have a relatively strong lock, for some definition of lock. It doesn't > preclude me from accepting contributions, soliciting contributions, etc. > It just means that if someone says "Ok, that's it, enough waiting, where > is it" I probably get a little more patience because people are aware the > work is on-going, before I get timed out and blown off. > > Robert N M Watson FreeBSD Core Team, TrustedBSD Project > [EMAIL PROTECTED] NAI Labs, Safeport Network Services > > > > To Unsubscribe: send mail to [EMAIL PROTECTED] > with "unsubscribe freebsd-current" in the body of the message > To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
>In the past week, a number of comments have been made both for and against >additional version control mechanisms being used to supplement the FreeBSD >Project official CVS server. Proponents of additional mechanisms, such as It's my view that work that happens outside of our official CVS repo is private work no matter what source control system is used (or if one is used at all). It is always a problem for one developer to say "I've got changes to , so don't touch that part of the system." This might be justified for a very short period (measured in a few days at most), but anything longer term will almost certainly put off other developers that may wish to work in the same or related area. This isn't a new problem, either. We've had similar turf conflicts before that very much resembled the one at hand. So...What's that phrase? - "Either take a dump or get off the pot". Something like that. Developers need to develop in ways that their work gets out as soon as possible so that 1) Other developers can review the work as it happens, make comments - perhaps influencing the design at an early stage when it really needs to be, and 2) So that you don't end up with some buggy mega-commit that has so much stuff wound up in it that it's nearly impossible to isolate the bugs. Anyway, my point is that the Perforce repo itself isn't the problem. The problem is that people are maintaining private patch sets for long periods and making claims to the areas that their patches cover. Step-wise evolution is the only way to go in this distributed development model and in order to acheive this, private development trees need to be minimized as much as possible. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org President, TeraSolutions, Inc. - http://www.terasolutions.com President, Download Technologies, Inc. - http://www.downloadtech.com Pave the road of life with opportunities. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms (fwd)
On Tue, 26 Feb 2002, Garance A Drosihn wrote: > I think the main issue here is how long the real repository can be > "locked" while waiting for some change to show up. If work can keep > going into the main repository, then what does anyone care if someone is > tracking their own personal work using CVS, or perforce, or a bunch of > home-grown shell scripts? I think the answer varies a great deal by the work that's being done. Suppose it's an implementation of mandatory access control on FreeBSD. Would it be reasonable for me to request that anyone who is interested in doing MAC on FreeBSD talk to me first before starting, so we can sync up and make sure we're coordinating, even if it's going to take 16 months to implement? Probably. Especially given that we'll probably end up investing at least four man years (probably more) developing it, meaning that people are unlikely to want to replicate that work when they could just wait :-). On the other hand, you could easily argue that the expectations might be much lower for smaller pieces of work. For example, the move to td_ucred required a substantial amount of infrastructure, but the patches themselves are relatively sane and small. Once the pieces are all in place (which they mostly are), knowing that someone has a lock on it is probably worth a "are you still working on this" followed by a wait of a week or two to see if it turns up before forging ahead. Imagine it's an initial port to IA64. If someone has announced they're working on it, you might expect people who would like to assist to checkpoint, and even wait a couple of months if substantial work has been done, before cutting it from scratch. Imagine it's an announced set of patches to clean up some #define's in a single C file. That's probably worth a couple of days wait to see if they're serious about doing it, since it's unlikely to stand in the way of much, but probably not much more. This suggests three rules of thumb that might make sense: (1) The timeout begins when contention occurs, of the lock has been declared. This means that if you seriously intend to do some work, you can say "I'm going to do the work", but you don't risk losing the lock until someone comes to you and says "Hey did you ever...". (2) The strength (timeout) of the lock depends on the level of investment by the person holding the lock. That means if it's a trivial patch, the time to break the lock is short, perhaps nil, but if it's a large piece of work, there's the opportunity to say "Yes, it's a work in progress, is that OK?", "Give me a week to finish it up", or "It's going to take me a long time, why don't you take over", or in the worst case "I'm doing it" followed by a timeout. (3) Common sense applies. It also suggests that there be a way to register intent and interest relating to a topic. For example, I maintain a web page expressing intent and on-going work regarding the TrustedBSD Project. It's linked from the FreeBSD projects page. I've posted about it on lists. That suggests that I have a relatively strong lock, for some definition of lock. It doesn't preclude me from accepting contributions, soliciting contributions, etc. It just means that if someone says "Ok, that's it, enough waiting, where is it" I probably get a little more patience because people are aware the work is on-going, before I get timed out and blown off. Robert N M Watson FreeBSD Core Team, TrustedBSD Project [EMAIL PROTECTED] NAI Labs, Safeport Network Services To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: Discussion of guidelines for additional version control mechanisms
In message <[EMAIL PROTECTED]>, Robe rt Watson writes: >Question 1: How should the presence of on-going work in an external >repository be announced to the broader community? On the project.freebsd.org web-page and the regular status emails generated from the contents of that web-page. >Question 2: How should the status of on-going work be announced to the >broader community? > >Suggestion: Bi-monthly developer status report >Suggestion: Status web page for the project >Suggestion: Regular status reports on work to relevant mailing lists All of the above with a s/Regular/As warranted/ on the third line. >Question 3: How should the results of the on-going work be made available >to the broader community? > >Suggestion: cvsup10 export of the Perforce tree >Suggestion: patchsets sent to appropriate mailing lists with status >Suggestion: patchsets generated automatically and posted to the mailing >list Whichever fits the particular project but it would be wonderful if a patch file for all projects could be accessed via the web-page. >Question 4: How agressively should on-going work be pushed back into the >base tree? > >Suggestion: For work requiring large source tree sweeps, API changes, etc, >only when the work is ready to commit. Panic: recursion: "commit only when ready to commit" Doesn't this one hinge on the stability/compilability goal of current and only that ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 [EMAIL PROTECTED] | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message