Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On Fri, May 17, 2013 at 6:29 PM, Keith Moore mo...@network-heretics.comwrote: On 05/17/2013 04:36 PM, Yoav Nir wrote: On May 17, 2013, at 6:37 PM, Dave Crocker d...@dcrocker.net wrote: On 5/17/2013 7:01 AM, Keith Moore wrote: But WGs should be able to periodically summarize what they're doing - what problem they're trying to solve, what approach they're taking, what technologies they're using, what major decisions they've made, what the current sticking points seem to be, what problems are as yet unresolved, what potential for cross-group and cross-area effects have been identified, and what efforts have been made to get the affected parties in the loop. For most groups that summary should be maybe 2-3 pages. The ADs should be able to verify that those summaries are accurate and reasonably complete, or appoint a trusted WG observer other than the chair to review each summary. ADs and other members of the community should be able to view those summaries and comment on their accuracy. The idea that working groups should be required to issue periodic project progress reports seems strikingly reasonable and useful. This makes the folks who are the most knowledgeable responsible for assessing their work, and should facilitate public review. Recording the sequence of reports into the wg datatracker could nicely allow evaluating progress over time. It also, of course, nicely distributes the work. d/ From: WG Chair To: ietf@ietf.org Sunbject: Progress Report - Foo WG There has been zero activity on the FOO list in the last three months (except for that Fake Conference message everybody got last month). I've tried emailing the WG document authors twice, but they're not answering my emails. So, the WG has 2 documents: draft-ietf-foo-use-cases-03, and draft-ietf-foo-proto-01. The use case document is just about done, but we haven't really started discussing the proto document. We haven't met in Orlando, and are unlikely to meet in Berlin That's it for this report. Cheers WGC Instead of a WG progress report, what I had in mind was a separate report for each work item. The report should briefly describe 1. The purpose of the work being undertaken 2. A description of the work being undertaken (including mention of major technologies on which the work is based) 3. All known parties and interests likely to be affected by the work 4. The current state of the work (what's been done, what hasn't been done) 5. Any major issues that have been identified but not resolved 6. Pointers to the WG's charter, the datatracker pages for each of the current draft document(s) associated with that work item, and any other useful material (e.g. open issues list, summaries of design decisions already taken and the rationale for each) A person who is knowledgeable about current Internet protocols should be able to read that single document and understand what the WG is doing with this particular work item, what state it's in, whether or not it will affect that person's are of interest, and where to look for detailed information. This seems like a good idea, although maybe a bit formal. IMO, big surprises at the tail end that cause lots of work to be thrown out are evidence of a management failure. The IESG and WG chairs should be more proactive wrt/ avoiding late surprises from ever happening. I notice that nowhere on this list is any mention of the charter milestones or dates. Is the Foo Proto draft due in 14 months or is it 14 months behind schedule? If the latter, why isn't the Foo WG meeting at the IETF? Keith Andy
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On Fri, May 17, 2013 at 7:29 PM, Keith Moore mo...@network-heretics.comwrote: On 05/17/2013 10:21 PM, Andy Bierman wrote: I notice that nowhere on this list is any mention of the charter milestones or dates. Is the Foo Proto draft due in 14 months or is it 14 months behind schedule? If the latter, why isn't the Foo WG meeting at the IETF? I don't think milestones will be useful unless and until: (a) they're defined in terms of not only concrete but also meaningful goals (e.g. complete problem definition, identify affected parties and groups representing their interests, complete outline of initial design, but NOT revise document X); (b) we start automatically suspending the activities of groups that fail to meet them (no meetings, no new I-Ds accepted, mailing list traffic blocked), until such groups are formally rechartered; and (c) IESG is reluctant to recharter groups that have repeatedly failed to meet milestones, especially if those groups haven't produced evidence of significant progress. I think we can find some middle ground between ignore charter milestones completely and autobot to terminate WGs behind schedule. :-) Keith Andy
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
The problem is that WG participants SHOULD follow/update their milestones and take responsibility to progress work to thoes goals direction. The Chair SHOULD follow the WG requests, or the Chair SHOULD encourage discussing the milestones. I already requested before that all WGs SHOULD discuss their milestones and update it in each meeting or on the list. If no one cares then the result is WG failing some-goals which no one realise until long, but some people outside the IETF are watching such performance. AB On Fri, May 17, 2013 at 7:29 PM, Keith Moore moore at network-heretics.com wrote: I don't think milestones will be useful unless and until: (a) they're defined in terms of not only concrete but also meaningful goals (e.g. complete problem definition, identify affected parties and groups representing their interests, complete outline of initial design, but NOT revise document X); (b) we start automatically suspending the activities of groups that fail to meet them (no meetings, no new I-Ds accepted, mailing list traffic blocked), until such groups are formally rechartered; and (c) IESG is reluctant to recharter groups that have repeatedly failed to meet milestones, especially if those groups haven't produced evidence of significant progress.
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
Instead of a WG progress report, what I had in mind was a separate report for each work item. The report should briefly describe I agree with you totally, that work-item-report SHOULD be copied to AD and WG. That report is needed mostly when the work does not target its milestone, requesting new milestone with plans/reason. AB
RE: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
I already requested before that all WGs SHOULD discuss their milestones and update it in each meeting or on the list. No-one cares what you requested. Didn't you get banned from the MANET list for lack of useful content? Lloyd Wood http://sat-net.com/L.Wood
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 16, 2013, at 11:55 PM, Stephen Farrell stephen.farr...@cs.tcd.ie wrote: I think Dave's idea is worth looking at, but have one comment: On 05/16/2013 09:46 PM, Yoav Nir wrote: There is a problem, though, that this will increase the load on ADs. There is that. But don't forget that ADs mostly read everything in IESG review and often comment. Even leaving aside DISCUSSes, if every IETF LC draft got a bunch of AD review comments, then the IETF LC would be much busier than now. And its in the nature of IETFers to chime in. So this would I suspect increase everyone's load, if it works, and not just ADs. More community participation at IETF last call is a good thing. The problem with AD's time is that they will need to review at IETF last call, and then may need to review again after some changes have been made (including changes not related to their own issue). That's where the increased AD load comes from. Yoav
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 17, 2013, at 12:58 AM, Keith Moore mo...@network-heretics.com wrote: On 05/16/2013 04:46 PM, Yoav Nir wrote: The time for asking whether the group has considered making this field fixed length instead of variable, or whether RFC 2119 language is used in an appropriate way, or whether the protocol is extensible enough is at IETF last call. Actually the time for asking these questions is long before IETF-wide Last Call. We need widespread review of proposals for standards-track documents long before a WG thinks it's finished with those documents. It's a gaping hole in our process. Sure. But we have opinionated ADs who read every draft that comes to the IESG. There is no way they have time to participate in all of the working groups. I, as a participant, can read drafts as they are discussed in working groups, because I'm free to ignore all the drafts that are not interesting to me. ADs don't have that luxury. Fix that problem, and most of the conflicts between IESG and WGs that surround DISCUSS votes will go away. Good reviewers are a scarce resource, and there are 500(*) working group drafts competing for their attention. That's a hard problem to fix. Yoav (*) Went to datatracker, and searched for active drafts that start with draft-ietf-. I probably hit some things that have already progressed, but OTOH the 500 number is too round, and may be a limitation of datatracker.
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 17, 2013, at 1:38 AM, Fred Baker (fred) f...@cisco.com wrote: On May 16, 2013, at 1:46 PM, Yoav Nir y...@checkpoint.com wrote: There is a problem, though, that this will increase the load on ADs. Other concerns raised during IETF LC may lead to revised I-Ds, which the ADs will need to re-read (or at least look at the diff). I don't know how significant this extra work would be, but it would come at a time that we're thinking of ways to reduce AD workload. It might also require prolonging the LC time, because there would be actual discussion in it. If they raise the issue later in a discuss, will they not have to do this anyway? How does this relate to the timing of the comment or the vehicle by which it is conveyed? If you review early, you later might feel like you need to review again, because the document has changed some. Hence - more work. Yoav
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 05/17/2013 10:18 AM, Yoav Nir wrote: On May 16, 2013, at 11:55 PM, Stephen Farrell stephen.farr...@cs.tcd.ie wrote: I think Dave's idea is worth looking at, but have one comment: On 05/16/2013 09:46 PM, Yoav Nir wrote: There is a problem, though, that this will increase the load on ADs. There is that. But don't forget that ADs mostly read everything in IESG review and often comment. Even leaving aside DISCUSSes, if every IETF LC draft got a bunch of AD review comments, then the IETF LC would be much busier than now. And its in the nature of IETFers to chime in. So this would I suspect increase everyone's load, if it works, and not just ADs. More community participation at IETF last call is a good thing. The problem with AD's time is that they will need to review at IETF last call, and then may need to review again after some changes have been made (including changes not related to their own issue). That's where the increased AD load comes from. I'm not so sure that's a problem. The additional time I think (for me) would be in reading the 100's of ietf-discuss mails that'd get triggered by my or some other AD's comments. Not all comments would trigger that of course, but if you look at the minutes for any IESG telechat [1] and imagine that all those comments had been sent to ietf-discuss during IETF LC then you might worry about what'd happen on this list. I'm not against considering this idea at all, but we should bear in mind that any random thread on this list has a non-zero probability of turning into major amounts of traffic. Most of that traffic would doubtless be excellent IETF LC stuff, but we maybe need to be careful in what we wish for too. Cheers, S. [1] https://www.ietf.org/iesg/minutes/2013/index.html Yoav
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
Dave, Ralph, Jari has expressed the goal of having AD concerns be raised more publicly. Moving AD review and comment to the IETF Last Call venue nicely accomplishes this, too. I just posted elsewhere a suggestion to move this review even earlier, to WG last call. Accomplishes most of the same ends, while putting the discussion in front of the IETF participants who are, presumably, most invested in the resulting document. We've also said that we'd like to move the directorate reviews earlier, to WGLC time. (Subject to us finding a way to do that without increasing directorate workload too much.) See the thread ways forward with the tail-heavy aspects of the IETF process. But overall, we have to be careful that we don't move the same processes to a process step that has a different name but in reality will not impact how things are done. Part of the idea for moving directorate reviews earlier is that more of the responsibility for dealing with these could reside with the WGs rather than ADs. And that is a change that affects substance and may improve scalability. Similar reasons are needed for other changes. I'm not opposed to moving AD reviews to IETF LC time or even earlier, but we'd have to find a way to accommodate both technical comments as well as those relating to taking LC feedback into account. (draft-foo LC feedback from Joe was was not appropriately addressed and similar end-of-process checks.) Jari
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 05/17/2013 05:31 AM, Yoav Nir wrote: On May 17, 2013, at 12:58 AM, Keith Moore mo...@network-heretics.com wrote: On 05/16/2013 04:46 PM, Yoav Nir wrote: The time for asking whether the group has considered making this field fixed length instead of variable, or whether RFC 2119 language is used in an appropriate way, or whether the protocol is extensible enough is at IETF last call. Actually the time for asking these questions is long before IETF-wide Last Call. We need widespread review of proposals for standards-track documents long before a WG thinks it's finished with those documents. It's a gaping hole in our process. Sure. But we have opinionated ADs who read every draft that comes to the IESG. There is no way they have time to participate in all of the working groups. I, as a participant, can read drafts as they are discussed in working groups, because I'm free to ignore all the drafts that are not interesting to me. ADs don't have that luxury. Unless things have changed a great deal since I was on IESG, ADs do have the luxury of not reading drafts. When I was an AD I tried to read every draft that was in my area (Applications), and every draft that seemed to have the ability to affect applications developers.The lower in the protocol stack, the less likely that I'd feel like I'd have anything useful to say about a draft. Even when I read a draft outside of my area, in many cases it was just skimming the draft looking for red flags. I developed a pretty good sense of whether a group had done due diligence or whether there were serious technical omissions that they were trying to ignore. Only in the latter (rare) cases did I feel like such drafts needed more of my attention. I certainly agree that ADs don't have time to participate in all working groups, or even probably 10% of our working groups. But WGs should be able to periodically summarize what they're doing - what problem they're trying to solve, what approach they're taking, what technologies they're using, what major decisions they've made, what the current sticking points seem to be, what problems are as yet unresolved, what potential for cross-group and cross-area effects have been identified, and what efforts have been made to get the affected parties in the loop. For most groups that summary should be maybe 2-3 pages. The ADs should be able to verify that those summaries are accurate and reasonably complete, or appoint a trusted WG observer other than the chair to review each summary. ADs and other members of the community should be able to view those summaries and comment on their accuracy. And I think it would be reasonable for everyone on IESG to read through those summaries once in awhile - at least for groups that seemed likely to affect their areas of concern. I think that such summaries could actually lessen IESG's workload. Fix that problem, and most of the conflicts between IESG and WGs that surround DISCUSS votes will go away. Good reviewers are a scarce resource, and there are 500(*) working group drafts competing for their attention. That's a hard problem to fix. That's a related but IMO somewhat different problem. Working groups are producing far too many documents. That's one reason (far from the only one) why WGs shouldn't undertake documents that aren't specifically authorized in their charters. Keith
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 05/17/2013 05:32 AM, Yoav Nir wrote: On May 17, 2013, at 1:38 AM, Fred Baker (fred) f...@cisco.com wrote: On May 16, 2013, at 1:46 PM, Yoav Nir y...@checkpoint.com wrote: There is a problem, though, that this will increase the load on ADs. Other concerns raised during IETF LC may lead to revised I-Ds, which the ADs will need to re-read (or at least look at the diff). I don't know how significant this extra work would be, but it would come at a time that we're thinking of ways to reduce AD workload. It might also require prolonging the LC time, because there would be actual discussion in it. If they raise the issue later in a discuss, will they not have to do this anyway? How does this relate to the timing of the comment or the vehicle by which it is conveyed? If you review early, you later might feel like you need to review again, because the document has changed some. Hence - more work. Yes, but you only need to review what has changed. It _can_ be more work, particularly for documents that have changed a lot, or if it's been too long since the last review.But I really wouldn't suggest that ADs should do several reviews of each document.I suspect that the trick is to review a document at the time the WG thinks that it's maybe 70-80% done (which is to say the WG is probably closer to 40-50% done), but when some issues still seem unresolved. That's when the ADs' input, and external input in general, can be the most valuable. That way, ADs should be able to say yes, but you're totally ignoring this very important issue here, and you really have to deal with it at a time when the WG isn't yet so exhausted or off in the weeds that it can't focus on it. Then the ADs just need to track the changes from that point forward, to make sure that the issues identified were dealt with satisfactorily. Of course new issues can still be identified, and WGs can address previously identified issues in unsatisfactory ways. But at some point (well prior to WGLC) it might be appropriate to raise the bar for new issues. At some point it should take a serious defect to significantly delay publication of a document, and at that point it might make more sense to consider other remedies for identified less-serious issues, especially if the existing protocol isn't seen to be actually harmful and it appears that the protocol can be extended to address the issues in a manner that is compatible with existing implementations. In those cases, it might make sense to go ahead and publish, but charter the WG to extend the protocol to address those issues. Keith
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 5/17/2013 7:01 AM, Keith Moore wrote: But WGs should be able to periodically summarize what they're doing - what problem they're trying to solve, what approach they're taking, what technologies they're using, what major decisions they've made, what the current sticking points seem to be, what problems are as yet unresolved, what potential for cross-group and cross-area effects have been identified, and what efforts have been made to get the affected parties in the loop. For most groups that summary should be maybe 2-3 pages. The ADs should be able to verify that those summaries are accurate and reasonably complete, or appoint a trusted WG observer other than the chair to review each summary. ADs and other members of the community should be able to view those summaries and comment on their accuracy. The idea that working groups should be required to issue periodic project progress reports seems strikingly reasonable and useful. This makes the folks who are the most knowledgeable responsible for assessing their work, and should facilitate public review. Recording the sequence of reports into the wg datatracker could nicely allow evaluating progress over time. It also, of course, nicely distributes the work. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
Dave, On 5/17/13 11:37 AM, Dave Crocker wrote: On 5/17/2013 7:01 AM, Keith Moore wrote: But WGs should be able to periodically summarize what they're doing - what problem they're trying to solve, what approach they're taking, what technologies they're using, what major decisions they've made, what the current sticking points seem to be, what problems are as yet unresolved, what potential for cross-group and cross-area effects have been identified, and what efforts have been made to get the affected parties in the loop. For most groups that summary should be maybe 2-3 pages. The ADs should be able to verify that those summaries are accurate and reasonably complete, or appoint a trusted WG observer other than the chair to review each summary. ADs and other members of the community should be able to view those summaries and comment on their accuracy. The idea that working groups should be required to issue periodic project progress reports seems strikingly reasonable and useful. This makes the folks who are the most knowledgeable responsible for assessing their work, and should facilitate public review. Recording the sequence of reports into the wg datatracker could nicely allow evaluating progress over time. It also, of course, nicely distributes the work. d/ We (the INT ADs) have been trying to do that for the past year. See http://trac.tools.ietf.org/area/int/trac/wiki/IETF86 as an example. The goal is to provide a place for INT area WGs to summarize what they have been doing so that others within INT can get an idea of what is going on. It is not to the level of a progress report, but it does provide a useful summation (at least based on the feedback that I have received to date). Regards, Brian
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 17, 2013, at 6:37 PM, Dave Crocker d...@dcrocker.net wrote: On 5/17/2013 7:01 AM, Keith Moore wrote: But WGs should be able to periodically summarize what they're doing - what problem they're trying to solve, what approach they're taking, what technologies they're using, what major decisions they've made, what the current sticking points seem to be, what problems are as yet unresolved, what potential for cross-group and cross-area effects have been identified, and what efforts have been made to get the affected parties in the loop. For most groups that summary should be maybe 2-3 pages. The ADs should be able to verify that those summaries are accurate and reasonably complete, or appoint a trusted WG observer other than the chair to review each summary. ADs and other members of the community should be able to view those summaries and comment on their accuracy. The idea that working groups should be required to issue periodic project progress reports seems strikingly reasonable and useful. This makes the folks who are the most knowledgeable responsible for assessing their work, and should facilitate public review. Recording the sequence of reports into the wg datatracker could nicely allow evaluating progress over time. It also, of course, nicely distributes the work. d/ From: WG Chair To: ietf@ietf.org Sunbject: Progress Report - Foo WG There has been zero activity on the FOO list in the last three months (except for that Fake Conference message everybody got last month). I've tried emailing the WG document authors twice, but they're not answering my emails. So, the WG has 2 documents: draft-ietf-foo-use-cases-03, and draft-ietf-foo-proto-01. The use case document is just about done, but we haven't really started discussing the proto document. We haven't met in Orlando, and are unlikely to meet in Berlin That's it for this report. Cheers WGC
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 05/17/2013 04:36 PM, Yoav Nir wrote: On May 17, 2013, at 6:37 PM, Dave Crocker d...@dcrocker.net wrote: On 5/17/2013 7:01 AM, Keith Moore wrote: But WGs should be able to periodically summarize what they're doing - what problem they're trying to solve, what approach they're taking, what technologies they're using, what major decisions they've made, what the current sticking points seem to be, what problems are as yet unresolved, what potential for cross-group and cross-area effects have been identified, and what efforts have been made to get the affected parties in the loop. For most groups that summary should be maybe 2-3 pages. The ADs should be able to verify that those summaries are accurate and reasonably complete, or appoint a trusted WG observer other than the chair to review each summary. ADs and other members of the community should be able to view those summaries and comment on their accuracy. The idea that working groups should be required to issue periodic project progress reports seems strikingly reasonable and useful. This makes the folks who are the most knowledgeable responsible for assessing their work, and should facilitate public review. Recording the sequence of reports into the wg datatracker could nicely allow evaluating progress over time. It also, of course, nicely distributes the work. d/ From: WG Chair To: ietf@ietf.org Sunbject: Progress Report - Foo WG There has been zero activity on the FOO list in the last three months (except for that Fake Conference message everybody got last month). I've tried emailing the WG document authors twice, but they're not answering my emails. So, the WG has 2 documents: draft-ietf-foo-use-cases-03, and draft-ietf-foo-proto-01. The use case document is just about done, but we haven't really started discussing the proto document. We haven't met in Orlando, and are unlikely to meet in Berlin That's it for this report. Cheers WGC Instead of a WG progress report, what I had in mind was a separate report for each work item. The report should briefly describe 1. The purpose of the work being undertaken 2. A description of the work being undertaken (including mention of major technologies on which the work is based) 3. All known parties and interests likely to be affected by the work 4. The current state of the work (what's been done, what hasn't been done) 5. Any major issues that have been identified but not resolved 6. Pointers to the WG's charter, the datatracker pages for each of the current draft document(s) associated with that work item, and any other useful material (e.g. open issues list, summaries of design decisions already taken and the rationale for each) A person who is knowledgeable about current Internet protocols should be able to read that single document and understand what the WG is doing with this particular work item, what state it's in, whether or not it will affect that person's are of interest, and where to look for detailed information. Keith
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 05/17/2013 10:21 PM, Andy Bierman wrote: I notice that nowhere on this list is any mention of the charter milestones or dates. Is the Foo Proto draft due in 14 months or is it 14 months behind schedule? If the latter, why isn't the Foo WG meeting at the IETF? I don't think milestones will be useful unless and until: (a) they're defined in terms of not only concrete but also meaningful goals (e.g. complete problem definition, identify affected parties and groups representing their interests, complete outline of initial design, but NOT revise document X); (b) we start automatically suspending the activities of groups that fail to meet them (no meetings, no new I-Ds accepted, mailing list traffic blocked), until such groups are formally rechartered; and (c) IESG is reluctant to recharter groups that have repeatedly failed to meet milestones, especially if those groups haven't produced evidence of significant progress. Keith
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 05/17/2013 10:37 PM, Andy Bierman wrote: On Fri, May 17, 2013 at 7:29 PM, Keith Moore mo...@network-heretics.com mailto:mo...@network-heretics.com wrote: I don't think milestones will be useful unless and until: (a) they're defined in terms of not only concrete but also meaningful goals (e.g. complete problem definition, identify affected parties and groups representing their interests, complete outline of initial design, but NOT revise document X); (b) we start automatically suspending the activities of groups that fail to meet them (no meetings, no new I-Ds accepted, mailing list traffic blocked), until such groups are formally rechartered; and (c) IESG is reluctant to recharter groups that have repeatedly failed to meet milestones, especially if those groups haven't produced evidence of significant progress. I think we can find some middle ground between ignore charter milestones completely and autobot to terminate WGs behind schedule. :-) Actually I think it might require an autobot. Because someone (probably the responsible AD) has to evaluate a WG's progress, and ADs don't want to take the heat for shutting WGs down. Better to put the responsibility on the chairs for completing the milestones and reporting to the AD before the shutdown deadline. (of course, there could be a generous grace period between the milestone deadline and the actual shutdown, with warning messages sent to the WG chairs and ADs, etc.) Keith
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 15, 2013, at 4:30 PM, Adrian Farrel adrian at olddog.co.uk wrote: The claim (or one of the claims) is that some ADs may place Discusses that are intended to raise a discussion with the authors/WG that could equally have been raised with a Comment or through direct email. This, it is claimed, may unnecessarily delay the document from completing the publication process. Discussions should have a time limit (can be one week), like we have in meetings (2hours), if there is time we can know when things are needed to respond to, I usually ignore when there is no milestones or planing-time. Does IESG have milestones for documents processing/discussions? Now the dangerous bit, Suppose the AD raised her concern by writing a Comment or sending an email and balloting No Objection. That would mean that the I-D would be approved for publication. At this point either: - the discussion goes on, but the document becomes an RFC anyway or - the responsible AD holds the document pending satisfactory completion of the discussion. That AD SHOULD not hold for unlimited time, also should discuss the issue with the WG related in limited time. I suggest that the former is a bad result. Not that the authors/WG will ignore the discussion, but if they disagree on something the AD considers very important, the authors/WG have no incentive to participate in the discussion. Only community rough concensus will decide the final result, Of course, all participants in this thread so far would never behave like that, but there is a possibility that this will happen for some authors. Yes only if there is no time limits for work that should be done, I also suggest that the latter introduces exactly the same amount of delay as the Discuss. There is always possibility of large delay in systems that have no time limits for processing or responds. Our time/work used is important for IETF, IMO, no one should hold work/time only if able to decide/notify/plan when/how to leave it go for all reaction possibilities. AB
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 2013-05-16 14:38, Abdussalam Baryun wrote: Discussions should have a time limit (can be one week), I totally disagree, DISCUSSES are our friends, they need to be discussed until we have rough consensus; it seems to be a manifestly bad idea to draw a deadline after seven days, if someone come up with a satisfactory solution on the eighth. 99,9% of the DISCUSSES improve our documents or the understanding of them. /Loa like we have in meetings (2hours), if there is time we can know when things are needed to respond to, I usually ignore when there is no milestones or planing-time. Does IESG have milestones for documents processing/discussions? -- Loa Anderssonemail: l...@mail01.huawei.com Senior MPLS Expert l...@pi.nu Huawei Technologies (consultant) phone: +46 739 81 21 64
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
Please distinguish between (1) making the system efficient and (2) making individual documents go through it quickly. If you put time limits on parts of the process, you're not allowing them to function correctly. Putting arbitrary time limits on will hurry things up but it will not produce higher quality results.
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
Hi Loa, I agree with you discussions are our friend. I was focusing on processing time, not document quality. No dought if you stay longer time you will get better quality, but what about progress. So I mean call for discussions is for a time limit, as if no discussion happends then the call matures and the holding of the work stops as well. If DISCUSS is continue I never like to close it as long as it is continuous, but not delayed. In practice I never seen that DISCUSS of ADs with the WGs are having much continuous communication, it is delay with no time schedule. 99,9% of the DISCUSSES improve our documents or the understanding of them. I know that DISCUSSES with WGs improves our documents (don't forget that WGs are following milestones and WGLC periods), but DISCUSSES that have no time limit makes more delays. I hope we see similar times of WG into the IESG, so the communication can improve the processing time. AB On 5/16/13, Loa Andersson l...@pi.nu wrote: On 2013-05-16 14:38, Abdussalam Baryun wrote: Discussions should have a time limit (can be one week), I totally disagree, DISCUSSES are our friends, they need to be discussed until we have rough consensus; it seems to be a manifestly bad idea to draw a deadline after seven days, if someone come up with a satisfactory solution on the eighth. 99,9% of the DISCUSSES improve our documents or the understanding of them. /Loa like we have in meetings (2hours), if there is time we can know when things are needed to respond to, I usually ignore when there is no milestones or planing-time. Does IESG have milestones for documents processing/discussions? -- Loa Anderssonemail: l...@mail01.huawei.com Senior MPLS Expert l...@pi.nu Huawei Technologies (consultant) phone: +46 739 81 21 64
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 5/16/13, Scott Brim scott.b...@gmail.com wrote: Please distinguish between (1) making the system efficient and (2) making individual documents go through it quickly. If you put time limits on parts of the process, you're not allowing them to function correctly. Putting arbitrary time limits on will hurry things up but it will not produce higher quality results. Ok, so do you agree, that if who holds the work, at least should tell us HOW long he is holding or what is the time PLAN. Do you think working without plan is efficient and gives good quality. AB
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
Putting arbitrary time limits on will hurry things up but it will not produce higher quality results. Ok, so do you agree, that if who holds the work, at least should tell us HOW long he is holding or what is the time PLAN. Do you think working without plan is efficient and gives good quality. We generally rely on judgment in this regard, not on time limits and deadlines. We do sometimes set deadlines (milestones in charters, timeouts on last calls, fixed telechat dates, alerts when things take too long), but we also give leeway (we know how solid charter milestones aren't, and no one will ignore important, useful input just because it came in after last call). Someone is always responsible for determining when a discussion is no longer productive (shepherds, chairs, responsible ADs), and someone is responsible for moving things forward. Reminders to those who are responsible, telling them that things seem to be dragging, are fine. Strict time limits are generally not how we prefer to work. Barry
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 5/15/2013 1:30 PM, Adrian Farrel wrote: Suppose the AD raised her concern by writing a Comment or sending an email and balloting No Objection. That would mean that the I-D would be approved for publication. At this point either: - the discussion goes on, but the document becomes an RFC anyway or - the responsible AD holds the document pending satisfactory completion of the discussion. I suggest that the former is a bad result. Personally (but this may reflect my Discusses) I find that an active engagement by the authors and the Discussing AD on the issue and the potential resolution, always leads to speedy progression of the document either with the AD feeling stupid, or the document improved. Both are adequate results. Adrian, I suggest we all take a look at the original text of the Subject field. The problem here is that basic reviewing is being done by the ADs too late in the process. We are making the mistake of having ADs be exempt from IETF Last Call, and allowing them to be unprepared for the IESG vote. So we are combining education with voting. That's a paradigm error. By the time the IESG schedules the vote, ADs need to already have educated themselves about the document. Of course, the IESG discussion during the voting process well might uncover an actual, serious issue with the document; this should be exceptional and it should be an issue that the /IESG/ agrees needs to be resolved and it means that voting should not take place until it is. But that is quite different from the usual let's talk about it kind of Discuss imposed by individual ADs. So here's a simple proposal that pays attention to AD workload and includes a simple efficiency hack: When the IETF Last Call is issued, wait a few days, to see whether any serious issues are raised by the community. The really serious ones usually are raised quickly. If there are none, it's pretty certain the document will advance to an IESG vote. That leaves 7-10 days of IETF Last Call for ADs to get educated and ask questions, just like everyone else. Jari has expressed the goal of having AD concerns be raised more publicly. Moving AD review and comment to the IETF Last Call venue nicely accomplishes this, too. On 5/15/2013 8:55 AM, Thomas Narten wrote: 1) Discuss criteria should be principles, not rigid rules. The details of the issue at hand always matter, and it will sometimes come down to judgement calls where reasonable individuals just might disagree. We've been doing protocol specification for 40 years. If we can't codify the major concerns that warrant blocking advancement of a protocol, we're just being lazy. Really. That a given situation might cause the IESG to decide to enhance the list does not mean that the list shouldn't seek to be complete and precise and that ADs be held to it. At every turn, the approach we've taken to the IESG review and approval process is to limit actual accountability to the community. Everyone is well-intentioned, but really this makes the process a matter of personalities and not the rigor of serious professionalism. The IESG's process is far, far better than it was 10-15 years ago, but it still lacks meaningful predictability and serious accountability; it is not reasonable for the process to require that authors and chairs take the initiative of complaining when an AD is being problematic. In terms of quality assurance, the idea that we have a process that relies on the sudden insight of a single AD, at the end of a many-month process, is broken. It's fine if that person sees something that everyone else has missed until then, but that is quite different from designing a process that is claimed to rely on it. And of course, the reality is that we allow bad specs out the door all the time; we just allow fewer of them than many/most other standards bodies... d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
RE: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
That's a good question Dave. The community might like to comment. On the whole, I am told that if an AD weighs in with her comments during working group last call, her fearsome personality may overwhelm some of the WG participants and she may dominate the WG consensus. Is it possible that the same would happen in IETF last call? I know, of course, that a number of you have a robust ability to defend your opinions, but how would you (the community, not Dave) feel if ADs (speaking as ADs, not speaking as individual contributors) made their review comments during IETF last call? I agree that this would bring the tail forward a little (not a lot). I don't believe it would reduce AD work-load (which is another issue entirely). I have mixed feelings about whether it would change the processing time for a draft. That might depend on how the review is handled in IETF last call. Lastly, I think I disagree with you about really serious IETF last call comments coming in the first few days. It seems to me that we also get an number of late comments of substance resulting from people who are not familiar with the document reviewing it (such as directorate reviews and people who didn't notice the I-D sneaking through). Still thinking, Adrian -Original Message- From: Dave Crocker [mailto:d...@dcrocker.net] Sent: 16 May 2013 17:23 To: adr...@olddog.co.uk Cc: ietf@ietf.org Subject: Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process] On 5/15/2013 1:30 PM, Adrian Farrel wrote: Suppose the AD raised her concern by writing a Comment or sending an email and balloting No Objection. That would mean that the I-D would be approved for publication. At this point either: - the discussion goes on, but the document becomes an RFC anyway or - the responsible AD holds the document pending satisfactory completion of the discussion. I suggest that the former is a bad result. Personally (but this may reflect my Discusses) I find that an active engagement by the authors and the Discussing AD on the issue and the potential resolution, always leads to speedy progression of the document either with the AD feeling stupid, or the document improved. Both are adequate results. Adrian, I suggest we all take a look at the original text of the Subject field. The problem here is that basic reviewing is being done by the ADs too late in the process. We are making the mistake of having ADs be exempt from IETF Last Call, and allowing them to be unprepared for the IESG vote. So we are combining education with voting. That's a paradigm error. By the time the IESG schedules the vote, ADs need to already have educated themselves about the document. Of course, the IESG discussion during the voting process well might uncover an actual, serious issue with the document; this should be exceptional and it should be an issue that the /IESG/ agrees needs to be resolved and it means that voting should not take place until it is. But that is quite different from the usual let's talk about it kind of Discuss imposed by individual ADs. So here's a simple proposal that pays attention to AD workload and includes a simple efficiency hack: When the IETF Last Call is issued, wait a few days, to see whether any serious issues are raised by the community. The really serious ones usually are raised quickly. If there are none, it's pretty certain the document will advance to an IESG vote. That leaves 7-10 days of IETF Last Call for ADs to get educated and ask questions, just like everyone else. Jari has expressed the goal of having AD concerns be raised more publicly. Moving AD review and comment to the IETF Last Call venue nicely accomplishes this, too. On 5/15/2013 8:55 AM, Thomas Narten wrote: 1) Discuss criteria should be principles, not rigid rules. The details of the issue at hand always matter, and it will sometimes come down to judgement calls where reasonable individuals just might disagree. We've been doing protocol specification for 40 years. If we can't codify the major concerns that warrant blocking advancement of a protocol, we're just being lazy. Really. That a given situation might cause the IESG to decide to enhance the list does not mean that the list shouldn't seek to be complete and precise and that ADs be held to it. At every turn, the approach we've taken to the IESG review and approval process is to limit actual accountability to the community. Everyone is well-intentioned, but really this makes the process a matter of personalities and not the rigor of serious professionalism. The IESG's process is far, far better than it was 10-15 years ago, but it still lacks meaningful predictability and serious accountability; it is not reasonable for the process to require that authors and chairs take the initiative of complaining when an AD is being problematic. In terms of quality
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 5/16/2013 9:40 AM, Adrian Farrel wrote: That's a good question Dave. The community might like to comment. On the whole, I am told that if an AD weighs in with her comments during working group last call, her fearsome personality may overwhelm some of the WG participants and she may dominate the WG consensus. Only women ADs? [1] But seriously... Is it possible that the same would happen in IETF last call? I think that the public sway of ADs on the IETF list is less of a risk than on wg mailing lists. And note that my suggestion has ADs waiting to weigh in until community-generated active disagreement with the spec, or its passive agreement, has been established. In any event, the current reality of having an AD weigh in with a a process-blocking Discuss is not exactly /under-/whelming... Simply put: We are starting with the reality that an AD is going to be expressing their opinions. The question is when and how. It's not going to be /less/ intimidating to have it done as a Discuss. Contrary to some of the mythology that has been expressed about this, the frequent reality is that the typical wg goal is to clear the Discuss, not really to engage in debate with an AD who is blocking document progress. (I've watched this reality many times over the years.) That is far less healthy than having the AD's concern become part of the /public/ review process. I agree that this would bring the tail forward a little (not a lot). I don't believe it would reduce AD work-load (which is another issue entirely). Yes, the timing change is small. But the context change is enormous. Lastly, I think I disagree with you about really serious IETF last call comments coming in the first few days. It seems to me that we also get an number Some, sure. But I said it was an efficiency hack. The goal isn't for it to be perfect, just helpful. d/ [1] The question of proper referential language is a continuing surprise to me, given that the currently-popular conventions create more problems than they solve -- and it's even a question on a PSAT preparatory example that I saw yesterday. There's a pretty straightforward alternative that works nearly all the time: http://dcrocker.net/#gender -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 16, 2013, at 1:01 PM, Dave Crocker d...@dcrocker.net wrote: http://dcrocker.net/#gender That's what I do. It gets a bit awkward with verb agreement and constructs like themself, which elicits the dreaded red snake underline of doom. But I find it more comfortable than just subverting the sexist paradigm with a differently sexist paradigm.
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
Dave Crocker wrote: And of course, the reality is that we allow bad specs out the door all the time; we just allow fewer of them than many/most other standards bodies... But different to (at least some) other standards bodies, we lack an official means to publish defect reports (aka errata) to document defects in a _timely_(!!) fashion. (Timely = can be found where the RFC says that it can be found, and within at most a few weeks after the defect/omission has been found by an implementor). In theory, we have the errata process, and recent RFC even include a direct URL pointer to the RFC-Editors errata page on the title page: Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at http://www.rfc-editor.org/info/rfc. ( containing the real number of the RFC on which this appears). However, the Errata process is currently working poorly, primarily because a number of folks (including some IETF leadership) currently thinks that posting something a trivial as a missing vital one-line clarification to a published RFC as a substantial change that can only be performed by publishing a whole new RFC. -Martin
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On Thursday, May 16, 2013, Dave Crocker wrote: By the time the IESG schedules the vote, ADs need to already have educated themselves about the document. Oh, so you're suggesting adding another phase to the process: IESG education. OK. So here's a simple proposal that pays attention to AD workload and includes a simple efficiency hack: When the IETF Last Call is issued, wait a few days, to see whether any serious issues are raised by the community. The really serious ones usually are raised quickly. If there are none, it's pretty certain the document will advance to an IESG vote. That leaves 7-10 days of IETF Last Call for ADs to get educated and ask questions, just like everyone else. Placing a time limit on some phase of the process doesn't produce quality. The process should take as long as it must in order to produce a good outcome. The question shouldn't be how long the process takes (that's just a cheap shortcut), it should be how to make it more efficient in doing well. I have an idea: let's place a time limit on working groups: they need to finish drafts in six weeks or there will be penalties. Why not? Scott
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 5/16/13 10:01 AM, Dave Crocker wrote: On 5/16/2013 9:40 AM, Adrian Farrel wrote: That's a good question Dave. The community might like to comment. On the whole, I am told that if an AD weighs in with her comments during working group last call, her fearsome personality may overwhelm some of the WG participants and she may dominate the WG consensus. Only women ADs? [1] But seriously... If Tiresias prophet of Thebes is in your wg, I'd listen. Is it possible that the same would happen in IETF last call? I think that the public sway of ADs on the IETF list is less of a risk than on wg mailing lists. And note that my suggestion has ADs waiting to weigh in until community-generated active disagreement with the spec, or its passive agreement, has been established. Early participation AD's weighing in on draft's generally takes the form of just another participant unless you're asking for an opinion on a process point. wearing different hats on the course of the lifecycle is normal imho. opinion at the mic or on the list is not iesg review. The sponsoring AD (who is presumably the most involved) is not likely the one with the discuss later in the process since they had to do the initial review, put it through the ietf last call and then ballot it, so fundamentally they should have a document they can live with when it gets to that stage. In any event, the current reality of having an AD weigh in with a a process-blocking Discuss is not exactly /under-/whelming... Simply put: We are starting with the reality that an AD is going to be expressing their opinions. The question is when and how. It's not going to be /less/ intimidating to have it done as a Discuss. Contrary to some of the mythology that has been expressed about this, the frequent reality is that the typical wg goal is to clear the Discuss, not really to engage in debate with an AD who is blocking document progress. (I've watched this reality many times over the years.) That is far less healthy than having the AD's concern become part of the /public/ review process. I agree that this would bring the tail forward a little (not a lot). I don't believe it would reduce AD work-load (which is another issue entirely). Yes, the timing change is small. But the context change is enormous. Lastly, I think I disagree with you about really serious IETF last call comments coming in the first few days. It seems to me that we also get an number Some, sure. But I said it was an efficiency hack. The goal isn't for it to be perfect, just helpful. d/ [1] The question of proper referential language is a continuing surprise to me, given that the currently-popular conventions create more problems than they solve -- and it's even a question on a PSAT preparatory example that I saw yesterday. There's a pretty straightforward alternative that works nearly all the time: http://dcrocker.net/#gender
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 16, 2013, at 9:08 PM, Scott Brim scott.b...@gmail.commailto:scott.b...@gmail.com wrote: On Thursday, May 16, 2013, Dave Crocker wrote: By the time the IESG schedules the vote, ADs need to already have educated themselves about the document. Oh, so you're suggesting adding another phase to the process: IESG education. OK. I don't speak for Dave, but it makes sense that when a document reaches the IESG telechat, the questions to be asked are (1) Did this get sufficient review by sufficiently capable people, and (2) Has the process been followed, including have all issues been addressed. The time for asking whether the group has considered making this field fixed length instead of variable, or whether RFC 2119 language is used in an appropriate way, or whether the protocol is extensible enough is at IETF last call. This is true of any participants, and ADs can do this too, regardless of whether their presence might intimidate the crowd. It could even be that if ADs voice their concerns at that time, others might join in, making the LC time more useful and less about just waiting two or four weeks. There is a problem, though, that this will increase the load on ADs. Other concerns raised during IETF LC may lead to revised I-Ds, which the ADs will need to re-read (or at least look at the diff). I don't know how significant this extra work would be, but it would come at a time that we're thinking of ways to reduce AD workload. It might also require prolonging the LC time, because there would be actual discussion in it. Yoav
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
I think Dave's idea is worth looking at, but have one comment: On 05/16/2013 09:46 PM, Yoav Nir wrote: There is a problem, though, that this will increase the load on ADs. There is that. But don't forget that ADs mostly read everything in IESG review and often comment. Even leaving aside DISCUSSes, if every IETF LC draft got a bunch of AD review comments, then the IETF LC would be much busier than now. And its in the nature of IETFers to chime in. So this would I suspect increase everyone's load, if it works, and not just ADs. Still worth a look though. S.
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 16, 2013, at 9:40 AM, Adrian Farrel adr...@olddog.co.uk wrote: On the whole, I am told that if an AD weighs in with her comments during working group last call, her fearsome personality may overwhelm some of the WG participants and she may dominate the WG consensus. There may be places where that happens, but I would be surprised if it happened in my working group. I think it is fair to say that the AD (or an IAB member, or someone who has recognized expertise on the topic) is likely to be listened to more carefully than some others might. Heck, I'm careful when I make a technical comment on a document in my working group, flagging it with /chair to ensure that it is seen as intended - a comment by a competent practitioner of the art, not a process remark or an attempt to trump some other view. Speaking personally, I would prefer to see those comments in the WGLC, not IETF Last Call, if we can make that happen. For example, I'd like to get directorate reviews done (gen-art, security directorate, etc) in the timeframe of WGLC.
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
Dave, On 17/05/2013 04:23, Dave Crocker wrote: ... The problem here is that basic reviewing is being done by the ADs too late in the process. You are making a lot of assumptions in that sentence. At least these: 1. Basic reviewing means 2. At some stage before approval, ADs should perform basic reviewing. 3. Doing basic reviewing after the document has completed IETF LC is too late. Now, if basic reviewing means (A) looking for fundamental flaws that is one thing. If it means (B) getting a general understanding of the topic it's another. I'm really not sure which you mean. We are making the mistake of having ADs be exempt from IETF Last Call, and allowing them to be unprepared for the IESG vote. So we are combining education with voting. That's a paradigm error. By the time the IESG schedules the vote, ADs need to already have educated themselves about the document. Ah, maybe you mean basic (B). Well, I think this is a totally unrealistic expectation. There are too many drafts on too diverse a range of subjects for ADs to educate themselves at the time of IETF LC, which may be weeks or months before a draft hits the agenda. (I'm not talking about the sponsoring AD or her co-AD, of course: they are presumed to be aware of what's going on in their area.) Busy people are deadline driven, and the IESG agenda is an imminent deadline for ADs in a way that an IETF LC in another Area isn't. Of course, the IESG discussion during the voting process well might uncover an actual, serious issue with the document; And that's basic (B). I think we all agree that it's unfortunate if a basic flaw shows up during the IESG ballot phase, but please don't blame the IESG for that. Blame the WG and the IETF as a whole for failing to pick it up during WG LC and IETF LC. Thank the IESG for finding it. this should be exceptional and it should be an issue that the /IESG/ agrees needs to be resolved and it means that voting should not take place until it is. But that is quite different from the usual let's talk about it kind of Discuss imposed by individual ADs. I'm not quite sure what you're saying here, but I think you're saying that the majority of clarity and cross-area issues should be picked up earlier. I couldn't agree more, but don't expect the over-worked IESG to fix that. The rest of us need to fix that. So here's a simple proposal that pays attention to AD workload and includes a simple efficiency hack: When the IETF Last Call is issued, wait a few days, to see whether any serious issues are raised by the community. The really serious ones usually are raised quickly. If there are none, it's pretty certain the document will advance to an IESG vote. That leaves 7-10 days of IETF Last Call for ADs to get educated and ask questions, just like everyone else. Which, as I said above, is a totally unrealistic expectation. Jari has expressed the goal of having AD concerns be raised more publicly. Moving AD review and comment to the IETF Last Call venue nicely accomplishes this, too. Thanks but no thanks. My mail is full enough with discussion of drafts I will never read as it is. AD reviews should certainly go to the WG list unless they are only nits, but isn't that what everybody does anyway? On 5/15/2013 8:55 AM, Thomas Narten wrote: 1) Discuss criteria should be principles, not rigid rules. The details of the issue at hand always matter, and it will sometimes come down to judgement calls where reasonable individuals just might disagree. We've been doing protocol specification for 40 years. If we can't codify the major concerns that warrant blocking advancement of a protocol, we're just being lazy. Really. That a given situation might cause the IESG to decide to enhance the list does not mean that the list shouldn't seek to be complete and precise and that ADs be held to it. I agree with Thomas. We need to be able to apply common sense. Rules and targets are the enemy of common sense. At every turn, the approach we've taken to the IESG review and approval process is to limit actual accountability to the community. Everyone is well-intentioned, but really this makes the process a matter of personalities and not the rigor of serious professionalism. The IESG's process is far, far better than it was 10-15 years ago, but it still lacks meaningful predictability and serious accountability; it I find that the tracker in its present state has substantially improved both visibility and predictability. is not reasonable for the process to require that authors and chairs take the initiative of complaining when an AD is being problematic. Huh? Who else will complain? In terms of quality assurance, the idea that we have a process that relies on the sudden insight of a single AD, at the end of a many-month process, is broken. It's fine if that person sees something that everyone else has missed until then, but that is quite
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 16, 2013, at 5:00 PM 5/16/13, Fred Baker (fred) f...@cisco.com wrote: On May 16, 2013, at 9:40 AM, Adrian Farrel adr...@olddog.co.uk wrote: On the whole, I am told that if an AD weighs in with her comments during working group last call, her fearsome personality may overwhelm some of the WG participants and she may dominate the WG consensus. There may be places where that happens, but I would be surprised if it happened in my working group. I think it is fair to say that the AD (or an IAB member, or someone who has recognized expertise on the topic) is likely to be listened to more carefully than some others might. Heck, I'm careful when I make a technical comment on a document in my working group, flagging it with /chair to ensure that it is seen as intended - a comment by a competent practitioner of the art, not a process remark or an attempt to trump some other view. Speaking personally, I would prefer to see those comments in the WGLC, not IETF Last Call, if we can make that happen. For example, I'd like to get directorate reviews done (gen-art, security directorate, etc) in the timeframe of WGLC. I think Fred is returning to an earlier theme here, when he asks for earlier review. Perhaps, as has been already suggested in this thread, we should think about SIRSbis? First, from draft-carpenter-icar-sirs-01: The procedure described in this document is intended to solve, or palliate, a number of related problems that have been observed in the IETF process [PROBLEM]: *submission of documents to the IESG that still have significant problems (leading to delay) *failure to detect fundamental problems and Internet- wide issues at an early stage Particularly because of the second point, it is impossible to resolve these problems simply by giving additional responsibility to working groups themselves. An additional procedure is needed. In my opinion, it's important to assign responsibility (and accountability) to all WGs for producing publication-ready documents. I agree that some additional work is needed before WGs send documents to the IESG. Perhaps we can accomplish these goals through reorganizing the work we are I suggest we might want to combine the need for more responsibility with the discussion of a new really close to being ready document state. However, rather than a new document state, suppose we codify the expectation that a document that has passed WG last call is essentially ready-to-publish? Correspondingly, any significant problems found in a document after WG last call would be considered a serious defect. Discussion: I realize that, elsewhere in this thread, it has been asserted (or at least implied), that WGs already have this responsibility and DISCUSSes on document are usually unnecessary. In practice, while there may still be unnecessary DISCUSSes, my experience as AD was that most DISCUSSes were appropriate and each one referred to a problem that the WG had missed. Let's get all the expert review possible - directorate, AD, cross-area - in the WG last call review. What pops out *should* be ready for publication. Any issues raised by these reviews in WG last call will be exposed to and can be discussed by the WG at large, rather than being buried in the noise of IETF last call discussions or being fixed in more focused discussions among the IESG and the document authors. This procedure diverges some from draft-carpenter-icar-sirs-01, in that it doesn't add a new form of review process. Instead, it reschedules reviews that were going to take place anyway earlier in the process, so there is little or no new work added to the document publication process. Perhaps the WG chairs would want to assign document shepherds earlier in the process, as well, investing the document shepherds with the responsibility of getting the right reviews and advising the WG chairs as to the readiness of the document for advancement. Any WGs willing to volunteer as experimental subjects? There is really no new process to invent ... it's mostly a matter of realigning expectations and responsibilities out to - Ralph
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 05/16/2013 04:46 PM, Yoav Nir wrote: The time for asking whether the group has considered making this field fixed length instead of variable, or whether RFC 2119 language is used in an appropriate way, or whether the protocol is extensible enough is at IETF last call. Actually the time for asking these questions is long before IETF-wide Last Call. We need widespread review of proposals for standards-track documents long before a WG thinks it's finished with those documents. It's a gaping hole in our process. Fix that problem, and most of the conflicts between IESG and WGs that surround DISCUSS votes will go away. Keith
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
Dave - I hope you'll indulge my selective quoting as I have a couple of specific points to address. My apologies if I end up quoting you out of context... On May 16, 2013, at 12:23 PM 5/16/13, Dave Crocker d...@dcrocker.net wrote: [...] So here's a simple proposal that pays attention to AD workload and includes a simple efficiency hack: When the IETF Last Call is issued, wait a few days, to see whether any serious issues are raised by the community. The really serious ones usually are raised quickly. If there are none, it's pretty certain the document will advance to an IESG vote. That leaves 7-10 days of IETF Last Call for ADs to get educated and ask questions, just like everyone else. Jari has expressed the goal of having AD concerns be raised more publicly. Moving AD review and comment to the IETF Last Call venue nicely accomplishes this, too. I just posted elsewhere a suggestion to move this review even earlier, to WG last call. Accomplishes most of the same ends, while putting the discussion in front of the IETF participants who are, presumably, most invested in the resulting document. [...] In terms of quality assurance, the idea that we have a process that relies on the sudden insight of a single AD, at the end of a many-month process, is broken. It's fine if that person sees something that everyone else has missed until then, but that is quite different from designing a process that is claimed to rely on it. As you and I have discussed in person, I am 100% in agreement with this comment. As much as I liked to think of myself, when I was an AD, as a rock-god Network Expert with complete and in-depth insight into every document I reviewed, I know the reality was that any problems I might have found were related to the old observation that even a blind squirrel finds an acorn occasionally. And of course, the reality is that we allow bad specs out the door all the time; we just allow fewer of them than many/most other standards bodies... You're such an optimist. - Ralph d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 16, 2013, at 5:58 PM 5/16/13, Keith Moore mo...@network-heretics.com wrote: On 05/16/2013 04:46 PM, Yoav Nir wrote: The time for asking whether the group has considered making this field fixed length instead of variable, or whether RFC 2119 language is used in an appropriate way, or whether the protocol is extensible enough is at IETF last call. Actually the time for asking these questions is long before IETF-wide Last Call. We need widespread review of proposals for standards-track documents long before a WG thinks it's finished with those documents. It's a gaping hole in our process. Hear, hear. Fix that problem, and most of the conflicts between IESG and WGs that surround DISCUSS votes will go away. Well, you might be a little optimistic here, but I agree in theory. Keith - Ralph
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 5/16/13 2:58 PM, Keith Moore wrote: On 05/16/2013 04:46 PM, Yoav Nir wrote: The time for asking whether the group has considered making this field fixed length instead of variable, or whether RFC 2119 language is used in an appropriate way, or whether the protocol is extensible enough is at IETF last call. Actually the time for asking these questions is long before IETF-wide Last Call. We need widespread review of proposals for standards-track documents long before a WG thinks it's finished with those documents. It's a gaping hole in our process. As a Chair and as an AD I have asked for external and cross area reviews of some documents before they were considered for WG acceptance. this doesn't apply to all work we processed but it does apply to those where we were clear that such input was going to be useful. One case you can see for that today is with capwap extensions that are potentially in opsawg. Fix that problem, and most of the conflicts between IESG and WGs that surround DISCUSS votes will go away. Maybe but I wouldn't take that as an article of faith. You're going to get pressure for more changes when fresh eyes review something. Keith
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 5/16/13 4:07 PM, Ralph Droms wrote: On May 16, 2013, at 5:58 PM 5/16/13, Keith Moore mo...@network-heretics.com wrote: On 05/16/2013 04:46 PM, Yoav Nir wrote: The time for asking whether the group has considered making this field fixed length instead of variable, or whether RFC 2119 language is used in an appropriate way, or whether the protocol is extensible enough is at IETF last call. Actually the time for asking these questions is long before IETF-wide Last Call. We need widespread review of proposals for standards-track documents long before a WG thinks it's finished with those documents. It's a gaping hole in our process. Hear, hear. One suggestion I've heard several times is a kind of cross-area buddy system (e.g., ask or assign someone with clue in Area X to help WG Y in Area Z early and often with regard to specific issues that are often addressed in Area X). Unfortunately, this suffers from the same problem that too many WGs suffer from on their own: participant burnout. But I still think it's a good idea... Peter -- Peter Saint-Andre https://stpeter.im/
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 16, 2013, at 1:46 PM, Yoav Nir y...@checkpoint.com wrote: There is a problem, though, that this will increase the load on ADs. Other concerns raised during IETF LC may lead to revised I-Ds, which the ADs will need to re-read (or at least look at the diff). I don't know how significant this extra work would be, but it would come at a time that we're thinking of ways to reduce AD workload. It might also require prolonging the LC time, because there would be actual discussion in it. If they raise the issue later in a discuss, will they not have to do this anyway? How does this relate to the timing of the comment or the vehicle by which it is conveyed? The ignorance of how to use new knowledge stockpiles exponentially. - Marshall McLuhan
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On 05/16/2013 06:09 PM, joel jaeggli wrote: Fix that problem, and most of the conflicts between IESG and WGs that surround DISCUSS votes will go away. Maybe but I wouldn't take that as an article of faith. You're going to get pressure for more changes when fresh eyes review something. Yeah, every new set of eyes that looks at a document is going to have some new ideas for what the document should be. The trick is to get those eyes to look at the document earlier in the process, when it's easier to fix problems. Maybe if we did the early review well enough, the scope of Last Call comments could be limited in some way. Keith
Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
The claim (or one of the claims) is that some ADs may place Discusses that are intended to raise a discussion with the authors/WG that could equally have been raised with a Comment or through direct email. This, it is claimed, may unnecessarily delay the document from completing the publication process. Now the dangerous bit, Suppose the AD raised her concern by writing a Comment or sending an email and balloting No Objection. That would mean that the I-D would be approved for publication. At this point either: - the discussion goes on, but the document becomes an RFC anyway or - the responsible AD holds the document pending satisfactory completion of the discussion. I suggest that the former is a bad result. Not that the authors/WG will ignore the discussion, but if they disagree on something the AD considers very important, the authors/WG have no incentive to participate in the discussion. Of course, all participants in this thread so far would never behave like that, but there is a possibility that this will happen for some authors. I also suggest that the latter introduces exactly the same amount of delay as the Discuss. Personally (but this may reflect my Discusses) I find that an active engagement by the authors and the Discussing AD on the issue and the potential resolution, always leads to speedy progression of the document either with the AD feeling stupid, or the document improved. Both are adequate results. Adrian
Re: Is this an elephant? [Was: call for ideas: tail-heavy IETF process]
On May 15, 2013, at 4:30 PM, Adrian Farrel adr...@olddog.co.uk wrote: I suggest that the former is a bad result. Not that the authors/WG will ignore the discussion, but if they disagree on something the AD considers very important, the authors/WG have no incentive to participate in the discussion. Of course, all participants in this thread so far would never behave like that, but there is a possibility that this will happen for some authors. This can also happen purely by accident—it need not involve malice on anyone's part.
