Re: Conclusion of the last call on draft-housley-two-maturity-levels
+4 and rotfl Brian On 2011-09-16 17:17, Hadriel Kaplan wrote: I thought the counting of votes was finished on this topic but people seem to keep emailing their support/lack-of, so naturally I will be a good lemming and do the same. 1) I am in favor of the two-maturity-levels draft and change. I have consulted a textbook on Euclidean geometry and determined that the distance from level 2 to 1 is shorter than 3 to 1, getting us closer to the actual location most of us are at (which is of course 1 maturity level). 2) I am strongly opposed to draft-loughney-newtrk-one-size-fits-all-01, because it is far too rational and sane, and would prevent this topic from continuing forever. Furthermore, I am against any move to 1 maturity level because apparently there are one or two people with so much free time or posterity they actually bother moving PS to higher levels these days, and who are we to squash their hobby/passion/disorder? (In fact, I was almost going to suggest we go to a 4 or 5 maturity level process just to give these people more harmless things to do, but I digress...) 3) The IESG should be applauded/thanked for recognizing there is only one maturity level (PS), and taking the steps necessary to treat potential RFCs as such from a quality perspective. But they should be denigrated for not telling us they did that. So they come out even. 4) Regarding the discussion in this thread about what types of comments should be counted or not: I believe we should produce a new RFC concerning what response phrases in emails are going to be counted or not for consensus counting, so that we may know what to say in the future to get our votes counted. (Of course the big question everyone wants to know is when will such a new RFC reach the second maturity level?) -hadriel p.s. in all seriousness, I'm in favor of this two-maturiy-level draft. I do not think it is change for change's sake, but rather a change attempting to accommodate differing viewpoints of our present location and where we want to be. If it fails to change the status-quo of 1 level, that's *OK*, we can try again - the Internet won't collapse because of this document, and neither will the IETF. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
I thought the counting of votes was finished on this topic but people seem to keep emailing their support/lack-of, so naturally I will be a good lemming and do the same. 1) I am in favor of the two-maturity-levels draft and change. I have consulted a textbook on Euclidean geometry and determined that the distance from level 2 to 1 is shorter than 3 to 1, getting us closer to the actual location most of us are at (which is of course 1 maturity level). 2) I am strongly opposed to draft-loughney-newtrk-one-size-fits-all-01, because it is far too rational and sane, and would prevent this topic from continuing forever. Furthermore, I am against any move to 1 maturity level because apparently there are one or two people with so much free time or posterity they actually bother moving PS to higher levels these days, and who are we to squash their hobby/passion/disorder? (In fact, I was almost going to suggest we go to a 4 or 5 maturity level process just to give these people more harmless things to do, but I digress...) 3) The IESG should be applauded/thanked for recognizing there is only one maturity level (PS), and taking the steps necessary to treat potential RFCs as such from a quality perspective. But they should be denigrated for not telling us they did that. So they come out even. 4) Regarding the discussion in this thread about what types of comments should be counted or not: I believe we should produce a new RFC concerning what response phrases in emails are going to be counted or not for consensus counting, so that we may know what to say in the future to get our votes counted. (Of course the big question everyone wants to know is when will such a new RFC reach the second maturity level?) -hadriel p.s. in all seriousness, I'm in favor of this two-maturiy-level draft. I do not think it is change for change's sake, but rather a change attempting to accommodate differing viewpoints of our present location and where we want to be. If it fails to change the status-quo of 1 level, that's *OK*, we can try again - the Internet won't collapse because of this document, and neither will the IETF. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Keith == Keith Moore mo...@network-heretics.com writes: Keith 2) This will not do any good Keith IMO, that is a valid objection. Stability in our process is Keith desirable; therefore change merely for the sake of change is Keith undesirable. This will not do any good, stability is important, so this should not be done, is an objection. This will not do any good, is neutral. You believe that stability is important. Others believe that forward progress and being seen to do something are good. I do tend to come down on your side, and if I think something isn't going to do do good I'm likely to actually state an objection. However for a lot of reasons, I think the IESG should actually require people to present something that is constructionally supportive or an objection before counting it as such. This will not do any good, is not such. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 12, 2011, at 7:32 AM, Sam Hartman wrote: Keith == Keith Moore mo...@network-heretics.com writes: Keith 2) This will not do any good Keith IMO, that is a valid objection. Stability in our process is Keith desirable; therefore change merely for the sake of change is Keith undesirable. This will not do any good, stability is important, so this should not be done, is an objection. This will not do any good, is neutral. You believe that stability is important. Others believe that forward progress and being seen to do something are good. I do tend to come down on your side, and if I think something isn't going to do do good I'm likely to actually state an objection. However for a lot of reasons, I think the IESG should actually require people to present something that is constructionally supportive or an objection before counting it as such. This will not do any good, is not such. I agree that a statement of the form this will not do any good is more compelling if it is supported by an argument as to _why_ it won't do any good. Such a statement by itself should count against consensus, but it shouldn't sway anyone else into changing his opinion. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Sam Hartman wrote: 1) I support moving to a two level process. I don't, and there were some other things in the document (last time I read) that to which I dissent. I do not think the following types of comments should be considered as objections when judging this sort of consensus: 1) You are not solving the most important problem 2) This will not do any good These are not objection, but when gauging consent, then counting them as consent rather than no consent or dissent would IMO be inappropriate. objections is stuff that needs to be addressed by issue resolution prior to any gauging/judging of consent. -Martin ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On 9/9/11 6:33 PM, Thomas Narten wrote: I am surely going to regret posting, because I have largely tuned out of this discussion due to the endless repetition, etc. I am not supportive of the current document, because I don't think it solves anything. To me, it smack a bit of change for changes sake. One of the key problems that isn't being addressed is that mixing advancement of a spec with revising a spec are fundmentally at odds with each other. Advancing a spec is done for marketing, political, process and other reasons. E.g., to give a spec more legitimacy. Or to more clear replace an older one. Nothing wrong with that. But the real reason that the IETF *should* be revising specs is to fix bugs and improve protocol quality. By definition, you cannot revise a spec (in a real, meaningful way) and advance at the same time. The spirit (if not letter) of advancement says you advance a spec, when there are implementations *based on the spec being advanced*. That means you can't revise a spec and at the same time have implementations derived from the revised spec. (You can have implementations based on mailing list discussions, but that is NOT the same thing.) The IETF is about making the Internet work better. That means revising specs (from a technical perpective) when they need to be revised. If we want to fix what's broken, we should focus on getting documents revised (without simultaneously advancing them). But once you do that, one quickly finds out that there are real and sometimes complicated reasons why revising documents is hard. In many cases, widely deployed protocols really need to have a revised spec developed (and the authors will readily admit that). But that just doesn't happen, not because of process, but because of other much more fundamental problems. E.g., Not enough energy from the relevant experts. key people who know a spec have moved on to other newer technologies or other higher priority things. Fixing specs can also be painful because some vendors won't or can't change their deployed implementations, so don't really want an updated spec that invalidates their implementation. etc., etc. It can be very hard for a WG to walk the line between we need to fix this and can we tweak the spec without invalidating various deployed implementations. IMO, these sorts of issues are the real reasons documents don't advance more. It's not just about process. Agreed. Well said. -Doug ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Thomas, I am in full agreement that document revision and bug fixing is the more important activity for IETF. Not just in my opinion, but I think we can also see it from the numbers of bis documents versus numbers of advancing documents. But I think some amount of bug fixing and revision is compatible with advancing a document. Clarifications. Removing cruft that no one ever implemented. Documenting security or other concerns better. I don't think it is quite as black and white as you wrote (you cannot revise a spec in a real meaningful way). Its just a question of how big surgery the specification needs. That being said, I do agree with you that the reasons behind lack of work are complex and often have little to do with IETF processes. It is indeed often the case that people with real capability to do something about a specification are either not interested in too minor changes or have vested interests in not causing their implementations to become non-compliant due to bigger changes. There is little that we can do in the IETF process about this, except maybe make the overall RFC publication easier. The industry will come to the IETF when they have a common incentive for a standard or an update thereof *and* they believe they can actually get the work done. But bringing ourselves back to the topic of process changes, I agree that Russ' draft is not solving our biggest problems. (I personally view it as a tiny effort to remove an unused feature from our process. It could have removed two levels instead of one, but would that have made it easier or harder to find consensus for the change?) But more importantly, I know that you have driven several process and administration related efforts. Do you have a suggestion on what kind of useful thing we could do to address some of the bigger issues? Or better, a draft in your desk drawer that I could take forward? Jari ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
I think you will see that this question was discussed at least once. We asked about moving to a one-level maturity model instead. The conclusion was that it was possible to go from a two-level to a one-level in the future if that is appropriate. However, if we go straight to a one-level now, and then learn that a two-level would have been better, we would be stuck. Russ On Sep 11, 2011, at 7:50 AM, Jari Arkko wrote: I personally view it as a tiny effort to remove an unused feature from our process. It could have removed two levels instead of one, but would that have made it easier or harder to find consensus for the change? ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
--On Sunday, September 11, 2011 11:57 -0400 Russ Housley hous...@vigilsec.com wrote: I think you will see that this question was discussed at least once. We asked about moving to a one-level maturity model instead. The conclusion was that it was possible to go from a two-level to a one-level in the future if that is appropriate. However, if we go straight to a one-level now, and then learn that a two-level would have been better, we would be stuck. But, if we go from a three-level to a two-level now, without compelling evidence that it would make an improvement, and then learn that a three-level would have been better, we would also be stuck. I'm not sure I see the difference between the two cases. But I also don't see any advantage in prolonging the discussion. If I correctly understand Jari's note, the IESG has decided that there is adequate consensus for this move. Either people will appeal that decision after it is formally announced or they won't. I would hope that, even if there is an appeal, it would not reopen the discussions we have been having over and over again. If no one does, or if any appeal that is ultimately filed is ultimately rejected, my hope is that we can all pull together to try to make this work. john ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
John C Klensin wrote: --On Sunday, September 11, 2011 11:57 -0400 Russ Housley However, if we go straight to a one-level now, and then learn that a two-level would have been better, we would be stuck. But, if we go from a three-level to a two-level now, without compelling evidence that it would make an improvement, and then learn that a three-level would have been better, we would also be stuck. I'm not sure I see the difference between the two cases. But I also don't see any advantage in prolonging the discussion. If I correctly understand Jari's note, the IESG has decided that there is adequate consensus for this move. Either people will appeal that decision after it is formally announced or they won't. I would hope that, even if there is an appeal, it would not reopen the discussions we have been having over and over again. If no one does, or if any appeal that is ultimately filed is ultimately rejected, my hope is that we can all pull together to try to make this work. Do you have list of documents that might immediately benefit or would be top candidates to be moved to IS? Top ten? FWIW, I was wondering which BIS documents with no RFC publication dates would be candidates. 93 total. - DRAFT INITDATEDAYS REVS CSTATE - draft-ietf-appsawg-rfc3536bis 2011-05-06 128 4 RFC Ed Queue draft-ietf-yam-rfc4409bis 2011-05-06 128 0 Publication Requested draft-ietf-appsawg-rfc3536bis 2011-05-02 132 0 RFC Ed Queue draft-iesg-rfc1150bis 2011-04-22 142 0 RFC Ed Queue draft-iesg-rfc1150bis 2011-04-22 142 0 RFC Ed Queue draft-faibish-nfsv4-pnfs-block-disk-protection 2011-03-07 188 0 ID Exists draft-hoffman-rfc3536bis 2011-03-07 188 0 ID Exists draft-hoeneisen-rfc5333bis 2011-03-06 189 0 ID Exists draft-ietf-6man-flow-3697bis 2011-01-30 224 0 IESG Evaluation - AD Followup draft-ietf-6man-flow-3697bis 2011-01-30 224 3 IESG Evaluation - AD Followup draft-crocker-dkim-rfc4871bis-doseta 2011-01-13 241 0 ID Exists draft-yevstifeyev-httpbis-http-warning-registry2011-01-11 243 0 AD is watching - External Party draft-gont-tcpm-rfc1948bis 2011-01-03 251 0 ID Exists draft-gellens-mime-bucket-bis 2010-12-23 262 5 IESG Evaluation - AD Followup draft-ietf-httpbis-authscheme-registrations2010-11-09 306 0 AD is watching draft-ietf-eai-5738bis 2010-11-09 306 0 AD is watching draft-ietf-ccamp-asymm-bw-bidir-lsps-bis 2010-10-18 328 1 IESG Evaluation draft-ietf-ccamp-asymm-bw-bidir-lsps-bis 2010-10-18 328 0 IESG Evaluation draft-henderson-tcpm-rfc3782-bis 2010-10-18 328 0 ID Exists draft-faltstrom-5892bis2010-10-18 328 1 RFC Ed Queue draft-ah-dnsext-rfc1995bis-ixfr2010-10-18 328 0 ID Exists draft-blanton-tcpm-3517bis 2010-10-18 328 0 ID Exists draft-ietf-eai-rfc5337bis-dsn 2010-10-18 328 0 AD is watching draft-cui-ippm-rfc5136bis 2010-10-16 330 0 ID Exists draft-ietf-iri-4395bis-irireg 2010-09-30 346 0 AD is watching draft-ietf-eai-rfc5721bis 2010-09-27 349 0 AD is watching draft-ietf-dkim-rfc4871bis 2010-08-16 391 3 RFC Ed Queue draft-ietf-eai-rfc5335bis 2010-07-05 433 0 AD is watching - AD Followup draft-cooper-pkix-rfc2560bis 2010-06-28 440 0 ID Exists draft-ietf-ccamp-rfc5787bis2010-06-25 443 0 ID Exists draft-ietf-eai-frmwrk-4952bis 2010-06-25 443 4 RFC Ed Queue draft-barnes-sipcore-rfc4244bis-callflows 2010-06-24 444 0 ID Exists draft-henderson-hip-rfc5206-bis2010-01-03 616 0 ID Exists draft-ietf-yam-5321bis-smtp-pre-evaluation 2009-11-11 669 2 DNP-waiting for AD note draft-law-rfc4869bis 2009-11-10 670 0 Waiting for AD Go-Ahead draft-huang-behave-rfc3338bis 2009-10-19 692 0 ID Exists draft-huang-behave-rfc2767bis 2009-10-19 692 0 ID Exists draft-bishnoi-mpls-ldp-node-group 2009-10-19 692 0 ID Exists draft-faibish-nfsv4-pnfs-access-permissions-check 2009-10-18 693 0
Re: Conclusion of the last call on draft-housley-two-maturity-levels
--On Sunday, September 11, 2011 18:01 -0400 Hector sant9...@gmail.com wrote: John C Klensin wrote: --On Sunday, September 11, 2011 11:57 -0400 Russ Housley However, if we go straight to a one-level now, and then learn that a two-level would have been better, we would be stuck. ... But I also don't see any advantage in prolonging the discussion. If I correctly understand Jari's note, the IESG has decided that there is adequate consensus for this move. Either people will appeal that decision after it is formally announced or they won't. I would hope that, even if there is an appeal, it would not reopen the discussions we have been having over and over again. If no one does, or if any appeal that is ultimately filed is ultimately rejected, my hope is that we can all pull together to try to make this work. Do you have list of documents that might immediately benefit or would be top candidates to be moved to IS? Top ten? FWIW, I was wondering which BIS documents with no RFC publication dates would be candidates. 93 total. Based on the first few, there are a bunch of errors in your list. I'd think the likely candidates of this type would be documents that have been approved, or are near approval, for Draft Standard. The more interesting case would be documents like 5321. There is no question about either interoperability or wide deployment and use, but, given outstanding errata, the list of proposed changes in the YAM preevaluation document, and other issues that have come up, I doubt that there would be consensus for moving the existing document to full standard. On the other hand, this change does not increase the motivation to do more work on it and may decrease it, so it is, as I said, and interesting case. A few comments about part of your list below... -- DRAFT INITDATE DAYS REVS CSTATE -- --- draft-ietf-appsawg-rfc3536bis 2011-05-06 128 4 RFC Ed Queue Already published as RFC 6365, a BCP and hence irrelevant to the maturity level discussion. draft-ietf-yam-rfc4409bis 2011-05-06 128 0 Publication Requested In queue for publication as full/Internet standard. Therefore presumably unaffected by this change. draft-ietf-appsawg-rfc3536bis 2011-05-02 132 0 RFC Ed Queue See above. You appear to have counted this document at least three times. draft-iesg-rfc1150bis 2011-04-22 142 0 RFC Ed Queue draft-iesg-rfc1150bis 2011-04-22 142 0 RFC Ed Queue Published as RFC 6360, an informational document and hence unaffected by this change. Note: probably counted twice. draft-faibish-nfsv4-pnfs-block-disk-protection 2011-03-07 188 0 ID Exists I have no personal information on this document. draft-hoffman-rfc3536bis 2011-03-07 188 0 ID Exists This document was replaced by draft-ietf-appsawg-rfc3536bis and then by RFC 6365, for which see above. And so on. The number using these criteria would appear to be somewhat smaller than you believe. john ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
John C Klensin wrote: --On Sunday, September 11, 2011 18:01 -0400 Hector sant9...@gmail.com wrote: FWIW, I was wondering which BIS documents with no RFC publication dates would be candidates. 93 total. Based on the first few, there are a bunch of errors in your list. Yes, just imported into SQL database what Jari provided for the raw data and for the output shown below, the SQL query was: SELECT draft, initdate, datediff(Now(),initdate) as days, revs, cstate FROM idpub WHERE rfcdate = -00-00 AND draft like %bis% ORDER by days; Sure, there seems to have some book keeping aging in the data. i.e. duplicate records or new ones not replacing others (hence the dupes you saw), verifying some of the drafts with the actual, some had real revisions but the revision count field was 0. I just wanted to give a rough output of those labeled as bis but did not have a RFC publishing date, sorted by the number of days from start to today. I'd think the likely candidates of this type would be documents that have been approved, or are near approval, for Draft Standard. The more interesting case would be documents like 5321. There is no question about either interoperability or wide deployment and use, but, given outstanding errata, the list of proposed changes in the YAM preevaluation document, and other issues that have come up, I doubt that there would be consensus for moving the existing document to full standard. On the other hand, this change does not increase the motivation to do more work on it and may decrease it, so it is, as I said, and interesting case. I would assume those documents that have been in the IESG hands will get flushed out quickly. For example, lets used DKIM, a query of: select draft, initdate, iesgdays as iesg, datediff(Now(),initdate) as days, rfcdate, revs, cstate from idpub where draft like %dkim% order by days; Produces: --- DRAFT INITDATEIESG DAYS RFCDATE REVS CSTATE --- draft-crocker-dkim-doseta 2011-01-13 114 241 -00-00 0ID Exists draft-crocker-dkim-rfc4871bis-doseta 2011-01-13 114 241 -00-00 0ID Exists draft-ietf-dkim-rfc4871bis2010-08-16 304 391 -00-00 3RFC Ed Queue draft-ietf-dkim-mailinglists 2010-05-08 344 491 -00-00 2RFC Ed Queue draft-fenton-dkim-reputation-hint 2009-02-12 180 941 -00-00 0ID Exists draft-ietf-dkim-rfc4871-errata2009-01-26 87958 2009-08-27 3RFC Published draft-hallambaker-dkim-extensions 2008-07-02 196 1166 -00-00 0ID Exists draft-ietf-marf-dkim-reporting2007-12-03 468 1378 -00-00 0AD is watching draft-ietf-dkim-deployment2007-11-11 716 1400 2010-06-01 2RFC Published draft-ietf-dkim-ssp-requirements 2006-08-10 317 1858 2007-10-17 1RFC Published draft-hallambaker-dkimpolicy 2006-06-22 180 1907 -00-00 0ID Exists draft-ietf-dkim-overview 2006-06-22 857 1907 2009-07-08 2RFC Published draft-fenton-dkim-threats 2005-09-28 2136 2174 -00-00 0Dead draft-ietf-dkim-threats 2005-09-28 102 2174 2006-10-04 1RFC Published draft-dkim-pkix 2005-09-09 185 2193 -00-00 0ID Exists draft-ietf-dkim-base 2005-07-12 225 2252 2007-05-28 5RFC Published draft-ietf-dkim-ssp 2005-07-12 452 2252 2009-08-12 4RFC Published --- With my familiarity with DKIM, the above seems to reflect pretty close to the history. I venture among those yet to be published but in the hands of the IESG, seem to be good candidates. draft-ietf-dkim-rfc4871bisI expect this one for sure draft-ietf-dkim-mailinglists Maybe draft-crocker-dkim-doseta don't know enough about it but its Crocker :) draft-crocker-dkim-rfc4871bis-doseta don't know enough about it but its Crocker :) draft-fenton-dkim-reputation-hint don't know enough about it draft-hallambaker-dkim-extensions I don't recall this one draft-ietf-marf-dkim-reportingSeems like a moving target, so don't know. and these probably not. draft-hallambaker-dkimpolicy POLICY is a deadly sin for DKIM! draft-dkim-pkix don't know enough about it. All pure SWAGGING on my part. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
1) Did the IESG consider processing this as RFC 3933 process experiment? How on Earth could that possibly work? First, simply the fact of the experiment will almost certainly prompt people to participate, resulting in a number of specs upgrading from PS to IS during the experiment... regardless of whether that pattern would continue afterward. The experiment would be entirely tainted by its own existence, and would appear to succeed regardless of whether the change will actually help in the long term or not. Second, should we decide that the experiment failed and we do not want to continue the process change, what happens to all the IS documents that advanced from PS during the experiment? Are they rolled back to DS? Do they stay at IS? In the former case, what does that do to assumptions that had been made on the basis of IS status? If the latter case, how is that fair to documents that didn't have the opportunity to skip a rung in the ladder? And wouldn't that be even *more* of an incentive for people to push their docs from PS to IS during the experiment, exacerbating the first effect? Doing this as a process experiment doesn't make any sense; we either make this change, or we don't. And we need to stop wasting time arguing about it. Barry ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Mykyta: Taking into account the controversy we all are able to observe on the mailing list, I'd like to point out several points. 1) Did the IESG consider processing this as RFC 3933 process experiment? (I actually don't know whether such approach has already been proposed during the discussions, and whether there has been some outcome, so, if this has already been proposed, just re-consider.) I personally see no consensus or rough consensus for this document being approved as BCP, i. e., on the permanent basis, but as some people claimed that this might be useful, processing the document as Experimental process RFC will allow to make the final judgment based on the actual experience, not the assumptions. As soon as we find out that two-tier maturity levels system works, the BCP will be simple to be written and published; if we reach the agreement that this is a bad idea, then the proposed experimental change will be rescinded, and the maturity system will be returned to the 2026 model. I do not recall the whole IESG discussing this idea, but I did consider it. I did not consider it a good fit, and until now, no one else seemed to even consider it. 2) How do we make the consensus judgment (in this particular case)? As IETF is based on rough consensus model, rough consensus may only be claimed if the consensus-qualifier (a WG chair or Sponsoring AD, in case of Individual Submission) reaches the inner conviction that the idea proposed in the document satisfies the community, or at least its predominant part. Since IETF has no formal membership, the community size may not be determined precisely, and we take into account those folks who participate in the discussion (who, correspondingly, found themselves interested in the discussed topic and are assumed to be knowledgeable in it) in the WG, or elsewhere. So now, when many of the most experienced and most knowledgeable members of our community claim that the proposed change is not a good idea, or is a bad idea, or there is no actual problem, or there is a problem but its proposed solution isn't fine and has some omissions, or there are a number of other problems w hich are also to be fixed, or something else, I actually have no idea how the consensus-qualifier (in our case, Sponsoring AD - Jari Arkko) may claim consensus or rough consensus for, at least, adopting it as BCP. (I'm not following the thread closely, but this is what I see from those messages I eventually read.) I think that Jari explained his thought process, at least twice. 3) Do we actually need to make cosmetic changes to our process documentation? RFC 2026 was published in 1996, and precisely 15 years have passed. RFC 2026 is really morally obsolete, and, in presence of RFC 4844, that defines RFC submission streams, was to be revised closely after it was published. I see a number of drafts proposing revisions of RFC 2026 at https://datatracker.ietf.org/doc/search/?name=Internet+Standards+Processrfcs=onactiveDrafts=onoldDrafts=on, but none of them were processed. BTW, RFC 1310 was published in 1992, RFC 1602 - in 1994, and RFC 2026 - in 1996. I don't believe something happened in 1996 which made the procedures unnecessary to be aligned with the current practice. The only changes made were IPR documents, PS-DS reports reqs, and IESG procedures for review of Independent Submissions and IRTF stream submissions. More precisely, don't we need to revise RFC 2026 rather than make separate changes to it? Please take a look at the minutes of the PUFI BOF held at IETF 71. The IETF community seems to think that process changes fall into one of two piles. Either they are too insignificant to be worth the trouble or they are too big and onerous to consider. The experience with this draft does not disprove those results. That said, at the plenary at IETF 81, Harald suggested that we have waited so long that incremental improvements may not be the right approach, rather it was time for 2026bis. I agree that it is needed, but I am not sure the IETF community is willing to tackle a project of that size and complexity. Russ ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 10, 2011, at 10:44 AM, Russ Housley wrote: That said, at the plenary at IETF 81, Harald suggested that we have waited so long that incremental improvements may not be the right approach, rather it was time for 2026bis. I agree that it is needed, but I am not sure the IETF community is willing to tackle a project of that size and complexity. I suspect that there are two underlying problems, both significant. 1. Many people would see an attempt to change the process as a threat of one kind or another, feeling like the result of such change under current circumstances would likely be worse than the status quo. 2. There seems to be wide perceptual variation among IETF participants as to the organization's purpose and/or what's wrong with the current process. I think there's a need for consensus-building around these two topics (and perhaps others) before even considering whether and how to revise 2026. In the first case, we need to find out what people see as threats and use that to bound the charter of the rewrite effort so that people won't feel like they have to be in damage control mode. But the second topic is even more important. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
10.09.2011 16:56, Barry Leiba wrote: 1) Did the IESG consider processing this as RFC 3933 process experiment? How on Earth could that possibly work? Those who are interested in something working will certainly find the way to make it work. First, simply the fact of the experiment will almost certainly prompt people to participate, resulting in a number of specs upgrading from PS to IS during the experiment... regardless of whether that pattern would continue afterward. The experiment would be entirely tainted by its own existence, and would appear to succeed regardless of whether the change will actually help in the long term or not. Second, should we decide that the experiment failed and we do not want to continue the process change, what happens to all the IS documents that advanced from PS during the experiment? Are they rolled back to DS? Do they stay at IS? In the former case, what does that do to assumptions that had been made on the basis of IS status? If the latter case, how is that fair to documents that didn't have the opportunity to skip a rung in the ladder? And wouldn't that be even *more* of an incentive for people to push their docs from PS to IS during the experiment, exacerbating the first effect? So, for the purpose of the experiment, there will be a 2nd level, which is Draft Standard now and would be Internet Standard. People wanting to participate in the experiment will be warned that a 2nd level of Internet Standard might be of temporary nature and therefore, in the case of experiment failure, will be Draft Standard. Correspondingly, RFCs advanced to 2nd level of IS, in the case of permanent adaptation, will be Internet Standard, and in case of returning to 2026 model, will be Draft Standard. The term experiment implies that something will go in some way which is unusual; this is that unusual way. Anyway, do we need to care what maturity level has some spec reached? Eg., YAM has undertaken an effort to advance RFC 4409, mail submission protocol spec, to Full Standard (now 4409bis is approved as FS), but, I'm sure, this won't affect the many implementations which use and will continue to use mail submission protocol independent on its current maturity level. SMTP spec is now on Draft Standard, and I don't think it is now more widespread that when its spec was on Proposed Standard. Does that matter? Doing this as a process experiment doesn't make any sense; we either make this change, or we don't. And we need to stop wasting time arguing about it. We either make change, or we don't - unless there is a stable consensus on the change, the second variant is going to be the right one. BTW, stopping arguing is indeed a good idea; starting discussing is indeed a better idea (if we haven't yet). So what we're doing - discussing or arguing? Mykyta Yevstifeyev Barry ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
10.09.2011 17:44, Russ Housley wrote: Mykyta: Taking into account the controversy we all are able to observe on the mailing list, I'd like to point out several points. 1) Did the IESG consider processing this as RFC 3933 process experiment? (I actually don't know whether such approach has already been proposed during the discussions, and whether there has been some outcome, so, if this has already been proposed, just re-consider.) I personally see no consensus or rough consensus for this document being approved as BCP, i. e., on the permanent basis, but as some people claimed that this might be useful, processing the document as Experimental process RFC will allow to make the final judgment based on the actual experience, not the assumptions. As soon as we find out that two-tier maturity levels system works, the BCP will be simple to be written and published; if we reach the agreement that this is a bad idea, then the proposed experimental change will be rescinded, and the maturity system will be returned to the 2026 model. I do not recall the whole IESG discussing this idea, but I did consider it. I did not consider it a good fit, and until now, no one else seemed to even consider it. See my previous message. 2) How do we make the consensus judgment (in this particular case)? As IETF is based on rough consensus model, rough consensus may only be claimed if the consensus-qualifier (a WG chair or Sponsoring AD, in case of Individual Submission) reaches the inner conviction that the idea proposed in the document satisfies the community, or at least its predominant part. Since IETF has no formal membership, the community size may not be determined precisely, and we take into account those folks who participate in the discussion (who, correspondingly, found themselves interested in the discussed topic and are assumed to be knowledgeable in it) in the WG, or elsewhere. So now, when many of the most experienced and most knowledgeable members of our community claim that the proposed change is not a good idea, or is a bad idea, or there is no actual problem, or there is a problem but its proposed solution isn't fine and has some omissions, or there are a number of other problems which are also to be fixed, or something else, I actually have no idea how the consensus-qualifier (in our case, Sponsoring AD - Jari Arkko) may claim consensus or rough consensus for, at least, adopting it as BCP. (I'm not following the thread closely, but this is what I see from those messages I eventually read.) I think that Jari explained his thought process, at least twice. Well, what Jari explained is that (how I understand), there were thoughts like good minor change, and other thoughts that don't support the change but, due to lack of reasonable reasons for people opposed to the change to think so, such thoughts may be overlooked. Whereas there might have been such positions, I don't think such approach is good to claim community consensus. Status of This Memo boilerplate for BCPs include the following statement: It represents the consensus of the IETF community. Does that mean IETF community consensus? 3) Do we actually need to make cosmetic changes to our process documentation? RFC 2026 was published in 1996, and precisely 15 years have passed. RFC 2026 is really morally obsolete, and, in presence of RFC 4844, that defines RFC submission streams, was to be revised closely after it was published. I see a number of drafts proposing revisions of RFC 2026 athttps://datatracker.ietf.org/doc/search/?name=Internet+Standards+Processrfcs=onactiveDrafts=onoldDrafts=on, but none of them were processed. BTW, RFC 1310 was published in 1992, RFC 1602 - in 1994, and RFC 2026 - in 1996. I don't believe something happened in 1996 which made the procedures unnecessary to be aligned with the current practice. The only changes made were IPR documents, PS-DS reports reqs, and IESG procedures for review of Independent Submissions and IRTF stream submissions. More precisely, don't we need to revise RFC 2026 rather than make separate changes to it? Please take a look at the minutes of the PUFI BOF held at IETF 71. The IETF community seems to think that process changes fall into one of two piles. Either they are too insignificant to be worth the trouble or they are too big and onerous to consider. The experience with this draft does not disprove those results. That said, at the plenary at IETF 81, Harald suggested that we have waited so long that incremental improvements may not be the right approach, rather it was time for 2026bis. I agree that it is needed, but I am not sure the IETF community is willing to tackle a project of that size and complexity. I concur here entirely. (Were we revising process documentation on the regular basis, this wouldn't be a problem, though.) Mykyta Yevstifeyev Russ ___ Ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Hi. I feel it's reasonable for me to speak up since I have not done so in over a year on this document so my opinion probably has not been counted. 1) I support moving to a two level process. 2) I've generally supported versions of this document I have read. I have not read this version in detail. In regard to more global issues. I do not think the following types of comments should be considered as objections when judging this sort of consensus: 1) You are not solving the most important problem 2) This will not do any good Statements of those forms can be combined with objections. This will not do any good and might do harm so I don't support it, clearly is an objection. Objections can be as simple and non-specific as I don't like it, or more actionable. More thought out objections carry more weight in some senses. I certainly would hate to see us block on someone simply saying I don't like it. Either enough people say that that we fail to have rough consensus or not enough people say that and we move on. More detailed objections may be worth blocking on for a time to try and resolve; we seem to be past point of diminishing returns here. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
The question I have is who are the beneficiaries? My input as a implementator. For 30 or so years, I chose to followed the IETF output as an commercial implementator based on sound engineering trust and faith on follow peers, no reason to suspect or otherwise feel I need to appeal anything. For the most part, it was all good positive and successful adventure. No naiveness in separating what was for common good against those what were isolated vendor specific, proprietary in nature - yet, I do have the GM/WORLD conservative mindset - Whats good for IETF, is good for the World! I don't wish to buck the system. The wind has changed that for sure, and I wonder what is the end result. I do know one thing: I don't like the idea that IETF Appeal is something new (to me) I have to consider these days and when that seems like an insurmountable mountain to climb when it comes to even security conflicts and back down because of it, it becomes even more vexing to whats going on. I am not use to this and the older one gets, you seem to accept more even when you know (or feel) there are problems. Only the young have the energy to fight for a cause and even then less these days accepting things once considered problematic. Just I hope the IETF leaders here make the right decision. From an engineering standpoint, when you couple this two-step with the RFC2119bis efforts, it will most likely; a) make people think more about implementing what they had trouble with implementing in the first place, b) raise the barriers to adoption so you have less implementators. With my marketing hat on, rest assured this is leveraging material. -- HLS Sam Hartman wrote: Hi. I feel it's reasonable for me to speak up since I have not done so in over a year on this document so my opinion probably has not been counted. 1) I support moving to a two level process. 2) I've generally supported versions of this document I have read. I have not read this version in detail. In regard to more global issues. I do not think the following types of comments should be considered as objections when judging this sort of consensus: 1) You are not solving the most important problem 2) This will not do any good Statements of those forms can be combined with objections. This will not do any good and might do harm so I don't support it, clearly is an objection. Objections can be as simple and non-specific as I don't like it, or more actionable. More thought out objections carry more weight in some senses. I certainly would hate to see us block on someone simply saying I don't like it. Either enough people say that that we fail to have rough consensus or not enough people say that and we move on. More detailed objections may be worth blocking on for a time to try and resolve; we seem to be past point of diminishing returns here. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Whether they were planned as part of revamp plan or not, I don't see the two-step and the RFC2119bis efforts and recent debates here as a mere coincidence. Here is how Pareto Optimality design decisions are made such that no-one could be made better off without making someone else worse off. I only point this out that we sometimes we to accept change where one might not be better off. http://en.wikipedia.org/wiki/Pareto_efficiency#Pareto_frontier The Pareto frontier is particularly useful in engineering: by restricting attention to the set of choices that are Pareto-efficient, a designer can make tradeoffs within this set, rather than considering the full range of every parameter. One we view we are an optimal state right now where we don't wish to benefit one without making someone else worst and I see the two-step and RFC2119bis as attempts to making Pareto Frontier Efficient engineering decisions. The question is always how much change we want without ignoring those that can be made worst off. -- Keith Moore wrote: On Sep 10, 2011, at 10:44 AM, Russ Housley wrote: That said, at the plenary at IETF 81, Harald suggested that we have waited so long that incremental improvements may not be the right approach, rather it was time for 2026bis. I agree that it is needed, but I am not sure the IETF community is willing to tackle a project of that size and complexity. I suspect that there are two underlying problems, both significant. 1. Many people would see an attempt to change the process as a threat of one kind or another, feeling like the result of such change under current circumstances would likely be worse than the status quo. 2. There seems to be wide perceptual variation among IETF participants as to the organization's purpose and/or what's wrong with the current process. I think there's a need for consensus-building around these two topics (and perhaps others) before even considering whether and how to revise 2026. In the first case, we need to find out what people see as threats and use that to bound the charter of the rewrite effort so that people won't feel like they have to be in damage control mode. But the second topic is even more important. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On 2011-09-11 08:11, Sam Hartman wrote: snip I do not think the following types of comments should be considered as objections when judging this sort of consensus: 1) You are not solving the most important problem 2) This will not do any good Exactly. A very large part of the discussion has not been relevant to the Last Call of this particular version of this particular draft. In deciding the question, the IESG only needs to look at comments that are relevant. There are many other issues in 2026 that probably need attention, but as we've learnt over the last N years, trying to tackle them simultaneously is manifestly impossible. Brian ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On 2011-09-11 08:11, Sam Hartman wrote: snip I do not think the following types of comments should be considered as objections when judging this sort of consensus: 2) This will not do any good now lets see, this argument seems to be that the fact that a particular process change is useless should not stop the IETF from adopting the change this argument would be nonsense if applied to a proposal for a technical standard - i.e. the IETF should adopt a technical standard that is known to be useless -- it is no less nonsense when applied in this case - changes for the sake of publishing new bits should not be what the IETF spends its time on Scott ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 10, 2011, at 4:11 PM, Sam Hartman wrote: I do not think the following types of comments should be considered as objections when judging this sort of consensus: 1) You are not solving the most important problem I don't think that was anybody's objection. Rather, the objection were against change for the sake of change without any convincing argument that the change would improve things; and/or that the changes would actually make it more difficult to improve things. 2) This will not do any good IMO, that is a valid objection. Stability in our process is desirable; therefore change merely for the sake of change is undesirable. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Brian E Carpenter wrote: On 2011-09-11 08:11, Sam Hartman wrote: snip I do not think the following types of comments should be considered as objections when judging this sort of consensus: 1) You are not solving the most important problem 2) This will not do any good Exactly. A very large part of the discussion has not been relevant to the Last Call of this particular version of this particular draft. In deciding the question, the IESG only needs to look at comments that are relevant. There are many other issues in 2026 that probably need attention, but as we've learnt over the last N years, trying to tackle them simultaneously is manifestly impossible. Please trust there is no disrespect in stating this but this *sounds* like there is a realization that the reasons for problematic stagnant proposals are now going to be view as non-valid reasons. I don't have a particular concern if that is used for progressing and labeling proposals with a new status, but I am weary if that idea is used to circumvent the initial concerns. -- HLS ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
--On Saturday, September 10, 2011 16:11 -0400 Sam Hartman hartmans-i...@mit.edu wrote: I do not think the following types of comments should be considered as objections when judging this sort of consensus: 1) You are not solving the most important problem Sure. As long as we differentiate between than and this will not solve the problem it claims to solve, the problem this addresses is not worth solving, and this may make things worse. While I agree with many of the meta-comments that have been made about this discussion, I think the some of the supporters of the proposal have tended to lump comments along the lines of those three together with the one you cite and its close relatives, you should solve some other problem instead and you aren't solving all of the problem. They are different, at least IMO. 2) This will not do any good Again, I agree, but I suggest that is different from the argument Dave Crocker makes frequently (and better than I can) which, if I may paraphrase, is that any process change has significant coats and some risks and therefore it should be a requirement that anyone advocating such a change show a compelling need for it. I can accept this will not do any good as a category of position that should not be considered an objection iff no compelling requirement and/or advantage for this has been demonstrated still counts as legitimate objection. If that were not the case, I can think of any number of protocol specifications that could be decorated with a sufficient number of optional-and-useless features to make the most option- and profile-happy of SDOs dance with joy. Such decorations would be of particular interest in the security area where we have traditionally felt that options that are useless and likely to be badly-implemented and little-used often increase risk by providing unnecessary attack vectors. :-( john john ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
So should we move to a one-step process? On Sep 9, 2011, at 9:33 PM, Thomas Narten wrote: Advancing a spec is done for marketing, political, process and other reasons. E.g., to give a spec more legitimacy. Or to more clear replace an older one. Nothing wrong with that. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Eric, Thomas may well have a different answer but, speaking personally, if we have a choice between a nominal three-step process that is actually one-step with a few exceptions and a nominal two-step process that is actually one-step with a few exceptions, I think we would be much better off with a one-step process. Ideally, we should be able to annotate that one-step process with how mature we think a spec is, but the facing reality situation is that, unless we can change how we and the marketplace do things, we have a one-step process today and trying to cut things from unused-three to unused-two accomplishes nothing other than giving us an extra opportunity to confirm our failure to be able to use a multi-step process. I'd find a change to one-step a lot easier to support than a change to two-step, if only because moving to one-step is not only closer to present reality but also would give us a starting part for new work to express maturity (if we still care). Two-step neither gets us to present reality nor gets us away from the idea that the multistep model actually expresses maturity and other useful information. john --On Saturday, September 10, 2011 21:26 -0400 Eric Burger eburge...@standardstrack.com wrote: So should we move to a one-step process? On Sep 9, 2011, at 9:33 PM, Thomas Narten wrote: Advancing a spec is done for marketing, political, process and other reasons. E.g., to give a spec more legitimacy. Or to more clear replace an older one. Nothing wrong with that. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On 2011-09-11 13:26, Eric Burger wrote: So should we move to a one-step process? There is a detailed proposal for that at http://tools.ietf.org/id/draft-loughney-newtrk-one-size-fits-all-01.txt I don't think the arguments have changed much since 2006. Brian On Sep 9, 2011, at 9:33 PM, Thomas Narten wrote: Advancing a spec is done for marketing, political, process and other reasons. E.g., to give a spec more legitimacy. Or to more clear replace an older one. Nothing wrong with that. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
--On Sunday, September 11, 2011 14:49 +1200 Brian E Carpenter brian.e.carpen...@gmail.com wrote: On 2011-09-11 13:26, Eric Burger wrote: So should we move to a one-step process? There is a detailed proposal for that at http://tools.ietf.org/id/draft-loughney-newtrk-one-size-fits-a ll-01.txt I don't think the arguments have changed much since 2006. Well, yes and no. To some extent, that proposal evolved into draft-ietf-newtrk-repurposing-isd which, without John Loughney's direct involvement, evolved into draft-klensin-isdbis. Both of those can be construed as one level and explanation proposals, rather than multiple levels and trying to make the situation fit the categories ones. john ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Amazing, this all seems to be a rehash of the RFC3844 WG (2002 - 2004) debates and discussions: http://www.alvestrand.no/pipermail/problem-statement The June 2003 quarter seems to be an interesting period of discussions touching based with many of the central issues. -- John C Klensin wrote: Eric, Thomas may well have a different answer but, speaking personally, if we have a choice between a nominal three-step process that is actually one-step with a few exceptions and a nominal two-step process that is actually one-step with a few exceptions, I think we would be much better off with a one-step process. Ideally, we should be able to annotate that one-step process with how mature we think a spec is, but the facing reality situation is that, unless we can change how we and the marketplace do things, we have a one-step process today and trying to cut things from unused-three to unused-two accomplishes nothing other than giving us an extra opportunity to confirm our failure to be able to use a multi-step process. I'd find a change to one-step a lot easier to support than a change to two-step, if only because moving to one-step is not only closer to present reality but also would give us a starting part for new work to express maturity (if we still care). Two-step neither gets us to present reality nor gets us away from the idea that the multistep model actually expresses maturity and other useful information. john --On Saturday, September 10, 2011 21:26 -0400 Eric Burger eburge...@standardstrack.com wrote: So should we move to a one-step process? On Sep 9, 2011, at 9:33 PM, Thomas Narten wrote: Advancing a spec is done for marketing, political, process and other reasons. E.g., to give a spec more legitimacy. Or to more clear replace an older one. Nothing wrong with that. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
11.09.2011 5:49, Brian E Carpenter wrote: On 2011-09-11 13:26, Eric Burger wrote: So should we move to a one-step process? There is a detailed proposal for that at http://tools.ietf.org/id/draft-loughney-newtrk-one-size-fits-all-01.txt This draft is what actually matches reality. I'd better consider publishing it rather than draft-housley-two-maturity-levels, unless the latter or other similar proposal can demonstrate that it will find the way to affect the understanding of people whom RFCs are aimed to (which is a Standards Track RFC is a Standard; maturity level doesn't matter), predominantly implementors, and the real-life reality (which is one-step process) effectively (if the don't want to fit themselves to this understanding and this reality). Mykyta Yevstifeyev I don't think the arguments have changed much since 2006. Brian On Sep 9, 2011, at 9:33 PM, Thomas Narten wrote: Advancing a spec is done for marketing, political, process and other reasons. E.g., to give a spec more legitimacy. Or to more clear replace an older one. Nothing wrong with that. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
I am surely going to regret posting, because I have largely tuned out of this discussion due to the endless repetition, etc. I am not supportive of the current document, because I don't think it solves anything. To me, it smack a bit of change for changes sake. One of the key problems that isn't being addressed is that mixing advancement of a spec with revising a spec are fundmentally at odds with each other. Advancing a spec is done for marketing, political, process and other reasons. E.g., to give a spec more legitimacy. Or to more clear replace an older one. Nothing wrong with that. But the real reason that the IETF *should* be revising specs is to fix bugs and improve protocol quality. By definition, you cannot revise a spec (in a real, meaningful way) and advance at the same time. The spirit (if not letter) of advancement says you advance a spec, when there are implementations *based on the spec being advanced*. That means you can't revise a spec and at the same time have implementations derived from the revised spec. (You can have implementations based on mailing list discussions, but that is NOT the same thing.) The IETF is about making the Internet work better. That means revising specs (from a technical perpective) when they need to be revised. If we want to fix what's broken, we should focus on getting documents revised (without simultaneously advancing them). But once you do that, one quickly finds out that there are real and sometimes complicated reasons why revising documents is hard. In many cases, widely deployed protocols really need to have a revised spec developed (and the authors will readily admit that). But that just doesn't happen, not because of process, but because of other much more fundamental problems. E.g., Not enough energy from the relevant experts. key people who know a spec have moved on to other newer technologies or other higher priority things. Fixing specs can also be painful because some vendors won't or can't change their deployed implementations, so don't really want an updated spec that invalidates their implementation. etc., etc. It can be very hard for a WG to walk the line between we need to fix this and can we tweak the spec without invalidating various deployed implementations. IMO, these sorts of issues are the real reasons documents don't advance more. It's not just about process. Thomas ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Taking into account the controversy we all are able to observe on the mailing list, I'd like to point out several points. 1) Did the IESG consider processing this as RFC 3933 process experiment? (I actually don't know whether such approach has already been proposed during the discussions, and whether there has been some outcome, so, if this has already been proposed, just re-consider.) I personally see no consensus or rough consensus for this document being approved as BCP, i. e., on the permanent basis, but as some people claimed that this might be useful, processing the document as Experimental process RFC will allow to make the final judgment based on the actual experience, not the assumptions. As soon as we find out that two-tier maturity levels system works, the BCP will be simple to be written and published; if we reach the agreement that this is a bad idea, then the proposed experimental change will be rescinded, and the maturity system will be returned to the 2026 model. 2) How do we make the consensus judgment (in this particular case)? As IETF is based on rough consensus model, rough consensus may only be claimed if the consensus-qualifier (a WG chair or Sponsoring AD, in case of Individual Submission) reaches the inner conviction that the idea proposed in the document satisfies the community, or at least its predominant part. Since IETF has no formal membership, the community size may not be determined precisely, and we take into account those folks who participate in the discussion (who, correspondingly, found themselves interested in the discussed topic and are assumed to be knowledgeable in it) in the WG, or elsewhere. So now, when many of the most experienced and most knowledgeable members of our community claim that the proposed change is not a good idea, or is a bad idea, or there is no actual problem, or there is a problem but its proposed solution isn't fine and has some omissions, or there are a number of other problems which are also to be fixed, or something else, I actually have no idea how the consensus-qualifier (in our case, Sponsoring AD - Jari Arkko) may claim consensus or rough consensus for, at least, adopting it as BCP. (I'm not following the thread closely, but this is what I see from those messages I eventually read.) 3) Do we actually need to make cosmetic changes to our process documentation? RFC 2026 was published in 1996, and precisely 15 years have passed. RFC 2026 is really morally obsolete, and, in presence of RFC 4844, that defines RFC submission streams, was to be revised closely after it was published. I see a number of drafts proposing revisions of RFC 2026 at https://datatracker.ietf.org/doc/search/?name=Internet+Standards+Processrfcs=onactiveDrafts=onoldDrafts=on, but none of them were processed. BTW, RFC 1310 was published in 1992, RFC 1602 - in 1994, and RFC 2026 - in 1996. I don't believe something happened in 1996 which made the procedures unnecessary to be aligned with the current practice. The only changes made were IPR documents, PS-DS reports reqs, and IESG procedures for review of Independent Submissions and IRTF stream submissions. More precisely, don't we need to revise RFC 2026 rather than make separate changes to it? Mykyta Yevstifeyev 30.08.2011 23:17, Jari Arkko wrote: I have reviewed the discussion from the last call on this document. My conclusion as the sponsoring AD is that we have consensus to move forward. There was clearly a constituency who believed this is a good (albeit small) step forward. A number of other people did not care so much; did not believe there was either harm or benefit. I also saw a couple of opposing opinions, though some of them were more about a desire to do something else than specific objections about this proposal. I will be recommending that the IESG approve the draft. There were a number of smaller details raised in the discussion. But I did not see an overwhelming consensus on any specific issue to make changes. But I will ask Russ to take a look at the issue raised by Scott, whether he wants to add an informative reference to RFC 5657. Another issue that I wanted to highlight is the question of what kinds of discusses are desirable/acceptable for documents that move from PS to IS. It is outside the scope of Russ' document, but generated nevertheless some interest. The IESG has discussed this matter and drafted some suggested guidelines. Look for a different thread on this list, under Discuss criteria for documents that advance on the standards track. Feedback on the guidelines would be appreciated. Jari ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
- Original Message - From: Keith Moore mo...@network-heretics.com To: Ted Hardie ted.i...@gmail.com Cc: ned+i...@mauve.mrochek.com; ietf@ietf.org Sent: Wednesday, September 07, 2011 1:55 AM On Sep 6, 2011, at 7:33 PM, Ted Hardie wrote: but it means we are changing out a standard that doesn't accurately reflect what we do now for one that doesn't accurately reflect what we will do. Yup. And IMO we're better off with the old incorrect statement than a new one. At least with the current situation a great many people know how things work. tp Yes, even. The purchasers, RFPers and like that I have worked with just about understand the difference between an I-D and an RFC, but have no concept of the classifications used inside the IETF (PS, DS, IS) so I cannot see any comprehension of this new scheme making it into the world at large for a long time. We may not be following RFC2026 but at least there is some understanding of what we are doing. Tom Petch /tp Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Face it, we've effectively had a one-step process pretty much ever since 2026 was approved. For the most part, the documents that have advanced have been those that were buggy enough to need to be fixed, but not so buggy that they had to recycle at Proposed. Just one small problem here - every document advancement exercise I've seen in the past two decades - and I've seen a bunch - directly contradicts your assertions here. In essentially every case advancement occurred when some individual or some subgroup believed doing so was important and pushed the issue. The most common reason for believing that is probably that the document in question replaces some other document that's already at draft or full, and IETF rules require advancement before the original document can be marked historic. The SNMP and MSGFMT/SMTP specifications are both good examples of this. In other cases advancement was the result of a strong sense that it was needed to legitimize the specification. This is why MIME was advanced so quickly. And please note that in the case of MIME we advanced it with minimal changes first (RFCs 1521/1522 aren't that far from 1341/1342), and only then revised it extensively (RFCs 2045-2049) without further advancement. So much for the notion of using advancment as an excuse to fix stuff. Another common reason to advance a specification has been the simple belief that following out processes is the right thing to do. I assure you that was my only real motivation for working to advance the SMTP extensions framework, the SMTP size extension, and the PIPELINING extension. And finally, there are cases where the motivations for pushing for advancment were frankly mysterious to me. The FAX WG experience comes to mind here. To this day I have no idea why advancment of those specifications was so important, especially when many similar groups clearly didn't give a crap about following the process. Now, I have no doubt that if you looked hard enough you can find a case or two where the primary motivation was to pry something open and fix it. However, the opposite sentiment is far more common: Opening that up would be a real can of worms so please don't try and advance it. Nevertheless, the assertion that this is the only reason for advancment is demonstrably false. We've been using advancement as a proxy for maintenance for about as long as I've been in IETF. Wow, you really think that? I'm frankly amazed at the degree of disconnect here. (Which is why what I think we need is to restructure our processes so that they actually are designed to _maintain_ our specifications rather than pretending that there's ever a situation when those specifications are mature in this constantly changing world.) Well, now you're shifting to talking about a fundamental change of philosophies. Tell you what - let's see if even a small change like this one is possible first, because if it isn't a shift like this isn't even worth wasting the electrons to discuss. Will the imposition of a two step process change this? It certainly won't do so immediately, so the likely outcome is that yes indeed, the bar will continue to go up, at least initially, irrespective of whether or not this document is approved. But if more documents start advancing - and yes, that's an if - that will lessen the pressure on the initial step, and perhaps break the cycle we're currently stuck in. You might turn out be right, but I don't see things happening that way. The reason is that I don't think that either implementors or the consumers of hardware and software that implement these protocols care about whether we label something as a Proposed Standard or an Internet Standard. Proposed Standards are still going to get implemented and widely deployed. And when they break, it's still going to be a big mess. IESG is still going to feel a responsibility to try to do something about it. As they should. There are things we have control over and things we don't. We have no control over this. The best we can do is to make our labels meaningful - and they aren't currently. So perhaps we should fix that, you know? And please don't try trotting out a bunch of additional what ifs about how if this proposal fails we can then get past this distraction (or however you would characterize it) and address whatever it is you think the actual problems are. The actual problem is that people think that deploying products based on Proposed Standards is a good idea, and our process doesn't consistently produce documents of sufficient quality to warrant that.There are two ways to fix that problem. One is to stop labeling our initially published specifications (intended for prototyping and testing) as either Proposed Standards or RFCs. The other is to impose more engineering rigor on the process that leads to the creation of Proposed Standards. That presuppoes we have the ability to
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 7, 2011, at 10:17 AM, Ned Freed wrote: Face it, we've effectively had a one-step process pretty much ever since 2026 was approved. For the most part, the documents that have advanced have been those that were buggy enough to need to be fixed, but not so buggy that they had to recycle at Proposed. Just one small problem here - every document advancement exercise I've seen in the past two decades - and I've seen a bunch - directly contradicts your assertions here. In essentially every case advancement occurred when some individual or some subgroup believed doing so was important and pushed the issue. The most common reason for believing that is probably that the document in question replaces some other document that's already at draft or full, and IETF rules require advancement before the original document can be marked historic. The SNMP and MSGFMT/SMTP specifications are both good examples of this. I am aware of those, but I consider those the rare exceptions to the general rule. I also think that the email community has a couple of influential individuals who really believe in following the process all the way through, but that belief is not typical of IETF as a whole. We've been using advancement as a proxy for maintenance for about as long as I've been in IETF. Wow, you really think that? I'm frankly amazed at the degree of disconnect here. Yes, I really think that holds as a general rule. Again, to me the email advancement efforts look like the exceptional cases. (I applaud those individuals for their diligence!) (Which is why what I think we need is to restructure our processes so that they actually are designed to _maintain_ our specifications rather than pretending that there's ever a situation when those specifications are mature in this constantly changing world.) Well, now you're shifting to talking about a fundamental change of philosophies. Tell you what - let's see if even a small change like this one is possible first, because if it isn't a shift like this isn't even worth wasting the electrons to discuss. If it were generally clear that this change was a Good Idea, I might buy that argument. But what you seem to be saying is that if people don't back this change even though it appears to many people to be useless at best (and harmful to some), they'll never back any change to our process even if it appears to be more useful. You might turn out be right, but I don't see things happening that way. The reason is that I don't think that either implementors or the consumers of hardware and software that implement these protocols care about whether we label something as a Proposed Standard or an Internet Standard. Proposed Standards are still going to get implemented and widely deployed. And when they break, it's still going to be a big mess. IESG is still going to feel a responsibility to try to do something about it. As they should. There are things we have control over and things we don't. We have no control over this. The best we can do is to make our labels meaningful - and they aren't currently. So perhaps we should fix that, you know? FIne. Let's rename Proposed Standard to something that doesn't contain the word standard (call them frobs for the sake of argument). And let's not publish them as RFCs, but instead leave them as Internet-Drafts, and amend the rules for Internet-Drafts to allow frobs (once approved by IESG) to expire in two years rather than six months. The actual problem is that people think that deploying products based on Proposed Standards is a good idea, and our process doesn't consistently produce documents of sufficient quality to warrant that.There are two ways to fix that problem. One is to stop labeling our initially published specifications (intended for prototyping and testing) as either Proposed Standards or RFCs. The other is to impose more engineering rigor on the process that leads to the creation of Proposed Standards. That presuppoes we have the ability to actually perform such analysis without actually trying things at some sort of scale. I'm sorry, but I've seen no evidence that the necessary skills for this actually exists. I agree at least somewhat with that. I do think that our processes in general need more use of engineering discipline, but I also think that there will always be things that slip through the cracks. Which is why we need a process that recognizes the need for specifications to be maintained in light of experience. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
I can't help but see the conflicts of three/four QA ideas in action: Who moved my Cheese? Getting it right... the first time! Customers are always right! including the follow up Customers are always right sometimes. Improvement come by taking risk and don't settle with just old way of doing things. We try to do work with maximum quality, minimum cost and strive to avoid future problems or revisiting old one so that all parties are winners. Customers are satisfied, yet, there is a point where you can't satisfy all. Its a balance. IMV, as long as the the new two maturity level process does not change the IETF QA process negatively, I don't see a problem with it but it does sound it will necessitate a higher, more rigorous document reviews. SM wrote: At 13:17 30-08-2011, Jari Arkko wrote: There were a number of smaller details raised in the discussion. But I did not see an overwhelming consensus on any specific issue to make changes. But I will ask Russ to take a look at the issue raised by Scott, whether he wants to add an informative reference to RFC 5657. I read draft-housley-two-maturity-levels-09. I read the messages which might be interpreted as statements of support. Mr Burger offered that we are moving a baby step forward. Mr Resnick asked A baby step toward what exactly to which Mr Saint-Andre pointed out that we are more closely aligning our documentation with our organizational running code. The organizational running code actually sets a higher bar than what is documented in RFC 2026 for the publication of a Proposed Standard. The draft does not even discuss about that. Mr Carpenter believes that the present situation is confusing both to IETF newcomers (who may falsely believe that the IETF actually follows the 3 stage process) and, worse, confusing to users of IETF standards (who may falsely believe that a document isn't useful until it's advanced). We, and those users, gain by reducing the confusion. In terms of document clarity, RFC 2026 taken together with draft-housley-two-maturity-levels-09 only reinforces the confusion for anyone who takes the time to read BCP 9. Mr Atkinson pointed out that a change in perception alone is sufficient to increase [his] own motivation. Mr Burger confirmed that the intent of the proposal is to change the perception. Mr Halpern mentioned that the draft tries to align what we document with what we do. In a response, Mr Klensin mentioned that the draft addresses one provision of our processes in which documentation and practice don't align, a provision about which there is no subtlety or confusion within the community at all (even though new participants may be confused). Mr Housley in response to one of my comments mentioned that the argument I raised was for the status quo and added that We have decades of experience with that not working. That is essentially an argument for a single maturity level; that is how the process is really working today. As a note to the reader, I may have quoted Mr Housley out of context. I presume that members of the IESG have followed the discussions surrounding this draft to understand the context. The Sponsoring Area Director mentioned that the opposing opinions were more about a desire to do something else than specific objections about this proposal. An Area Director generally sponsors documents that he or she believes in. I would like to point out that even if a desire to do something else was tabled as a proposal, my perception is that it would be difficult to have such a proposal sponsored by the relevant Area Director. Mr Crocker and Mr Housley are listed as authors of STD 71 and STD 70 respectively. It would be informative if they could comment on the impediments they came across in advancing their documents to Full Standard. Mr Gellens and Mr Klensin might also be able to comment on the impediments given that they are listed as the authors of a soon to be published STD. Regards, -sm ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf -- Sincerely Hector Santos http://www.santronics.com ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
RE: Conclusion of the last call on draft-housley-two-maturity-levels
Its a balance. IMV, as long as the the new two maturity level process does not change the IETF QA process negatively, I don't see a problem with it but it does sound it will necessitate a higher, more rigorous document reviews. I haven't heard anyone currently on the IESG say that the two step process would require higher more rigorous document reviews. Ross ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Tue, Sep 06, 2011 at 11:38:14AM -0400, Ross Callon wrote: I haven't heard anyone currently on the IESG say that the two step process would require higher more rigorous document reviews. That particular refusal to recognize part of reality is the thing that annoys me about this draft and about the discussions leading to its modification of the official process. The causal claim asserted early in the I-D's life was that, since many RFCs effectively live forever today at step 1 of the standards track, IESG members feel a responsibility to make sure that an I-D is right before publication as PS even though that requirement is much higher than the RFC 2026 process requires. As a result, proponents argued, the process would be made less onerous by moving to a two-step process in which initial publication at step 1 is the same as RFC 2026's step 1, except that it is even easier to go from that and get the honorific Internet Standard than it is today. I find it impossible to believe that this will not result in even more hard-line positions on the part of some IESG members when something with which they disagree is a candidate for PS. I see no way in which the draft solves this problem, which remains one of its implicit goals. I said before, I don't care if it is published, because I think it will have little effect. But I think we'd better be prepared for some IESG members to insist on the same high bar for PS that we have under RFC 2026, regardless of what the RFC says. A -- Andrew Sullivan a...@anvilwalrusden.com ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 6, 2011, at 12:11 PM, Andrew Sullivan wrote: On Tue, Sep 06, 2011 at 11:38:14AM -0400, Ross Callon wrote: I haven't heard anyone currently on the IESG say that the two step process would require higher more rigorous document reviews. That particular refusal to recognize part of reality is the thing that annoys me about this draft and about the discussions leading to its modification of the official process. The causal claim asserted early in the I-D's life was that, since many RFCs effectively live forever today at step 1 of the standards track, IESG members feel a responsibility to make sure that an I-D is right before publication as PS even though that requirement is much higher than the RFC 2026 process requires. As a result, proponents argued, the process would be made less onerous by moving to a two-step process in which initial publication at step 1 is the same as RFC 2026's step 1, except that it is even easier to go from that and get the honorific Internet Standard than it is today. I find it impossible to believe that this will not result in even more hard-line positions on the part of some IESG members when something with which they disagree is a candidate for PS. I see no way in which the draft solves this problem, which remains one of its implicit goals. I said before, I don't care if it is published, because I think it will have little effect. But I think we'd better be prepared for some IESG members to insist on the same high bar for PS that we have under RFC 2026, regardless of what the RFC says. +1 Best statement of the problem with this document that I've seen so far. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
I find it impossible to believe that this will not result in even more hard-line positions on the part of some IESG members when something with which they disagree is a candidate for PS. I see no way in which the draft solves this problem, which remains one of its implicit goals. I said before, I don't care if it is published, because I think it will have little effect. But I think we'd better be prepared for some IESG members to insist on the same high bar for PS that we have under RFC 2026, regardless of what the RFC says. +1 Best statement of the problem with this document that I've seen so far. Except for one small problem: It's nonsensical. Why is it nonsensical? Because you're comparing the status quo with a possible future course of action. The one thing that's we can be certain of is things won't remain the same. They never do. So in order to make a reasonable comparison you have to project what's likely to happen if this document isn't approved, then compare that with what might happen if it is. And the future where this isn't approved is fairly easy to predict: As more and more documents become proposed standards and then fail to progress along the standards track - and the trend lines for this could not be clearer - we effectively move more and more to a one-step process. The IESG has only one rational response to that: Continue to raise the bar on the initial step to proposed. Will the imposition of a two step process change this? It certainly won't do so immediately, so the likely outcome is that yes indeed, the bar will continue to go up, at least initially, irrespective of whether or not this document is approved. But if more documents start advancing - and yes, that's an if - that will lessen the pressure on the initial step, and perhaps break the cycle we're currently stuck in. And please don't try trotting out a bunch of additional what ifs about how if this proposal fails we can then get past this distraction (or however you would characterize it) and address whatever it is you think the actual problems are. Given the time that has gone into trying to make this one simple and fairly obvious change, if it fails, the odds of someone attempting even more fundamental changes are pretty darned low. Ned ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
I actually have a lot of sympathy with Andrew's formulation, largely because the document wants you to infer something rather than making it explicit. Take this text: 2.1. The First Maturity Level: Proposed Standard The stated requirements for Proposed Standard are not changed; they remain exactly as specified in RFC 2026 [1]. No new requirements are introduced; no existing published requirements are relaxed. The document doesn't actually say out loud there that the requirements for Proposed Standard have been considerably increased by IESG practice over the years, nor does it charge subsequent IESGs to return to a faithful reading of the actual text. You can infer it from the refrain (stated requirements and published requirements), but I don't think you could fairly call it explicit. You can certainly get that from Russ in bar or on a mailing list, but we normally try to write our documents such that you don't have to have shared a bar with the author to get their real intent. If the IESG does not choose to follow-up that inference with action, we have effectively moved from a one step standards process pretending to be a three-step standards process to a one step standards process pretending to have two. That's hardly worth the electrons which have been spent in this argument. My personal opinion for some time has been that we ought to recognize that the previous PS moved into WG draft years ago and that anything named an RFC should be recognized as something that market will consider a standard. (My own somewhat-tongue-in-cheek proposal was to make advancement completely automatic, barring a stated objection, but that went nowhere.) If we are going to try to move the goalposts for PS back and retain more than one maturity level with the label RFC, I think we'd have a better chance of success if we fess up that they were moved *and put into the document that the community wants the IESG to use those and only those for PS*. Without that, the real requirements for PS remain up to the IESG of the moment--a matter of lore rather than documented practice. To add one final comment, I think what little consensus we have on this is was well characterized as by exhaustion in John's earlier note. It may be no less real for that, but let's not pretend to ourselves that this is anything but a much-needed removal of the requirement for annual review, along with a fond hope that the future may be different from the past. regards, Ted ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
While I disagree with the WG Draft part, partially because I think we do still derive significant value from cross-area review, I agree with the rest of this. FWIW, I am actually quite sympathetic to Ned's argument that the presence of two levels past proposed and the amount of nonsense that seems necessary with each is demoralizing and acts as an obstruction for some people to get to the second level. It hasn't persuaded me that this proposal is helpful only because, if the experience with Proposed is extended from a three levels to two, we will see the nonsense level at the second level increase on the basis of last shot, so we better get it right reasoning (which I believe is one of the things that has driven the threshold for Proposed to its current level). If we could somehow take the combination of a quest for perfection and the notion that, if the IESG has to review something, ADs have to find something to complain about or some way to impose their stamp on the document out of the system, then I'd completely believe Ned's argument... except that, with the nonsense removed from the 2nd and 3rd levels, that incentive to remove the third might be much reduced. As I indicated in my note last week, the way out of these problems seems to me to be is much less symbolic tuning of the standards process and much more having the community express clear support for the IESG changing the way things are done --within the existing rules or even closer to their intent-- and having the IESG do it. best, john --On Tuesday, September 06, 2011 14:35 -0700 Ted Hardie ted.i...@gmail.com wrote: I actually have a lot of sympathy with Andrew's formulation, largely because the document wants you to infer something rather than making it explicit. Take this text: 2.1. The First Maturity Level: Proposed Standard The stated requirements for Proposed Standard are not changed; they remain exactly as specified in RFC 2026 [1]. No new requirements are introduced; no existing published requirements are relaxed. The document doesn't actually say out loud there that the requirements for Proposed Standard have been considerably increased by IESG practice over the years, nor does it charge subsequent IESGs to return to a faithful reading of the actual text. You can infer it from the refrain (stated requirements and published requirements), but I don't think you could fairly call it explicit. You can certainly get that from Russ in bar or on a mailing list, but we normally try to write our documents such that you don't have to have shared a bar with the author to get their real intent. If the IESG does not choose to follow-up that inference with action, we have effectively moved from a one step standards process pretending to be a three-step standards process to a one step standards process pretending to have two. That's hardly worth the electrons which have been spent in this argument. ... ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 6, 2011, at 2:26 PM, Ned Freed wrote: I find it impossible to believe that this will not result in even more hard-line positions on the part of some IESG members when something with which they disagree is a candidate for PS. I see no way in which the draft solves this problem, which remains one of its implicit goals. I said before, I don't care if it is published, because I think it will have little effect. But I think we'd better be prepared for some IESG members to insist on the same high bar for PS that we have under RFC 2026, regardless of what the RFC says. +1 Best statement of the problem with this document that I've seen so far. Except for one small problem: It's nonsensical. Why is it nonsensical? Because you're comparing the status quo with a possible future course of action. The one thing that's we can be certain of is things won't remain the same. They never do. So in order to make a reasonable comparison you have to project what's likely to happen if this document isn't approved, then compare that with what might happen if it is. And the future where this isn't approved is fairly easy to predict: As more and more documents become proposed standards and then fail to progress along the standards track - and the trend lines for this could not be clearer - we effectively move more and more to a one-step process. Face it, we've effectively had a one-step process pretty much ever since 2026 was approved. For the most part, the documents that have advanced have been those that were buggy enough to need to be fixed, but not so buggy that they had to recycle at Proposed. We've been using advancement as a proxy for maintenance for about as long as I've been in IETF.(Which is why what I think we need is to restructure our processes so that they actually are designed to _maintain_ our specifications rather than pretending that there's ever a situation when those specifications are mature in this constantly changing world.) The IESG has only one rational response to that: Continue to raise the bar on the initial step to proposed. And that was indeed a perfectly rational, and reasonable, response. Nor is there anything that changing our process can really do about that, so long as our first publications intended for public consumption are labeled __ Standard and as RFCs (as people think the latter means standard no matter how much we protest to the contrary). Changing what happens _after_ Proposed Standard and/or RFC publication will never address this problem. Will the imposition of a two step process change this? It certainly won't do so immediately, so the likely outcome is that yes indeed, the bar will continue to go up, at least initially, irrespective of whether or not this document is approved. But if more documents start advancing - and yes, that's an if - that will lessen the pressure on the initial step, and perhaps break the cycle we're currently stuck in. You might turn out be right, but I don't see things happening that way. The reason is that I don't think that either implementors or the consumers of hardware and software that implement these protocols care about whether we label something as a Proposed Standard or an Internet Standard. Proposed Standards are still going to get implemented and widely deployed. And when they break, it's still going to be a big mess. IESG is still going to feel a responsibility to try to do something about it. As they should. And please don't try trotting out a bunch of additional what ifs about how if this proposal fails we can then get past this distraction (or however you would characterize it) and address whatever it is you think the actual problems are. The actual problem is that people think that deploying products based on Proposed Standards is a good idea, and our process doesn't consistently produce documents of sufficient quality to warrant that.There are two ways to fix that problem. One is to stop labeling our initially published specifications (intended for prototyping and testing) as either Proposed Standards or RFCs. The other is to impose more engineering rigor on the process that leads to the creation of Proposed Standards. Given the time that has gone into trying to make this one simple and fairly obvious change, if it fails, the odds of someone attempting even more fundamental changes are pretty darned low. Except that, for some reason, the obvious change is obviously wrong to many of us. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 6, 2011, at 5:35 PM, Ted Hardie wrote: The document doesn't actually say out loud there that the requirements for Proposed Standard have been considerably increased by IESG practice over the years, nor does it charge subsequent IESGs to return to a faithful reading of the actual text. Is IESG really misreading no known technical omissions with respect to the requirements placed on it? If the bar has been raised since the publication of 2026, might this actually be reasonable given that the Internet is much larger, more diverse, and more hostile than it used to be? Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Tue, Sep 6, 2011 at 3:13 PM, Keith Moore mo...@network-heretics.comwrote: On Sep 6, 2011, at 5:35 PM, Ted Hardie wrote: The document doesn't actually say out loud there that the requirements for Proposed Standard have been considerably increased by IESG practice over the years, nor does it charge subsequent IESGs to return to a faithful reading of the actual text. Is IESG really misreading no known technical omissions with respect to the requirements placed on it? Read further to: However, since the content of Proposed Standards may be changed if problems are found or better solutions are identified, deploying implementations of such standards into a disruption-sensitive environment is not recommended. The IESG has been working to the assumption that Proposed Standards will be widely deployed into all environments for a long time. That may well be an appropriate response to the deployment practice (heck, if the internet runs on internet drafts we're lucky that we don't have an IESG review step before i-d publication). But if the result of this exercise is that the bar for PS stays as-is and the bar for the second stage merges, we will retain what is a functionally a one-stage standards process. We can certainly live with that (we live with it now), but it means we are changing out a standard that doesn't accurately reflect what we do now for one that doesn't accurately reflect what we will do. regards, Ted If the bar has been raised since the publication of 2026, might this actually be reasonable given that the Internet is much larger, more diverse, and more hostile than it used to be? Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 6, 2011, at 4:33 PM, Ted Hardie wrote: The IESG has been working to the assumption that Proposed Standards will be widely deployed into all environments for a long time. That may well be an appropriate response to the deployment practice (heck, if the internet runs on internet drafts we're lucky that we don't have an IESG review step before i-d publication). But if the result of this exercise is that the bar for PS stays as-is and the bar for the second stage merges, we will retain what is a functionally a one-stage standards process. We can certainly live with that (we live with it now), but it means we are changing out a standard that doesn't accurately reflect what we do now for one that doesn't accurately reflect what we will do. Agreed. That has been my primary puzzlement with this entire discussion. The problem that cycling at Proposed solves is what if I need to change the technology in some way. Foo updates bar does the same. Generally speaking, we don't get a rewrite until A has been updated by B1, B2, and B3, B2 has been obsoleted by C, and C has been updated by D. What Draft Standard was supposed to fix was a raft of testing coupled with that rewrite of the specification that also removed cruft that wasn't used. Frankly, the only thing I ever figured out that Full Standard was useful for was obsolete (he ducks). I wonder if we would be better off discarding the concept of layers of standards, call PS Standards track, and instead specify a way to report interoperability tests. If we have a document A that has been updated by B and someone has tested several implementations of A+B, could they say I tested A+B in this configuration, which used features A.1, A.2, A.3, A.5, and B.1, with these results, without forcing the rewrite or the major conniption fits that DS involves. Folks in fact do interoperability tests with some regularity. They do them for equipment they want to buy, they do bake-offs, and they do other things. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 6, 2011, at 7:33 PM, Ted Hardie wrote: but it means we are changing out a standard that doesn't accurately reflect what we do now for one that doesn't accurately reflect what we will do. Yup. And IMO we're better off with the old incorrect statement than a new one. At least with the current situation a great many people know how things work. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 6, 2011, at 7:48 PM, Fred Baker wrote: I wonder if we would be better off discarding the concept of layers of standards, call PS Standards track, and instead specify a way to report interoperability tests. +1. Perhaps along with periodically updated applicability statements. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Keith Moore mo...@network-heretics.com wrote: On Sep 6, 2011, at 7:48 PM, Fred Baker wrote: I wonder if we would be better off discarding the concept of layers of standards, call PS Standards track, and instead specify a way to report interoperability tests. +1. Perhaps along with periodically updated applicability statements. This is a serious proposal... It has been a serious proposal ever since newtrk. Details differ, but really not significantly: some folks want to update 2026 to do this; others want to do it first and decide later whether to update 2026; others want to do it and let the IESG decide whether to use it as input to deciding about advancing levels. Many distinctions; no real difference... -- John Leslie j...@jlc.net ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Other proposals (Was: :Re: Conclusion of the last call on draft-housley-two-maturity-levels)
Hi Jari, At 15:05 02-09-2011, Jari Arkko wrote: But what I really wanted to say here was a response to your concern about those proposals to do something else. Let me just state this clearly. I know I would be VERY happy to sponsor many different kinds of improvement proposals. In sequence or in parallel with the current proposal. I'm already sponsoring another one (though it is destined to an IESG statement and somewhat related to this one, but it could have been something else too). I think I'm speaking also for the rest of the IESG in this. Please make the proposals. I sent you a message off-list. If I had to reach a conclusion, I'd say: it's one of this situations where you can be as fair as it gets but you cannot please everybody. You have put in more that enough effort into this. Regards, -sm ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Keith: The current IETF Standards Process has become essentially a one-step process. The goal, as I believe is stated in the document, is gather some benefit from implementation and deployment experience. We are not getting that today. When we do get it, the document recycles at the same maturity level due to problems that are not addressed in this proposal, must commonly dependencies on documents that are not ready to advance. So, the reader cannot tell when a document has the benefit of implementation and deployment experience. My reading of the comments on earlier proposals was to focus on one problem in the document, so the handling of the dependency problem and others was removed. I'm pleased to work on them in the future if this document gains rough consensus. Russ On Sep 2, 2011, at 4:37 PM, Keith Moore wrote: Honestly, the thing that is the most broken about this draft is the idea that there's something wrong with our process because few drafts make it to full standard, so the solution is to short-circuit the process. The transition from Draft to Full Standard is the least of the problems with our process. And the sooner we stop trying to fix irrelevancies, the better. Keith smime.p7s Description: S/MIME cryptographic signature ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
At 13:17 30-08-2011, Jari Arkko wrote: There were a number of smaller details raised in the discussion. But I did not see an overwhelming consensus on any specific issue to make changes. But I will ask Russ to take a look at the issue raised by Scott, whether he wants to add an informative reference to RFC 5657. I read draft-housley-two-maturity-levels-09. I read the messages which might be interpreted as statements of support. Mr Burger offered that we are moving a baby step forward. Mr Resnick asked A baby step toward what exactly to which Mr Saint-Andre pointed out that we are more closely aligning our documentation with our organizational running code. The organizational running code actually sets a higher bar than what is documented in RFC 2026 for the publication of a Proposed Standard. The draft does not even discuss about that. Mr Carpenter believes that the present situation is confusing both to IETF newcomers (who may falsely believe that the IETF actually follows the 3 stage process) and, worse, confusing to users of IETF standards (who may falsely believe that a document isn't useful until it's advanced). We, and those users, gain by reducing the confusion. In terms of document clarity, RFC 2026 taken together with draft-housley-two-maturity-levels-09 only reinforces the confusion for anyone who takes the time to read BCP 9. Mr Atkinson pointed out that a change in perception alone is sufficient to increase [his] own motivation. Mr Burger confirmed that the intent of the proposal is to change the perception. Mr Halpern mentioned that the draft tries to align what we document with what we do. In a response, Mr Klensin mentioned that the draft addresses one provision of our processes in which documentation and practice don't align, a provision about which there is no subtlety or confusion within the community at all (even though new participants may be confused). Mr Housley in response to one of my comments mentioned that the argument I raised was for the status quo and added that We have decades of experience with that not working. That is essentially an argument for a single maturity level; that is how the process is really working today. As a note to the reader, I may have quoted Mr Housley out of context. I presume that members of the IESG have followed the discussions surrounding this draft to understand the context. The Sponsoring Area Director mentioned that the opposing opinions were more about a desire to do something else than specific objections about this proposal. An Area Director generally sponsors documents that he or she believes in. I would like to point out that even if a desire to do something else was tabled as a proposal, my perception is that it would be difficult to have such a proposal sponsored by the relevant Area Director. Mr Crocker and Mr Housley are listed as authors of STD 71 and STD 70 respectively. It would be informative if they could comment on the impediments they came across in advancing their documents to Full Standard. Mr Gellens and Mr Klensin might also be able to comment on the impediments given that they are listed as the authors of a soon to be published STD. Regards, -sm ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Jari Arkko sed I also saw a couple of opposing opinions, though some of them were more about a desire to do something else than specific objections about this proposal. for the record in case Jari is confused - I have specific objection to this proposal imo - it fixes no known problem - it only adds additional fuzz to a surface understanding of what causes few RFCs to be advanced on the standards track - I see no reason to think that this proposal will have any significant effect on that problem (nor any insignificant effect) Scott ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
--On Tuesday, August 30, 2011 23:17 +0300 Jari Arkko jari.ar...@piuha.net wrote: I have reviewed the discussion from the last call on this document. My conclusion as the sponsoring AD is that we have consensus to move forward. There was clearly a constituency who believed this is a good (albeit small) step forward. A number of other people did not care so much; did not believe there was either harm or benefit. I also saw a couple of opposing opinions, though some of them were more about a desire to do something else than specific objections about this proposal. I will be recommending that the IESG approve the draft. Jari, Like Scott, I wonder if there is some misunderstanding here. Part of the problem is the way that this draft was developed and the discussion has been handled, despite your heroic efforts. For example: (1) If someone says we should be looking in this different direction instead, the response has been irrelevant to consideration to this proposal, so it should go forward. The irrelevancy is debatable, but that may be another issue. (2) If someone says the proposal claims to solve problem X, but there is no evidence for that, the assertion about what problems are being solved is removed, but there is no substantive change to the draft. (3) If someone says this solves no problem, the response has been that things have been broken for years and therefore this proposal should be approved. (The difficulty with that logic should be clear.) Sometimes that has been accompanied by a claim that it is the only proposal on the table and should therefore be adopted (even though that statement isn't true and, true or not, would never even be considered if we were considering a protocol specification). One of the co-authors has recently argued for a very high standard of compelling necessity to make changes to important processes or related documents, but that criterion obviously does not apply to this document. (4) There have been a few arguments made that making this sort of change --without compelling justification and without evidence that it would accomplish anything-- would actually be harmful. There has been no substantive response to those arguments. In normal Last Calls, such comments are known as unresolved issues and the sponsoring AD does not move the document forward until they are addressed (or even dismissed) in some substantive way. (5) This is probably off-topic unless someone decides to appeal, but, to a certain extent, the processing of this document points out a far more significant problem with the handling of process change suggestions in the IETF. The IESG holds a discussion. An IESG member prepares a draft. An IESG member (the same one or a different one - it makes a difference, but not much) decides what other process change proposals can be considered (either at the same time or otherwise). While the IESG would normally decide that anything that has produced this much controversy needs a Working Group to consider alternatives and get a real consensus determination, the IESG decides that no WG will be considered for this work (the claim that previous WGs addressed to process issues have been a problem --one with which I personally agree-- may not be relevant unless the IESG is ready to consider other review and mechanisms). The IESG gets to pick and choose which arguments for and against the proposal count -- normal, but see (4) above and the many solves no problem comments. And then the IESG decides to advance the document. (6) Unfortunately, although the document has improved significantly since -00 --by removing material for which there was little or no support and some question about relevance and by removing unsupportable claims-- the basic pattern outlined in (5) has been perceived as inevitable, i.e., that this AD-produced draft, produces in response to a discussion and conclusion already reached by the IESG, was going to go through because the IESG had prejudged the ultimate outcome before the draft was written. Whether that perception is correct or not, it leads all but the most persistent members of the community to tune out and stop making comments, either early on or after several rounds. I am not going to take a position about consensus among some silent majority because I simply don't know how those who are not speaking up feel, but I think the community should exercise caution about the possibility of consensus-by-exhaustion in any discussion that has dragged on as long as this document and its predecessors have been debated on the IETF list. regards, john ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
Honestly, the thing that is the most broken about this draft is the idea that there's something wrong with our process because few drafts make it to full standard, so the solution is to short-circuit the process. The transition from Draft to Full Standard is the least of the problems with our process. And the sooner we stop trying to fix irrelevancies, the better. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On 2 September 2011 22:20, John C Klensin wrote: I simply don't know how those who are not speaking up feel https://profiles.google.com/103449397114700758824/posts/1KALM7oLKzi ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
RE: Conclusion of the last call on draft-housley-two-maturity-levels
In looking through this discussion, I see: - People saying that moving from 3 steps to 2 steps is a small step in the right direction, lets do it. Many people who have said this (including I) have been silent for a while quite possibly because they have gotten frustrated with the endless discussion. - People saying that there are other more important problems that we should be focusing on. Therefore, rather than either making this simple change or discussing other possible improvements in the process, instead let's debate this simple step forever and never get anything done. - People saying that this step won't do anything. Two things that I don't seem to have picked up on: (i) Any consensus that a 3 step process is better than a 2 step process; (ii) Any hint of moving towards an agreement on other things that we might do to improve the process. I think that we should go to a two maturity level process, be done with this little step, and also enthusiastically encourage people to write drafts that propose *other* changes to the process. Then at least we can be debating something different 6 months from now than we were debating last year. Ross -Original Message- From: ietf-boun...@ietf.org [mailto:ietf-boun...@ietf.org] On Behalf Of John C Klensin Sent: Friday, September 02, 2011 4:20 PM To: Jari Arkko Cc: ietf@ietf.org Subject: Re: Conclusion of the last call on draft-housley-two-maturity-levels --On Tuesday, August 30, 2011 23:17 +0300 Jari Arkko jari.ar...@piuha.net wrote: I have reviewed the discussion from the last call on this document. My conclusion as the sponsoring AD is that we have consensus to move forward. There was clearly a constituency who believed this is a good (albeit small) step forward. A number of other people did not care so much; did not believe there was either harm or benefit. I also saw a couple of opposing opinions, though some of them were more about a desire to do something else than specific objections about this proposal. I will be recommending that the IESG approve the draft. Jari, Like Scott, I wonder if there is some misunderstanding here. Part of the problem is the way that this draft was developed and the discussion has been handled, despite your heroic efforts. For example: (1) If someone says we should be looking in this different direction instead, the response has been irrelevant to consideration to this proposal, so it should go forward. The irrelevancy is debatable, but that may be another issue. (2) If someone says the proposal claims to solve problem X, but there is no evidence for that, the assertion about what problems are being solved is removed, but there is no substantive change to the draft. (3) If someone says this solves no problem, the response has been that things have been broken for years and therefore this proposal should be approved. (The difficulty with that logic should be clear.) Sometimes that has been accompanied by a claim that it is the only proposal on the table and should therefore be adopted (even though that statement isn't true and, true or not, would never even be considered if we were considering a protocol specification). One of the co-authors has recently argued for a very high standard of compelling necessity to make changes to important processes or related documents, but that criterion obviously does not apply to this document. (4) There have been a few arguments made that making this sort of change --without compelling justification and without evidence that it would accomplish anything-- would actually be harmful. There has been no substantive response to those arguments. In normal Last Calls, such comments are known as unresolved issues and the sponsoring AD does not move the document forward until they are addressed (or even dismissed) in some substantive way. (5) This is probably off-topic unless someone decides to appeal, but, to a certain extent, the processing of this document points out a far more significant problem with the handling of process change suggestions in the IETF. The IESG holds a discussion. An IESG member prepares a draft. An IESG member (the same one or a different one - it makes a difference, but not much) decides what other process change proposals can be considered (either at the same time or otherwise). While the IESG would normally decide that anything that has produced this much controversy needs a Working Group to consider alternatives and get a real consensus determination, the IESG decides that no WG will be considered for this work (the claim that previous WGs addressed to process issues have been a problem --one with which I personally agree-- may not be relevant unless the IESG is ready to consider other review and mechanisms). The IESG gets to pick and choose which arguments for and against the proposal count -- normal, but see (4) above and the many solves no problem comments. And then the IESG decides to advance
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 2, 2011, at 5:29 PM, Ross Callon wrote: Two things that I don't seem to have picked up on: (i) Any consensus that a 3 step process is better than a 2 step process; (ii) Any hint of moving towards an agreement on other things that we might do to improve the process. (iii) Any consensus that a 2 step process is better than a 3 step process. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
RE: Conclusion of the last call on draft-housley-two-maturity-levels
In looking through this discussion, I see: - People saying that moving from 3 steps to 2 steps is a small step in the right direction, lets do it. Many people who have said this (including I) have been silent for a while quite possibly because they have gotten frustrated with the endless discussion. Ross, I'm right there with you. I fully support this document at worse a small incremental step that clears away the some brush (at best it may actually turn out to be quite valuable) and I'm completely frustrated that this discussion is continuing. This really needs to stop now. And yes, some people aren't happy with the outcome. Thems the breaks. - People saying that there are other more important problems that we should be focusing on. Therefore, rather than either making this simple change or discussing other possible improvements in the process, instead let's debate this simple step forever and never get anything done. At least part of the problem is lack of agreement on what the issues are. Even if this step is a waste of time - I think that's unlikely but let's suppose - at least it will make it clear where the problems *aren't*. - People saying that this step won't do anything. Two things that I don't seem to have picked up on: (i) Any consensus that a 3 step process is better than a 2 step process; (ii) Any hint of moving towards an agreement on other things that we might do to improve the process. Well, that's the real problem, isn't it? Even if you believe this is a distraction and even actively harmful, it's not like we've been able to move past it either. The running code result here seems pretty clear, and it does not argue in favor of another round of discussion. I think that we should go to a two maturity level process, be done with this little step, and also enthusiastically encourage people to write drafts that propose *other* changes to the process. Then at least we can be debating something different 6 months from now than we were debating last year. +100 Ned ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
for the record in case Jari is confused - I have specific objection to this proposal I might well be, it would not be a surprise :-) but let me just clarify that I said that there were objections, *some* of which were not specific. Not all. Jari ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
RE: Conclusion of the last call on draft-housley-two-maturity-levels
At 04:29 PM 9/2/2011, Ross Callon wrote: In looking through this discussion, I see: - People saying that moving from 3 steps to 2 steps is a small step in the right direction, lets do it. Many people who have said this (including I) have been silent for a while quite possibly because they have gotten frustrated with the endless discussion. I think there are many that have voiced their frustrations that this draft isn't addressing the more important issue (or issues) in their minds. I don't see a consensus on what that 1 issue is, but many (including I) have said it's the problem of such a high hurdle to get a draft to PS. Because this draft isn't addressing that problem, I'm frustrated with this draft - because - I don't know that if this draft were to RFC that the high hurdle for PSs is the next thing tackled. OTOH, if the high hurdle for PSs were what we worked on initially, and solve it, then I'd probably be much more comfortable with this draft progressing (then having started to appreciate what this means as a second step where getting a draft to PS is the first step). just my opinion James - People saying that there are other more important problems that we should be focusing on. Therefore, rather than either making this simple change or discussing other possible improvements in the process, instead let's debate this simple step forever and never get anything done. - People saying that this step won't do anything. Two things that I don't seem to have picked up on: (i) Any consensus that a 3 step process is better than a 2 step process; (ii) Any hint of moving towards an agreement on other things that we might do to improve the process. I think that we should go to a two maturity level process, be done with this little step, and also enthusiastically encourage people to write drafts that propose *other* changes to the process. Then at least we can be debating something different 6 months from now than we were debating last year. Ross -Original Message- From: ietf-boun...@ietf.org [mailto:ietf-boun...@ietf.org] On Behalf Of John C Klensin Sent: Friday, September 02, 2011 4:20 PM To: Jari Arkko Cc: ietf@ietf.org Subject: Re: Conclusion of the last call on draft-housley-two-maturity-levels --On Tuesday, August 30, 2011 23:17 +0300 Jari Arkko jari.ar...@piuha.net wrote: I have reviewed the discussion from the last call on this document. My conclusion as the sponsoring AD is that we have consensus to move forward. There was clearly a constituency who believed this is a good (albeit small) step forward. A number of other people did not care so much; did not believe there was either harm or benefit. I also saw a couple of opposing opinions, though some of them were more about a desire to do something else than specific objections about this proposal. I will be recommending that the IESG approve the draft. Jari, Like Scott, I wonder if there is some misunderstanding here. Part of the problem is the way that this draft was developed and the discussion has been handled, despite your heroic efforts. For example: (1) If someone says we should be looking in this different direction instead, the response has been irrelevant to consideration to this proposal, so it should go forward. The irrelevancy is debatable, but that may be another issue. (2) If someone says the proposal claims to solve problem X, but there is no evidence for that, the assertion about what problems are being solved is removed, but there is no substantive change to the draft. (3) If someone says this solves no problem, the response has been that things have been broken for years and therefore this proposal should be approved. (The difficulty with that logic should be clear.) Sometimes that has been accompanied by a claim that it is the only proposal on the table and should therefore be adopted (even though that statement isn't true and, true or not, would never even be considered if we were considering a protocol specification). One of the co-authors has recently argued for a very high standard of compelling necessity to make changes to important processes or related documents, but that criterion obviously does not apply to this document. (4) There have been a few arguments made that making this sort of change --without compelling justification and without evidence that it would accomplish anything-- would actually be harmful. There has been no substantive response to those arguments. In normal Last Calls, such comments are known as unresolved issues and the sponsoring AD does not move the document forward until they are addressed (or even dismissed) in some substantive way. (5) This is probably off-topic unless someone decides to appeal, but, to a certain extent, the processing of this document points out a far more significant problem with the handling of process change suggestions in the IETF. The IESG holds a discussion. An IESG member prepares a draft
Other proposals (Was: :Re: Conclusion of the last call on draft-housley-two-maturity-levels)
SM, The Sponsoring Area Director mentioned that the opposing opinions were more about a desire to do something else than specific objections about this proposal. An Area Director generally sponsors documents that he or she believes in. I would like to point out that even if a desire to do something else was tabled as a proposal, my perception is that it would be difficult to have such a proposal sponsored by the relevant Area Director. I already responded to Scott, clarifying what I had said about the objections. But what I really wanted to say here was a response to your concern about those proposals to do something else. Let me just state this clearly. I know I would be VERY happy to sponsor many different kinds of improvement proposals. In sequence or in parallel with the current proposal. I'm already sponsoring another one (though it is destined to an IESG statement and somewhat related to this one, but it could have been something else too). I think I'm speaking also for the rest of the IESG in this. Please make the proposals. Of course, this does not mean that I would sponsor anything -- I do have to believe in it, I do have to think it has a reasonable chance of success. And I'm not too eager to take draft-grand-unified-theory-of-ietf-process-00.txt while everything else has to sit still. But if you have been sending e-mail to the list that we should not waste our time with Russ' draft and that we should do something else instead -- here's an idea. Send that something to us, and we'll get it done! Simple, no? Jari ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 2, 2011, at 5:36 PM, ned+i...@mauve.mrochek.com wrote: In looking through this discussion, I see: - People saying that moving from 3 steps to 2 steps is a small step in the right direction, lets do it. Many people who have said this (including I) have been silent for a while quite possibly because they have gotten frustrated with the endless discussion. Ross, I'm right there with you. I fully support this document at worse a small incremental step that clears away the some brush (at best it may actually turn out to be quite valuable) and I'm completely frustrated that this discussion is continuing. This really needs to stop now. And yes, some people aren't happy with the outcome. Thems the breaks. As far as our process is concerned, the question is, do we have rough consensus to accept it? I think it's dubious that we have such consensus, and apparently so do others. Personally I think this proposal is Mostly Harmless, so I'm willing to hold my nose about it. But I'm very concerned about the argument that the default assumption should be that we change our process even in the absence of consensus to do so. Regarding the proposal, I get the impression that people are mostly in three camps: 1) Even if this is a baby step, it's a step in the right direction. Or even if it's not a step in the right direction, taking some step will at least make it possible to make some changes in our process. Maybe we'll not like the results of taking this step, but at least then we'll have learned something, and if the result is clearly worse we'll be motivated to change it. (I call this change for the sake of change) 2) Fixing the wrong problem doesn't do anything useful, and will/may serve as a distraction from doing anything useful. (I call this rearranging the deck chairs) 3) People should stop arguing about this and just hold their noses about it, because the arguing will make it harder to do anything else in this space. (I call this Oceana has always been at war with Eurasia. Ok, that's probably too harsh, but it's what immediately comes to mind.) All of these are defensible theories.As it happens, I don't believe #1 applies in this space, I do believe #2, and I have to admit that #3 does happen. The arguments that people are giving in favor of approving this bother me more than the proposal itself does. (I'm a firm believer that good decisions are highly unlikely to result from flawed assumptions, and flawed assumptions often affect many decisions. So challenging a widely-held flawed assumption is often more important than challenging any single decision.) The core problem, I suspect, is that we don't really have any consensus on what IETF's role is. Is it to help ensure that the Internet works well? Or is it to enable vendors to ship products that use new/updated protocols as quickly as possible? These two aren't diametrically opposed, but there's a fair amount of tension between them. Keith ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
First, I'm in full agreement with Ross. Second, for the record and as a response to Keith, my read of the discussion on the last call was the biggest group of responses said that we should move forward with the draft. There were two smaller groups, those with a clear objection and those with roughly a no-objection or it does not cause harm opinion (and a group who seemed to discuss orthogonal issues and not respond to the question). I could of course have made mistakes in this determination, but I thought it was rough (perhaps very rough) consensus. Of course, it gets more interesting if you start thinking about the reasons why people wanted to move forward. Keith's latest e-mail has interesting theories about those. I don't think anyone thinks this is the priority #1 process fix for the IETF. For me, cleaning cruft from the IETF process RFCs is a big reason for supporting this work. And I must admit that we seem to be in a place where its very, very hard to make _any_ process RFC changes. Getting one done, even if its a small change would by itself be useful, IMO. Finally, I think two levels are enough. Jari On 03.09.2011 00:34, Keith Moore wrote: (iii) Any consensus that a 2 step process is better than a 3 step process. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
My main conclusion for the moment is that Last Call comments should indicate first of all a definite Support or Oppose for the decision at hand if they are to be counted for or against consensus. I'm not going to state a position, which means that you should not count me as either for or against the proposal. Keith On Sep 2, 2011, at 6:31 PM, Jari Arkko wrote: First, I'm in full agreement with Ross. Second, for the record and as a response to Keith, my read of the discussion on the last call was the biggest group of responses said that we should move forward with the draft. There were two smaller groups, those with a clear objection and those with roughly a no-objection or it does not cause harm opinion (and a group who seemed to discuss orthogonal issues and not respond to the question). I could of course have made mistakes in this determination, but I thought it was rough (perhaps very rough) consensus. Of course, it gets more interesting if you start thinking about the reasons why people wanted to move forward. Keith's latest e-mail has interesting theories about those. I don't think anyone thinks this is the priority #1 process fix for the IETF. For me, cleaning cruft from the IETF process RFCs is a big reason for supporting this work. And I must admit that we seem to be in a place where its very, very hard to make _any_ process RFC changes. Getting one done, even if its a small change would by itself be useful, IMO. Finally, I think two levels are enough. Jari On 03.09.2011 00:34, Keith Moore wrote: (iii) Any consensus that a 2 step process is better than a 3 step process. ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
I'm in the group that likes this document, thinks it will help us move forward, and thinks we should stop babbling and just do it. That said, I have a issue with what Ross says: Two things that I don't seem to have picked up on: (i) Any consensus that a 3 step process is better than a 2 step process; (ii) Any hint of moving towards an agreement on other things that we might do to improve the process. The issue is that we aren't looking for consensus that what we have is better than what's being proposed. What we're looking for is consensus that what's being proposed is better than what we have. That is, a proposal for change needs to establish consensus FOR THE CHANGE, not the other way around. It's up to Jari and the rest of the IESG to determine whether we have that rough consensus. I'm not sure. In numbers, we seem to -- it's mostly a vocal few who are urging us not to do it. The thing is that, as we know, rough consensus isn't just a numbers game, and one or two reasonable minority arguments can derail things. Barry ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
RE: Conclusion of the last call on draft-housley-two-maturity-levels
--On Friday, September 02, 2011 14:36 -0700 Ned Freed ned.fr...@mrochek.com wrote: ... Well, that's the real problem, isn't it? Even if you believe this is a distraction and even actively harmful, it's not like we've been able to move past it either. The running code result here seems pretty clear, and it does not argue in favor of another round of discussion. To make one thing clear that I may not have been clear enough about: I felt a need to respond to Jari's summary. I did not want to set off another round of discussion. I do not think another round of discussion would be in the least useful: as far as I can tell, no one who is expressing opinions about whether or not we should adopt this change now has changed opinions on that subject (sometimes the reasons stated have changed). That doesn't suggest that more discussion would change any minds. From that point of view, I believe the _only_ question at this point, with this process, is whether there is sufficient consensus that this proposal meets the community's norms for making process changes. I've heard all of the following argued to be those norms for this type of proposal: (1) No worse than what we have (i.e., no proof that three levels are better than two) (2) Likely useless, but the burden is on those who don't like it to prove that rather than on those who advocate it to demonstrate its utility. (3) Possibly useful and harmless at worst. (4) Possibly useful but possibly harmful. (5) Definitely useful and a compelling argument established for this change. For the first, I think that consensus probably exists, but I'm not on the IESG and may not have a correct perspective. For (2) - (4), I really don't have a good estimate of how the community feels: figuring that out is why we pay the IESG the big bucks. And I don't think anyone has really made a compelling case for the 5th. YMMV. And, fwiw, I think it would be a pity to prolong this discussion by debating those categories, either as to whether they are the correct categories or as to which one we should be using. To paraphrase a recent note from Russ, if someone feels a need to do that, at least start another thread. john ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 2, 2011, at 5:36 PM, ned+i...@mauve.mrochek.com wrote: In looking through this discussion, I see: - People saying that moving from 3 steps to 2 steps is a small step in the right direction, lets do it. Many people who have said this (including I) have been silent for a while quite possibly because they have gotten frustrated with the endless discussion. Ross, I'm right there with you. I fully support this document at worse a small incremental step that clears away the some brush (at best it may actually turn out to be quite valuable) and I'm completely frustrated that this discussion is continuing. This really needs to stop now. And yes, some people aren't happy with the outcome. Thems the breaks. As far as our process is concerned, the question is, do we have rough consensus to accept it? I think it's dubious that we have such consensus, and apparently so do others. Simply put, I've watched the responses to this fairly closely, and I completely disagree with your assessment. Personally I think this proposal is Mostly Harmless, so I'm willing to hold my nose about it. But I'm very concerned about the argument that the default assumption should be that we change our process even in the absence of consensus to do so. Regarding the proposal, I get the impression that people are mostly in three camps: Well, none of these describe my own position, which is that eliminating the three step process will at a minimum act as an incentive to move more documents along. (You, and most others engaging in this debate, routinely neglect the psychological factors involved.) I can easily name a dozen RFCs, all currently at proposed, that I for one will be strongly incented to work to advance if this step is taken. And there isn't a chance in hell that I'll bother with any of them if this step doesn't happen, especially after the recent debacle surrounding the attempt to move 4409bis to full standard, and more generally given how the entire YAM experiment played out. I'm sorry, but passing down the advancement gauntlet is plenty hard enough to do once. Requiring it be done twice? Been there, done that, not remotely interested in doing it again. Additionally, by simplifying the process, we will gain essential insight into where other problems lie. Without such simplification I see no chance at all at making progress on any of these issues. 1) Even if this is a baby step, it's a step in the right direction. Or even if it's not a step in the right direction, taking some step will at least make it possible to make some changes in our process. Maybe we'll not like the results of taking this step, but at least then we'll have learned something, and if the result is clearly worse we'll be motivated to change it. (I call this change for the sake of change) That last substantially and obviously mischaracterizes this position. In fact I strongly recommend that you stop trying to summarize complex position with cute - and utterly wrong - phrases like this. This is annoying and quite unhelpful. 2) Fixing the wrong problem doesn't do anything useful, and will/may serve as a distraction from doing anything useful. (I call this rearranging the deck chairs) 3) People should stop arguing about this and just hold their noses about it, because the arguing will make it harder to do anything else in this space. (I call this Oceana has always been at war with Eurasia. Ok, that's probably too harsh, but it's what immediately comes to mind.) Actually, I think there are a substantial numer of people who believe exactly the opposite of this. All of these are defensible theories.As it happens, I don't believe #1 applies in this space, I do believe #2, and I have to admit that #3 does happen. The arguments that people are giving in favor of approving this bother me more than the proposal itself does. (I'm a firm believer that good decisions are highly unlikely to result from flawed assumptions, and flawed assumptions often affect many decisions. So challenging a widely-held flawed assumption is often more important than challenging any single decision.) Well, the main argument I'm giving is based on my own perception of the effect this will have on myself and similarly minded people as a contributor. If you think that assessment is incorrect, then I'm sorry, but I think you're being extraordinarily foolish. ned ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
First, I'm in full agreement with Ross. Second, for the record and as a response to Keith, my read of the discussion on the last call was the biggest group of responses said that we should move forward with the draft. There were two smaller groups, those with a clear objection and those with roughly a no-objection or it does not cause harm opinion (and a group who seemed to discuss orthogonal issues and not respond to the question). I could of course have made mistakes in this determination, but I thought it was rough (perhaps very rough) consensus. FWIW, this matches my own assessment almost exactly. Of course, it gets more interesting if you start thinking about the reasons why people wanted to move forward. Keith's latest e-mail has interesting theories about those. I don't think anyone thinks this is the priority #1 process fix for the IETF. Agreed. For me, cleaning cruft from the IETF process RFCs is a big reason for supporting this work. And I must admit that we seem to be in a place where its very, very hard to make _any_ process RFC changes. Getting one done, even if its a small change would by itself be useful, IMO. Finally, I think two levels are enough. Cruft elimination is also a Good Thing. Ned ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On 2011-09-03 09:29, Ross Callon wrote: ... I think that we should go to a two maturity level process, be done with this little step, and also enthusiastically encourage people to write drafts that propose *other* changes to the process. Then at least we can be debating something different 6 months from now than we were debating last year. +1. Let's please not hear all over again the arguments that were made during the *concluded* last call. The consensus is rough; time to move on. Brian ___ Ietf mailing list Ietf@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 2, 2011, at 7:07 PM, Ned Freed wrote: As far as our process is concerned, the question is, do we have rough consensus to accept it? I think it's dubious that we have such consensus, and apparently so do others. Simply put, I've watched the responses to this fairly closely, and I completely disagree with your assessment. ok. Personally I think this proposal is Mostly Harmless, so I'm willing to hold my nose about it. But I'm very concerned about the argument that the default assumption should be that we change our process even in the absence of consensus to do so. Regarding the proposal, I get the impression that people are mostly in three camps: Well, none of these describe my own position, which is that eliminating the three step process will at a minimum act as an incentive to move more documents along. (You, and most others engaging in this debate, routinely neglect the psychological factors involved.) I can easily name a dozen RFCs, all currently at proposed, that I for one will be strongly incented to work to advance if this step is taken. At the risk of playing devil's advocate, how will that help? Will the specifications significantly improve in quality and interoperability improve as a result? Will the blessing of these documents as Internet Standards result in wider implementation and thus greater benefit to users? (not knowing which RFCs you're talking about, I can't even guess) From my perspective there's little problem with implementing and deploying at Proposed and having documents stay at Proposed indefinitely, provided we can ensure that the specifications are of high quality by the time they get to Proposed. And given that people do tend to implement and deploy at Proposed, there's only marginal benefit to promoting them to anything else - except on those occasions where this serves as a carrot to fix bugs in the original spec that people might otherwise live with. And it's not clear to me that the proposed change increases the incentive to do either. Additionally, by simplifying the process, we will gain essential insight into where other problems lie. Without such simplification I see no chance at all at making progress on any of these issues. Okay, I can see that as a possibility. Sometimes when undertaking a great task, it doesn't matter what subtask you pick to do next, as long as you do something. Momentum is often more important than doing things in order of importance. My question is then, how many people think that we need to undertake a great task where our process is concerned, and how many of those think that given current political conditions, if we undertake such a task, we're likely to end up with something substantially better than we have now? (I'm open to the idea but skeptical) 1) Even if this is a baby step, it's a step in the right direction. Or even if it's not a step in the right direction, taking some step will at least make it possible to make some changes in our process. Maybe we'll not like the results of taking this step, but at least then we'll have learned something, and if the result is clearly worse we'll be motivated to change it. (I call this change for the sake of change) That last substantially and obviously mischaracterizes this position. In fact I strongly recommend that you stop trying to summarize complex position with cute - and utterly wrong - phrases like this. This is annoying and quite unhelpful. There are definitely cases where a journey of a thousand miles begins with a single step, I'm just skeptical that that argument applies in this specific case. 2) Fixing the wrong problem doesn't do anything useful, and will/may serve as a distraction from doing anything useful. (I call this rearranging the deck chairs) 3) People should stop arguing about this and just hold their noses about it, because the arguing will make it harder to do anything else in this space. (I call this Oceana has always been at war with Eurasia. Ok, that's probably too harsh, but it's what immediately comes to mind.) Actually, I think there are a substantial numer of people who believe exactly the opposite of this. I'm not sure I understand what you mean here. Are you saying that there are a substantial number of people who wish to make it harder to do anything at all in this space, so they keep arguing about it? Or something else? The arguments that people are giving in favor of approving this bother me more than the proposal itself does. (I'm a firm believer that good decisions are highly unlikely to result from flawed assumptions, and flawed assumptions often affect many decisions. So challenging a widely-held flawed assumption is often more important than challenging any single decision.) Well, the main argument I'm giving is based on my own perception of the effect this will have on myself and similarly minded people as a
Re: Conclusion of the last call on draft-housley-two-maturity-levels
On Sep 2, 2011, at 7:07 PM, Ned Freed wrote: As far as our process is concerned, the question is, do we have rough consensus to accept it? I think it's dubious that we have such consensus, and apparently so do others. Simply put, I've watched the responses to this fairly closely, and I completely disagree with your assessment. ok. Personally I think this proposal is Mostly Harmless, so I'm willing to hold my nose about it. But I'm very concerned about the argument that the default assumption should be that we change our process even in the absence of consensus to do so. Regarding the proposal, I get the impression that people are mostly in three camps: Well, none of these describe my own position, which is that eliminating the three step process will at a minimum act as an incentive to move more documents along. (You, and most others engaging in this debate, routinely neglect the psychological factors involved.) I can easily name a dozen RFCs, all currently at proposed, that I for one will be strongly incented to work to advance if this step is taken. At the risk of playing devil's advocate, how will that help? Will the specifications significantly improve in quality and interoperability improve as a result? First of all, since when is improving specifications the sole purpose of the IETF standards advancement process? As I understand it, the goal behind having different document statuses is for the labels to tell implementors how well things are likely to actually work. In other words, I categorically reject your implied assumption here that the IETF advancement process only makes sense when specifications are improved along the way. Plenty of specifications currently at proposed are of sufficient quality to be full standards right now. Our problem is right now the process is so onerous few people bother, so the presence of the higher label is useful but the absence of it is not. But even if I were to accept your tacit assumptions about the purpose of the process - which I most assuredly do not - the answer is yes, some and perhaps all of these specifications will be improved along the way, and at least a couple are likely to be significantly improved. Features that in hindsight turned out to be problematic will be removed, prose will be checked and tightend, etc. etc. If you think specification advancement doesn't receive this level of scrutiny and resulting improvement currently, you haven't been paying attention. Will the blessing of these documents as Internet Standards result in wider implementation and thus greater benefit to users? (not knowing which RFCs you're talking about, I can't even guess) Keith, you know as well as I do that there can be no guarantess that the Internet will care about anything we do, let alone see value in such advancements. But given past experience it seems very likely that at least some benefits along these lines will be accrued. From my perspective there's little problem with implementing and deploying at Proposed and having documents stay at Proposed indefinitely, provided we can ensure that the specifications are of high quality by the time they get to Proposed. And given that people do tend to implement and deploy at Proposed, there's only marginal benefit to promoting them to anything else - except on those occasions where this serves as a carrot to fix bugs in the original spec that people might otherwise live with. And it's not clear to me that the proposed change increases the incentive to do either. And what happens in this world of only proposed standards when, as is often the case despite all the absurd added scrutiny we now apply to proposed standards, something problematic slips past us all? The simple fact that we almost always make changes, and often significant ones, to documents that advance, shows we don't always get it right. And when this happens what's the alternative? Once something is approved it can be very difficult to get consensus to move it historic. You appear to be assuming that if we stare at stuff long enough and review it hard enough we'll find all the issues in one step. Sorry, it never worked this way in the past, and as the complexity of specifications has increased, it works even less well now than ever before. Specifications have to be tried, and tried at scale, to tell if they are any good. Indeed, past experience indicates that the sorts of additional review you have repeatedly advocoted for in these discussions usually backfires: It has an uncanny way of turning the process into a beauty contest were important work gets stifled just because it ran contrary to some higher-up's beliefs about something. Once again you're failing to take psychological factors into account, and they *matter*. Additionally, by simplifying the process, we will gain essential insight into where other problems lie. Without such simplification I see no
