Aw: What do we mean when we standardize something?
Hi John, this is a great summary. Regarding the question about the type of standardization we want I would argue for amixture of both since inpractice there are, of course,a lot of grey areas. I suspect that setting the expectations right at the beginning of starting the work (in a group or on a specific document) are important. I guess it is just the surprises that folks get concerned about when they suddenly realize that the last call period isnt actually something were they can change a lot (even if they dislike something) because the technology is widely deployed already and any change in the spec is likely going to cause a disconnect with the deployment. Ciao Hannes Gesendet:Mittwoch, 29. Mai 2013 um 19:23 Uhr Von:John C Klensin john-i...@jck.com An:ietf@ietf.org Betreff:What do we mean when we standardize something? Hi. A number of recent discussions, specifically including the Last Calls on DKIM and standardizing RRTYPEs and even to some extent the meeting location ones, have started me thinking that perhaps we need to review what business the IETF is actually in and to be sure that we actually still agree about it. The note that follows is an attempt to take the isolated parts (and symptoms) of that discussion up a level and discuss it as a general issue. The key issues go to the very nature of standardization. While there are many variations on each, there are two fundamentally different models for what standards are about. Each is legitimate in its own way and each has its advocates, but they are different. One is a focus on quality: an engineering consensus on the best technical solution given physics or consensus goals and boundary conditions. The other is sometimes called consensus of industry practice in which a standard reflects what is already deployed, possibly picking among different implemented options in a few cases but mostly requiring that a common model appears before standards are issued. Two variations on the second theme were extremely common in the earlier days of computer and telecommunciations standardization but are less popular today (at least in theory) due to IPR and antitrust concerns. Both start with a single vendor (or closed consortium) implementation. That implementation might then be offered for standardization (sometimes with a no major changes restriction) and adopted more generally or endorsed by traditional SDOs. Or that single implementation might be reverse-engineered by others and then the result treated as common industry practice. Despite the occasional claim that a strange (to me) idea called anticipatory standardization is a variation on the second model, that combination makes standards in the absence of, or prior to, implementation and deployment an impossible contradiction. It is basically incompatible with innovation of any type. The IETF has traditionally chosen the first model and a great deal of our thinking and public rhetoric are based on it. Even when we have adopted proposals, sometimes implemented ones, from elsewhere, we have insisted on change control and usually taken that seriously. One could claim that an adaptation of the second model would make the Internet more uniform, but a consensus of existing practice cannot aspire to working better except in the narrow sense of substitutability of conforming products. It is interesting that the multi-stage model of RFC 2026 (particularly the original Proposed Standard definition), the IEEEs Trial-Use standards (the current Section 5.7 of their Standards Board Operations Manual, http://standards.ieee.org/develop/policies/opman/sect5.html#5.7, makes instructive reading), and ISO Technical Specification all try to combine the two models by allowing a preliminary specification --one that is intended for trial implementation but not deployment-- to be designed on an engineering basis and then standardized only when implementations exist and can be compared (our original Draft Standard). In theory, only when both an industry common practice and consensus about value emerges does true standardization (including our original definition for Internet Standard) move forward. We know how well that has worked for us in practice. While there have been exceptions, I dont believe it has worked much better for other SDOs. If we are not going to move significantly in the direction of consensus of industry practice, then it seems to me that we need to be careful about the arguments that are made (or at least those that are considered legitimate) in support of advancement into or on the standards track. For example, if we agree on a relaxed registration procedure for some protocol parameters, it is not reasonable to later turn around and ask that that those parameters be standardized, unchanged, because they are already registered (and maybe deployed). For standardization, the IETF generally needs not only change control but an opportunity to comment on and affect the actual design and
Re: What do we mean when we standardize something?
Hi John, At 10:23 29-05-2013, John C Klensin wrote: The IETF has traditionally chosen the first model and a great deal of our thinking and public rhetoric are based on it. Even when we have adopted proposals, sometimes implemented ones, from elsewhere, we have insisted on change control and usually taken that seriously. One could claim that an adaptation of the second model would make the Internet more uniform, but a consensus of existing practice cannot aspire to working better except in the narrow sense of substitutability of conforming products. The note is thoughtful. It's the first time I see a message where the working better angle is discussed. If we are not going to move significantly in the direction of consensus of industry practice, then it seems to me that we need to be careful about the arguments that are made (or at least those that are considered legitimate) in support of advancement into or on the standards track. If the IETF were to move in the direction of consensus of industry practice it would be moving towards consensus of industry practice based on the opinions of people in the industry who are active in the IETF. Michael Richardson asked the following question: Given that the [Country X] mandated them, why wasn't the IETF involved earlier? The regulator really should have reached out to the IETF here. If people from Country X do not come to the IETF with the problem the IETF won't know about the problem. If people from Country X do not provide input about their industry practice the IETF will know about it. For example, if we agree on a relaxed registration procedure for some protocol parameters, it is not reasonable to later turn around and ask that that those parameters be standardized, unchanged, because they are already registered (and maybe deployed). For standardization, the IETF generally needs not only change control but an opportunity to comment on and affect the actual design and specification of whatever is being standardized. Country X would also have to allow the IETF to have change control on the specification or else it is like asking the IETF to rubber-stamp the specification. If I am not mistaken the relaxed registration procedure is so that there can be a registry of what is being used on the Internet. The difference here is that the IETF does not review the design, or it may set some minimal criteria for the review before allocating the code point. Similarly, we sometimes hear it argued that we should accept a specification for Proposed Standard unchanged because it has been extensively developed and reviewed elsewhere. That may be reasonable in some cases although I'd hope we wouldn't make it a common practice. But, if a specification adopted for Proposed Standard on that basis is then proposed for advancement to Internet Standard, I think the review should be comprehensive --perhaps even more comprehensive than the usual such review-- because the Internet Standard is unambiguously an IETF product and recommendation not that of the other body. If we allow a specification to advance to Proposed Standard without an open and critical review because it was developed and reviewed comprehensively by outside expert organizations and then allow the Last Call review for Internet Standard to be constrained by existing deployment, we would have gone into the rubber stamping business that we regularly say is incompatible with our standardization model. That doesn't mean, of course, that changes during review for Internet Standard are likely to be a good idea. Those who propose changes to the spec, and especially to the protocol, should have to convince the community that those changes are important enough to overcome the costs, including those of incompatibility with deployed versions. But I don't think we can reasonably exclude them from raising the issues and making those arguments. If an intended Proposed Standard has actually been developed and reviewed elsewhere the Last Call and IESG Evaluation is not a problem, i.e. any problem in the specification has already been identified and addressed. One of the issues here is the rhetoric (the undue use of exaggeration, or from an IETF perspective, making unpleasant comments to discourage people from identifying problems). It is difficult to have an open and critical review of an Internet Standard. There is a significant cost for any change made at that level as it may cause interoperability issues. What we mean when we standardize something is that we will think carefully before making a change as we would like the specification to be stable even if that means not fixing some minor errors. Internet Standard could have been used to say we are not going to tinker with this specification and it is your best bet if you want to deploy the code and forget about it for a few years. As an unrelated comment, what could we mean? It could mean: 1. doing what is
Re: Fixing: the standards track or RFC series (was: Re: What do we mean when we standardize something?)
On 5/30/13, John C Klensin john-i...@jck.com wrote: difficult problems arise when someone comes to us with a spec that might be ok but isn't how we would do it and tries to say you can have this and we will turn over change control as long as you don't really want to make any changes. When a statement equivalent to that is justified on the basis of either being in a hurry or not invalidating existing implementations, we find ourselves in the middle of the contradiction between consensus of industry practice and best engineering solution for standardization. If the standards proposed are reviewed well I don't think there will be contradiction, I don't recommending always sticking to best engineering solutions, because it is difficult to guarantee best solutions in present/future (best practices ok). For industry request work, IMO its better that our standards get into between *best engineering practices* and *good engineering practices*, that will not contradict with *consensus* and *industry practices*. AB
What do we mean when we standardize something?
Hi. A number of recent discussions, specifically including the Last Calls on DKIM and standardizing RRTYPEs and even to some extent the meeting location ones, have started me thinking that perhaps we need to review what business the IETF is actually in and to be sure that we actually still agree about it. The note that follows is an attempt to take the isolated parts (and symptoms) of that discussion up a level and discuss it as a general issue. The key issues go to the very nature of standardization. While there are many variations on each, there are two fundamentally different models for what standards are about. Each is legitimate in its own way and each has its advocates, but they are different. One is a focus on quality: an engineering consensus on the best technical solution given physics or consensus goals and boundary conditions. The other is sometimes called consensus of industry practice in which a standard reflects what is already deployed, possibly picking among different implemented options in a few cases but mostly requiring that a common model appears before standards are issued. Two variations on the second theme were extremely common in the earlier days of computer and telecommunciations standardization but are less popular today (at least in theory) due to IPR and antitrust concerns. Both start with a single vendor (or closed consortium) implementation. That implementation might then be offered for standardization (sometimes with a no major changes restriction) and adopted more generally or endorsed by traditional SDOs. Or that single implementation might be reverse-engineered by others and then the result treated as common industry practice. Despite the occasional claim that a strange (to me) idea called anticipatory standardization is a variation on the second model, that combination makes standards in the absence of, or prior to, implementation and deployment an impossible contradiction. It is basically incompatible with innovation of any type. The IETF has traditionally chosen the first model and a great deal of our thinking and public rhetoric are based on it. Even when we have adopted proposals, sometimes implemented ones, from elsewhere, we have insisted on change control and usually taken that seriously. One could claim that an adaptation of the second model would make the Internet more uniform, but a consensus of existing practice cannot aspire to working better except in the narrow sense of substitutability of conforming products. It is interesting that the multi-stage model of RFC 2026 (particularly the original Proposed Standard definition), the IEEE's Trial-Use standards (the current Section 5.7 of their Standards Board Operations Manual, http://standards.ieee.org/develop/policies/opman/sect5.html#5.7, makes instructive reading), and ISO Technical Specification all try to combine the two models by allowing a preliminary specification --one that is intended for trial implementation but not deployment-- to be designed on an engineering basis and then standardized only when implementations exist and can be compared (our original Draft Standard). In theory, only when both an industry common practice and consensus about value emerges does true standardization (including our original definition for Internet Standard) move forward. We know how well that has worked for us in practice. While there have been exceptions, I don't believe it has worked much better for other SDOs. If we are not going to move significantly in the direction of consensus of industry practice, then it seems to me that we need to be careful about the arguments that are made (or at least those that are considered legitimate) in support of advancement into or on the standards track. For example, if we agree on a relaxed registration procedure for some protocol parameters, it is not reasonable to later turn around and ask that that those parameters be standardized, unchanged, because they are already registered (and maybe deployed). For standardization, the IETF generally needs not only change control but an opportunity to comment on and affect the actual design and specification of whatever is being standardized. Similarly, we sometimes hear it argued that we should accept a specification for Proposed Standard unchanged because it has been extensively developed and reviewed elsewhere. That may be reasonable in some cases although I'd hope we wouldn't make it a common practice. But, if a specification adopted for Proposed Standard on that basis is then proposed for advancement to Internet Standard, I think the review should be comprehensive --perhaps even more comprehensive than the usual such review-- because the Internet Standard is unambiguously an IETF product and recommendation not that of the other body. If we allow a specification to advance to Proposed Standard without an open and critical review because it was developed and reviewed comprehensively by outside expert
Re: What do we mean when we standardize something?
I think this is one of the best discussions of what we're about that I've seen anywhere, and I'm grateful to John for working this through. One thing I'd like to take up further is this: On 5/29/13 9:23 AM, John C Klensin wrote: Similarly, we sometimes hear it argued that we should accept a specification for Proposed Standard unchanged because it has been extensively developed and reviewed elsewhere. That may be reasonable in some cases although I'd hope we wouldn't make it a common practice. But, if a specification adopted for Proposed Standard on that basis is then proposed for advancement to Internet Standard, I think the review should be comprehensive --perhaps even more comprehensive than the usual such review-- because the Internet Standard is unambiguously an IETF product and recommendation not that of the other body. I'd actually much prefer to see these go to informational, if they're to be published. Otherwise I agree - if something's going to be an IETF standard it needs to go through the IETF standards development review and revision process, which is probably not what the authors want. Melinda
Re: What do we mean when we standardize something?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 5/29/13 11:38 AM, Melinda Shore wrote: I think this is one of the best discussions of what we're about that I've seen anywhere, and I'm grateful to John for working this through. One thing I'd like to take up further is this: On 5/29/13 9:23 AM, John C Klensin wrote: Similarly, we sometimes hear it argued that we should accept a specification for Proposed Standard unchanged because it has been extensively developed and reviewed elsewhere. That may be reasonable in some cases although I'd hope we wouldn't make it a common practice. But, if a specification adopted for Proposed Standard on that basis is then proposed for advancement to Internet Standard, I think the review should be comprehensive --perhaps even more comprehensive than the usual such review-- because the Internet Standard is unambiguously an IETF product and recommendation not that of the other body. I'd actually much prefer to see these go to informational, if they're to be published. Otherwise I agree - if something's going to be an IETF standard it needs to go through the IETF standards development review and revision process, which is probably not what the authors want. /me wonders if we need a separate series for informational documentation Peter - -- Peter Saint-Andre https://stpeter.im/ -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.19 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJRpj3xAAoJEOoGpJErxa2pAiYP/jvIR0UI55/HBq4wUeGW4rWJ LXJLvEpxuKZL056F6cNFajksDlOVyL3oknHacc7OKPt+3d8ClXyQTzisAsjzxYdT XyQynTk0x0AmyFTvwDQ99HnPnpe+FJoBBdPvKM0KXvrGSDLQQviPvgBol/J/jBrz fBGq/sf/qDRbuORDyK30Dr9g3iDPvuWtz8umVGKZVH0NU0SuWcBx+UqZlMxYDh62 T4X/HZXWEHcooGviiTLfpMRi+wzWJc16OlRW9Ncq1DwgITFXAuDchp5xmXoP20Q2 DU6K+pqCGZpO/AU2XBH72pno7cwi92nZPatAVhvNnwuC3IrtwcACH/mEvf6LJZ7q B1ZmlMuhNA22Bu1lyLBzvJ/udwK84wtiCV64QGRnY5fzTxP3j8E00BjabjBMdkKM ctwNaYI9JplVKUiW2NdOBVlsDwVYKpEBXNcZGPP4BmaaO9BFhUnK21heyD+TNf+o gUC10MOkOLgByKN21EFMc8VTNettZLSM90MlOx06ToN/xaAL5Cgh0Qn18nvIisxg IMCfEdTSdsxlBNgD01X/6MNBTzb6gFVS7nRm74UX4R+HA9eTrBhB1BYx6y4PhVhY UOPVZkNuHngjth21TM3U62UZdIgT/AWZ/EvVFQFlTuB25vLXOV7tVfWbWa/FW4AT 6rgPVAQbyPvJZm4kSX7R =sjcp -END PGP SIGNATURE-
Re: What do we mean when we standardize something?
On 29 May 2013 18:42, Peter Saint-Andre stpe...@stpeter.im wrote: /me wonders if we need a separate series for informational documentation Or maybe multiple paths, with multiple entry points. Perhaps instead of Proposed Standard, we have a Engineering Proposal for an engineering consensus, and a Submitted Proposal for an industry submission. Both would move to Internet Standard from there, if appropriate. I admit to picking names without the word standard in on purpose, but that's because I think we should reclaim PS... I know I'm in the rough. Dave.
Re: What do we mean when we standardize something?
On 30/05/2013 08:04, Dave Cridland wrote: On 29 May 2013 18:42, Peter Saint-Andre stpe...@stpeter.im wrote: /me wonders if we need a separate series for informational documentation Or maybe multiple paths, with multiple entry points. We already do have exactly that, and there are many instances of proprietary or local protocols being documented, but not standardised, as Informational RFCs in the Independent stream. Sometimes they squeeze through as Informational RFCs in the IETF stream. We also have BCPs, when there is robust consensus on good operational practice. Again, we have Informational RFCs when somebody wants to document current practice without seeking consensus. I'm not sure what we need to change. Where we get into trouble seems to be when people want a rubber stamp for something that doesn't make the cut for IETF consensus. We have a little trouble saying No. But I think we have a duty to say No when something works but is believed to be bad for the Internet as a whole. Brian Perhaps instead of Proposed Standard, we have a Engineering Proposal for an engineering consensus, and a Submitted Proposal for an industry submission. Both would move to Internet Standard from there, if appropriate. I admit to picking names without the word standard in on purpose, but that's because I think we should reclaim PS... I know I'm in the rough. Dave.
Fixing: the standards track or RFC series (was: Re: What do we mean when we standardize something?)
Melinda, Peter, Dave, and Brian, I wanted to try to describe some fundamental models and their implications but avoid leaping head first into either the fix the standards process or change the categories or content of the RFC Series rat holes.However... --On Wednesday, May 29, 2013 09:38 -0800 Melinda Shore melinda.sh...@gmail.com wrote: Proposed Standard on that basis is then proposed for advancement to Internet Standard, I think the review should be comprehensive --perhaps even more comprehensive than the usual such review-- because the Internet Standard is unambiguously an IETF product and recommendation not that of the other body. I'd actually much prefer to see these go to informational, if they're to be published. Otherwise I agree - if something's going to be an IETF standard it needs to go through the IETF standards development review and revision process, which is probably not what the authors want. While I agree with the preference, I didn't mention it because I assumed, for the purpose of my analysis, that the choice of standards track or not was outside my logical scope. I suggest we've done it both ways, e.g., with NFS, we published Version 3 as Informational (RFC 1813) and then produced Version 4 on the standards track (RFC 3010). HTTP was pretty much the same: 1.0 was pretty much taken over from W3C and published as Informational (RFC 1945); 1.1 was a Proposed Standards in RFC 2068. DKIM, IIR, went directly to Proposed Standard (RFC 4871ff) based mostly on an external spec (RFC 4870 appears to represent a somewhat different protocol). There are probably better examples of the latter. My guess is that cases will differ and trying to make a hard rule would just get us into trouble. --On Wednesday, May 29, 2013 11:42 -0600 Peter Saint-Andre stpe...@stpeter.im wrote: /me wonders if we need a separate series for informational documentation I'm not sure what you are suggesting. If it is time to get informational documents out of the RFC Series, the community has considered variations on that theme endless times and been unable to reach consensus that it is a good idea (and has sometimes reached consensus that it isn't). In particular, putting them in a separate series would complicate the model of publishing externally-produced specifications as Informational and then following up with Standards Track, IETF designed and vetted, specs discussed above. If it is time to rethink the categories of RFCs and/or those of the standards track as Dave suggests... well, I've tried floating several of those proposals and gotten absolutely no traction. If you want to try, I promise to cheer supportively from the sidelines. --On Thursday, May 30, 2013 08:44 +1200 Brian E Carpenter brian.e.carpen...@gmail.com wrote: Where we get into trouble seems to be when people want a rubber stamp for something that doesn't make the cut for IETF consensus. We have a little trouble saying No. But I think we have a duty to say No when something works but is believed to be bad for the Internet as a whole. Yes, and the latter is implicit in my long note. The even more difficult problems arise when someone comes to us with a spec that might be ok but isn't how we would do it and tries to say you can have this and we will turn over change control as long as you don't really want to make any changes. When a statement equivalent to that is justified on the basis of either being in a hurry or not invalidating existing implementations, we find ourselves in the middle of the contradiction between consensus of industry practice and best engineering solution for standardization. best, john
