Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Just as a follow-up here ... I was John's co-author on RFC 3933 (http://tools.ietf.org/html/rfc3933). When we were working on the draft, the problem I thought we were solving, was that the IESG needs to update the IETF's BCP processes from time to time, but (1) it was like 32 simultaneous root canals to actually get community consensus to modify the IETF's process BCPs including untried changes that might be improvements, so (2) the IESG was using informal IESG statements developed with much less community involvement than was expected for process BCP changes, in order to get anything done at all. I (and, I think, John as well) saw RFC 3933 process experiments as a middle path between lightweight IESG decisions and full process BCP revisions. That's what I thought Section 2 of RFC 3933 was trying to say. I had assumed that if the IESG clearly had discretion to do something under the current process BCPs, and thought it was the right thing to do, they would just do it, rather than set up a process experiment. For what that's worth. Spencer On 1/24/2013 12:34 PM, John C Klensin wrote: --On Tuesday, January 22, 2013 16:31 -0500 Thomas Narten nar...@us.ibm.com wrote: This document seems to have a bit of missing the forest for the trees. In the overall scheme of things, I don't believe the draft will materially help, and is at best a distraction from dealing with meaningful problems. The crux of the issue is that any attempt at fast tracking is fundamentally about short-circuiting some aspect of our review processes. We should only do that very carefully. In almost all cases, individual judgement is needed as to when it is appropriate to do that. ADs/WG chairs already have some flexibility here. e.g., a WG can skip WG LC if they think its ... Hi. First of all, I am completely in agreement with Thomas and his analysis. Anything that can reasonably and appropriately be done by this sort of effort can be done by discretion without adoption of a formal procedure and adoption of a formal procedure creates additional and unnecessary risks. As co-author of the process experiment model, I also object to its use when it is not demonstrably necessary, for example to give extra status and force IETF-wide use where the actions suggested can be carried out as a matter of normal discretion (see Section 5 of the document).
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
I agree with your approach. However, if it should be tested by community and reported successful then why we need to go through 5 years, just publish fast ways, AB Sun, 27 Jan 2013 20:27:17 -0800 If this is an experiment, then you presumably answers to the following questions: 1- what is your an hypothesis? 2- what you intend to measure? 3- what is your 'control' against which to compare the results? 4- what is your objective metric for success/failure? I've heard only one hypothesis - that this reduces time to publication. I disagree that this is a useful hypothesis to test for the following reasons: - time to publication isn't a goal of the IETF IMO, any doc that isn't useful in 5 years ought to not be published here; we don't need to document every sneeze - thorough review ought to be a requirement and this 'experiment' potentially compromises that by reducing the overall time of review - community resources ought to be considered and this 'experiment' burns group resources due to having a broad group concurrently review a doc that could have been reviewed by smaller groups first Given the limited cycles this community has to review docs, I cannot see a benefit to this experiment that is worth the cost. Having this entire community burn cycles on this document speaks for itself. It should have been vetted in a smaller, more invested community first. Calling something an 'experiment' doesn't make it worthwhile to test. Joe
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
I agree with your concerns and may suggest that the tests' results of the running code SHOULD be reported inside a mandatory section of the Fast-Tracked I-D to RFC. AB On 01/26/2013 Martin Rex wrote: Stephen Farrell wrote: On 01/25/2013 09:36 PM, Martin Rex wrote: I don't know about the last time it happened, but I know about one time in Nov-2009 in the TLS WG (now rfc5746). I recall that and agree with the sequence of events you describe, but I'm not sure that that situation is relevant when considering this draft, for two reasons: Uh-oh! - First, that was the IETF in security-incident-handling mode, and that's just different from normal process for us, whether fast-tracked or not. If I hadn't stumbled upon the TLS renegotiation issue while discussing the proposal for TLS channel bindings, then the solution would have been developed and shipped under NDA and entirely with involvement of the IETF TLS WG (just some selected TLS WG members), and likely there would have been a request for rubber stamping the (half-baked) solution *AFTER* deploymend of the fixes by major vendors. http://www.ietf.org/mail-archive/web/tls/current/msg03928.html http://www.ietf.org/mail-archive/web/tls/current/msg03942.html Myself, I'm worried by the idea of shortcutting review and marginalization of comments that do not meet the discuss criteria. - Second, there was significant controversy within the WG before the last calls, (with many hundreds of mails;-) so a set of WG chairs that chose to try a fast-track experiment in such circumstances would be crazy basically. (Remember, we're only talking about an experiment here.) Admittedly, I deserve part of the blame for the heated discussion. But if occasional heated discussions turn out to be a serious problem for creating or improving technical solutions, then we may need more procedural safeguards (such as independent leadership), not less. The true essence of most of the 2009 TLS WG discussion was actually fairly small and simple: http://www.ietf.org/mail-archive/web/tls/current/msg05365.html -Martin
RE: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hello, Sorry I missed your last paragraph in the snow storm. So, Adrian, noting the ratio between discussion of this draft on the IETF list in the last few weeks and discussions of everything else, how long does professional courtesy to another IESG member (presumably in combination with your thinking this is a good idea) encourage you to allow the unintentional DoS attack on the IETF list to continue? No-one is obliged to continue discussions beyond expressing their irreconcilable discontent with the document. The document progressed this far not out of professional courtesy to another IESG member, but according to the rules of RFC3933. Page 3 bullet 1 had completed, and the conditional part of bullet 2 was completed, so the next step was IETF last call. The IETF last call is four weeks (according to RFC3933). It ends on 2/8/13. I am unaccustomed to calling consensus before the end of a last call. My personal view, in sponsoring this document, is that the community needs the opportunity to test whether there is support for running an experiment. I also have a view that experimentation can be made safe and non-damaging to the rest of the IETF. In that way we can often be surprised that something we thought would never work, or was pointless, or potentially harmful, turns out to be harmless/OK/beneficial. Nevertheless, whatever my views on this experiment, or on experimentation in general, when the last call completes I will summarise the last call comments to the IESG so that they can act on bullet 3 (i.e. decide whether the proposed experiment is plausible and appropriate. Thanks, Adrian
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Thomas said: The crux of the issue is that any attempt at fast tracking is fundamentally about short-circuiting some aspect of our review processes. Speaking as a Gen-ART reviewer, I am indeed worried by this aspect. I feel I would have to spend much longer reviewing a draft if I knew it had not been through WGLC. The fact that a spec is implementable does not mean it is correct, which is exactly why we want review by subject matter experts. I would expect other late reviewers, including ADs, to react the same way. I welcome anything that encourages early implementations; interop challenges are a good way to do that. But in the end, is the IETF process the right vehicle for this? Can we start a new tradition of interop challenges? Brian
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
On 1/22/13 10:31 PM, Thomas Narten wrote: a WG can skip WG LC if they think its not needed. ??? When was the last time that happened? Did it require a consensus call to determine? Eliot
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
--On Friday, January 25, 2013 14:36 +0100 Eliot Lear l...@cisco.com wrote: On 1/22/13 10:31 PM, Thomas Narten wrote: a WG can skip WG LC if they think its not needed. ??? When was the last time that happened? Did it require a consensus call to determine? Chair discretion. It is seems clear that the WG is in agreement on all relevant issues and has adequately reviewed the document, a WG LC is just another way to waste time. Keep in mind that, whether there is a WG LC or not, WG members are permitted to comment in IETF LC. Although it may be bad taste for a WG member to disagree during IETF LC about issues already settled, we don't forbid that and, because of a lot of edge cases and the vague possibility of abuse, it would be a terrible idea to forbid it. Other than added bureaucracy, a WG LC serves only the following purposes: * If the WG participants haven't adequately reviewed the document(s), a WG LC can provide a deadline and forcing function for more reviews. * Surprises during IETF LC and long debates on the IETF list waste the community's time and distract from other discussions. If a WG LC might contribute to identifying and understanding the issues before the document reaches IETF LC and possibly even resolving them in the WG, that is almost certainly A Good Thing. A WG Chair who can't make an guess about whether such issues are likely to arise (or issues that were thought settled really were not) probably needs to retire (and should certainly err in the direction of doing a WG LC). In the context of draft-farrell-ft, the above makes the idea of WG LC in parallel with IETF LC either irrelevant or bad news. If the WG Chair (or AD) concludes that a WG LC is needed, then the procedure should not be invoked. If a WG LC is not needed, then organizing things so that one is conducted in parallel with the IETF LC risks having two discussions going on in parallel with comments made to the WG list not being exposed to the community (remember the days when IETF LC comments were routinely sent to the IESG and not to the IETF list?). And, if the WG does conduct a WG LC, the document can be read as forbidding use of the FT procedure to condense and set timelines on the rest of the process (something I'd suggest fixing if we are going to go ahead with this). All of this points out one of my main concerns. Almost as a side-effect, the proposal formalizes a number of informal procedures and mechanisms work pretty well most of the time but, because they are informal, can be used flexibly without a big fuss. If we are going to formalize them, we really should be asking questions about consensus, exception cases, and so on... and risk another several steps in the direction of trying to substitute procedures for judgment, responsibility, and accountability. john
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
John, On 1/25/13 4:27 PM, John C Klensin wrote: a WG can skip WG LC if they think its not needed. ??? When was the last time that happened? Did it require a consensus call to determine? Chair discretion [... and five of paragraphs of text] None of which answered my above questions. When was the last time chairs USED that discretion?! Eliot
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
On 01/25/2013 03:27 PM, John C Klensin wrote: In the context of draft-farrell-ft, the above makes the idea of WG LC in parallel with IETF LC either irrelevant or bad news. If the WG Chair (or AD) concludes that a WG LC is needed, then the procedure should not be invoked. If a WG LC is not needed, then organizing things so that one is conducted in parallel with the IETF LC risks having two discussions going on in parallel with comments made to the WG list not being exposed to the community (remember the days when IETF LC comments were routinely sent to the IESG and not to the IETF list?). And, if the WG does conduct a WG LC, the document can be read as forbidding use of the FT procedure to condense and set timelines on the rest of the process (something I'd suggest fixing if we are going to go ahead with this). That's a fair point and I'd be open to incorporating a change to make that better if we are going ahead with the experiment. I'll add a pointer to this mail to the working copy [1] in a bit. All of this points out one of my main concerns. Almost as a side-effect, the proposal formalizes a number of informal procedures and mechanisms work pretty well most of the time but, because they are informal, can be used flexibly without a big fuss. If we are going to formalize them, we really should be asking questions about consensus, exception cases, and so on... and risk another several steps in the direction of trying to substitute procedures for judgment, responsibility, and accountability. I disagree that doing an experiment formalises these things. I do agree that more thought would be needed if the experiment turned up something that we did want to make permanent. S. [1] http://down.dsg.cs.tcd.ie/misc/draft-farrell-ft-04.txt
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
--On Friday, January 25, 2013 16:31 +0100 Eliot Lear l...@cisco.com wrote: John, On 1/25/13 4:27 PM, John C Klensin wrote: a WG can skip WG LC if they think its not needed. ??? When was the last time that happened? Did it require a consensus call to determine? Chair discretion [... and five of paragraphs of text] None of which answered my above questions. When was the last time chairs USED that discretion?! I didn't answer because I don't know. It isn't something I've tried to track and my first-hand experience is limited because I've tried to avoid WG Chair roles in favor of advising for the last decade or so. The times when I didn't do a WG LC are both long ago and probably before much of the community assumed they were required. We did short ones in EAI, not because they were required but because we were trying to encourage additional internal reviews and to verify that any issues that were missed were identified. But that WG was quite difficult in terms of getting explicit reviews and discussion so WG LC was used as a tool to address known problems. john
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
--On Friday, January 25, 2013 15:34 + Stephen Farrell stephen.farr...@cs.tcd.ie wrote: ... All of this points out one of my main concerns. Almost as a side-effect, the proposal formalizes a number of informal procedures and mechanisms work pretty well most of the time but, because they are informal, can be used flexibly without a big fuss. If we are going to formalize them, we really should be asking questions about consensus, exception cases, and so on... and risk another several steps in the direction of trying to substitute procedures for judgment, responsibility, and accountability. I disagree that doing an experiment formalises these things. I do agree that more thought would be needed if the experiment turned up something that we did want to make permanent. If I correctly understand the above, it lies at the root of the problem I was trying to describe. This is really an experiment if the effect of deciding we didn't want to make it permanent was that we were at status quo ante, i.e., as if the experiment had never been initiated. If the experimental procedure doesn't turn out to be useful but we then need to rethink things that have changed, then we are in rather different territory. In other language, if the worst possible consequence of this proposed experiment would be a harmless waste of time, then a number of us who are now expressing reservations or opposition would be saying ok, try it and let us know how it works out. To the degree to which you can modify the document to be clear than the experiment has no permanent procedural side-effects whether it is judged to be a success or not, the specification becomes more attractive. As an example, Barry has initiated an experiment in a different shepherd write-up regime that moves away from the yes, I checked this and yes, I checked that too format and toward narrative about notable points. Because the shepherd write-up format and requirements have never been part of a formal procedure, that particular experiment could be initiated without a big fuss (and, in particular, without a 3933 procedure). I don't know if the requirements on shepherd write-ups in item 11 of Section 4 is compatible with Barry's template, would require modifications to it, or would require use of some other template if one intended to use the Fast Track procedure. Nor do I know whether having this provision would frustrate other experiments with shepherd templates during the course of the experiment. Similarly, this experiment is heavily dependent on retention of the IESG's current voting categories and DISCUSS criteria. Does it prevent changing those categories and criteria during the course of the experiment? Does it give them a status they don't now have (I think it does, YMMD) and, if it does so, does their status revert at the end of the experiment? I think you could add language to the document that would clarify these things and either eliminate those concerns or bring them more clearly into focus. All of that said, I'm about to go back to lurking but want to make a prediction about a likely outcome of this experiment (assuming it is approved) that the document doesn't mention. A document is moved into this procedure and fast-tracked. Someone concludes, as Thomas, Brian, and others have suggested, that some aspect of the procedure has frustrated adequate review and that there are bad associated consequences. They appeal on the grounds that the things that this procedure shortcuts are insufficient to ensure fairness given the things that the claim that there is a running implementation doesn't demonstrate (e.g., as others have suggested, while cannot be implemented would certainly indicate there is something wrong with a spec, can be implemented doesn't show that the spec is correct or does what is wanted). The moment that appeal is filed, the document in question moves from fast track to the slowest track of all. john
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hiya, On 01/25/2013 04:37 PM, John C Klensin wrote: If I correctly understand the above, it lies at the root of the problem I was trying to describe. This is really an experiment if the effect of deciding we didn't want to make it permanent was that we were at status quo ante, i.e., as if the experiment had never been initiated. That is definitely my intention. I'm happy to make it clear in the draft, though I guess I'd assumed it was inherent in the use of 3933. Anyway, yes, I'll add text saying that explicitly. For now, I've put in a note in the working copy. [1] ... The moment that appeal is filed, the document in question moves from fast track to the slowest track of all. I think I recall someone sometime hoping there'd be more appeals:-) Yes, if this generated more appeals then that'd be a bad outcome. Would it be more likely? I don't know to be honest but I'd hope not, though perhaps you're right that it sort of paints a target on a draft. I think this is similar to points made about picking suitable drafts/implementations and would be a good thing to say in a wiki rather than put into the draft. I've added a reminder note to do that also in [1]. Thanks, S. [1] http://down.dsg.cs.tcd.ie/misc/draft-farrell-ft-04.txt
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Eliot Lear wrote: On 1/25/13 4:27 PM, John C Klensin wrote: a WG can skip WG LC if they think its not needed. When was the last time that happened? Did it require a consensus call to determine? Chair discretion [... and five of paragraphs of text] None of which answered my above questions. When was the last time chairs USED that discretion?! I don't know about the last time it happened, but I know about one time in Nov-2009 in the TLS WG (now rfc5746). The WG chair *did* a WG consensus call before AD WG chair short-cutted to IETF LC. IMHO, at the very minimum, a WG consensus call with a specific question or choice is necessary to gauge consensus in a WG. WG Consensus call (20-Nov-2009): http://www.ietf.org/mail-archive/web/tls/current/msg04571.html AD taking over (27-Nov-2009): http://www.ietf.org/mail-archive/web/tls/current/msg04928.html IETF LC (30-Nov-2009): http://www.ietf.org/mail-archive/web/tls/current/msg04962.html While I was OK with skipping the WGLC, I really didn't like some of the AD's shortcut, in particular with respect to standardizing two methods for signaling, and his refusal to perform a consensus call for mandating the simpler version of the signaling, which would have significantly facilitated the creation of fixes for the installed base. http://www.ietf.org/mail-archive/web/tls/current/msg05287.html There is a REAL issue with rushing a document pointing to implementations, when it turns out pretty quickly that what has been implemented is really just half of a solution. I'm actually afraid that rushing things on the process side is going to worsen problems. What I was slighly missing in that process was (a) leadership, in recognizing that mandating one signaling scheme will simplify the spec, implementations and interop effort and (b) true consensus calls on issues that are obviously contentious. Even running a consensus call during / in parallel to an IETF LC is better than blaming time-constraints and the AD infering his desired outcome from prior discussions of *MUCH* broader topics. Process-wise, we need to be extra careful about collision of interest, each time when short-cutting processes. -Martin
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hi Martin, On 01/25/2013 09:36 PM, Martin Rex wrote: I don't know about the last time it happened, but I know about one time in Nov-2009 in the TLS WG (now rfc5746). I recall that and agree with the sequence of events you describe, but I'm not sure that that situation is relevant when considering this draft, for two reasons: - First, that was the IETF in security-incident-handling mode, and that's just different from normal process for us, whether fast-tracked or not. Its in the nature of things that the vast majority of security incidents don't directly affect IETF protocols so as to require a backwards incompatible change. So I think that was a highly unusual case. (And let's hope things stay that way.) - Second, there was significant controversy within the WG before the last calls, (with many hundreds of mails;-) so a set of WG chairs that chose to try a fast-track experiment in such circumstances would be crazy basically. (Remember, we're only talking about an experiment here.) As for Eliot's question, I don't recall any case when a WG skipped WGLC. Even if its not part of 2026, right now it's a de-facto but mandatory part of the process as far as I can see. I'd be interested if there are cases where WGLC was skipped, esp. if its been regularly done. S.
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Stephen Farrell wrote: On 01/25/2013 09:36 PM, Martin Rex wrote: I don't know about the last time it happened, but I know about one time in Nov-2009 in the TLS WG (now rfc5746). I recall that and agree with the sequence of events you describe, but I'm not sure that that situation is relevant when considering this draft, for two reasons: Uh-oh! - First, that was the IETF in security-incident-handling mode, and that's just different from normal process for us, whether fast-tracked or not. If I hadn't stumbled upon the TLS renegotiation issue while discussing the proposal for TLS channel bindings, then the solution would have been developed and shipped under NDA and entirely with involvement of the IETF TLS WG (just some selected TLS WG members), and likely there would have been a request for rubber stamping the (half-baked) solution *AFTER* deploymend of the fixes by major vendors. http://www.ietf.org/mail-archive/web/tls/current/msg03928.html http://www.ietf.org/mail-archive/web/tls/current/msg03942.html Myself, I'm worried by the idea of shortcutting review and marginalization of comments that do not meet the discuss criteria. - Second, there was significant controversy within the WG before the last calls, (with many hundreds of mails;-) so a set of WG chairs that chose to try a fast-track experiment in such circumstances would be crazy basically. (Remember, we're only talking about an experiment here.) Admittedly, I deserve part of the blame for the heated discussion. But if occasional heated discussions turn out to be a serious problem for creating or improving technical solutions, then we may need more procedural safeguards (such as independent leadership), not less. The true essence of most of the 2009 TLS WG discussion was actually fairly small and simple: http://www.ietf.org/mail-archive/web/tls/current/msg05365.html -Martin
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
--On Tuesday, January 22, 2013 16:31 -0500 Thomas Narten nar...@us.ibm.com wrote: FWIW, I share Joe's concerns. And Stephen's responses don't really change my mind. This document seems to have a bit of missing the forest for the trees. In the overall scheme of things, I don't believe the draft will materially help, and is at best a distraction from dealing with meaningful problems. The crux of the issue is that any attempt at fast tracking is fundamentally about short-circuiting some aspect of our review processes. We should only do that very carefully. In almost all cases, individual judgement is needed as to when it is appropriate to do that. ADs/WG chairs already have some flexibility here. e.g., a WG can skip WG LC if they think its ... Hi. First of all, I am completely in agreement with Thomas and his analysis. Anything that can reasonably and appropriately be done by this sort of effort can be done by discretion without adoption of a formal procedure and adoption of a formal procedure creates additional and unnecessary risks. As co-author of the process experiment model, I also object to its use when it is not demonstrably necessary, for example to give extra status and force IETF-wide use where the actions suggested can be carried out as a matter of normal discretion (see Section 5 of the document). In this particular case, a better experiment would be to use a process like this for some otherwise-relevant drafts and in some areas only and compare the results. The force IETF-wide use part doesn't apply as long as its invocation is a matter of WG Chair and AD discretion and belief that the right conditions are met, so, other than consuming a rather large amount of community time since it was first proposed almost two months ago, I just don't see what it accomplishes other than adding to the collection of process documents and constraining the review process, including about issues that running code doesn't help to inform. The process experiment mechanism is, IMO, an entirely inappropriate way to simply make a statement that the IETF really, really, does believe in running code and interoperability... especially since one would hope that such a statement would not expire after a year. Another thing this draft accomplishes is to shift the responsibility for determining whether a document is good enough from an IESG interpretation of community consensus to the decision of a single AD as to whether or not changes are required (Section 3, item 5). While it would probably improve speed of processing, it is not, in general, a desirable change, especially if one believes in the first part of the Clark quote. It is probably just a nit in the grand scheme of things but, if the responsible AD doesn't have time to do a sufficient review to be confident in voting yes (Section 3, item 7) how can she possibly sign off on the belief than no changes are required (Section 3, item 5)? It seems to me that situation creates a situation in which an AD can initiate this process, take the word of the WG that everything is in order, and then not only avoid responsibility by abstaining but use the provisions of Section 3 item 8 to let the document go through (as long as some AD is willing to say yes) without taking real responsibility. Especially because we don't have firm rules against a WG Chair and responsible AD coming from the same organization, I would hope that those who are concerned about IETF CoI and antitrust policies would take a careful look at that scenario because it would seem to avoid many of the protections that are inherent in wide community review and individual responsibility. Other than that, it is nothing more than a specific scenario consistent with Thomas's concern that this procedure could be used to short-circuit effective review. I'm not going to quote extensively from Thomas's other arguments or waste bits trying to summarize them, but let me add a few observations to Thomas's about the document and how it is being handled. If it is really urgent to get a document done, it is far better to take steps to make sure the editors are engaged and responsive. That is more likely the real issue!!! And that no AD is going to screw around nit-picking or delaying things in other ways. Based on experience, I have absolutely no confidence that a prohibition against anything that isn't important (such as the one in this draft) will stop that behavior if, e.g., someone thinks the document is incomprehensible or ignores important issues. (And, as Thomas points out, it shouldn't). Running code is valuable. Always has been, always will be. But we need to resist the temptation of making running code more equal than other criteria or putting it on a pedestal as some sort of holy grail. Running code by itself is just a sound bite. The importance of running code is what it tells us about a protocol specification. Just the mere fact that there is ... Moreover, Dave Clark
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
I do not really have time or desire to enter an extended discussion on this document. It's pretty clear to me we just disagree. But I did want to be on record as not supporting this document so that silence wouldn't be taken as agreement or support. A few specific followups below. This document seems to have a bit of missing the forest for the trees. In the overall scheme of things, I don't believe the draft will materially help, and is at best a distraction from dealing with meaningful problems. Do you mean the experiment or if this became a permanent part of the process? Your comments read as if you're concerned mostly about the latter, and you make no mention of the former. Mostly the latter. But given I don't see this experiment being successful, I think the community has better ways of spending its limited resources. As it happens, I agree that this is not going to produce a huge improvement. But its not intended to, and modest improvements (as opposed to modest proposals:-) should also be ok. If not then we're in a worse process-rut than even I think. If there is an improvement, I expect it will be too small to be worth bothering with. And given the community's limited resources (a not insubstantial amount of collective cycles has already gone into this document), it seems better to use those resources where there is some hope of making a real difference. The crux of the issue is that any attempt at fast tracking is fundamentally about short-circuiting some aspect of our review processes. I'd prefer s/short-circuiting/speeding-up/ Sorry, I think short-circuiting is the right term. Saying you are just speeding things up is just spin. The reality is that it takes time to do quality standards/technical work. And, it takes iteration. The only kind of iteration that makes sense is sequential. Get some review, revise, then review the new version. When one tries to overlap sequential reviews, the result is almost always a poorer quality document. It is important that the revisions get reviewed by a fresh set of eyes. If you short-circuit such iteration, you inevitably get poorer quality documents. Also, it's really unfair to reviewers to have all of them stumble across the same issues. This document risks substituting process for judgement. I'd rather see more of the latter and less of the former. Disagree. The WG chairs exercise judgement to kick off the process. The IESG exercise judgement as usual with no changes. This document would use a different kind of judgement. One that is more formulaic. The kind of judgement I'd rather see is the WG/WG chairs/ADs getting together and making the case for a particular document being so important/urgent that it needs to be accelerated. And then have them work out a plan to actually get reviews in a more timely manner, etc. E.g., get committments from folk to review and revise quickly. Have a realistic workplan that folk commit to. If its not possible to get folk to commit to being responsive, doesn't that implicitely say something about the overall urgency of the document? But make that case on a number of factors (importance of having the RFC in place, etc.) rather than just on the fact that there was some sort of running code associated with it. I think incorporating the discuss criteria should be an interesting part of the experiment. (I've found them really helpful in reviewing documents these last couple of years.) Discuss criteria are an IESG-level construct, developed to find balance between nit picking by the IESG very late in the process vs. correcting real problems. That same criteria is not appropriate to apply to a document that hasn't gone through WG LC, which is at a much earlier stage of the process. No doubt, when a AD or reviewer suggests this needs fixing, the proponents will hold up this document and say you shouldn't do this, per the RFC -- you're violating the spirit of this document, only really really critical stuff needs to get addressed... No thanks. That is not what the IETF is about. I don't see how you get to No doubt... I do entirely doubt that'll happen for most all editors. This community is really good at citing process to justify whatever action (or inaction) they would like to see. Like it or not, this could become another such tool. Thomas
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hi, all, On 1/11/2013 8:21 AM, Adrian Farrel wrote: Hi Alexa, Please be aware of this document that has just entered a four-week IETF last call. The document describes a proposed IETF process experiment under the rules of RFC 3933. The proposed experiment calls on the IETF Secretariat to take specific actions under certain circumstances in corner cases of the experiment. C This is a silly idea. First, running code should already be considered as part of the context of review. Second, running code is not correlated to correctness, appropriateness, or safety. See Linux for numerous examples. Third, running code doesn't mean the doc is sufficient that multiple parties can generate interoperable instances. It's merely the sound of one hand clapping ;-) Finally, NOTHING should circumvent the multi-tiered review process. That process helps reduce the burden on the community at large via the presumption that smaller groups with more context have already reviewed proposals before they get to the broader community. This is a bad idea even as an experiment. Joe
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hi Joe, On 01/22/2013 04:39 PM, Joe Touch wrote: Hi, all, On 1/11/2013 8:21 AM, Adrian Farrel wrote: Hi Alexa, Please be aware of this document that has just entered a four-week IETF last call. The document describes a proposed IETF process experiment under the rules of RFC 3933. The proposed experiment calls on the IETF Secretariat to take specific actions under certain circumstances in corner cases of the experiment. C This is a silly idea. So you're in two minds about it eh:-) First, running code should already be considered as part of the context of review. Second, running code is not correlated to correctness, appropriateness, or safety. See Linux for numerous examples. Third, running code doesn't mean the doc is sufficient that multiple parties can generate interoperable instances. It's merely the sound of one hand clapping ;-) Your second and third and points seem opposed to your first. The latter ones imply that running code is useless, the first one says its not. I don't believe any of us have any quantitative basis on which to base assertions that this will improve or dis-improve our processes or output, or be neutral. (Hence proposing it as an experiment.) Finally, NOTHING should circumvent the multi-tiered review process. That process helps reduce the burden on the community at large via the presumption that smaller groups with more context have already reviewed proposals before they get to the broader community. I disagree with the shouted NOTHING - if there are non-silly ways in which we figure we can improve our processes then we ought be open to trying 'em out. You may or may not be right that this is silly, but merely asserting that it is doesn't make it so. Being stuck with current processes or only ever adding more review tiers would IMO be sillier than this proposal. But that seems to be where we're mostly at. This is a bad idea even as an experiment. Sorry, I don't get the bad aspect - rhetoric aside, in what way do you see running this experiment doing harm? S. Joe
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
On 1/22/2013 9:00 AM, Stephen Farrell wrote: Hi Joe, On 01/22/2013 04:39 PM, Joe Touch wrote: ... This is a silly idea. So you're in two minds about it eh:-) First, running code should already be considered as part of the context of review. Second, running code is not correlated to correctness, appropriateness, or safety. See Linux for numerous examples. Third, running code doesn't mean the doc is sufficient that multiple parties can generate interoperable instances. It's merely the sound of one hand clapping ;-) Your second and third and points seem opposed to your first. The latter ones imply that running code is useless, the first one says its not. I never said useless; I explained several ways in which it alone is correlated to any of the issues relevant to speeding up the review process. Multiple interoperable implementations helps ensure a doc sufficiently describes a protocol - nothing less, but also NOTHING MORE. I don't believe any of us have any quantitative basis on which to base assertions that this will improve or dis-improve our processes or output, or be neutral. (Hence proposing it as an experiment.) It takes more than an unknown to make an experiment. There has to be an hypothesis. Near as I can tell, yours is running code means it's OK to run concurrent review at multiple levels. Please explain why you think that is true. I gave multiple reasons why it is not. Finally, NOTHING should circumvent the multi-tiered review process. That process helps reduce the burden on the community at large via the presumption that smaller groups with more context have already reviewed proposals before they get to the broader community. I disagree with the shouted NOTHING - if there are non-silly ways in which we figure we can improve our processes then we ought be open to trying 'em out. You may or may not be right that this is silly, but merely asserting that it is doesn't make it so. Being stuck with current processes or only ever adding more review tiers would IMO be sillier than this proposal. But that seems to be where we're mostly at. OK, so let's try an experiment where authors with the first name Stephen pay everyone $1,000 to review their docs. It certainly hasn't been tried, so - by your metric - it's worth considering? Some things are simply not. This is a bad idea even as an experiment. Sorry, I don't get the bad aspect - rhetoric aside, in what way do you see running this experiment doing harm? It puts more work on the community at large to review an idea that could have been either rejected or significantly improved in a smaller community before wasting the larger communities time. This document is a prime example of such. Joe
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
On 01/22/2013 05:14 PM, Joe Touch wrote: On 1/22/2013 9:00 AM, Stephen Farrell wrote: Hi Joe, On 01/22/2013 04:39 PM, Joe Touch wrote: ... This is a silly idea. So you're in two minds about it eh:-) First, running code should already be considered as part of the context of review. Second, running code is not correlated to correctness, appropriateness, or safety. See Linux for numerous examples. Third, running code doesn't mean the doc is sufficient that multiple parties can generate interoperable instances. It's merely the sound of one hand clapping ;-) Your second and third and points seem opposed to your first. The latter ones imply that running code is useless, the first one says its not. I never said useless; That's how I interpreted one hand clapping. I explained several ways in which it alone is correlated to any of the issues relevant to speeding up the review process. I can't parse the above. But sure, there are many things that running code does not do, and for which running code is insufficient. Some of those are noted in the draft. Multiple interoperable implementations helps ensure a doc sufficiently describes a protocol - nothing less, but also NOTHING MORE. Multiple interoperable implementations are not required for PS or experimental RFCs. Perhaps you've misread something. Or perhaps there is some other process-experiment that you'd prefer we try that's not this one? If so, writing a draft would be the way to go. I don't believe any of us have any quantitative basis on which to base assertions that this will improve or dis-improve our processes or output, or be neutral. (Hence proposing it as an experiment.) It takes more than an unknown to make an experiment. There has to be an hypothesis. Near as I can tell, yours is running code means it's OK to run concurrent review at multiple levels. Please explain why you think that is true. I gave multiple reasons why it is not. The above is an oversimplification of what the draft proposes but see below. Finally, NOTHING should circumvent the multi-tiered review process. That process helps reduce the burden on the community at large via the presumption that smaller groups with more context have already reviewed proposals before they get to the broader community. I disagree with the shouted NOTHING - if there are non-silly ways in which we figure we can improve our processes then we ought be open to trying 'em out. You may or may not be right that this is silly, but merely asserting that it is doesn't make it so. Being stuck with current processes or only ever adding more review tiers would IMO be sillier than this proposal. But that seems to be where we're mostly at. OK, so let's try an experiment where authors with the first name Stephen pay everyone $1,000 to review their docs. It certainly hasn't been tried, so - by your metric - it's worth considering? Some things are simply not. Rhetoric to the fore then. Feel free to cut the text above this line if responding;-) This is a bad idea even as an experiment. Sorry, I don't get the bad aspect - rhetoric aside, in what way do you see running this experiment doing harm? It puts more work on the community at large to review an idea that could have been either rejected or significantly improved in a smaller community before wasting the larger communities time. The proposal actually doesn't formally change what's required of a WG before a publication request. So I think you're just incorrect there. You would perhaps have been correct had you argued that the proposal might speed things up and hence give the community less time for review. But seeing if such a speed-up happens and whether that has other good or bad impacts under these conditions is an explicit goal of the experiment. I'm guessing we just disagree about the likelihood that going faster under these conditions might improve or dis-improve things. But that's ok. If we get to run the experiment we might find out. If the draft doesn't become an RFC, then we won't. S. This document is a prime example of such. Joe
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Joe Touch to...@isi.edu wrote: On 1/11/2013 8:21 AM, Adrian Farrel wrote: The proposed experiment calls on the IETF Secretariat to take specific actions under certain circumstances in corner cases of the experiment. Specifically: ] ] 8. If at any point in the fast-track process the responsbile AD does ]not act in a timely fashion then the IESG Secretary should ensure ]that the draft enters IESG evaluation and is scheduled for the ]the soonest IESG telechat that is more than one week after the ]end of the two-week post-IETF LC period. That is, if the ]responsible AD does not act, then the default action applies ]which is that the draft enters IESG evaluation and is placed on ]the earliest IESG telechat. I honestly don't know who or what the IESG Secretary might be: several of us have assumed this is a typo, and Secretariat was intended. (There is one mention of Secretariat in the draft, but it speaks of what WG Chairs must do, not what the Secretariat should do.) This is a silly idea. Assuming that Joe refers to the item quoted above, I find myself forced to agree with Joe (which is scary ;^)... (There is, BTW, an email address iesg-secret...@ietf.org, which feeds into an automated ticket-generation system. I have been asked not to respond to it because it tends to get overloaded.) If we actually wanted the Secretariat to take the actions listed above, they would need an automated tool to discover the inaction of the AD involved. From the paragraph above, I frankly have no idea how to program such a tool. Enters IESG evaluation is a simple state variable, mostly harmless. Scheduled for an IESG telechat is a little vague. Usually this means creating a ballot where each IESG member can enter a position, but there are other ways to get on the agenda. Who gets to decide whether this paragraph calls upon the Secretariat to create that ballot? What time-limit applies between adding the item to the agenda and the actual start of the telechat? I have seen items appear _after_ the FINAL Agenda is posted the afternoon before (and have done my best to discourage such a practice). First, running code should already be considered as part of the context of review. Considered how? Second, running code is not correlated to correctness, appropriateness, or safety. See Linux for numerous examples. Exactly! Evaluating the correlation of running code to the printed standard-to-be is a long-winded process. Third, running code doesn't mean the doc is sufficient that multiple parties can generate interoperable instances. It's merely the sound of one hand clapping ;-) Here I disagree with Joe (familiar territory!) -- the existence of one open-source implementation decidely helps other implementors to produce interoperable instances. (Alas, at the same time it tends to hide ambiguities of the spec.) Finally, NOTHING should circumvent the multi-tiered review process. That process helps reduce the burden on the community at large via the presumption that smaller groups with more context have already reviewed proposals before they get to the broader community. This multi-tiered review process is somewhat mythical. To the extent it actually exists, I believe it benefits us; but accomplishing it in limited time is iffy at best. This is a bad idea even as an experiment. I'd like to call folks' attention to one paragraph in the Introduction: ] ] In summary, this experiment collapses three stages of the publication ] process to run concurrently: working group last call, AD review, and ] IETF last call. Additionally, the experiment will only require ] issues raised during these three stages to be addressed if they meet ] the IESG's Discuss criteria. The former is not formally a process ] change, although it is a change to the normal conventions. The ] latter is a process change and requires this experiment to be run ] under the auspices of [RFC3933]. All the other verbiage aside, _this_ is the point of the experiment: to remove any implied requirement to respond to LastCall comments that don't meet (current?) IESG Discuss criteria. (There is, fortunately, a normative reference to these, but being a web-page, one wonders if it will be entirely stable...) It is unreasonable to expect every commenter to read that web-page; and I can testify that IESG members sometimes disagree on how it should be interpreted. In practice, I believe this change will mean that the effort now spent responding to comments from respected IETFers will tend to be replaced by justifying why no response is needed. :^( To me, it seems unfortunate we should even be experimenting with paying less attention to comments from the more-experienced IETFers. The existence of running code doesn't make evaluation simpler. :^( I wonder if we might do better to encourage the Experimental status for potential standards _with_
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
FWIW, I share Joe's concerns. And Stephen's responses don't really change my mind. This document seems to have a bit of missing the forest for the trees. In the overall scheme of things, I don't believe the draft will materially help, and is at best a distraction from dealing with meaningful problems. The crux of the issue is that any attempt at fast tracking is fundamentally about short-circuiting some aspect of our review processes. We should only do that very carefully. In almost all cases, individual judgement is needed as to when it is appropriate to do that. ADs/WG chairs already have some flexibility here. e.g., a WG can skip WG LC if they think its not needed. And nothing stops an AD from going to WGLC before doing a careful review. But I'd rather that be done because the case at hand justifies such an optimization, not because there is some running code checkmark that allows a special case to be fast tracked.. This document risks substituting process for judgement. I'd rather see more of the latter and less of the former. Some specifics from the doc: In summary this experiment collapses three stages of the publication process to run concurrently: working group last call, AD review, and IETF last call. Additionally, the experiment will only require IMO, it is almost never appropriate to collapse WG LC and IETF LC. In cases where it is appropriate to do so, there presumably isn't a need for a WG LC to start with. And an AD has always been able to start an IETF LC without doing a detailed review. The point, is judgement is needed, and all such cases are best handled on a case-by-case basis. Surely, we don't need a document to justify doing this in those cases where it makes sense... IETF last call. Additionally, the experiment will only require issues raised during these three stages to be addressed if they meet the IESG's Discuss criteria. The former is not formally a process and later: 4. Only comments that would be DISCUSS-worthy according to the IESG Discuss Criteria [DCRIT] need be handled during last call. Other comments can be handled or not, at the authors/editors discretion. The above can easily become a way to dismiss legitimate review comments. No doubt, when a AD or reviewer suggests this needs fixing, the proponents will hold up this document and say you shouldn't do this, per the RFC -- you're violating the spirit of this document, only really really critical stuff needs to get addressed... No thanks. That is not what the IETF is about. If it is really urgent to get a document done, it is far better to take steps to make sure the editors are engaged and responsive. That is more likely the real issue!!! It is understood that the savings in time for the end-to-end delivery of a proposed standard or experimental RFC may be modest, however, the modifications to normal IETF process will also serve as an indication of the importance that the IETF places in running code. Running code is valuable. Always has been, always will be. But we need to resist the temptation of making running code more equal than other criteria or putting it on a pedestal as some sort of holy grail. Running code by itself is just a sound bite. The importance of running code is what it tells us about a protocol specification. Just the mere fact that there is running code doesn't mean there is anything particularly enlightening to learn from an implementation. For example, running code of a router function in software doesn't necessarily say anything as to whether the code can be implemented efficiently using ASICs, which may be the real requirement. 5. The responsible AD for the WG concerned makes the decision as to whether changes are required as a result of review comments, and determines whether or not those have been completed. If significant change or extended discussion is required, or if changes are not complete within two weeks after the end of fast- track last call, then the draft is returned to the WG by the responsible AD and the document is withdrawn from the experiment. The two week figure is too stringent. The above is too prescriptive to be practical. For starters, how will the IESG telechat align with the ending of LC? Or what if an IESG member issues a defer (or will those be disallowed?). Section 4 has a bit too much detail and process in it. I'm sure it has both too much and too little actually. I.e., it probably misses some corner cases, and then what do you do given how prescriptive the rest of the section is? But overall, I see this document at best as a no-op. However, I fear that it can be used to short-circuit our review processes in a way that doesn't help the IETF and that won't really speed things up enough to justify the downsides. Thomas
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hiya, On 01/22/2013 05:14 PM, Joe Touch wrote: It puts more work on the community at large to review an idea that could have been either rejected or significantly improved in a smaller community before wasting the larger communities time. Actually it occurs to me that there might be something here of note (while continuing to disagree with you about the overall idea)... If we do the experiment and a draft changes significantly during last-call based on new comments from someone active in the relevant WG then that would imply that either that draft wasn't suited for fast-tracking or perhaps that the experiment indicates a problem with fast-tracking. I'd be surprised if that happens at all to be honest, unless lots of WGs choose to try the experiment, but it could. I don't think there's really a high probability that a draft from a WG will be rejected - almost all docs that get to the start of WGLC tend to arrive at the IESG as far as I can see, though I'm sure there'll be some WG that operates differently. S.
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hi Thomas, On 01/22/2013 09:31 PM, Thomas Narten wrote: FWIW, I share Joe's concerns. And Stephen's responses don't really change my mind. Ah well. I'm willing to keep trying:-) This document seems to have a bit of missing the forest for the trees. In the overall scheme of things, I don't believe the draft will materially help, and is at best a distraction from dealing with meaningful problems. Do you mean the experiment or if this became a permanent part of the process? Your comments read as if you're concerned mostly about the latter, and you make no mention of the former. As it happens, I agree that this is not going to produce a huge improvement. But its not intended to, and modest improvements (as opposed to modest proposals:-) should also be ok. If not then we're in a worse process-rut than even I think. The crux of the issue is that any attempt at fast tracking is fundamentally about short-circuiting some aspect of our review processes. I'd prefer s/short-circuiting/speeding-up/ We should only do that very carefully. For example as an experiment? I think we are being careful. In almost all cases, individual judgement is needed as to when it is appropriate to do that. ADs/WG chairs already have some flexibility here. e.g., a WG can skip WG LC if they think its not needed. And nothing stops an AD from going to WGLC before doing a careful review. But I'd rather that be done because the case at hand justifies such an optimization, not because there is some running code checkmark that allows a special case to be fast tracked.. This document risks substituting process for judgement. I'd rather see more of the latter and less of the former. Disagree. The WG chairs exercise judgement to kick off the process. The IESG exercise judgement as usual with no changes. Some specifics from the doc: In summary this experiment collapses three stages of the publication process to run concurrently: working group last call, AD review, and IETF last call. Additionally, the experiment will only require IMO, it is almost never appropriate to collapse WG LC and IETF LC. In cases where it is appropriate to do so, there presumably isn't a need for a WG LC to start with. And an AD has always been able to start an IETF LC without doing a detailed review. The point, is judgement is needed, and all such cases are best handled on a case-by-case basis. What's wrong with the judgement of a set of WG chairs? AFAIK, ADs are no wiser in general and this one is certainly dumber than a lot of WG chairs. Surely, we don't need a document to justify doing this in those cases where it makes sense... IETF last call. Additionally, the experiment will only require issues raised during these three stages to be addressed if they meet the IESG's Discuss criteria. The former is not formally a process and later: 4. Only comments that would be DISCUSS-worthy according to the IESG Discuss Criteria [DCRIT] need be handled during last call. Other comments can be handled or not, at the authors/editors discretion. The above can easily become a way to dismiss legitimate review comments. Equally, gratuitous nit-picking can be used to try kill something with a thousand cuts. Neither are at all common that I can see, during IETF LC. I think incorporating the discuss criteria should be an interesting part of the experiment. (I've found them really helpful in reviewing documents these last couple of years.) No doubt, when a AD or reviewer suggests this needs fixing, the proponents will hold up this document and say you shouldn't do this, per the RFC -- you're violating the spirit of this document, only really really critical stuff needs to get addressed... No thanks. That is not what the IETF is about. I don't see how you get to No doubt... I do entirely doubt that'll happen for most all editors. If it is really urgent to get a document done, it is far better to take steps to make sure the editors are engaged and responsive. That is more likely the real issue!!! Urgency isn't a part of this. Going faster doesn't imply urgency. It is understood that the savings in time for the end-to-end delivery of a proposed standard or experimental RFC may be modest, however, the modifications to normal IETF process will also serve as an indication of the importance that the IETF places in running code. Running code is valuable. Always has been, always will be. But we need to resist the temptation of making running code more equal than other criteria or putting it on a pedestal as some sort of holy grail. I agree. And this draft doesn't do that. Or are you saying it does? Running code by itself is just a sound bite. The importance of running code is what it tells us about a protocol specification. Just the mere fact that there is running code doesn't mean there is anything particularly enlightening to
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hi John, Bits and pieces below... On 01/22/2013 07:04 PM, John Leslie wrote: Joe Touch to...@isi.edu wrote: On 1/11/2013 8:21 AM, Adrian Farrel wrote: The proposed experiment calls on the IETF Secretariat to take specific actions under certain circumstances in corner cases of the experiment. Specifically: ] ] 8. If at any point in the fast-track process the responsbile AD does ]not act in a timely fashion then the IESG Secretary should ensure ]that the draft enters IESG evaluation and is scheduled for the ]the soonest IESG telechat that is more than one week after the ]end of the two-week post-IETF LC period. That is, if the ]responsible AD does not act, then the default action applies ]which is that the draft enters IESG evaluation and is placed on ]the earliest IESG telechat. I honestly don't know who or what the IESG Secretary might be: several of us have assumed this is a typo, and Secretariat was intended. (There is one mention of Secretariat in the draft, but it speaks of what WG Chairs must do, not what the Secretariat should do.) Fair enough. I think the terms effectively mean the same set of folks though. This is a silly idea. Assuming that Joe refers to the item quoted above, I find myself forced to agree with Joe (which is scary ;^)... (There is, BTW, an email address iesg-secret...@ietf.org, which feeds into an automated ticket-generation system. I have been asked not to respond to it because it tends to get overloaded.) If we actually wanted the Secretariat to take the actions listed above, they would need an automated tool to discover the inaction of the AD involved. Perhaps ultimately, were this to be a part of our permanent process. From the paragraph above, I frankly have no idea how to program such a tool. AD didn't hit button to move forward or backward? Absent a tool, my guess is a WG chair would complain and some other AD would tell 'em to talk to their AD or ask the secretariat to push the button. This is very much a corner case. Enters IESG evaluation is a simple state variable, mostly harmless. Scheduled for an IESG telechat is a little vague. Usually this means creating a ballot where each IESG member can enter a position, but there are other ways to get on the agenda. Who gets to decide whether this paragraph calls upon the Secretariat to create that ballot? What time-limit applies between adding the item to the agenda and the actual start of the telechat? I have seen items appear _after_ the FINAL Agenda is posted the afternoon before (and have done my best to discourage such a practice). I'm not seeing the vagueness there sorry. First, running code should already be considered as part of the context of review. Considered how? Second, running code is not correlated to correctness, appropriateness, or safety. See Linux for numerous examples. Exactly! Evaluating the correlation of running code to the printed standard-to-be is a long-winded process. Third, running code doesn't mean the doc is sufficient that multiple parties can generate interoperable instances. It's merely the sound of one hand clapping ;-) Here I disagree with Joe (familiar territory!) -- the existence of one open-source implementation decidely helps other implementors to produce interoperable instances. (Alas, at the same time it tends to hide ambiguities of the spec.) Finally, NOTHING should circumvent the multi-tiered review process. That process helps reduce the burden on the community at large via the presumption that smaller groups with more context have already reviewed proposals before they get to the broader community. This multi-tiered review process is somewhat mythical. To the extent it actually exists, I believe it benefits us; but accomplishing it in limited time is iffy at best. This is a bad idea even as an experiment. I'd like to call folks' attention to one paragraph in the Introduction: ] ] In summary, this experiment collapses three stages of the publication ] process to run concurrently: working group last call, AD review, and ] IETF last call. Additionally, the experiment will only require ] issues raised during these three stages to be addressed if they meet ] the IESG's Discuss criteria. The former is not formally a process ] change, although it is a change to the normal conventions. The ] latter is a process change and requires this experiment to be run ] under the auspices of [RFC3933]. All the other verbiage aside, _this_ is the point of the experiment: to remove any implied requirement to respond to LastCall comments that don't meet (current?) IESG Discuss criteria. (There is, fortunately, a normative reference to these, but being a web-page, one wonders if it will be entirely stable...) Stable enough for an experiment I think. That might be something
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
I have two concerns and comments: - How will success or failure be measured? Number of appeal increases or lesser amount? I have a concern that once this door is open, there will be increase appeals and also apathy of outcomes. There should be a statement of what sort of problems or issues are possible and envisioned and the plans to deal with them or not. - The drafts indicates that fast tracking probably applies better towards BIS work. I cited concerns that this ideas should only be used for new work, not BIS work where there is already established implementators and working code. I prefer to work in No Surprises environment where it doesn't require me to pay attention to all protocol change work to make sure nothing fell thru the crack or some group of vendors with their questionable code implementation different from the norm being mandated in new BIS work. I only see this RFC being used to negate voiced concerns in the name of fast tracking, not win people over if they already have a problem with the work, especially if its BIS WORK. But then again, maybe that is the intent. I believe the experiment should only be applied first to new protocol work, not BIS work and if so, iff there are absolutely no question of possible conflict of interest, rush to judgment and no new code change impacts. Existing protocol implementers should not be put into a position, and worst within a shorter time period, to argue or just review what BIS changes made entail and force them into appeal actions. Perhaps the experiment can be done in two phases. I believe there is good in fast tracking new ideas and methods, especially if simple and one that most people seem to agree with with little concerns. Many vendors have much to contribute in this area. So the first phase can explore only with fast tracking new concepts. If this works out well using some measuring tool, then BIS work can be better explored for fast tracking by gaining some insights from the first phase. -- HLS Adrian Farrel wrote: Hi Alexa, Please be aware of this document that has just entered a four-week IETF last call. The document describes a proposed IETF process experiment under the rules of RFC 3933. The proposed experiment calls on the IETF Secretariat to take specific actions under certain circumstances in corner cases of the experiment. Could you please have someone in the Secretariat look at the draft and comment on the practicalities of the actions. Note that, at this stage, no changes to the tools are proposed so any actions would require manual intervention (if the experiment were successful and resulted in permanent changes to IETF process we might make changes to the tools at some future time). Thanks, Adrian -Original Message- From: ietf-announce-boun...@ietf.org [mailto:ietf-announce- boun...@ietf.org] On Behalf Of The IESG Sent: 11 January 2013 15:15 To: IETF-Announce Subject: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC The IESG has received a request from an individual submitter to consider the following document: - 'A Fast-Track way to RFC with Running Code' draft-farrell-ft-03.txt as Experimental RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2013-02-08. Exceptionally, comments may be sent to i...@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract This memo describes an optional, fast-track way to progress a working group document to IESG review. It is provided as a process experiment as defined in RFC 3933 for use when working group chairs believe that there is running code that implements a working group Internet-Draft. The motivation is to have the IETF process explicitly consider running code, consistent with the IETF's overall philosophy of running code and rough consensus. In this process all of working group last call, IETF last call, and Area Director review are carried out in the same two week period. Only comments that meet IESG Discuss criteria need to be addressed during this stage, and authors are required to make any changes within two weeks. This experiment will run for one year. The file can be obtained via http://datatracker.ietf.org/doc/draft-farrell-ft/ IESG discussion can be tracked via http://datatracker.ietf.org/doc/draft-farrell-ft/ballot/ No IPR declarations have been submitted directly on this I-D. -- HLS
Re: FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hi Hector, On 01/14/2013 05:05 PM, Hector Santos wrote: I have two concerns and comments: - How will success or failure be measured? Number of appeal increases or lesser amount? I have a concern that once this door is open, there will be increase appeals and also apathy of outcomes. There should be a statement of what sort of problems or issues are possible and envisioned and the plans to deal with them or not. RFC 3933 describes that I guess. Basically, if the IESG consider it a success then they could make the RFC into a BCP or more likely find a victim to write an I-D for that. That'd go through the usual process. I don't think this'll have any impact on appeals (hope not anyway), and for me the main metric of success will be whether or not it gets used, and if used, if the people involved in those drafts think its worth making permanent or not. - The drafts indicates that fast tracking probably applies better towards BIS work. Not really. Its says that -bis RFCs might be suitable. I cited concerns that this ideas should only be used for new work, not BIS work where there is already established implementators and working code. I prefer to work in No Surprises environment where it doesn't require me to pay attention to all protocol change work to make sure nothing fell thru the crack or some group of vendors with their questionable code implementation different from the norm being mandated in new BIS work. I don't get the concern. I only see this RFC being used to negate voiced concerns in the name of fast tracking, not win people over if they already have a problem with the work, especially if its BIS WORK. But then again, maybe that is the intent. I believe the experiment should only be applied first to new protocol work, not BIS work and if so, iff there are absolutely no question of possible conflict of interest, rush to judgment and no new code change impacts. Existing protocol implementers should not be put into a position, and worst within a shorter time period, to argue or just review what BIS changes made entail and force them into appeal actions. Perhaps the experiment can be done in two phases. I believe there is good in fast tracking new ideas and methods, especially if simple and one that most people seem to agree with with little concerns. Many vendors have much to contribute in this area. So the first phase can explore only with fast tracking new concepts. If this works out well using some measuring tool, then BIS work can be better explored for fast tracking by gaining some insights from the first phase. Ah, you're worried some WG will shoot out a -bis draft this way and that existing implementers will miss that because of the shorter timescale and no longer be compliant with the latest RFC? Is that it? To be honest, I can't really see that this'll have any impact there. If the existing coder who'd be caught out isn't aware that a WG is doing stuff then this will make no difference. If they are aware that the WG is doing a -bis, then they can comment as always before and during IETF LC. I really can't see this experiment causing any real surprise in cases like that. S. -- HLS Adrian Farrel wrote: Hi Alexa, Please be aware of this document that has just entered a four-week IETF last call. The document describes a proposed IETF process experiment under the rules of RFC 3933. The proposed experiment calls on the IETF Secretariat to take specific actions under certain circumstances in corner cases of the experiment. Could you please have someone in the Secretariat look at the draft and comment on the practicalities of the actions. Note that, at this stage, no changes to the tools are proposed so any actions would require manual intervention (if the experiment were successful and resulted in permanent changes to IETF process we might make changes to the tools at some future time). Thanks, Adrian -Original Message- From: ietf-announce-boun...@ietf.org [mailto:ietf-announce- boun...@ietf.org] On Behalf Of The IESG Sent: 11 January 2013 15:15 To: IETF-Announce Subject: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC The IESG has received a request from an individual submitter to consider the following document: - 'A Fast-Track way to RFC with Running Code' draft-farrell-ft-03.txt as Experimental RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2013-02-08. Exceptionally, comments may be sent to i...@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract This memo describes an optional, fast-track way to progress a working group document to IESG review. It is provided as a process experiment as defined in RFC 3933 for use
FW: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC
Hi Alexa, Please be aware of this document that has just entered a four-week IETF last call. The document describes a proposed IETF process experiment under the rules of RFC 3933. The proposed experiment calls on the IETF Secretariat to take specific actions under certain circumstances in corner cases of the experiment. Could you please have someone in the Secretariat look at the draft and comment on the practicalities of the actions. Note that, at this stage, no changes to the tools are proposed so any actions would require manual intervention (if the experiment were successful and resulted in permanent changes to IETF process we might make changes to the tools at some future time). Thanks, Adrian -Original Message- From: ietf-announce-boun...@ietf.org [mailto:ietf-announce- boun...@ietf.org] On Behalf Of The IESG Sent: 11 January 2013 15:15 To: IETF-Announce Subject: Last Call: draft-farrell-ft-03.txt (A Fast-Track way to RFC with Running Code) to Experimental RFC The IESG has received a request from an individual submitter to consider the following document: - 'A Fast-Track way to RFC with Running Code' draft-farrell-ft-03.txt as Experimental RFC The IESG plans to make a decision in the next few weeks, and solicits final comments on this action. Please send substantive comments to the ietf@ietf.org mailing lists by 2013-02-08. Exceptionally, comments may be sent to i...@ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. Abstract This memo describes an optional, fast-track way to progress a working group document to IESG review. It is provided as a process experiment as defined in RFC 3933 for use when working group chairs believe that there is running code that implements a working group Internet-Draft. The motivation is to have the IETF process explicitly consider running code, consistent with the IETF's overall philosophy of running code and rough consensus. In this process all of working group last call, IETF last call, and Area Director review are carried out in the same two week period. Only comments that meet IESG Discuss criteria need to be addressed during this stage, and authors are required to make any changes within two weeks. This experiment will run for one year. The file can be obtained via http://datatracker.ietf.org/doc/draft-farrell-ft/ IESG discussion can be tracked via http://datatracker.ietf.org/doc/draft-farrell-ft/ballot/ No IPR declarations have been submitted directly on this I-D.
