Re: IONs discuss criteria
All of this depends on the quality of the review and how it's followed up on. Having to push back on insistent nonsense is a problem. A good review that engenders a lot of discussion on substantial issues is very worthwhile. We should foster those -- they are important. This is no different than what happens within a WG. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Thomas Narten wrote: IMO, one of the biggest causes of problems (and most under-appreciated process weakness) in the IETF (and any consensus based organization for that matter) is poor handling of review comments. Whereas all of my own experiences with groups having problematic handling of reviews (except for a recent one where I happened to be the reviewer), is with the reviewer. In all of the groups I've been involved with, reviewers were taken and pursued seriously by the group. Problems occurred when the reviewer was vague, misguided and/or intractable. Diligent reviews are hugely helpful, especially so the earlier they occur. But not all reviews (including AD Discuss vetoes, which frequently are part of a form of review) are offered so helpfully. These repeated discussions about reviews are forceful in demanding acknowledgment of the former, helpful type, while vigorously denying the damaging reality of the latter and the need to deal with the pattern of strategic problems they cause. One of the reasons I'm such a fan of issue trackers is that it tends to remove a lot of the above stuff by simply not allowing stuff to fall through the cracks. Sure, trackers have overhead and are overkill in some cases. But if one could somehow analyze the number of documents that have been delayed for some time due to poor handling of review comments... Mostly, we agree on these points. Handled properly, placing review items in an issues list can be helpful to all parties, as long as each issue is clearly stated and possible resolutions or constructive guidance are included. One caveat: Sometimes it is the aggregate review that is most significant and breaking it into constituent 'issues' loses the broader concerns. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
--On Sunday, March 09, 2008 22:45:33 -0400 Ted Hardie [EMAIL PROTECTED] wrote: This particular ION had a life as an internet-draft with an intent to publish it as an RFC before the ION series existed. It was draft-iesg at one time, and no one came up with a draft-ietfer- counter proposal. But our mechanisms for allowing that kind of publication have years of experience behind them. Not so much for community commentary on IONs, IESG statements, or the like, which have tended to be perceived as changeable only by replacing the sitting IESG. my memory says that discuss criteria started out as an I-D, but with no consensus (or even a clear idea) on how to get a referenceable publication of it. It was one of the exampels that made me suggest IONs, exactly for that reason - RFC just didn't seem to fit. Harald ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
At 9:18 PM -0700 3/9/08, Russ Housley wrote: I really disagree. Gen-ART Reviews begin this way: I have been selected as the General Area Review Team (Gen-ART) reviewer for this draft (for background on Gen-ART, please see _http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html_). Please resolve these comments along with any other Last Call comments you may receive. This tells the recipients that the review fits exactly the role you describe above. But your behavior does not tell the recipient that. If they were being treated as general Last Call comments, it would be up to the shepherd and sponsoring AD to resolve them, not up to the General Area AD. That leaves one set of people on the hook for making sure they are done and deciding when they are, and it is the same set no matter how the Last Call comment is generated. Your mechanism privileges one set over others (in that they are more likely to be held as blocking until resolved), is likely to be slower (since yet another busy person must be informed that something is resolved, and may miss it when it was), and does not encourage things to push earlier than Last Call (which is the opportunity I think you're missing). Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted: I think you completely misunderstand my point. A reviewer can make a comment, and the authors or WG can say that they disagree. This is important for an AD to see. The AD now needs to figure out whether the reviewer is in the rough part of the rough consensus or whether the reviewer is representative of a larger part of the community. If the review is ignored, then the AD get no indication that further investigation is needed. snip True. As you have heard from Tim and me, these reviews are very helpful to the ADs, and I am saying that the discussion that follows such a review is helpful in judging consensus. Creating an environment where all these additional reviews and reviewers essentially block document progress is not the right direction. I agree, and requiring a response to the review is substantially different than blocking a document. The alternative is silence, and silence cannot aid in judging consensus. I think you and Tim (and potentially other ADs in areas that have review teams) are missing an opportunity here. Over time, these review teams have been grown to the point where they do their reviews at Last Call or before. That's a very good thing. One of the reasons it *could* be a good thing is to foster a culture of general cross area review. If the Last Call reviews by SAAG, Transport, Applications, and so on were seen as positive activities of the areas, they could help encourage even earlier cross area review, either by those teams or the areas as a whole. Since that is one of the main selling points of the IETF, that would be, let us say, nice. To make that happen, though, you'd have to see them as your areas feeding Last Call comments into the general Last Call commentary stream. Those are resolved by the shepherds and the area advisor, not by the area directors for the areas. The way you're doing it now treats these reviews differently, as advice to the area director of a relevant area, to be resolved differently. In other words, it continues to make the individual IESG folks the focus of the activity. That limits the benefits this review can provide, pretty much, to the benefit the IESG can absorb. If the IESG isn't doing the early review, the review teams don't either. To put this another way, having vibrant, active review teams for an area could be an area of leadership. Right now, it looks like they are being used soley as time-management aids for the ADs instead. That's a real opportunity missed. I really disagree. Gen-ART Reviews begin this way: I have been selected as the General Area Review Team (Gen-ART) reviewer for this draft (for background on Gen-ART, please see _http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html_). Please resolve these comments along with any other Last Call comments you may receive. This tells the recipients that the review fits exactly the role you describe above. Russ ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted: I really disagree. Gen-ART Reviews begin this way: I have been selected as the General Area Review Team (Gen-ART) reviewer for this draft (for background on Gen-ART, please see _http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html_). Please resolve these comments along with any other Last Call comments you may receive. This tells the recipients that the review fits exactly the role you describe above. But your behavior does not tell the recipient that. If they were being treated as general Last Call comments, it would be up to the shepherd and sponsoring AD to resolve them, not up to the General Area AD. That leaves one set of people on the hook for making sure they are done and deciding when they are, and it is the same set no matter how the Last Call comment is generated. Your mechanism privileges one set over others (in that they are more likely to be held as blocking until resolved), is likely to be slower (since yet another busy person must be informed that something is resolved, and may miss it when it was), and does not encourage things to push earlier than Last Call (which is the opportunity I think you're missing). I disagree with this characterization. IETF Last Call (hopefully) generates comments. These are usually resolved before IESG Evaluation, which is what you advocate in your note. This is the normal case in my experience. The issue seems to come up when they are not resolved. As I said in my previous note, there are two cases. (1) The Gen-ART Review or other Last Call comments were ignored. If someone takes the time to review the document at Last Call, they deserve the respect of a response. Failure to respond is a procedural objection. This is usually handled by the PROTO Shepherd, WG Chair, or document author. If by the time the document reaches IESG Evaluation, I have put a DISCUSS on documents to ensure that a response does happen. (I did not say that the comments are accepted; I said that a response is provided.) I have entered DISCUSS positions like this for Gen-ART Reviews, SecDir Reviews, and reviews from individual IETF participants. I've been careful to say that the authors do not need to accept all of the comments, but then need to answer them. (2) I agree with one or more concerns raised in the Last Call comments that was not resolved. Thus, a very often a very small portion of last Call comments become blocking comments. I tend to break the unresolved review comments into DISCUSS and COMMENT, giving credit to the source of the review. (I'm not trying to take credit for someone else's work.) AD judgement is needed here, and I consider the DISCUSS Criteria in making that judgement. Russ ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Speaking only as a Gen-ART reviewer, what Russ said is how I think it works, and Ted's concern that I might be privileged as a Gen-ART reviewer at last call time is the reason we're having that conversation. Gen-ART reviewers have had that concern since we were writing reviews for Harald. We don't WANT to be privileged, and we've worked consistently to head that off. I provided this text that's in the Gen-ART FAQ: 'And always remember that the IESG ballot position is called DISCUSS, not IMPERIAL EDICT or BLACKMAIL'. This should be doubly so, when a review team reviewer raised a concern. Thanks, Spencer From: Russ Housley [EMAIL PROTECTED] Ted: I really disagree. Gen-ART Reviews begin this way: I have been selected as the General Area Review Team (Gen-ART) reviewer for this draft (for background on Gen-ART, please see _http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html_). Please resolve these comments along with any other Last Call comments you may receive. This tells the recipients that the review fits exactly the role you describe above. But your behavior does not tell the recipient that. If they were being treated as general Last Call comments, it would be up to the shepherd and sponsoring AD to resolve them, not up to the General Area AD. That leaves one set of people on the hook for making sure they are done and deciding when they are, and it is the same set no matter how the Last Call comment is generated. Your mechanism privileges one set over others (in that they are more likely to be held as blocking until resolved), is likely to be slower (since yet another busy person must be informed that something is resolved, and may miss it when it was), and does not encourage things to push earlier than Last Call (which is the opportunity I think you're missing). I disagree with this characterization. IETF Last Call (hopefully) generates comments. These are usually resolved before IESG Evaluation, which is what you advocate in your note. This is the normal case in my experience. The issue seems to come up when they are not resolved. As I said in my previous note, there are two cases. (1) The Gen-ART Review or other Last Call comments were ignored. If someone takes the time to review the document at Last Call, they deserve the respect of a response. Failure to respond is a procedural objection. This is usually handled by the PROTO Shepherd, WG Chair, or document author. If by the time the document reaches IESG Evaluation, I have put a DISCUSS on documents to ensure that a response does happen. (I did not say that the comments are accepted; I said that a response is provided.) I have entered DISCUSS positions like this for Gen-ART Reviews, SecDir Reviews, and reviews from individual IETF participants. I've been careful to say that the authors do not need to accept all of the comments, but then need to answer them. (2) I agree with one or more concerns raised in the Last Call comments that was not resolved. Thus, a very often a very small portion of last Call comments become blocking comments. I tend to break the unresolved review comments into DISCUSS and COMMENT, giving credit to the source of the review. (I'm not trying to take credit for someone else's work.) AD judgement is needed here, and I consider the DISCUSS Criteria in making that judgement. Russ ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
It is my experience as well that Gen-ART or other organized reviews are not given any more weight than other Last Call comments. However, I at least weight different comments in different ways, based on whether I agree with the issue, whether I believe the issue is a major problem or a minor nit, whether the RFC Editor would take care of the edit anyway, whether the fix can be just done or if it requires careful guidance and discussion with someone, what the implications of the problem are if left unsolved, etc. At the end of the day, for some comments I would ask the documents to go back to the WG, some require further discussion with the reviewer and draft revision, some require an ack from the authors, some can be left for the authors to decide what they do with it, and some can be ignored (with a response, of course). Jari ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On 2008-03-11 03:42, Russ Housley wrote: Ted: I really disagree. Gen-ART Reviews begin this way: I have been selected as the General Area Review Team (Gen-ART) reviewer for this draft (for background on Gen-ART, please see _http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html_). Please resolve these comments along with any other Last Call comments you may receive. This tells the recipients that the review fits exactly the role you describe above. But your behavior does not tell the recipient that. If they were being treated as general Last Call comments, it would be up to the shepherd and sponsoring AD to resolve them, not up to the General Area AD. That leaves one set of people on the hook for making sure they are done and deciding when they are, and it is the same set no matter how the Last Call comment is generated. Your mechanism privileges one set over others (in that they are more likely to be held as blocking until resolved), is likely to be slower (since yet another busy person must be informed that something is resolved, and may miss it when it was), and does not encourage things to push earlier than Last Call (which is the opportunity I think you're missing). I disagree with this characterization. IETF Last Call (hopefully) generates comments. These are usually resolved before IESG Evaluation, which is what you advocate in your note. This is the normal case in my experience. FWIW, that's my experience as General AD too - I left the handling of LC reviews to the responsible people, but I certainly looked back to check that they had been responded to, when the draft reached the IESG. I believe that one reason for this apparent disagreement is a missing tool: an IETF-wide system for logging Last Call comments (including reviews) on each draft that's in Last Call. The last time we discussed this, we ended up slightly tuning the text of the Last Call message, to say Please send substantive comments to the ietf at ietf.org mailing lists by date. Exceptionally, comments may be sent to iesg at ietf.org instead. In either case, please retain the beginning of the Subject line to allow automated sorting. But that just dumps comments into a messy email archive, at best. I'd really like to see a more organised tool for this, and perhaps some more precision about when a comment to the IESG alone is appropriate. Brian ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On Mar 9, 2008, at 10:56 PM, Ted Hardie wrote: I think you and Tim (and potentially other ADs in areas that have review teams) are missing an opportunity here. Over time, these review teams have been grown to the point where they do their reviews at Last Call or before. That's a very good thing. One of the reasons it *could* be a good thing is to foster a culture of general cross area review. If the Last Call reviews by SAAG, Transport, Applications, and so on were seen as positive activities of the areas, they could help encourage even earlier cross area review, either by those teams or the areas as a whole. Since that is one of the main selling points of the IETF, that would be, let us say, nice. To make that happen, though, you'd have to see them as your areas feeding Last Call comments into the general Last Call commentary stream. Those are resolved by the shepherds and the area advisor, not by the area directors for the areas. The way you're doing it now treats these reviews differently, as advice to the area director of a relevant area, to be resolved differently. In other words, it continues to make the individual IESG folks the focus of the activity. That limits the benefits this review can provide, pretty much, to the benefit the IESG can absorb. If the IESG isn't doing the early review, the review teams don't either. To put this another way, having vibrant, active review teams for an area could be an area of leadership. Right now, it looks like they are being used soley as time-management aids for the ADs instead. That's a real opportunity missed. Ted Ted, There is no intention to treat Last Call comments from individuals differently than those that come from a review team. ADs *do* submit the same type of procedural discuss to ensure a response to Last Call comments that *weren't* generated by a review team. I'll agree that such discusses are more commonly associated with Last Call comments from review teams. That shouldn't be surprising. For many documents, the only cross area reviews come from the review teams. However, I also expect that Last Call comments that *weren't* generated by a review team are more likely to fall through the cracks. I know that I am more sensitive to the security directorate reviews than Gen-ART reviews or other Last Call comments. In addition to being assigned more or less on my behalf, they are focused on issues near and dear to my heart. I try to read all the Last Call comments, but when reviews focus on issues I don't understand or don't find compelling, I move on. In that case, I will not notice that a response did not occur. I suspect that other ADs suffer from similar human foibles. Thanks, Tim Polk ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Dave == Dave Crocker [EMAIL PROTECTED] writes: Dave Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. Dave How? You can update an IESG statement mor easily than a BCP. As you find areas where the text is unclear and you have to interpret you can actually go back and update the text. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Lakshminath: It's a fair thing to say that the ADs need to see a response. I also agree that cross-area review is important and at times unearths issues that may not have been raised in WG-level reviews. Personally, I prefer cross-area reviews to take place prior to the LC process and hope that the the LC process is for those issues that may have been really overlooked despite the best efforts of the WG chairs and ADs. I do not however quite understand the idea that we have to get consensus in the context of each GenART/Sec-dir/fill-in-the-blank review. It is of course quite plausible that one or two of those reviewers will never be satisfied with any level of revision of a given specification. In other cases, it may be that the reviewer has his or her personal preference on how to write documents and will never come out and say that the document they reviewed is ready. I think you completely misunderstand my point. A reviewer can make a comment, and the authors or WG can say that they disagree. This is important for an AD to see. The AD now needs to figure out whether the reviewer is in the rough part of the rough consensus or whether the reviewer is representative of a larger part of the community. If the review is ignored, then the AD get no indication that further investigation is needed. I have winced when some authors wrote to me after a review saying something along the lines of does the following revised text sound better. I would have been merely pointing out something I thought was not clear or might cause interoperability issues that they may have overlooked or missed. How they might fix it is entirely up to them (or the ADs involved). If their take was that they considered the comment and thought what they wrote is much more meaningful in the context of their work or the interoperability issue I raised does not apply within the scope of their specification, well so be it. Next, I trust them to do that. I don't need to see a dialog. I am willing to clarify my comments, and if I cannot articulate the issues I am raising clearly, well then the document needs to move on in the process. In the example you give, I think the response is good as it ensures that the comment was understood. Either the shepherding AD or the AD who solicited the review needs to determine what if anything specific needs to be done in all these cases. They are the judges of consensus. No matter who the reviewer might be they were not selected by the community to do the AD's job. True. As you have heard from Tim and me, these reviews are very helpful to the ADs, and I am saying that the discussion that follows such a review is helpful in judging consensus. Creating an environment where all these additional reviews and reviewers essentially block document progress is not the right direction. I agree, and requiring a response to the review is substantially different than blocking a document. The alternative is silence, and silence cannot aid in judging consensus. Russ ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Lakshimnath, On 2008-03-08 21:12, Lakshminath Dondeti wrote: ... Reviewers are not accountable for delays. Well, at least for Gen-ART there is a deadline: the end of Last Call for LC reviews, and a day or so before the telechat for pre-IESG reviews. Obviously, reviewers are human and sometimes miss those deadlines, but they exist. Brian ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
John, On 2008-03-09 05:56, John C Klensin wrote: I definitely do not want to see a discussion between authors and reviewers --especially Area-selected reviewers-- during Last Call. It too easily deteriorates into a satisfy him situation, and those reviewers are not anything special (or, unlike the ADs themselves, selected by some community mechanism). Well, speaking as a reviewer who has often exchanged email with authors during LC, I find that hard to accept. Very often the dialogue is about clarity issues (either in the the draft, or in the review) and ends with either the reviewer or the author saying Oh, I see what you mean. But there's a reason why the Gen-ART review boilerplate says Please resolve these comments along with any other Last Call comments you may receive. You're correct that there is no special status, and I'd be rather disturbed if my review was the only community review. Brian ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
At 6:38 AM -0700 3/9/08, Sam Hartman wrote: Dave == Dave Crocker [EMAIL PROTECTED] writes: Dave Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. Dave How? You can update an IESG statement mor easily than a BCP. As you find areas where the text is unclear and you have to interpret you can actually go back and update the text. No, Dave cannot update an IESG statement or an ION issued by the IESG. He can, however, put forward a proposed replacement document for a BCP as an Internet draft. So could the IESG. See the difference? This particular ION had a life as an internet-draft with an intent to publish it as an RFC before the ION series existed. It was draft-iesg at one time, and no one came up with a draft-ietfer- counter proposal. But our mechanisms for allowing that kind of publication have years of experience behind them. Not so much for community commentary on IONs, IESG statements, or the like, which have tended to be perceived as changeable only by replacing the sitting IESG. Ted Ted ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
At 1:42 PM -0800 3/8/08, Russ Housley wrote: I think you completely misunderstand my point. A reviewer can make a comment, and the authors or WG can say that they disagree. This is important for an AD to see. The AD now needs to figure out whether the reviewer is in the rough part of the rough consensus or whether the reviewer is representative of a larger part of the community. If the review is ignored, then the AD get no indication that further investigation is needed. snip True. As you have heard from Tim and me, these reviews are very helpful to the ADs, and I am saying that the discussion that follows such a review is helpful in judging consensus. Creating an environment where all these additional reviews and reviewers essentially block document progress is not the right direction. I agree, and requiring a response to the review is substantially different than blocking a document. The alternative is silence, and silence cannot aid in judging consensus. I think you and Tim (and potentially other ADs in areas that have review teams) are missing an opportunity here. Over time, these review teams have been grown to the point where they do their reviews at Last Call or before. That's a very good thing. One of the reasons it *could* be a good thing is to foster a culture of general cross area review. If the Last Call reviews by SAAG, Transport, Applications, and so on were seen as positive activities of the areas, they could help encourage even earlier cross area review, either by those teams or the areas as a whole. Since that is one of the main selling points of the IETF, that would be, let us say, nice. To make that happen, though, you'd have to see them as your areas feeding Last Call comments into the general Last Call commentary stream. Those are resolved by the shepherds and the area advisor, not by the area directors for the areas. The way you're doing it now treats these reviews differently, as advice to the area director of a relevant area, to be resolved differently. In other words, it continues to make the individual IESG folks the focus of the activity. That limits the benefits this review can provide, pretty much, to the benefit the IESG can absorb. If the IESG isn't doing the early review, the review teams don't either. To put this another way, having vibrant, active review teams for an area could be an area of leadership. Right now, it looks like they are being used soley as time-management aids for the ADs instead. That's a real opportunity missed. Ted ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On 3/9/2008 1:30 PM, Brian E Carpenter wrote: Lakshimnath, On 2008-03-08 21:12, Lakshminath Dondeti wrote: ... Reviewers are not accountable for delays. Well, at least for Gen-ART there is a deadline: the end of Last Call for LC reviews, and a day or so before the telechat for pre-IESG reviews. Obviously, reviewers are human and sometimes miss those deadlines, but they exist. Brian Brian, Thanks for the clarification. I know of those deadlines. Sam W also sets deadlines on sec-dir deadlines. My reference to the delays was in the context of overall delays that Thomas was also referring to, that is if and when a dialog ensues between one or more reviewers and authors and the goal becomes 'achieving some kind of a consensus with the reviewers' and there is really no accountability for delays and such. regards, Lakshminath ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Spencer, On Mar 7, 2008, at 8:56 AM, Spencer Dawkins wrote: (stuff deleted) So, for example, it probably IS worth finding out if the rest of the ADs who sponsor reviewing bodies As an AD who sponsors a reviewing body (the Security Directorate), I guess it is my turn to step into the fray. Yikes! Before I get started, I would like to note that I would not have survived my first year as an AD without the support of the Security Directorate. I continue to be impressed by the quality of the reviews that are performed. It is not an easy task when you are assigned a document from an unfamiliar area. I am sure that ADs sponsoring other reviewing bodies feel the same way. also use Russ's division into - did you consider the reviewer's comments?, especially when the review was issued as part of IETF Last Call, and - how did you address this specific comment, which I agreed with?, whether the AD entered the comment as a non-blocking COMMENT or as a DISCUSS. I considerate it my responsibility to ensure that reviews from the security directorate are considered thoughtfully. I requesting that members of the community devote their precious time to these reviews, and I don't want to see them ignored. So yes, I have filed process discusses of the form The authors have not responded to John Doe's secdir review. If the review had unsuccessfully attempted to initiate a conversation with the authors and they were unresponsive, that may even be the only appropriate path to take. If the review was in the form of concrete suggestions (the security considerations section needs to address man-in-the-middle attacks) I may choose to issue a DISCUSS using their text instead of involving the reviewer. If the review was acknowledged, then I review the email thread. I attempt to verify that any agreed upon actions are included in the current draft or implemented by a note to the RFC Editor. Modifications promised but not executed merit a discuss just to ensure they aren't forgotten. (I believe that this is consistent with the DICUSS criteria ION, under the IETF process for document advancement. Regardless, it doesn't present a burden or unduly complicate the process. At least, no one has complained about this type of discuss.) If issues were raised in the review but agreement was not reached, I try to decide whether I agree with the reviewer's comment *and* its relative importance. So, these residual issues get addressed using the second method. The more difficult problems come when a review is submitted just days before a telechat. IETF Last Call has typically closed, and the authors may not have even seen the review yet. This is dangerous territory, since the temptation is to cut and paste the entire review into my discuss to ensure that it isn't overlooked. Of course, the more appropriate and more helpful course of action is to determine which comments I support and separate them into non-blocking comments and discuss worthy buckets. [Confession time: The temptation of cut-and-paste is sometimes too strong for a mere mortal, though. I just revised my discuss on a document from yesterday's telechat, where I had cut and pasted a secdir review, to separate the issues into the discuss and comment buckets. Knowing better isn't the same as doing better. My thanks to the sponsoring AD, who kept me honest and asked me to review and revise!] I am not particularly methodical by nature, so I can't claim I perform this exact process in every case. However, that is a good overview of the process I try to follow. Tim Polk ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On 3/7/2008 11:18 AM, Thomas Narten wrote: Lakshminath Dondeti [EMAIL PROTECTED] writes: I have reviewed documents as a Gen-ART reviewer (during Brian's tenure I think), sec-dir reviewer and also provided IETF LC comments on some documents. As a reviewer, I am not sure whether I was expecting answers all those times. I am pretty sure I have not always stated whether or not the answers are satisfactory. On this one point. IMO, one of the biggest causes of problems (and most under-appreciated process weakness) in the IETF (and any consensus based organization for that matter) is poor handling of review comments. The ideal way to deal with them is to always respond, and to get an I am satisfied with your response to close the thread. Ideal being the keyword though. Not everyone, for any number of reasons, including cultural reasons, will come out and state all clear. It is also asking too much to ask the reviewer to get into a debate with the authors. It also fosters an environment where the reviewer starts becoming an authority. Anything else runs the risk of: - author says I dealt with those comments in the revised ID, with the reviewer saying Nope, I was ignored. - author says I dealt with those comments in the revised ID, and this actually was the case, except that they accidentally missed one or two important issues. Reviewer is left wondering was I blown off, or was this just an oversight, or... - author thinking they are done, because they responded on the list. But, no changes in the document and/or reviewer is still not happy. - reviewer having invested a not-insignificant amount of time doing a quality review feeling what's the point, which doesn't help to motivate a volunteer organization. This is especially problematic from the cross-area review perspective. I am not sure this is the right kind of motivation here. As one such reviewer, all I look for is thanks from the AD whose directorate or team I am serving on. And they have always been thankful. No, I do not constitute representative population. I am just offering one data point. Repeat above several times and intersperse with long periods of time where nothing happens on a document. You now have an idea of why it seems to take a long time to get documents through the system. Indeed. What started out as a great idea -- I volunteered to be a GenART reviewer 3-4 years ago now -- is beginning to become yet another burden in the process. One of the reasons I'm such a fan of issue trackers is that it tends to remove a lot of the above stuff by simply not allowing stuff to fall through the cracks. Sure, trackers have overhead and are overkill in some cases. But if one could somehow analyze the number of documents that have been delayed for some time due to poor handling of review comments... Yeah, it would be interesting. Although, I wonder what we will do with that information. Reviewers are not accountable for delays. There is no expectation of time commitment from them, for instance. Thomas ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On 3/7/2008 10:56 AM, Russ Housley wrote: Lakshminath: So, I'll tell everyone how I deal with Gen-ART Reviews. Other General ADs may have done things slightly different. When I use a Gen-ART Review as the basis of a DISCUSS, I put it in one of two categories. (1) The Gen-ART Review was ignored. Like any other Last Call comment, it deserves an answer. So, this is a procedural objection. In this situation, I've been careful to say that the authors do not need to accept all of the comments, but then need to answer them. I have reviewed documents as a Gen-ART reviewer (during Brian's tenure I think), sec-dir reviewer and also provided IETF LC comments on some documents. As a reviewer, I am not sure whether I was expecting answers all those times. I am pretty sure I have not always stated whether or not the answers are satisfactory. Next, I can imagine an author not wanting to respond to something I may have said because it was totally bogus or inappropriate and does not deserve a response. That might very well happen when I review documents on a topic that I am not familiar with and haven't had the time to read related references (that varies depending on the time available, etc.). Perhaps that is not such a bad thing; being blissfully ignorant on some topics keeps me, well, blissful. I use somewhat of a hyperbole for obvious reasons. I am sure many other situations are much more nuanced. I hope ADs don't continue to hold a DISCUSS in those situations waiting for a dialog to take place or waiting for a consensus to emerge. I sometimes hint in my reviews that the topic may be at the border of my knowledge and if I have a bias. Perhaps that is helpful. Even if the response does not go to the person making the comments, ADs need to see a response. Silence does not help us understand if consensus has been achieved. Last Call is the only point in the development and review of many documents where review from other IETF Areas takes place. It is very important that this cross-Area review take place. Russ Russ, Thank you for your note. It's a fair thing to say that the ADs need to see a response. I also agree that cross-area review is important and at times unearths issues that may not have been raised in WG-level reviews. Personally, I prefer cross-area reviews to take place prior to the LC process and hope that the the LC process is for those issues that may have been really overlooked despite the best efforts of the WG chairs and ADs. I do not however quite understand the idea that we have to get consensus in the context of each GenART/Sec-dir/fill-in-the-blank review. It is of course quite plausible that one or two of those reviewers will never be satisfied with any level of revision of a given specification. In other cases, it may be that the reviewer has his or her personal preference on how to write documents and will never come out and say that the document they reviewed is ready. I have winced when some authors wrote to me after a review saying something along the lines of does the following revised text sound better. I would have been merely pointing out something I thought was not clear or might cause interoperability issues that they may have overlooked or missed. How they might fix it is entirely up to them (or the ADs involved). If their take was that they considered the comment and thought what they wrote is much more meaningful in the context of their work or the interoperability issue I raised does not apply within the scope of their specification, well so be it. Next, I trust them to do that. I don't need to see a dialog. I am willing to clarify my comments, and if I cannot articulate the issues I am raising clearly, well then the document needs to move on in the process. Either the shepherding AD or the AD who solicited the review needs to determine what if anything specific needs to be done in all these cases. They are the judges of consensus. No matter who the reviewer might be they were not selected by the community to do the AD's job. Creating an environment where all these additional reviews and reviewers essentially block document progress is not the right direction. best regards, Lakshminath ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
--On Saturday, 08 March, 2008 00:12 -0800 Lakshminath Dondeti [EMAIL PROTECTED] wrote: The ideal way to deal with them is to always respond, and to get an I am satisfied with your response to close the thread. Ideal being the keyword though. Not everyone, for any number of reasons, including cultural reasons, will come out and state all clear. It is also asking too much to ask the reviewer to get into a debate with the authors. It also fosters an environment where the reviewer starts becoming an authority. ... Repeat above several times and intersperse with long periods of time where nothing happens on a document. You now have an idea of why it seems to take a long time to get documents through the system. Indeed. What started out as a great idea -- I volunteered to be a GenART reviewer 3-4 years ago now -- is beginning to become yet another burden in the process. I have to agree with this. One of the biggest risks we have to quality in standards is the dark side of the review process, a situation in which the effort to get a document as nearly correct as is reasonable given its maturity level and possible turns into deal with that objection. A half-dozen years ago we had extensive discussions of a concern in which ADs would object to particular text and authors, exhausted from the process of getting documents produced, would agree to any suggested changes --as long as they were not really offensive-- in order to make the objection go away. Put differently, there is a tendency for satisfy him (or her) and make the DISCUSS go away to become a more important objecting in practice than get things right. In at least some ways, the DISCUSS criteria were an attempt to constrain that problem, at as as far as the ADs were concerned. If we replace the opportunity to have to individually satisfy a dozen or so ADs with the opportunity to have to individually satisfy them plus a dozen more area-related reviewers, we are in big trouble. I am greatly in favor of these invited reviews if they ensure that every document is carefully reviewed by someone who was not part of its development process. I think that is where we started out. But the IESG has been selected by the community to take responsibility for these evaluations. If the IESG isn't going to do it, or can't, we need to be looking at our basic processes and structure, not at who is reviewing what for whom. If a review is done for a particular area, I expect that review to be treated primarily as advice to the relevant ADs. I expect those ADs to evaluate that advice carefully, not just to critically accept it. I definitely do not want to see a discussion between authors and reviewers --especially Area-selected reviewers-- during Last Call. It too easily deteriorates into a satisfy him situation, and those reviewers are not anything special (or, unlike the ADs themselves, selected by some community mechanism). I think this whole process needs a little refocusing. Especially for WG documents, no matter how many reviewers, shepherds, and lions, tigers, and bears we introduce into the system, the IESG should have one primary focus, which is making a go / no-go decision on whether a document is ready for approval and publication given maturity levels and any other relevant issues. If the answer is no-go for anything but an obvious matter about which there is no dissent or ambiguity, the document ought to go back to the WG for resolution of issues (which should clearly be identified as clearly as possible), not turned into a negotiation process between whomever happened to generate a comment and the authors about whether that person's view of the comment can be satisfied. There is just too much risk in the satisfy comments model of getting something important wrong or of responding to the comment but missing the main point of which the comment is a symptom. I certainly don't object to Thomas's idea of using issue trackers more and I think that making reviews public is an important safeguard. But those issue trackers should be used to * inform the IESG's decision about whether the document is ready and to * help inform the WG, presumably along with a careful IESG summary, about the issues (not the specific comments) it needs to address if the document is bounced back to it. Except perhaps for editorial matters and for clarification, a dialogue between an author and someone who makes a comment should have no place in the consensus process. john ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Wow. This was an interesting thread (that developed quickly). Why aren't you guys all on airplanes yet? So, to summarize a couple of points that other people made, but I didn't want to lose in the forest... ... DISCUSS has no BCP process standing today. I'm not sure that giving it process standing is the best use of any time we're willing to spend on process discussions. ... if people would feel better if the IESG reissued the discuss criteria document as an IESG statement, go for it. ... I understand what Ted is saying about this needs to be a BCP. If we were, in any way, capable of making changes to BCP text during the same decade that we discover a problem, I might even agree. We're not. Pete's heavy sigh about the amount of stuff that's stacking up for PUFI is real, but is also the result of the community's decision to ignore process topics after Montreal. That existing stack does not make me want to put anything in BCP text that's not already in BCP text. Look. We can't even come up with new names for three levels of standards track that aren't stupid, and that really doesn't matter. What makes us think we can come up with a set of non-criteria that will be solid enough to put in a BCP that might last for more than a decade (please check the dates on most text in 2026, 2418, etc), especially if the downside is that an AD can then say but that's not on the list of non-criteria? If the concern really is a rogue AD, I'm pretty sure our ability to predict and preempt the path of rogue-ivity isn't up to this challenge (it would be the IETF BCP equivalent of proposing a Law Prohibiting Cooking Intelligence to Justify Invading the Country Next to the Country that Hosted the Group that Attacked Us for the USA in 2002). ... I'm thinking that unreasonable DISCUSSes will be unreasonable whether they specifically map onto a non-criteria in the current ION, or in some future BCP, or not. Unreasonable DISCUSSes should always be challenged. ... there is a pre-existing escalation process for dealing with unreasonable DISCUSSes, but it doesn't sound like we're particularly good at using it. ... there is a pre-existing process for dealing with an IESG that goes against the current in the community, whether they are actually violating BCP text or not. Various folks (who I consider smart friends) have pointed out that we've never used the recall process (also in RFC 3777). That's true, but in private discussions, I've heard about two recall petitions that were filled out and then shown to the subjects of the petitions, and the recall petition initiators were satisfied with the results, without going public. So I'm not as concerned about a rogue IESG that says yeah, that's what the ION says, but we don't care (or even yeah, that's what the IESG Statement says, but we don't care) as other people seem to be. If you're rogue enough, you can blow off BCP text, too. On the specific topic of unfiltered reviews, (as one of the original Gen-ART reviewers) I do see authors talking about satisfying reviewers. Please don't. Satisfy the COMMENTing/DISCUSSing AD, and satisfy the community. Always remember that http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html explicitly addresses the question, What if the Gen-ART reviewer is just flat wrong?. That is not an accident... I think it would be nice for the IESG to collect, and maybe even put in one place, the way reviews work. It's worth noting that even for Gen-ART, the process has changed significantly since we started writing reviews for Harald. Those reviews were done ONLY for documents on a telechat agenda, and they went ONLY to Harald, who either laughed, COMMENTed, or DISCUSSed. The reviewers weren't involved in further resolution, and I'm not even sure authors knew that Harald's comments came from someone else's review. Eventually, we started reviewing at IETF Last Call time, and copying shepherding ADs, and then authors, and then WG chairs, but that's not where we started out. So, for example, it probably IS worth finding out if the rest of the ADs who sponsor reviewing bodies also use Russ's division into - did you consider the reviewer's comments?, especially when the review was issued as part of IETF Last Call, and - how did you address this specific comment, which I agreed with?, whether the AD entered the comment as a non-blocking COMMENT or as a DISCUSS. Thanks, Spencer ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On Mar 6, 2008, at 9:43 PM, Ted Hardie wrote: It was later that I suggested someone else hold the discuss, because I thought Cullen would want to recuse, since he is a patent author on a patent his company has filed related to this document. This is a reasonable action given the conflict of interest here. If another IESG member could take up the discuss, the author team and document shepherd would work to resolve it. But Cullen should recuse. -andy ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Lakshminath: So, I'll tell everyone how I deal with Gen-ART Reviews. Other General ADs may have done things slightly different. When I use a Gen-ART Review as the basis of a DISCUSS, I put it in one of two categories. (1) The Gen-ART Review was ignored. Like any other Last Call comment, it deserves an answer. So, this is a procedural objection. In this situation, I've been careful to say that the authors do not need to accept all of the comments, but then need to answer them. I have reviewed documents as a Gen-ART reviewer (during Brian's tenure I think), sec-dir reviewer and also provided IETF LC comments on some documents. As a reviewer, I am not sure whether I was expecting answers all those times. I am pretty sure I have not always stated whether or not the answers are satisfactory. Next, I can imagine an author not wanting to respond to something I may have said because it was totally bogus or inappropriate and does not deserve a response. That might very well happen when I review documents on a topic that I am not familiar with and haven't had the time to read related references (that varies depending on the time available, etc.). Perhaps that is not such a bad thing; being blissfully ignorant on some topics keeps me, well, blissful. I use somewhat of a hyperbole for obvious reasons. I am sure many other situations are much more nuanced. I hope ADs don't continue to hold a DISCUSS in those situations waiting for a dialog to take place or waiting for a consensus to emerge. I sometimes hint in my reviews that the topic may be at the border of my knowledge and if I have a bias. Perhaps that is helpful. Even if the response does not go to the person making the comments, ADs need to see a response. Silence does not help us understand if consensus has been achieved. Last Call is the only point in the development and review of many documents where review from other IETF Areas takes place. It is very important that this cross-Area review take place. Russ ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Lakshminath Dondeti [EMAIL PROTECTED] writes: I have reviewed documents as a Gen-ART reviewer (during Brian's tenure I think), sec-dir reviewer and also provided IETF LC comments on some documents. As a reviewer, I am not sure whether I was expecting answers all those times. I am pretty sure I have not always stated whether or not the answers are satisfactory. On this one point. IMO, one of the biggest causes of problems (and most under-appreciated process weakness) in the IETF (and any consensus based organization for that matter) is poor handling of review comments. The ideal way to deal with them is to always respond, and to get an I am satisfied with your response to close the thread. Anything else runs the risk of: - author says I dealt with those comments in the revised ID, with the reviewer saying Nope, I was ignored. - author says I dealt with those comments in the revised ID, and this actually was the case, except that they accidentally missed one or two important issues. Reviewer is left wondering was I blown off, or was this just an oversight, or... - author thinking they are done, because they responded on the list. But, no changes in the document and/or reviewer is still not happy. - reviewer having invested a not-insignificant amount of time doing a quality review feeling what's the point, which doesn't help to motivate a volunteer organization. This is especially problematic from the cross-area review perspective. Repeat above several times and intersperse with long periods of time where nothing happens on a document. You now have an idea of why it seems to take a long time to get documents through the system. One of the reasons I'm such a fan of issue trackers is that it tends to remove a lot of the above stuff by simply not allowing stuff to fall through the cracks. Sure, trackers have overhead and are overkill in some cases. But if one could somehow analyze the number of documents that have been delayed for some time due to poor handling of review comments... Thomas ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
RE: IONs discuss criteria
Thomas, et al, Please see one (set of) comment(s) below... -- Eric Gray Principal Engineer Ericsson -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Thomas Narten Sent: Friday, March 07, 2008 2:18 PM To: Lakshminath Dondeti Cc: ietf@ietf.org; [EMAIL PROTECTED] Subject: Re: IONs discuss criteria Lakshminath Dondeti [EMAIL PROTECTED] writes: I have reviewed documents as a Gen-ART reviewer (during Brian's tenure I think), sec-dir reviewer and also provided IETF LC comments on some documents. As a reviewer, I am not sure whether I was expecting answers all those times. I am pretty sure I have not always stated whether or not the answers are satisfactory. On this one point. IMO, one of the biggest causes of problems (and most under-appreciated process weakness) in the IETF (and any consensus based organization for that matter) is poor handling of review comments. The ideal way to deal with them is to always respond, and to get an I am satisfied with your response to close the thread. Anything else runs the risk of: - author says I dealt with those comments in the revised ID, with the reviewer saying Nope, I was ignored. - author says I dealt with those comments in the revised ID, and this actually was the case, except that they accidentally missed one or two important issues. Reviewer is left wondering was I blown off, or was this just an oversight, or... - author thinking they are done, because they responded on the list. But, no changes in the document and/or reviewer is still not happy. - reviewer having invested a not-insignificant amount of time doing a quality review feeling what's the point, which doesn't help to motivate a volunteer organization. This is especially problematic from the cross-area review perspective. Actually both of these last points are especially problematic from a generalist review perspective. Those of us doing Gen-ART reviews can not possibly subscribe to every list and it is sometimes (often?) the case that some of the review comments are discussed at great length on the mailing list - completely (though unintentionally) leaving the commenter out of it. In addition to raising the what's the point attitude flag, this can even lead to the feeling that the comments actually went black-hole surfing, possibly even leading to those harrassing trouble tickets for lost E-Mail our IT people love to see so much. This is very disturbing given the non-insignificant amount of effort occasionally involved in just trying to understand a draft enough to make (presumed) intelligent comments about it and not even getting a sense of whether or not you actually succeeded. Minimally, as one of the people to whom that has happened, it would be nice if at least an initial (thanks for the review and comments) mail included the commenter, in every case. Even a I wish you would stop bothering us with all of these silly comments would be a response. Of course, I presonally would prefer that that sort of response was not addressed to the list, with or without me on it. :-) By the way, I agree with the comments below on issue tracker usage, and I feel there must have been some significant, noticeable, improvements in draft processing since they started being used. Perhaps with more (and wider) education on what is available, the gains may be even more appreciable. Repeat above several times and intersperse with long periods of time where nothing happens on a document. You now have an idea of why it seems to take a long time to get documents through the system. One of the reasons I'm such a fan of issue trackers is that it tends to remove a lot of the above stuff by simply not allowing stuff to fall through the cracks. Sure, trackers have overhead and are overkill in some cases. But if one could somehow analyze the number of documents that have been delayed for some time due to poor handling of review comments... Thomas ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted, Firstly, it's not for me to prejudge the IESG's conclusions about IONs, but I would suggest that any ION issued by the IESG implicitly carries the same status as any other IESG statement, unless rescinded, so I don't quite share your concern. However, the deeper question is whether the discuss criteria *should* be promoted to BCP (which effectively binds future IESGs as well as the current IESG). That is worth some discussion. My experience on both sides of the fence is that having the criteria spelled out has been extremely valuable to authors, WGs, reviewers and the IESG itself. I'm a bit less certain whether they should be made binding. Flexibility leaves space for applying common sense. Brian On 2008-03-07 09:01, Ted Hardie wrote: The call for comments on IONs seems to have ended without clarifying the effect of the end of the experiment on the standing of current IONs. For most of them, I honestly don't think the standing is much of an issue. But for the discuss criteria ION, I believe it is a serious issue. At this point, it is difficult to know whether the discuss criteria document is in force or not, and the extent to which the issuing body is bound by it. I think this is a very bad thing. I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious part of our process at this point. Having a community agreed standard to which IESG members could be held was always a better path than than a document approved only by the IESG. Now that the ION experiment is over and the status of its document is in limbo, things are even worse. The current document is here: http://www.ietf.org/IESG/content/ions/ion-discuss-criteria.html for those readers playing the home game. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
At 12:42 PM -0800 3/6/08, Brian E Carpenter wrote: Ted, Firstly, it's not for me to prejudge the IESG's conclusions about IONs, but I would suggest that any ION issued by the IESG implicitly carries the same status as any other IESG statement, unless rescinded, so I don't quite share your concern. I went and looked at RFC 4693 before I posted my first note. It says: If the IESG decides that the feedback warrants terminating the series, the repository will be closed for new documents, and the existing ION documents will be returned to having the same status as any other Web page or file on the IETF servers -- this situation will closely resemble the situation before the experiment started. The status this document had prior to being approved as an ION was Internet draft, which means it had no formal status at all and was followed by the IESG as a matter of lore. However, the deeper question is whether the discuss criteria *should* be promoted to BCP (which effectively binds future IESGs as well as the current IESG). That is worth some discussion. My experience on both sides of the fence is that having the criteria spelled out has been extremely valuable to authors, WGs, reviewers and the IESG itself. I'm a bit less certain whether they should be made binding. Flexibility leaves space for applying common sense. There is no reason for a community-agreed document not to have flexibility. There are strong reasons to make this a community agreed document. Making it something that the community can hold the IESG to, rather than something the IESG can modify by issuing an updated ION, is a critical part of this. Ted Hardie Brian On 2008-03-07 09:01, Ted Hardie wrote: The call for comments on IONs seems to have ended without clarifying the effect of the end of the experiment on the standing of current IONs. For most of them, I honestly don't think the standing is much of an issue. But for the discuss criteria ION, I believe it is a serious issue. At this point, it is difficult to know whether the discuss criteria document is in force or not, and the extent to which the issuing body is bound by it. I think this is a very bad thing. I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious part of our process at this point. Having a community agreed standard to which IESG members could be held was always a better path than than a document approved only by the IESG. Now that the ION experiment is over and the status of its document is in limbo, things are even worse. The current document is here: http://www.ietf.org/IESG/content/ions/ion-discuss-criteria.html for those readers playing the home game. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted, Speaking for myself here but I suspect that other ADs are in the same boat ... I'm keen to make sure my Discusses are within the parameters of the discuss criteria ION regardless of the official status of this document. Agree we need to sort out what we the end result is of several experiments. I believe Russ is working to get that some IESG agenda time. Cullen On Mar 6, 2008, at 12:01 PM, Ted Hardie wrote: The call for comments on IONs seems to have ended without clarifying the effect of the end of the experiment on the standing of current IONs. For most of them, I honestly don't think the standing is much of an issue. But for the discuss criteria ION, I believe it is a serious issue. At this point, it is difficult to know whether the discuss criteria document is in force or not, and the extent to which the issuing body is bound by it. I think this is a very bad thing. I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious part of our process at this point. Having a community agreed standard to which IESG members could be held was always a better path than than a document approved only by the IESG. Now that the ION experiment is over and the status of its document is in limbo, things are even worse. The current document is here: http://www.ietf.org/IESG/content/ions/ion-discuss-criteria.html for those readers playing the home game. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted: The call for comments has resulted in some input, and the IESG plans to discuss that input at our meeting on Sunday. In fact there is also an experiment on mail list suspension that we will be discussing as well. The two experiments are listed on the web page: http://www.ietf.org/IESG/content/experiments.html Once this discussion is over, the future of IONs should be clear, and I will share with the whole IETF community the outcome of the experiment. If IONs are not part of the future, then we need to figure out the best home for each of the things that has been posted as part of the experiment: http://www.ietf.org/IESG/content/ions/ Russ At 03:01 PM 3/6/2008, Ted Hardie wrote: The call for comments on IONs seems to have ended without clarifying the effect of the end of the experiment on the standing of current IONs. For most of them, I honestly don't think the standing is much of an issue. But for the discuss criteria ION, I believe it is a serious issue. At this point, it is difficult to know whether the discuss criteria document is in force or not, and the extent to which the issuing body is bound by it. I think this is a very bad thing. I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious part of our process at this point. Having a community agreed standard to which IESG members could be held was always a better path than than a document approved only by the IESG. Now that the ION experiment is over and the status of its document is in limbo, things are even worse. The current document is here: http://www.ietf.org/IESG/content/ions/ion-discuss-criteria.html for those readers playing the home game. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Cullen == Cullen Jennings [EMAIL PROTECTED] writes: Cullen Ted, Cullen Speaking for myself here but I suspect that other ADs are in the same Cullen boat ... I'm keen to make sure my Discusses are within the parameters Cullen of the discuss criteria ION regardless of the official status of this Cullen document. Agree we need to sort out what we the end result is of Cullen several experiments. I believe Russ is working to get that some IESG Cullen agenda time. Ted, as one of the ADs you have most recently interacted with on the discuss criteria document, I believe that I'm bound by that document and assume it has the same force as an IESG statement. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted: Not oall of the IONs were approved for posting by the IESG. There is one from the IAOC, for example. That was the point of figure out what to do. Russ At 04:01 PM 3/6/2008, Ted Hardie wrote: At 12:42 PM -0800 3/6/08, Brian E Carpenter wrote: Ted, Firstly, it's not for me to prejudge the IESG's conclusions about IONs, but I would suggest that any ION issued by the IESG implicitly carries the same status as any other IESG statement, unless rescinded, so I don't quite share your concern. I went and looked at RFC 4693 before I posted my first note. It says: If the IESG decides that the feedback warrants terminating the series, the repository will be closed for new documents, and the existing ION documents will be returned to having the same status as any other Web page or file on the IETF servers -- this situation will closely resemble the situation before the experiment started. The status this document had prior to being approved as an ION was Internet draft, which means it had no formal status at all and was followed by the IESG as a matter of lore. However, the deeper question is whether the discuss criteria *should* be promoted to BCP (which effectively binds future IESGs as well as the current IESG). That is worth some discussion. My experience on both sides of the fence is that having the criteria spelled out has been extremely valuable to authors, WGs, reviewers and the IESG itself. I'm a bit less certain whether they should be made binding. Flexibility leaves space for applying common sense. There is no reason for a community-agreed document not to have flexibility. There are strong reasons to make this a community agreed document. Making it something that the community can hold the IESG to, rather than something the IESG can modify by issuing an updated ION, is a critical part of this. Ted Hardie Brian On 2008-03-07 09:01, Ted Hardie wrote: The call for comments on IONs seems to have ended without clarifying the effect of the end of the experiment on the standing of current IONs. For most of them, I honestly don't think the standing is much of an issue. But for the discuss criteria ION, I believe it is a serious issue. At this point, it is difficult to know whether the discuss criteria document is in force or not, and the extent to which the issuing body is bound by it. I think this is a very bad thing. I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious part of our process at this point. Having a community agreed standard to which IESG members could be held was always a better path than than a document approved only by the IESG. Now that the ION experiment is over and the status of its document is in limbo, things are even worse. The current document is here: http://www.ietf.org/IESG/content/ions/ion-discuss-criteria.html for those readers playing the home game. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
--On Thursday, 06 March, 2008 12:01 -0800 Ted Hardie [EMAIL PROTECTED] wrote: I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious part of our process at this point. Having a community agreed standard to which IESG members could be held was always a better path than than a document approved only by the IESG. Now that the ION experiment is over and the status of its document is in limbo, things are even worse. The current document is here: http://www.ietf.org/IESG/content/ions/ion-discuss-criteria.html for those readers playing the home game. Hmm. If people believe that this document should be processed as a BCP, thereby presumably constraining long-term IESG behavior and adding to our procedural core, should it be added to the PUFI agenda for preliminary discussion? john ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
At 12:52 PM -0800 3/6/08, Russ Housley wrote: Once this discussion is over, the future of IONs should be clear, and I will share with the whole IETF community the outcome of the experiment. Russ, Whatever the fate of IONs in general, it is clear to me that this document does not belong in the series. It needs community consensus that binds the IESG and the rest of the community; as a statement of the IESG (and which is subject to update by the IESG) it does not have the force it needs. At the moment it is completely lacking in such force. I look forward to the IESG statements on the series as a whole. But please understand that if the IESG indicates an intent to retain this document at the end of that in this series I would appeal. I respect your work, but I believe the IESG has recently relaxed the vigilance it once held toward adherence to these criteria. I have seen at least two recent discusses that amounted to go satisfy that guy and several cut and paste external reviews where it was blindingly obvious the AD had not even looked at the most recent version of the text. I have also seen quite a few that amount to Disagreement with informed working group decisions where the AD is putting their preferences over any real acknowledgement that a working group has considered the issues. The only way I know of to make sure the IESG restores the focus on this issue (which took a lot of our energy several years ago) is to make it binding on the IESG. I hope that you, personally, agree that it should be community-based and binding on the IESG and that we are simply discussing the mechanism by which that occurs. If you do not agree that it should be binding on the IESG and a consensus statement of the community, I am interested to know why. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Cullen, Thank you for your statement that you are keen to make sure your DISCUSSes are within the parameters of the discuss criteria ION. I appreciate it. Perhaps I am naive or my understanding of the English language is poor (they are both probably true), but could you explain how one of your most recent DISCUSSes: Cullen Jennings: Discuss [2008-03-05]: There has been a lot of discussion about keying modes for SRTP, so I'm glad to see a document that covers this topic for MIKEY. For that reason, I think it's really important to get this right. It looks to me like some of the issues EKR raises need to be fixed in order to achieve that. does not fit into the DISCUSS non-criteria? Unfiltered external party reviews. While an AD is welcome to consult with external parties, the AD is expected to evaluate, to understand and to concur with issues raised by external parties. Blindly cut-and-pasting an external party review into a DISCUSS is inappropriate if the AD is unable to defend or substantiate the issues raised in the review. You chose to not even cut-and-paste the comments. I also wonder which of the DISCUSS criteria fit to advance that specific document to an informational RFC. Are we to guess which of the the issues EKR raises the authors need to fix? Needless to say, we don't need to debate the specifics of that document here, but whereas your intent is honorable, the externally observable behavior is unfortunately different. Sorry for picking on you; I can probably find other similar examples on other ADs, but I was thinking that the ION is not a BCP and so I have no basis to raise the issue. Perhaps I am misunderstanding the ION or perhaps as Brian says that there should be some flexibility in the application of the rules. regards, Lakshminath On 3/6/2008 1:04 PM, Cullen Jennings wrote: Ted, Speaking for myself here but I suspect that other ADs are in the same boat ... I'm keen to make sure my Discusses are within the parameters of the discuss criteria ION regardless of the official status of this document. Agree we need to sort out what we the end result is of several experiments. I believe Russ is working to get that some IESG agenda time. Cullen On Mar 6, 2008, at 12:01 PM, Ted Hardie wrote: The call for comments on IONs seems to have ended without clarifying the effect of the end of the experiment on the standing of current IONs. For most of them, I honestly don't think the standing is much of an issue. But for the discuss criteria ION, I believe it is a serious issue. At this point, it is difficult to know whether the discuss criteria document is in force or not, and the extent to which the issuing body is bound by it. I think this is a very bad thing. I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious part of our process at this point. Having a community agreed standard to which IESG members could be held was always a better path than than a document approved only by the IESG. Now that the ION experiment is over and the status of its document is in limbo, things are even worse. The current document is here: http://www.ietf.org/IESG/content/ions/ion-discuss-criteria.html for those readers playing the home game. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted == Ted Hardie [EMAIL PROTECTED] writes: Ted I respect your work, but I believe the IESG has recently Ted relaxed the vigilance it once held toward adherence to these criteria. Ted I have seen at least two recent discusses that amounted to Ted go satisfy that guy and several cut and paste external reviews where Ted it was blindingly obvious the AD had not even looked at the Ted most recent version of the text. I have also seen quite a few Ted that amount to Disagreement with informed working group Ted decisions where the AD is putting their preferences over Ted any real acknowledgement that a working group has considered Ted the issues. Ted The only way I know of to make sure the IESG restores Ted the focus on this issue (which took a lot of our energy several years Ted ago) is to make it binding on the IESG. I hope that you, personally, Ted agree that it should be community-based and binding on the IESG Ted and that we are simply discussing the mechanism by which that Ted occurs. If you do not agree that it should be binding on the IESG Ted and a consensus statement of the community, I am interested to know Ted why. Ted Ted Hardie If someone believes that a discuss is inappropriate, I recommend that they start both by contacting the discussing AD *and* the shepherding AD. I know that I would treat a request to rethink whether a discuss I held was consistent with the discuss criteria document from another IESG member very seriously. I would treat such a request from an author seriously, although not as seriously as from another IESG member. There are a number of reasons for this. The authors are often much more emotionally involved in a document than ADs. The ADs are likely more familiar with the discuss criteria than the author and are definitely more familiar with the current interpretation of the discuss criteria. No matter how frustrating it is, it's simply true that procedures like the discuss criteria are subject to interpretation, and the IESG is going to have an evolving interpretation of any such procedural document. I think it is reasonable for an author to expect to get a response back from an shepherding AD that either they think the discuss is reasonable, or they think it is unreasonable. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
At Thu, 06 Mar 2008 13:35:04 -0800, Lakshminath Dondeti wrote: Cullen, Thank you for your statement that you are keen to make sure your DISCUSSes are within the parameters of the discuss criteria ION. I appreciate it. Perhaps I am naive or my understanding of the English language is poor (they are both probably true), but could you explain how one of your most recent DISCUSSes: Cullen Jennings: Discuss [2008-03-05]: There has been a lot of discussion about keying modes for SRTP, so I'm glad to see a document that covers this topic for MIKEY. For that reason, I think it's really important to get this right. It looks to me like some of the issues EKR raises need to be fixed in order to achieve that. does not fit into the DISCUSS non-criteria? Unfiltered external party reviews. While an AD is welcome to consult with external parties, the AD is expected to evaluate, to understand and to concur with issues raised by external parties. Blindly cut-and-pasting an external party review into a DISCUSS is inappropriate if the AD is unable to defend or substantiate the issues raised in the review. You chose to not even cut-and-paste the comments. Doesn't this fall into the category of evaluate and concur? I also wonder which of the DISCUSS criteria fit to advance that specific document to an informational RFC. Are we to guess which of the the issues EKR raises the authors need to fix? If only some other area director had excerpted my review and identified the sections that he felt most clearly needed correction. Oh, wait, Sam did. -Ekr ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Lakshminath == Lakshminath Dondeti [EMAIL PROTECTED] writes: Lakshminath Cullen, Lakshminath Thank you for your statement that you are keen to make sure your Lakshminath DISCUSSes are within the parameters of the discuss criteria ION. I Lakshminath appreciate it. Perhaps I am naive or my understanding of the English Lakshminath language is poor (they are both probably true), but could you explain Lakshminath how one of your most recent DISCUSSes: Lakshminath Cullen Jennings: Lakshminath Discuss [2008-03-05]: Lakshminath There has been a lot of discussion about keying modes for Lakshminath SRTP, so I'm glad to see a document that covers this topic Lakshminath for MIKEY. For that reason, I think it's really important Lakshminath to get this right. It looks to me like some of the issues Lakshminath EKR raises need to be fixed in order to achieve that. Presumably Cullen is agreeing with the discuss position that I'm holding and that Russ is holding. If Cullen plans to hold his discuss position past the resolution of Russ's discuss (Russ has agreed to take on mine), then I agree his discuss is inappropriate. I'm not sure that Cullen made the best use of the tool, but I'm not sure he did anything wrong either. I believe that my discuss is consistent with the discuss criteria because while it is based on an external review, I've explained what parts of the review I consider blocking. I haven't read Russ's discuss. I believe that if he selected what parts of the review he considers are a valid discuss,or if he simply asked you to respond to Eric's comments (saying that he believes last call discussion is still ongoing), then it is a valid discuss. The second discuss (please respond to Eric and conclude the last call discussion) is a process discuss not a content discuss; he would be asking you to actually engage in a discussion. If he later believed that Tim had incorrectly evaluated the consensus of that discussion, he might change his position to another process discuss about a consensus problem. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Sam, I fail to understand why this has to be a guessing game. I also don't understand the argument about resolving DISCUSSes sequentially (in reference to your point about Cullen holding his DISCUSS beyond resolution of Russ's). I have seen better examples where for instance your DISCUSS quotes what you think needs to be resolved from the external review. Likewise, Russ states what needs to be resolved from the external review. Recently Tim put a DISCUSS on another document based on my comments stating what his specific concerns are. In all those cases, it is at least somewhat clear as to what the AD might want. In closing, perhaps some of us would like to be behind a DISCUSS that states resolve some of the issues of a 3rd party. I for one find it very hard to work in that kind of an environment. From my view point, here is how the process looks: First we have to beg to know what the issues are, then propose text, only to have it dismissed after some time, propose text again, repeat that for a non-deterministic number of times and eventually hope that the AD is satisfied; repeat that for the next AD, and so on. That is seriously broken!! All I am asking for though is to just remove the first step for starters. We shouldn't have to ask to know what the DISCUSS is about. best regards, Lakshminath On 3/6/2008 1:51 PM, Sam Hartman wrote: Lakshminath == Lakshminath Dondeti [EMAIL PROTECTED] writes: Lakshminath Cullen, Lakshminath Thank you for your statement that you are keen to make sure your Lakshminath DISCUSSes are within the parameters of the discuss criteria ION. I Lakshminath appreciate it. Perhaps I am naive or my understanding of the English Lakshminath language is poor (they are both probably true), but could you explain Lakshminath how one of your most recent DISCUSSes: Lakshminath Cullen Jennings: Lakshminath Discuss [2008-03-05]: Lakshminath There has been a lot of discussion about keying modes for Lakshminath SRTP, so I'm glad to see a document that covers this topic Lakshminath for MIKEY. For that reason, I think it's really important Lakshminath to get this right. It looks to me like some of the issues Lakshminath EKR raises need to be fixed in order to achieve that. Presumably Cullen is agreeing with the discuss position that I'm holding and that Russ is holding. If Cullen plans to hold his discuss position past the resolution of Russ's discuss (Russ has agreed to take on mine), then I agree his discuss is inappropriate. I'm not sure that Cullen made the best use of the tool, but I'm not sure he did anything wrong either. I believe that my discuss is consistent with the discuss criteria because while it is based on an external review, I've explained what parts of the review I consider blocking. I haven't read Russ's discuss. I believe that if he selected what parts of the review he considers are a valid discuss,or if he simply asked you to respond to Eric's comments (saying that he believes last call discussion is still ongoing), then it is a valid discuss. The second discuss (please respond to Eric and conclude the last call discussion) is a process discuss not a content discuss; he would be asking you to actually engage in a discussion. If he later believed that Tim had incorrectly evaluated the consensus of that discussion, he might change his position to another process discuss about a consensus problem. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Part of the reason I replied so quickly on this thread is that I think I currently have two discuss that do not meet the discuss criteria (this being one of them the other being on Lost). Totally fair to pick on me here. Both were entered as, excuse the pun, fairly fluffy comments because I believe fully stating each point by point things would have actually make it significantly harder for the editor of the documents to find a good solution. In both cases I believe the editor pretty much understands the issue and if I get requests to please rewrite to be a reasonable discuss, I'm glad to do that after the meeting. On Mar 6, 2008, at 1:35 PM, Lakshminath Dondeti wrote: Cullen, Thank you for your statement that you are keen to make sure your DISCUSSes are within the parameters of the discuss criteria ION. I appreciate it. Perhaps I am naive or my understanding of the English language is poor (they are both probably true), but could you explain how one of your most recent DISCUSSes: Cullen Jennings: Discuss [2008-03-05]: There has been a lot of discussion about keying modes for SRTP, so I'm glad to see a document that covers this topic for MIKEY. For that reason, I think it's really important to get this right. It looks to me like some of the issues EKR raises need to be fixed in order to achieve that. does not fit into the DISCUSS non-criteria? Unfiltered external party reviews. While an AD is welcome to consult with external parties, the AD is expected to evaluate, to understand and to concur with issues raised by external parties. Blindly cut-and-pasting an external party review into a DISCUSS is inappropriate if the AD is unable to defend or substantiate the issues raised in the review. You chose to not even cut-and-paste the comments. I also wonder which of the DISCUSS criteria fit to advance that specific document to an informational RFC. Are we to guess which of the the issues EKR raises the authors need to fix? Needless to say, we don't need to debate the specifics of that document here, but whereas your intent is honorable, the externally observable behavior is unfortunately different. Sorry for picking on you; I can probably find other similar examples on other ADs, but I was thinking that the ION is not a BCP and so I have no basis to raise the issue. Perhaps I am misunderstanding the ION or perhaps as Brian says that there should be some flexibility in the application of the rules. regards, Lakshminath On 3/6/2008 1:04 PM, Cullen Jennings wrote: Ted, Speaking for myself here but I suspect that other ADs are in the same boat ... I'm keen to make sure my Discusses are within the parameters of the discuss criteria ION regardless of the official status of this document. Agree we need to sort out what we the end result is of several experiments. I believe Russ is working to get that some IESG agenda time. Cullen On Mar 6, 2008, at 12:01 PM, Ted Hardie wrote: The call for comments on IONs seems to have ended without clarifying the effect of the end of the experiment on the standing of current IONs. For most of them, I honestly don't think the standing is much of an issue. But for the discuss criteria ION, I believe it is a serious issue. At this point, it is difficult to know whether the discuss criteria document is in force or not, and the extent to which the issuing body is bound by it. I think this is a very bad thing. I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious part of our process at this point. Having a community agreed standard to which IESG members could be held was always a better path than than a document approved only by the IESG. Now that the ION experiment is over and the status of its document is in limbo, things are even worse. The current document is here: http://www.ietf.org/IESG/content/ions/ion-discuss-criteria.html for those readers playing the home game. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Thanks for the clarification Cullen. I appreciate it. Speaking from the view point of someone on the other side, more often than not, a detailed DISCUSS is much more helpful. Thank you again. best wishes, Lakshminath On 3/6/2008 2:23 PM, Cullen Jennings wrote: Part of the reason I replied so quickly on this thread is that I think I currently have two discuss that do not meet the discuss criteria (this being one of them the other being on Lost). Totally fair to pick on me here. Both were entered as, excuse the pun, fairly fluffy comments because I believe fully stating each point by point things would have actually make it significantly harder for the editor of the documents to find a good solution. In both cases I believe the editor pretty much understands the issue and if I get requests to please rewrite to be a reasonable discuss, I'm glad to do that after the meeting. On Mar 6, 2008, at 1:35 PM, Lakshminath Dondeti wrote: Cullen, Thank you for your statement that you are keen to make sure your DISCUSSes are within the parameters of the discuss criteria ION. I appreciate it. Perhaps I am naive or my understanding of the English language is poor (they are both probably true), but could you explain how one of your most recent DISCUSSes: Cullen Jennings: Discuss [2008-03-05]: There has been a lot of discussion about keying modes for SRTP, so I'm glad to see a document that covers this topic for MIKEY. For that reason, I think it's really important to get this right. It looks to me like some of the issues EKR raises need to be fixed in order to achieve that. does not fit into the DISCUSS non-criteria? Unfiltered external party reviews. While an AD is welcome to consult with external parties, the AD is expected to evaluate, to understand and to concur with issues raised by external parties. Blindly cut-and-pasting an external party review into a DISCUSS is inappropriate if the AD is unable to defend or substantiate the issues raised in the review. You chose to not even cut-and-paste the comments. I also wonder which of the DISCUSS criteria fit to advance that specific document to an informational RFC. Are we to guess which of the the issues EKR raises the authors need to fix? Needless to say, we don't need to debate the specifics of that document here, but whereas your intent is honorable, the externally observable behavior is unfortunately different. Sorry for picking on you; I can probably find other similar examples on other ADs, but I was thinking that the ION is not a BCP and so I have no basis to raise the issue. Perhaps I am misunderstanding the ION or perhaps as Brian says that there should be some flexibility in the application of the rules. regards, Lakshminath On 3/6/2008 1:04 PM, Cullen Jennings wrote: Ted, Speaking for myself here but I suspect that other ADs are in the same boat ... I'm keen to make sure my Discusses are within the parameters of the discuss criteria ION regardless of the official status of this document. Agree we need to sort out what we the end result is of several experiments. I believe Russ is working to get that some IESG agenda time. Cullen On Mar 6, 2008, at 12:01 PM, Ted Hardie wrote: The call for comments on IONs seems to have ended without clarifying the effect of the end of the experiment on the standing of current IONs. For most of them, I honestly don't think the standing is much of an issue. But for the discuss criteria ION, I believe it is a serious issue. At this point, it is difficult to know whether the discuss criteria document is in force or not, and the extent to which the issuing body is bound by it. I think this is a very bad thing. I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious part of our process at this point. Having a community agreed standard to which IESG members could be held was always a better path than than a document approved only by the IESG. Now that the ION experiment is over and the status of its document is in limbo, things are even worse. The current document is here: http://www.ietf.org/IESG/content/ions/ion-discuss-criteria.html for those readers playing the home game. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Lakshminath == Lakshminath Dondeti [EMAIL PROTECTED] writes: Lakshminath Sam, Lakshminath I fail to understand why this has to be a guessing game. I also don't Lakshminath understand the argument about resolving DISCUSSes sequentially (in Lakshminath reference to your point about Cullen holding his DISCUSS beyond Lakshminath resolution of Russ's). I guess I was unclear. I think it's reasonable for Cullen to say I agree with that other discuss, and that's how I interpret his current position. I think it's kind of odd for him to stick that in the discuss box rather than the comment box, but I don't think it is particularly harmful provided that his discuss never blocks the document. I.E. he needs to make sure his discuss is removed before Russ clears. Put another way, it's fine for Cullen to tell other IESG members that he agrees with a discuss. It's fine for him to agree so strongly that he'd like to be given an opportunity to take on the discuss if for example the person holding the discuss gives up and wants to drop the issue. It's not fine for him to expect you to do anything based on a discuss that vague. It's not fine for his inaction to cause your document to get stuck based on a discuss that vague. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On 3/6/08 at 4:24 PM -0500, John C Klensin wrote: Hmm. If people believe that this document should be processed as a BCP, thereby presumably constraining long-term IESG behavior and adding to our procedural core, should it be added to the PUFI agenda for preliminary discussion? The PUFI BOF chair, who has not completed his list of currently desired items on the grand list of things to cover in this BOF, hereby groans at the thought of adding another. pr -- Pete Resnick http://www.qualcomm.com/~presnick/ Qualcomm Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102 ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
At 2:23 PM -0800 3/6/08, Cullen Jennings wrote: Part of the reason I replied so quickly on this thread is that I think I currently have two discuss that do not meet the discuss criteria (this being one of them the other being on Lost). Totally fair to pick on me here. Both were entered as, excuse the pun, fairly fluffy comments because I believe fully stating each point by point things would have actually make it significantly harder for the editor of the documents to find a good solution. In both cases I believe the editor pretty much understands the issue and if I get requests to please rewrite to be a reasonable discuss, I'm glad to do that after the meeting. Speaking as one of the editors of LoST, and not as a general statement on this problem, a clear discuss from the beginning would have been helpful. I have been willing to have phone calls, conduct a series of email exchanges, and chat via IM on this; I want this document to be right. But the amount of effort you are requiring from the authors to work towards a statement of the problem, much less a solution, has been pretty significant: 10 emails from me alone on your issues alone, as we worked toward an understanding of the issue you have. That boiled down to a DISCUSS that should be as simple as In non-emergency use cases, I am concerned that the provisions of 12.2 allow servers to use algorithms that will needlessly return null results. Please add guidance for non-emergency use cases. Speaking again as someone who thinks this is general problem, the issue I am raising is not that there are bad discusses. The issue I am raising is that the document which describes what discusses are or should be has no force at the moment at all, and should be a community document rather than a statement of the body which may hold discusses. Only the latter allows the community to hold the IESG accountable adequately. regards, Ted Hardie On Mar 6, 2008, at 1:35 PM, Lakshminath Dondeti wrote: Cullen, Thank you for your statement that you are keen to make sure your DISCUSSes are within the parameters of the discuss criteria ION. I appreciate it. Perhaps I am naive or my understanding of the English language is poor (they are both probably true), but could you explain how one of your most recent DISCUSSes: Cullen Jennings: Discuss [2008-03-05]: There has been a lot of discussion about keying modes for SRTP, so I'm glad to see a document that covers this topic for MIKEY. For that reason, I think it's really important to get this right. It looks to me like some of the issues EKR raises need to be fixed in order to achieve that. does not fit into the DISCUSS non-criteria? Unfiltered external party reviews. While an AD is welcome to consult with external parties, the AD is expected to evaluate, to understand and to concur with issues raised by external parties. Blindly cut-and-pasting an external party review into a DISCUSS is inappropriate if the AD is unable to defend or substantiate the issues raised in the review. You chose to not even cut-and-paste the comments. I also wonder which of the DISCUSS criteria fit to advance that specific document to an informational RFC. Are we to guess which of the the issues EKR raises the authors need to fix? Needless to say, we don't need to debate the specifics of that document here, but whereas your intent is honorable, the externally observable behavior is unfortunately different. Sorry for picking on you; I can probably find other similar examples on other ADs, but I was thinking that the ION is not a BCP and so I have no basis to raise the issue. Perhaps I am misunderstanding the ION or perhaps as Brian says that there should be some flexibility in the application of the rules. regards, Lakshminath On 3/6/2008 1:04 PM, Cullen Jennings wrote: Ted, Speaking for myself here but I suspect that other ADs are in the same boat ... I'm keen to make sure my Discusses are within the parameters of the discuss criteria ION regardless of the official status of this document. Agree we need to sort out what we the end result is of several experiments. I believe Russ is working to get that some IESG agenda time. Cullen On Mar 6, 2008, at 12:01 PM, Ted Hardie wrote: The call for comments on IONs seems to have ended without clarifying the effect of the end of the experiment on the standing of current IONs. For most of them, I honestly don't think the standing is much of an issue. But for the discuss criteria ION, I believe it is a serious issue. At this point, it is difficult to know whether the discuss criteria document is in force or not, and the extent to which the issuing body is bound by it. I think this is a very bad thing. I call on Russ to restore this document to its original status as an Internet Draft and to process it as a BCP. IESG DISCUSSes are a very serious
Re: IONs discuss criteria
Sam, There is no need to prolong this particular side of the discussion now that Cullen clarified his position. But, I have to say that this thread is but one example that we often don't clearly understand each other's positions. You interpret Cullen's DISCUSS as : I think it's reasonable for Cullen to say I agree with that other discuss, and that's how I interpret his current position. Cullen clarifies it as: I believe the editor pretty much understands the issue and if I get requests to please rewrite to be a reasonable discuss, I'm glad to do that after the meeting. Most of the IESG members' names have four letters or less :). It is not very hard to type Agree with even if someone is in a hurry. Next, I can't read Steffen and Dragan's minds, and so I don't know what their understanding of the issue is and whether they understand it as Cullen agreeing with the other discuss or something else. At this point, we have that additional step of saying please rewrite your discuss to be a reasonable discuss. It looks like my interpretation was right that I have to beg for clarification to go forward here. best, Lakshminath On 3/6/2008 2:32 PM, Sam Hartman wrote: Lakshminath == Lakshminath Dondeti [EMAIL PROTECTED] writes: Lakshminath Sam, Lakshminath I fail to understand why this has to be a guessing game. I also don't Lakshminath understand the argument about resolving DISCUSSes sequentially (in Lakshminath reference to your point about Cullen holding his DISCUSS beyond Lakshminath resolution of Russ's). I guess I was unclear. I think it's reasonable for Cullen to say I agree with that other discuss, and that's how I interpret his current position. I think it's kind of odd for him to stick that in the discuss box rather than the comment box, but I don't think it is particularly harmful provided that his discuss never blocks the document. I.E. he needs to make sure his discuss is removed before Russ clears. Put another way, it's fine for Cullen to tell other IESG members that he agrees with a discuss. It's fine for him to agree so strongly that he'd like to be given an opportunity to take on the discuss if for example the person holding the discuss gives up and wants to drop the issue. It's not fine for him to expect you to do anything based on a discuss that vague. It's not fine for his inaction to cause your document to get stuck based on a discuss that vague. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted == Ted Hardie [EMAIL PROTECTED] writes: Ted Speaking again as someone who thinks this is general problem, the Ted issue I am raising is not that there are bad discusses. The issue I Ted am raising is that the document which describes what discusses Ted are or should be has no force at the moment at all, Ted, I'd like to disagree with this point. I believe that you could appeal a discuss because it does not meet the discuss criteria. I believe you could ask the iesg as a body to evaluate whether a discuss fit the criteria. If you did appeal, I believe you could carry the appeal to the IAB; I believe that they would conclude that the IESG has chosen to bind itself to the discuss criteria document. There is some complexity. The IESG could in theory use some other mechanism rather than its balloting procedure to reach consensus on what to do with a document. Especially if that consensus were strong, I think it would be reasonable for the IESG to do that. In my time on the IESG that's never happened; I don't see it starting soon. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
At 1:43 PM -0800 3/6/08, Sam Hartman wrote: I know that I would treat a request to rethink whether a discuss I held was consistent with the discuss criteria document from another IESG member very seriously. I would treat such a request from an author seriously, although not as seriously as from another IESG member. There are a number of reasons for this. The authors are often much more emotionally involved in a document than ADs. The ADs are likely more familiar with the discuss criteria than the author and are definitely more familiar with the current interpretation of the discuss criteria. No matter how frustrating it is, it's simply true that procedures like the discuss criteria are subject to interpretation, and the IESG is going to have an evolving interpretation of any such procedural document. I can think of no stronger statement of why the discuss criteria document should be a community statement that represents the agreement of the IETF as a whole than that above. If the IESG can have an evolving interpretation of the criteria, then the community needs to know it and agree to the interpretation. That interpretation being evolved without community input in the context of an inward-looking IESG is not appropriate. And an inward-looking IESG is exactly what is described, with the trust given to IESG members being greater simply because they share that common context. As a side note, this view seems to relegate the document shepherd's role to invisible friend, something I would press further on if Sam were not so immanently leaving the IESG. Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
John C Klensin wrote: Hmm. If people believe that this document should be processed as a BCP, thereby presumably constraining long-term IESG behavior and adding to our procedural core, should it be added to the PUFI agenda for preliminary discussion? Yes. A series of postings by sitting area directors about their commitment to following a document says nothing about the commitment of any future area director. If the document is merely a reference to be used internally by the IESG, then it needs no formal standing. If the document is meant as formal criteria to ensure transparency and accountability of the IESG, then it needs formal standing, which means formal adoption by the IETF community. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted == Ted Hardie [EMAIL PROTECTED] writes: Ted At 1:43 PM -0800 3/6/08, Sam Hartman wrote: I know that I would treat a request to rethink whether a discuss I held was consistent with the discuss criteria document from another IESG member very seriously. I would treat such a request from an author seriously, although not as seriously as from another IESG member. There are a number of reasons for this. The authors are often much more emotionally involved in a document than ADs. The ADs are likely more familiar with the discuss criteria than the author and are definitely more familiar with the current interpretation of the discuss criteria. No matter how frustrating it is, it's simply true that procedures like the discuss criteria are subject to interpretation, and the IESG is going to have an evolving interpretation of any such procedural document. Ted I can think of no stronger statement of why the discuss criteria Ted document should be a community statement that represents the Ted agreement of the IETF as a whole than that above. If the IESG Ted can have an evolving interpretation of the criteria, then the Ted community needs to know it and agree to the interpretation. Ted, we have an evolving interpretation of every document we've ever written--BCPs, standards, IONs, webpages, email messages, presentations. I don't mean that the iesg should be able to interpret the discuss criteria document in a manner inconsistent with the text. however there are a lot of ways to read the text. Making it a BCP will make the interpretation problem worse not better. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. How? d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Dave, On 2008-03-07 12:34, Dave Crocker wrote: Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. How? To some extent that depends on how carefully the putative BCP is crafted, with should and when to disregard should being very precise. What I think we've seen, with 2026 over the years, and apparently this year with 3777, is that it's virtually impossible to write precise procedural text that deals with completely unexpected circumstances. Yet if the text has the force of a BCP, it becomes very hard to interpret it flexibly when flexibility is clearly needed. I don't know if that is Sam's point, of course. Brian ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Ted, Lakshminath, and the Rest of the IETF Community: I fail to understand why this has to be a guessing game. The handling of reviews by non-IESG members seems to be an important part of this discussion. So, I'll tell everyone how I deal with Gen-ART Reviews. Other General ADs may have done things slightly different. When I use a Gen-ART Review as the basis of a DISCUSS, I put it in one of two categories. (1) The Gen-ART Review was ignored. Like any other Last Call comment, it deserves an answer. So, this is a procedural objection. In this situation, I've been careful to say that the authors do not need to accept all of the comments, but then need to answer them. (2) I agree with one or more concerns raised in the Gen-ART Review that has not been resolved. I often break the unresolved review comments into DISCUSS and COMMENT. AD judgement is needed, and I consider the DISCUSS Criteria in making that judgement. Russ ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Hi Russ, Thanks for your response. Some notes inline: On 3/6/2008 4:09 PM, Russ Housley wrote: Ted, Lakshminath, and the Rest of the IETF Community: I fail to understand why this has to be a guessing game. The handling of reviews by non-IESG members seems to be an important part of this discussion. I agree and have contributed to that part of the solution, only now I am realizing that it may be becoming part of the problem, so to speak. The concern is that over time it seems to be degenerating into, I will use Ted's phrase here because that is what it feels like, go satisfy that guy. Consider how it sounds when trying to explain to an outsider: the document is held up in IESG processing because X, who is not an IESG or IAB member, does not like it. I think your own word is answer (I have heard respond) in lieu of satisfy. Some notes on that below: So, I'll tell everyone how I deal with Gen-ART Reviews. Other General ADs may have done things slightly different. When I use a Gen-ART Review as the basis of a DISCUSS, I put it in one of two categories. (1) The Gen-ART Review was ignored. Like any other Last Call comment, it deserves an answer. So, this is a procedural objection. In this situation, I've been careful to say that the authors do not need to accept all of the comments, but then need to answer them. I have reviewed documents as a Gen-ART reviewer (during Brian's tenure I think), sec-dir reviewer and also provided IETF LC comments on some documents. As a reviewer, I am not sure whether I was expecting answers all those times. I am pretty sure I have not always stated whether or not the answers are satisfactory. Next, I can imagine an author not wanting to respond to something I may have said because it was totally bogus or inappropriate and does not deserve a response. That might very well happen when I review documents on a topic that I am not familiar with and haven't had the time to read related references (that varies depending on the time available, etc.). Perhaps that is not such a bad thing; being blissfully ignorant on some topics keeps me, well, blissful. I use somewhat of a hyperbole for obvious reasons. I am sure many other situations are much more nuanced. I hope ADs don't continue to hold a DISCUSS in those situations waiting for a dialog to take place or waiting for a consensus to emerge. I sometimes hint in my reviews that the topic may be at the border of my knowledge and if I have a bias. Perhaps that is helpful. regards, Lakshminath (2) I agree with one or more concerns raised in the Gen-ART Review that has not been resolved. I often break the unresolved review comments into DISCUSS and COMMENT. AD judgement is needed, and I consider the DISCUSS Criteria in making that judgement. Russ ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
I believe Sam's discuss cover the issues I was concerned about and I have removed my discuss. On Mar 6, 2008, at 2:57 PM, Lakshminath Dondeti wrote: Sam, There is no need to prolong this particular side of the discussion now that Cullen clarified his position. But, I have to say that this thread is but one example that we often don't clearly understand each other's positions. You interpret Cullen's DISCUSS as : I think it's reasonable for Cullen to say I agree with that other discuss, and that's how I interpret his current position. Cullen clarifies it as: I believe the editor pretty much understands the issue and if I get requests to please rewrite to be a reasonable discuss, I'm glad to do that after the meeting. Most of the IESG members' names have four letters or less :). It is not very hard to type Agree with even if someone is in a hurry. Next, I can't read Steffen and Dragan's minds, and so I don't know what their understanding of the issue is and whether they understand it as Cullen agreeing with the other discuss or something else. At this point, we have that additional step of saying please rewrite your discuss to be a reasonable discuss. It looks like my interpretation was right that I have to beg for clarification to go forward here. best, Lakshminath On 3/6/2008 2:32 PM, Sam Hartman wrote: Lakshminath == Lakshminath Dondeti [EMAIL PROTECTED] writes: Lakshminath Sam, Lakshminath I fail to understand why this has to be a guessing game. I also don't Lakshminath understand the argument about resolving DISCUSSes sequentially (in Lakshminath reference to your point about Cullen holding his DISCUSS beyond Lakshminath resolution of Russ's). I guess I was unclear. I think it's reasonable for Cullen to say I agree with that other discuss, and that's how I interpret his current position. I think it's kind of odd for him to stick that in the discuss box rather than the comment box, but I don't think it is particularly harmful provided that his discuss never blocks the document. I.E. he needs to make sure his discuss is removed before Russ clears. Put another way, it's fine for Cullen to tell other IESG members that he agrees with a discuss. It's fine for him to agree so strongly that he'd like to be given an opportunity to take on the discuss if for example the person holding the discuss gives up and wants to drop the issue. It's not fine for him to expect you to do anything based on a discuss that vague. It's not fine for his inaction to cause your document to get stuck based on a discuss that vague. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Brian, A small clarification below on the reference to the interpretation problems related to 3777: On 3/6/2008 4:10 PM, Brian E Carpenter wrote: Dave, On 2008-03-07 12:34, Dave Crocker wrote: Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. How? To some extent that depends on how carefully the putative BCP is crafted, with should and when to disregard should being very precise. What I think we've seen, with 2026 over the years, and apparently this year with 3777, is that it's virtually I am not sure whether you have made it to the appendix in my report, but the disagreements in interpretation of 3777 have a history (see Page 37). The only thing special about the current nomcom is that we chose to bring it to the community's attention. In Ralph's case, he brought it to the IESG and IAB's attention in March 2006. thanks, Lakshminath Nomcom 2007-8 Chair impossible to write precise procedural text that deals with completely unexpected circumstances. Yet if the text has the force of a BCP, it becomes very hard to interpret it flexibly when flexibility is clearly needed. I don't know if that is Sam's point, of course. Brian ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Thanks Cullen. regards, Lakshminath On 3/6/2008 5:05 PM, Cullen Jennings wrote: I believe Sam's discuss cover the issues I was concerned about and I have removed my discuss. On Mar 6, 2008, at 2:57 PM, Lakshminath Dondeti wrote: Sam, There is no need to prolong this particular side of the discussion now that Cullen clarified his position. But, I have to say that this thread is but one example that we often don't clearly understand each other's positions. You interpret Cullen's DISCUSS as : I think it's reasonable for Cullen to say I agree with that other discuss, and that's how I interpret his current position. Cullen clarifies it as: I believe the editor pretty much understands the issue and if I get requests to please rewrite to be a reasonable discuss, I'm glad to do that after the meeting. Most of the IESG members' names have four letters or less :). It is not very hard to type Agree with even if someone is in a hurry. Next, I can't read Steffen and Dragan's minds, and so I don't know what their understanding of the issue is and whether they understand it as Cullen agreeing with the other discuss or something else. At this point, we have that additional step of saying please rewrite your discuss to be a reasonable discuss. It looks like my interpretation was right that I have to beg for clarification to go forward here. best, Lakshminath On 3/6/2008 2:32 PM, Sam Hartman wrote: Lakshminath == Lakshminath Dondeti [EMAIL PROTECTED] writes: Lakshminath Sam, Lakshminath I fail to understand why this has to be a guessing game. I also don't Lakshminath understand the argument about resolving DISCUSSes sequentially (in Lakshminath reference to your point about Cullen holding his DISCUSS beyond Lakshminath resolution of Russ's). I guess I was unclear. I think it's reasonable for Cullen to say I agree with that other discuss, and that's how I interpret his current position. I think it's kind of odd for him to stick that in the discuss box rather than the comment box, but I don't think it is particularly harmful provided that his discuss never blocks the document. I.E. he needs to make sure his discuss is removed before Russ clears. Put another way, it's fine for Cullen to tell other IESG members that he agrees with a discuss. It's fine for him to agree so strongly that he'd like to be given an opportunity to take on the discuss if for example the person holding the discuss gives up and wants to drop the issue. It's not fine for him to expect you to do anything based on a discuss that vague. It's not fine for his inaction to cause your document to get stuck based on a discuss that vague. ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
The part of the discuss on lost that I have problems with as a discuss was text that said: Ted and I have discussed this and he is going to propose some clarifying text before I try to evaluate this. I put that in before the IESG call where this document was on the Agenda - This was put in as the document editor, Ted in this case, had asked me not to put in a discuss until we tried to figure out a way to resolve this that did it without opening too many old wounds (this WG has plenty of wounds). Ted sent me the text before the call (he send it Tuesday) - I should have updated the discuss before the call this morning however for some odd reason there have been some other things using up my time this week. I have now updated the discuss and removed this part as I should have done this morning. Cullen On Mar 6, 2008, at 2:50 PM, Ted Hardie wrote: At 2:23 PM -0800 3/6/08, Cullen Jennings wrote: Part of the reason I replied so quickly on this thread is that I think I currently have two discuss that do not meet the discuss criteria (this being one of them the other being on Lost). Totally fair to pick on me here. Both were entered as, excuse the pun, fairly fluffy comments because I believe fully stating each point by point things would have actually make it significantly harder for the editor of the documents to find a good solution. In both cases I believe the editor pretty much understands the issue and if I get requests to please rewrite to be a reasonable discuss, I'm glad to do that after the meeting. Speaking as one of the editors of LoST, and not as a general statement on this problem, a clear discuss from the beginning would have been helpful. I have been willing to have phone calls, conduct a series of email exchanges, and chat via IM on this; I want this document to be right. But the amount of effort you are requiring from the authors to work towards a statement of the problem, much less a solution, has been pretty significant: 10 emails from me alone on your issues alone, as we worked toward an understanding of the issue you have. That boiled down to a DISCUSS that should be as simple as In non-emergency use cases, I am concerned that the provisions of 12.2 allow servers to use algorithms that will needlessly return null results. Please add guidance for non-emergency use cases. Speaking again as someone who thinks this is general problem, the issue I am raising is not that there are bad discusses. The issue I am raising is that the document which describes what discusses are or should be has no force at the moment at all, and should be a community document rather than a statement of the body which may hold discusses. Only the latter allows the community to hold the IESG accountable adequately. regards, Ted Hardie On Mar 6, 2008, at 1:35 PM, Lakshminath Dondeti wrote: Cullen, Thank you for your statement that you are keen to make sure your DISCUSSes are within the parameters of the discuss criteria ION. I appreciate it. Perhaps I am naive or my understanding of the English language is poor (they are both probably true), but could you explain how one of your most recent DISCUSSes: Cullen Jennings: Discuss [2008-03-05]: There has been a lot of discussion about keying modes for SRTP, so I'm glad to see a document that covers this topic for MIKEY. For that reason, I think it's really important to get this right. It looks to me like some of the issues EKR raises need to be fixed in order to achieve that. does not fit into the DISCUSS non-criteria? Unfiltered external party reviews. While an AD is welcome to consult with external parties, the AD is expected to evaluate, to understand and to concur with issues raised by external parties. Blindly cut-and-pasting an external party review into a DISCUSS is inappropriate if the AD is unable to defend or substantiate the issues raised in the review. You chose to not even cut-and-paste the comments. I also wonder which of the DISCUSS criteria fit to advance that specific document to an informational RFC. Are we to guess which of the the issues EKR raises the authors need to fix? Needless to say, we don't need to debate the specifics of that document here, but whereas your intent is honorable, the externally observable behavior is unfortunately different. Sorry for picking on you; I can probably find other similar examples on other ADs, but I was thinking that the ION is not a BCP and so I have no basis to raise the issue. Perhaps I am misunderstanding the ION or perhaps as Brian says that there should be some flexibility in the application of the rules. regards, Lakshminath On 3/6/2008 1:04 PM, Cullen Jennings wrote: Ted, Speaking for myself here but I suspect that other ADs are in the same boat ... I'm
Re: IONs discuss criteria
On 2008-03-07 14:06, Lakshminath Dondeti wrote: Brian, A small clarification below on the reference to the interpretation problems related to 3777: On 3/6/2008 4:10 PM, Brian E Carpenter wrote: Dave, On 2008-03-07 12:34, Dave Crocker wrote: Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. How? To some extent that depends on how carefully the putative BCP is crafted, with should and when to disregard should being very precise. What I think we've seen, with 2026 over the years, and apparently this year with 3777, is that it's virtually I am not sure whether you have made it to the appendix in my report, but the disagreements in interpretation of 3777 have a history (see Page 37). The only thing special about the current nomcom is that we chose to bring it to the community's attention. In Ralph's case, he brought it to the IESG and IAB's attention in March 2006. That's true, from my personal knowledge since I was in the IESG at that time. However, that supports my point ;-) . (Not to be defensive, but the only changes in RFC 3777 that Ralph specifically recommended were the ones covered in RFC 5078). Brian thanks, Lakshminath Nomcom 2007-8 Chair impossible to write precise procedural text that deals with completely unexpected circumstances. Yet if the text has the force of a BCP, it becomes very hard to interpret it flexibly when flexibility is clearly needed. I don't know if that is Sam's point, of course. Brian ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On Mar 6, 2008, at Mar 6, 2008,8:55 PM, Brian E Carpenter wrote: On 2008-03-07 14:06, Lakshminath Dondeti wrote: Brian, A small clarification below on the reference to the interpretation problems related to 3777: On 3/6/2008 4:10 PM, Brian E Carpenter wrote: Dave, On 2008-03-07 12:34, Dave Crocker wrote: Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. How? To some extent that depends on how carefully the putative BCP is crafted, with should and when to disregard should being very precise. What I think we've seen, with 2026 over the years, and apparently this year with 3777, is that it's virtually I am not sure whether you have made it to the appendix in my report, but the disagreements in interpretation of 3777 have a history (see Page 37). The only thing special about the current nomcom is that we chose to bring it to the community's attention. In Ralph's case, he brought it to the IESG and IAB's attention in March 2006. That's true, from my personal knowledge since I was in the IESG at that time. However, that supports my point ;-) . (Not to be defensive, but the only changes in RFC 3777 that Ralph specifically recommended were the ones covered in RFC 5078). Brian Brian - you might be right, but only on a technicality. I noted that a clarification in RFC 3777 in the definition of the term of a mid- term appointment was needed, but didn't give a specific recommendation. More to the point of Lakshminath's observation, I explicitly pointed out the conflict between RFC 3777 and the IAB requirements statement to the IAB and the IESG; I didn't recommend a change to RFC 3777 mostly because I thought it was the IAB requirements that needed to change. - Ralph thanks, Lakshminath Nomcom 2007-8 Chair impossible to write precise procedural text that deals with completely unexpected circumstances. Yet if the text has the force of a BCP, it becomes very hard to interpret it flexibly when flexibility is clearly needed. I don't know if that is Sam's point, of course. Brian ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
At 5:48 PM -0800 3/6/08, Cullen Jennings wrote: I put that in before the IESG call where this document was on the Agenda - This was put in as the document editor, Ted in this case, had asked me not to put in a discuss until we tried to figure out a way to resolve this that did it without opening too many old wounds (this WG has plenty of wounds). Ted sent me the text before the call (he send it Tuesday) - I should have updated the discuss before the call this morning however for some odd reason there have been some other things using up my time this week. I have now updated the discuss and removed this part as I should have done this morning. For the record, I suggested we talk after Cullen DEFERed on the previous telechat to see whether it was a misunderstanding on his part, rather than a serious issue. It was later that I suggested someone else hold the discuss, because I thought Cullen would want to recuse, since he is a patent author on a patent his company has filed related to this document. Given that I have now seen the discuss text, I can see that he has managed to open the old ones, bring in ones from a related working group that aren't really salient, and open significant new ones, all without having paid much attention to all of the text and effort that flowed in the attempt to get early discussion of this. To quote ekr, Outstanding! Ted Hardie ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On Mar 6, 2008, at 7:10 PM, Brian E Carpenter wrote: Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. How? To some extent that depends on how carefully the putative BCP is crafted, with should and when to disregard should being very precise. What does this mean? Is it an argument that as a BCP the shoulds carry weight whereas now they can be obeyed more conveniently? Or is it a general comment regarding the futility of formalizing procedures? -andy ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
On 2008-03-07 16:10, Andrew Newton wrote: On Mar 6, 2008, at 7:10 PM, Brian E Carpenter wrote: Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. How? To some extent that depends on how carefully the putative BCP is crafted, with should and when to disregard should being very precise. What does this mean? Is it an argument that as a BCP the shoulds carry weight whereas now they can be obeyed more conveniently? Or is it a general comment regarding the futility of formalizing procedures? I think it's both. It's harder to disregard a should in a BCP; it's easier to update an IESG-issued document than a BCP, and it's very hard to get either of them 100% right. We also have to remember that a DISCUSS position is not a formal part of the IETF process. It's simply the current method used by the IESG for logging lack of consensus. There's a lot of work in turning it into formal process language, and I wonder who has the appetite for that work? Brian ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
Re: IONs discuss criteria
Brian E Carpenter wrote: Dave, On 2008-03-07 12:34, Dave Crocker wrote: Sam Hartman wrote: Making it a BCP will make the interpretation problem worse not better. How? To some extent that depends on how carefully the putative BCP is crafted, with should and when to disregard should being very precise. What I think we've seen, with 2026 over the years, and apparently this year with 3777, is that it's virtually impossible to write precise procedural text ... I see your point. What I don't see is why the IETF's efforts at writing and enforcing procedural rules would be so much less successful than the efforts to create laws, contracts, and the formal rules that govern so many other organizations. The real irony, of course, is that we are community whose main work is to write formal procedures and make then into formal standards. So we probably ought to view it as a tad embarrassing that we can't do that adequately for our governing body. d/ d/ d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net ___ IETF mailing list IETF@ietf.org https://www.ietf.org/mailman/listinfo/ietf
