Re: Content-free Last Call comments
- Original Message - From: Pete Resnick presn...@qti.qualcomm.com To: Stephen Farrell stephen.farr...@cs.tcd.ie Cc: ietf@ietf.org Sent: Tuesday, June 11, 2013 8:45 PM Subject: Re: Content-free Last Call comments It's interesting to see that people are interpreting me to mean I want more text. I don't. I want less. Save your breath. There is no reason to send one line of support, and it only encourages the view that we're voting. Details below. tp Seems to me that there are more points of view on this thread than there are members of the IESG, in which case, to help followers of this list, perhaps they, when instigating a Last Call, should do as some WG Chairs do when instigating WG Last Call, spell out that expressions of support from those that have read it are welcome - or not, as the case may be. Thus the AD might add that any e-mail containing +1 in the body will be filtered into a spam trap and discarded without being read. Tom Petch /tp Specifically on Stephen's message: On 6/10/13 7:36 PM, Stephen Farrell wrote: I think you err when you say this: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. In fact, you do know Russ. If you did not, then the above would be far closer to correct. But in reality you do know a lot more than you claim below. With overwhelming probability, your choice #2 applies and I'm very surprised you don't also think that. If it made a significant difference and I wasn't sure I'd ask Russ to clarify. For me, I read Russ' mail and I concluded he meant #2. And I'm confident in that conclusion. So I'm really bemused when you say that you don't know how to interpret Russ' mail. Let's separate out a few issues. There's what the message means itself, there's its usefulness and how much weight it should be given, and then there's what's problematic about it. I think you've conflated a few things. As you know, on the earlier message I called out on the IESG list, I guessed wrong that the sender was claiming choice #3 (that is, I thought, incorrectly, that the sender disagreed with a recent argument against publication, but simply didn't explain why). In fact in that case, not only wasn't it choice #4, it was also neither choice #1 (an implementer claiming implementation) nor choice #2 (an expert on the topic), my other two guesses, but rather a generally smart IETFer who thought the work seemed like a good idea. So I'm a little disinclined to guess on motives. In the case of Russ's message, I did guess it was choice #2, but I wasn't really sure. But let's separate what Russ meant from the weight it should be given. The message was that a person (let's presume with some expertise in the area) had read it. So what? Didn't this document go through a WG? Weren't a good bunch of experts already reading and reviewing this document? If this was during the WG discussion and the chairs solicited some final check reviews, *maybe* the message could have some use. But during IETF Last Call? Even if the message was sent because it was trying to say, I'm an expert on this stuff and it looks technically sound, is that going to change anything that the IESG should do about the document? I just think at Last Call, these statements of support are either harmless-but-useless or they are nefarious. Declaring that mail problematic also seems quite purist to me. I always find it amusing that when Stephen and I disagree, it's almost always because he presumes I want purity (and usually a new process), which I don't, and it seems to me that he wants no rules-of-thumb at all and that everything should be on a case-by-case basis. (I guess that means that I'm into purity, where Stephen has no principles. ;-) ) I think general principles are a fine thing, and we should try to stick to them, but I've got no interest in purity. The single piece of mail isn't problematic at all. As I said, probably harmless but useless. But the overall impression it gives *is* problematic. It encourages the view that we are voting, that a simple statement of support is important to our process. (What really set my hair on end was the I support publication bit. That always sounds like a vote for a new RFC. From Russ, I have a pretty good idea that he didn't mean it that way, but it's a poor formulation and not what we should be encouraging.) It also indicates that during Last Call, we want to hear that a document is getting reviewed. I would like to presume that documents get serious reviews in the WG, and that Last Call time is for people who haven't been participating in the WG to do a final check, letting us know if there's something *wrong*. If the document wasn't getting good review in the WG and needs statements of support at Last Call time, something has seriously failed earlier in the process. The chair or responsible AD should have been saying early on, Dear Expert, can you please have a read over
Re: Content-free Last Call comments
On 6/12/13 9:42 PM, Ted Lemon wrote: On Jun 12, 2013, at 3:31 PM, Peter Saint-Andre stpe...@stpeter.im wrote: I think these messages are useless, not harmful. But perhaps I have more confidence in the inherent skepticism of your average IETF participant than Pete does... FWIW, until I read Pete's document on consensus, I thought that +1 statements were part of the consensus process. This was not a strongly held opinion—it was just my understanding of how consensus operated, from having watched other working group chairs run their working groups. I think the point Pete is making is very important, because the consensus process Pete describes is more in keeping with how I think the IETF ought to operate than the process in which +1 counts for something. +1 / -1 are conventions that crept in from other standards bodies, they don't have any particular place or meaning here, apart from what you can literally interpret them as e.g. the equivalent of hand raising in agreement or disagreement. (BTW, in case it wasn't obvious, I've been engaging in this discussion with my AD and working group chair experience in the back of my mind, but my AD hat off.)
Re: Content-free Last Call comments
Without agreeing with or disagreeing with Pete, I'll point out that Pete was talking about IETF last call. It's perfectly reasonable for a WG participant who has been actively involved to say, This one is ready. Ship it, and Pete isn't saying otherwise. In that case there is context that helps. Barry On Wednesday, June 12, 2013, Randy Presuhn wrote: Hi - From: Ted Lemon ted.le...@nominum.com javascript:; Sent: Jun 12, 2013 12:42 PM To: Peter Saint-Andre stpe...@stpeter.im javascript:; Cc: ned+i...@mauve.mrochek.com javascript:; ned+i...@mauve.mrochek.com javascript:;, Alexey Melnikov alexey.melni...@isode.com javascript:;, Pete Resnick presn...@qti.qualcomm.com javascript:;, ietf@ietf.org javascript:;Discussion ietf@ietf.org javascript:; Subject: Re: Content-free Last Call comments On Jun 12, 2013, at 3:31 PM, Peter Saint-Andre stpe...@stpeter.imjavascript:; wrote: I think these messages are useless, not harmful. But perhaps I have more confidence in the inherent skepticism of your average IETF participant than Pete does... FWIW, until I read Pete's document on consensus, I thought that +1 statements were part of the consensus process. This was not a strongly held opinion—it was just my understanding of how consensus operated, from having watched other working group chairs run their working groups. I think the point Pete is making is very important, because the consensus process Pete describes is more in keeping with how I think the IETF ought to operate than the process in which +1 counts for something. ... As a former WG chair who's had to deal with some very rough consensus calls... Not counting a +1 is more consistent with a classical definition of consensus. But, particularly at a WG level (less so, perhaps, at the IETF level) +1 is very helpful in determining whether the previously mentioned Abilene Paradox should be of concern. Randy
Re: Content-free Last Call comments
On Tue, Jun 11, 2013 at 11:20 PM, Ted Lemon ted.le...@nominum.com wrote: On Jun 11, 2013, at 6:03 PM, Dave Cridland d...@cridland.net wrote: ... and how would we judge IETF consensus on a document that doesn't get done under a charter (which would in turn have been granted consensus without any IETF comments?) I would expect that you'd start with a mailing list, see if there is interest, come up with a proposal for a BOF, come up with a working group charter, get IESG review on the charter, then get IETF consensus on the charter, and then start working on the document. That's how it's usually done. I think you misread entirely. But anyway, let's assume we follow those steps. I suspect the closest we get to getting an idea of IETF consensus is the interest gauging at the beginning of the process, though interestingly this is only positive interest - objections to doing the work at all aren't really relevant here. The IETF consensus on the charter is handled by Apathy Is Assent rules, so claiming that this consensus call becomes the default is an interesting argument to make. BTW, the fact that a few people think the process ought to work differently does not mean there is consensus for it to work differently. Also, what there may not be consensus on among the people who have weighed in on the topic is whether positive statements in favor of a document are relevant in IETF last call, but I don't really know how to reduce that to practice, because in reality I think it is rare for a quorum of IETF participants to read a document as a consequence of a last call announcement. Without that, I don't see how you can have any other last call process than the one we currently have. OK, so we don't have voting but we do have a quorum? How wonderful. And given that the majority of people are silent means they agree (with, mind, everything - not just that they're not reading), so any number of people are therefore few. But to loop around: I strongly feel that positive statements have value, as they allow the community to gauge the level of review and consensus, and I suspect that human nature means that we get more reviews if people get to brag about it. I suggest that if more than one bit of data is required, it's simply asked for. Given that the text of IETF Last Call announcements is not governed by any process RFC that I'm aware of (feel free to correct), I suggest simply putting a set of optional questions there. I note this practise has served the XSF very well. I do not think this needs an endless bikeshed discussion on what questions; the IESG can pick what it wants to know. If, on the other hand, only objections are sought, then the text (which simply asks for comments) also needs changing. And the GenArt, AppsDir, and SecDir reviews should only be send when they have objections to publication, of course. If you feel that the only way to make either change is to form a working group and publish an RFC to change something undocumented in the series, then I think we're stranded in a bureaucratic quagmire with no chance of escape, but I'll be happy to send comments, as requested, nonetheless.
Re: Content-free Last Call comments
perhaps we should go to the source of the problem and require a phd dissertation and defense from draft authors. A couple of years ago I worked with someone who completed his PhD thesis on a topic faster than it took to publish the RFC on the same topic… that was my wake-up call for IETF process needing some work :-) Things have improved since then, though, but probably not enough. But back to the topic. I, for one, would like to see responses on IETF last calls. It builds my confidence that we know enough about the topic to make an approval decision. Particularly when the input comes from people outside the working group. And I'd like to distinguish everyone thinks this is fine from no one read the document. And substance of the comments - be it positive or negative - is obviously important. But I find myself strangely between Stephen's opinion and Pete's opinion. Pete is of course right that blind support is bad and that we could easily guess wrong what the commenter meant, even when there is substance behind the support but the substance did not get explicitly articulated in the e-mail. But I can also see that there are cases where there's some context. Elwyn's mail illustrates an important example with Gen-ART reviewers. (Thank you by the way Elwyn and others for this great service. It is very much appreciated, and does improve RFC quality.) Anyway, we know why these reviews are being done, and what the expectation is. The expectation is that a generalist reads the spec and determines if it is understandable, tries to spot possible errors, and so on. But at the same time, the reviewers are unlikely to be in a position to say, for instance, that they have implemented the spec or plan to deploy the technology. So at the end of the day, if the spec is fine, it will only say Ready, We will interpret that as an Internet technology generalist outside the working group having read the document and indicating that it has no obvious problems. But there is no other implication - about the reviewer needing the work for himself or herself, for instance. In Russ' case I took the message to mean that he reviewed it as an expert on the technology. It would probably have helped if he said whether he only reviewed it for correctness or if he was also making a statement about the technology being needed in his opinion. I would have appreciated his opinion on it. That being said, it feels kind of odd to explicitly say things if you have nothing to say about the matter. I have read the document and it seems fine, but I am not implementing it and I don't know if it addresses a real need in the Internet. We tend to assume that if the commenter is not saying these things, it is because he has no statement about his implementation, for instance. Jari
Re: Content-free Last Call comments
On Wed, 12 Jun 2013, Jari Arkko wrote: But back to the topic. I, for one, would like to see responses on IETF last calls. It builds my confidence that we know enough about the topic to make an approval decision. Particularly when the input comes from people outside the working group. And I'd like to distinguish everyone thinks this is fine from no one read the document. Should people who supported the document within the WG LC generally avoid voicing support in the IETF LC discussion? -- Mikael Abrahamssonemail: swm...@swm.pp.se
Re: Content-free Last Call comments
On Jun 12, 2013, at 4:43 AM, Dave Cridland d...@cridland.netmailto:d...@cridland.net wrote: I suspect the closest we get to getting an idea of IETF consensus is the interest gauging at the beginning of the process, though interestingly this is only positive interest - objections to doing the work at all aren't really relevant here. The IETF consensus on the charter is handled by Apathy Is Assent rules, so claiming that this consensus call becomes the default is an interesting argument to make. Working group charters go by much less frequently than new drafts; the burden of checking them is about as low as any comprehensive review burden in the IETF can possibly get. So if you don't comment on a charter when it goes by, and you don't comment on the work the working group does, you have only yourself to blame. The IETF doesn't have members, so we can't say only 10% of IETFers like this idea, so we won't bother with it. We don't have voting, so we can't issue a ballot and count up the yeas and nays. If you want to limit the number of RFCs published, you need to voluntarily do the work that is required to make that happen. If you think there's a problem with the charter, and you raise it on the IETF mailing list, there will be no shortage of discussion. Trust me on this. OK, so we don't have voting but we do have a quorum? How wonderful. And given that the majority of people are silent means they agree (with, mind, everything - not just that they're not reading), so any number of people are therefore few. Right. We don't have members, so we can't have a quorum. That's just not how the IETF operates. If you prefer to operate in an SDO that operates that way, you can either change the IETF, or work in a different SDO. I strongly feel that positive statements have value, as they allow the community to gauge the level of review and consensus, and I suspect that human nature means that we get more reviews if people get to brag about it. If the only reason you are doing a review is so you can brag about it, that seems a bit useless to me. But in any case, as you say, Pete made his point, you don't agree with it, it's a matter of opinion, so we're bikeshedding. FWIW, my reason for responding to these questions on ietf@ietf.orgmailto:ietf@ietf.org is that before I became an AD, I actually didn't _know_ how IETF consensus was determined, and had to do quite a bit of looking around to figure it out when I suddenly needed to know. So I thought it was worth sharing; if in fact there are a lot of IETF participants who think this is the wrong way to handle last call, you really ought to get together and do a BOF. That too is how the IETF thinks about things—it is only a bureaucratic quagmire if you make it one. Should it really be _easy_ to change how the IETF evaluates consensus?
Re: Content-free Last Call comments
On 6/11/13 3:45 PM, Pete Resnick wrote: It's interesting to see that people are interpreting me to mean I want more text. I don't. I want less. Save your breath. There is no reason to send one line of support, and it only encourages the view that we're voting. Details below. And a lot of details they were. I will point out that circumstances probably matter. For a working group document that has already received some scrutiny, you've already addressed many questions, like whether the work is important, whether it's right for the IETF to do, and whether the solution is correct. For AD sponsored, I would expect you'd actually like a bit more. Yes, you've answered some of these questions in principle, but the rest of us haven't had our say. In these circumstances, I'd have to say that affirmative support with an eye toward answering those questions (as well as nits et al) would probably be useful for me to see as an individual. Eliot
Re: Content-free Last Call comments
Dave Cridland wrote: I strongly feel that positive statements have value, as they allow the community to gauge the level of review and consensus, and I suspect that human nature means that we get more reviews if people get to brag about it. Agreed 100%. But also consider the likely effect of calling certain comments useless. Discussions like this don't exactly fire me up with enthusiasm to expend additional time reviewing and commenting on documents not directly related to what I do. And I rather doubt I'm alone in this. Ned
Re: Content-free Last Call comments
On 6/12/2013 2:28 AM, Jari Arkko wrote: In Russ' case I took the message to mean that he reviewed it as an expert on the technology. It would probably have helped if he said whether he only reviewed it for correctness or if he was also making a statement about the technology being needed in his opinion. I would have appreciated his opinion on it. That being said, it feels kind of odd to explicitly say things if you have nothing to say about the matter. The act of posting something means they feel that they do have something to say. That's actually not the issue here. The issue here is how fully the posting should explain itself. When we write, we make assumptions about the reader. Russ talking to you, Jari, permits an especially rich set of assumptions to be easy and correct. You know each other. For informal exchange, that's not only acceptable, it's essential. The alternative is pathologically stilted and unworkably inefficient. We often write things in the IETF -- including formal things -- as if they are only for folks we know. But that's not really an appropriate model for a formal standards process. When contributing to the formal parts of a standards process, we need to assume the writer and the reader don't know each other, because they usually don't. The fact that some do is actually distracting. Write for the record; the record does not know the background or intent or implications of the writer. Neither will a random reader, now or later. So make them explicit. The fact that you, Jari, are still left with a guess about what Russ's comment fully meant should make obvious just how important it is for folks who post comments to provide context for their comment, where 'context' means demonstrating an understanding of what is being commented on and fully explaining what our comment means. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: Content-free Last Call comments
Pete, On Jun 10, 2013, at 1:37 PM, Pete Resnick presn...@qti.qualcomm.com wrote: Russ, our IAB chair and former IETF chair, just sent a message to the IETF list regarding a Last Call on draft-ietf-pkix-est. Here is the entire contents of his message, save quoting the whole Last Call request: On 6/10/13 1:45 PM, Russ Housley wrote: I have read the document, I a support publication on the standards track. Russ A month ago, we had another very senior member of the community post just such a message (in that case directly to the IESG) in response to a different Last Call. I took that senior member of the community to task for it. But apparently Russ either disagrees with my complaint or didn't notice that discussion on the IESG list, so I think it's worth airing here in public: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. Please describe the context of your email. Are you speaking for the IESG, yourself as an AD, or an individual? Is this something the IESG has discussed? Are you (or the IESG) creating rules for the community, that is, thou should not send one sentence responses to IETF last calls? Same question when you took that senior member of the community to task. I read your email that you are trying to control how people respond to IETF last calls and if they don't respond to your liking you can take them to task. Especially the Russ either disagrees with my complaint… text. This may not have been your intent, but your original email and responses can be read that way. Bob
Re: Content-free Last Call comments
Hi Dave, At 01:43 12-06-2013, Dave Cridland wrote: I strongly feel that positive statements have value, as they allow the community to gauge the level of review and consensus, and I suspect that human nature means that we get more reviews if people get to brag about it. I suggest that if more than one bit of data is required, it's simply asked for. Given that the text of IETF Last Call announcements is not governed by any process RFC that I'm aware of (feel free to correct), I suggest simply putting a set of optional questions there. I note this practise has served the XSF very well. I do not think this needs an endless bikeshed discussion on what questions; the IESG can pick what it wants to know. If, on the other hand, only objections are sought, then the text (which simply asks for comments) also needs changing. And the GenArt, AppsDir, and SecDir reviews should only be send when they have objections to publication, of course. If you feel that the only way to make either change is to form a working group and publish an RFC to change something undocumented in the series, then I think we're stranded in a bureaucratic quagmire with no chance of escape, but I'll be happy to send comments, as requested, nonetheless. An interesting point in the above is level of review and consensus. Here's what I know: there is going to be apathy, there might be attempts by a group to support a draft or even attempts to silence critics, there might be someone new to all this who might be commenting. If it was my decision to make (and it is not) I would take those factors and some other points into consideration in making a determination about a document. As I have read your reviews I have an approximate idea of the type of review you would do. I read the draft and I notice some obvious issues; I downgrade your statement of support to a tweeter comment. I read the draft and I notice an obvious issue; I consider your statement of support as good enough. I have read the reviews from the IAB Chair. I read the draft and I notice that it is not well-written; I downgrade the statement of support of the IAB Chair to a tweeter comment. I read the draft and I notice that it is good to go. However, I don't find any comments about it except for a statement of support from the IAB Chair. I don't say that there is consensus. Note that this is really a personal decision; someone else might say that there is consensus. It's not a problem unless the IESG is affected by the Abilene paradox. The XSF is likely a group of people who can write code. The IETF is a bunch of people who might discuss about content-free comments but won't comment about the draft. :-) Drawing up a set of optional questions will generate a bottom-up discussion and that would be against the values which the IETF cherishes. :-) There isn't anything preventing an Area Director or someone else from asking optional questions during a Last Call. Optional questions from an IETF participant might be ignored if such activity would turn a Last Call into the Tribunal del Santo Oficio de la Inquisición. If you are doing an AppsDir review, for example, and you state: The draft is ready for publication as a Proposed Standard. I presume that you can personally explain the meaning of that sentence. If you are an individual responding to a Last Call you can say anything you wish. If your message is littered with spelling mistakes or it does not contain any substantive comment, it won't bear much weight. If your English writing skills is not that good but your code is good the message will bear more weight. If your message is to show your management that you are participating in the IETF the message will not bear much weight. It's simple enough. I would send a message if I believe that it can affect the decision. It's up to me to know what will influence the content and fate of the draft. Regards, -sm
Re: Content-free Last Call comments
On 12/06/2013 15:16, ned+i...@mauve.mrochek.com wrote: Dave Cridland wrote: I strongly feel that positive statements have value, as they allow the community to gauge the level of review and consensus, and I suspect that human nature means that we get more reviews if people get to brag about it. Agreed 100%. But also consider the likely effect of calling certain comments useless. Discussions like this don't exactly fire me up with enthusiasm to expend additional time reviewing and commenting on documents not directly related to what I do. And I rather doubt I'm alone in this. You are not alone. I do occasional reviews of documents not directly related to what I do. But I hate paperworks. And if I would be required to fill in 10 pages questionnaire about why I think the document is in a good shape, I probably will stop doing such reviews altogether.
Re: Content-free Last Call comments
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/12/13 12:38 PM, Alexey Melnikov wrote: On 12/06/2013 15:16, ned+i...@mauve.mrochek.com wrote: Dave Cridland wrote: I strongly feel that positive statements have value, as they allow the community to gauge the level of review and consensus, and I suspect that human nature means that we get more reviews if people get to brag about it. Agreed 100%. But also consider the likely effect of calling certain comments useless. Discussions like this don't exactly fire me up with enthusiasm to expend additional time reviewing and commenting on documents not directly related to what I do. And I rather doubt I'm alone in this. You are not alone. I do occasional reviews of documents not directly related to what I do. But I hate paperworks. And if I would be required to fill in 10 pages questionnaire about why I think the document is in a good shape, I probably will stop doing such reviews altogether. Much as I love Pete, IMHO this thread started because in good curmudgeonly style he needed something to grump at. In this instance, I suggest we just keep doing what we've always been doing. Peter - -- Peter Saint-Andre https://stpeter.im/ -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.19 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJRuMF+AAoJEOoGpJErxa2pAkwQAK6dse/oJRAS/kNcjvaKvOvD elHpEX54kil8v01wfiJ8/kd0fHSsKItR0m0r8b/YTkH/8I0+2dy+tG2F/gchI6uK OrJsev4vFcSY2yZM/qh3QAWeJ2HbYDRjHKc7yMDrwAwS8zUPiNRTfecsw6p/lWNw qj2H3R5bCIIkIR0dQO5NvWhaEj6Edy/40rwLe1pOOKo5EiMGhut0lQWUV6Dqy3u/ SJAy/dYRexXLuZDuYkI1hGUMBDCmspqnWVOe/bX6oQ3MuC1wTFyZXNntgOlsWOnu QK7PkY9mNpSpd8KZ8Rt0LgMNEvsPKmNgggmD1Hx1RjBv5B/6SXPwKlT1jS3bZ2or eCcdHPxLbTD06Ec1Ukg2vNiFe+kgVz5rXHe4GuBm7RQFvMTkq2jO5AF8MddKjtoi wfW5ociPNQCqIjGkj+xM9CJS6aSQaQxUk0DPzxjwoUAYSIxcF4VIsB0uY/JUs5n0 41deCNgp26OAyvcxdTFW3pgD2vWX+BTA+H6w+0VzLd0msvousDi+tm5wPiQml3+y Us6wSZwNrxdBgCp7Ce5BAyEaLFQ+LhtOqW3ElJv+AiNgKOujH5lhnNPfsd7jRWHF Ca4m3XxhFsri+Hw9sr7D2PtUWWJZ4ntm32zfZVKRDM6SXXvUzpZoGAzzgwTOf2Zj UTBku2xoH8Khfloa29y1 =79mX -END PGP SIGNATURE-
Re: Content-free Last Call comments
After reading some of the criticisms, I wonder if folks who think they've been disagreeing with me are going to get to the end of this message and say, Oh, if that's all he's on about, who cares? But *I* of course think there is an important issue in here. Anyway, back into the breach. David's message seems like a good launching point. On 6/12/13 3:43 AM, Dave Cridland wrote: The IETF consensus on the charter is handled by Apathy Is Assent rules, so claiming that this consensus call becomes the default is an interesting argument to make. Partially tangential: We in the IETF use the phrase consensus call in, AFAICT, a unique (and IMO goofy) way. It implies that first we have a discussion, and then at the end of the discussion we engage in a short act we refer to as a consensus call, the completion of which *creates* the consensus. That isn't a consensus call. That's just an anonymous and imprecisely counted vote. I think I've got a new section for my consensus document to work on: Consensus isn't the destination; it's the journey. (I should get a job writing text for fortune cookies.) But back to the topic at hand: I strongly feel that positive statements have value, as they allow the community to gauge the level of review and consensus... Reviews are good. We need to do them. They are of value. They should be encouraged. People should post the results of reviews. They should brag about them as they see fit. No argument there. Public statements of negative reviews (with specific issues identified) are really important. If issues are identified that then go unanswered, that breaks consensus. Public statements (even short ones) of positive reviews have value in that we want to know that reviews have taken place. That's useful. Mind you, such statements don't help you gauge consensus: Sure, you *might* conclude that one more person is part of the consensus, but of course the number of people doesn't matter to the consensus, and the consensus hasn't moved at all -- you haven't resolved a new open issue. (There is a separate question about what to do when documents haven't gotten serious review prior to IETF Last Call and what that means for consensus. More on that in a moment.) So, none of the above things get me torqued. Reviews: Good. Public statements of reviews: Good. Issues to be resolved: Good. Here's what gets me torqued, that I think is useless, and starts people thinking that they are voting: I support publication of this document. I do not support publication of this document. Even prefacing those with I reviewed the document doesn't help. (Of course the I reviewed the document part itself is potentially useful if the person bothers to tell us the result of the review, like it's technically sound or it solves a problem that needs solving or it's technically crappy *and* here are the reasons why.) But the I do/don't support this document part is what gets my goat. It contains the apparent implication that someone is counting noses and that support matters. It shouldn't. What matters is that we as a community have worked this thing out, that it solves a problem that we have identified is real, and that nobody in the community has found serious fault in it. It's not the shortness of the statement that's the problem (though sometimes, especially with negative reviews, it is a problem); it's treating the call for consensus as a vote that bugs me. It's not useful to simply support a document. If, on the other hand, only objections are sought, then the text (which simply asks for comments) also needs changing. And the GenArt, AppsDir, and SecDir reviews should only be send when they have objections to publication, of course. Nope, I'm good with all that. In fact, the GenArt review says exactly what it should: It provides generalist reviews for the General Area director (currently the IETF Chair), providing an additional set of eyes for documents as they are being considered for publication. It's not looking to gauge consensus. It's looking for a review. That said, I do believe that objections are *way* more important than comments saying, It's fine. Once something has gone through a WG, the presumption is It's fine. Additional eyes are good, but they're good precisely because they might find problems. That brings us to one last thing which I mentioned above: What if documents *aren't* getting serious review prior to IETF Last Call? This goes back to our tail heavy discussion. The IETF used to be very good about getting lots of review (sometimes even cross-area review) *during* the consensus building process in the WG. We now have many instances where WGs are not really working toward consensus to solve a problem; they're doing reviews of a single person's idea of how to solve a problem. Even worse, we are seeing many individual submissions that are not the product of consensus building at all, and when that 4-week Last
Re: Content-free Last Call comments
On 6/12/13 10:33 AM, Bob Hinden wrote: Please describe the context of your email. Are you speaking for the IESG, yourself as an AD, or an individual? Oh, crap. And given that I'm usually the one giving people a hard time about *this* issue, I feel especially bad about not being clearer. This is Pete, hatless individual talking. There are (as you've seen) IESG people, past and present who disagree with me. Nothing I've said is anything we've come to an IESG consensus around. Insofar as I think that only saying I support this document is a bogus thing to say, I am revealing something about how I behave as an AD in the face of such statements, and my further explanations that I value reviews and take them seriously also indicates my views as an AD, but that doesn't indicate some policy that I'm instituting with WGs for which I am responsible. I am terribly sorry about not making this clear up front. My sincere apologies. I hope nobody took this as a other than my commentary. Sheepishly, pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: Content-free Last Call comments
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 6/12/13 12:44 PM, Peter Saint-Andre wrote: On 6/12/13 12:38 PM, Alexey Melnikov wrote: On 12/06/2013 15:16, ned+i...@mauve.mrochek.com wrote: Dave Cridland wrote: I strongly feel that positive statements have value, as they allow the community to gauge the level of review and consensus, and I suspect that human nature means that we get more reviews if people get to brag about it. Agreed 100%. But also consider the likely effect of calling certain comments useless. Discussions like this don't exactly fire me up with enthusiasm to expend additional time reviewing and commenting on documents not directly related to what I do. And I rather doubt I'm alone in this. You are not alone. I do occasional reviews of documents not directly related to what I do. But I hate paperworks. And if I would be required to fill in 10 pages questionnaire about why I think the document is in a good shape, I probably will stop doing such reviews altogether. Much as I love Pete, IMHO this thread started because in good curmudgeonly style he needed something to grump at. In this instance, I suggest we just keep doing what we've always been doing. To clarify: I reject kings, presidents, and voting. But I do believe in the delete key. If you think that these +1 messages are useless, don't factor them into your thinking about whether we have consensus. If you think that these +1 messages are actively harmful (e.g., by leading to groupthink or poisoning our very understanding of what it means for us to reach consensus), then proceed as Pete is proceeding. I think these messages are useless, not harmful. But perhaps I have more confidence in the inherent skepticism of your average IETF participant than Pete does... Peter - -- Peter Saint-Andre https://stpeter.im/ -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.19 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJRuMygAAoJEOoGpJErxa2p+FYP/iB8QDp6rQ8AdB+hcweMUWFF lJ1zhGlI0KIaHFzvzLm68FUyjNv4373nRninqq1biS5Uj3wjKgm4Q1rM0kL46mM0 ETVkD916HB3u11nQd0akjtR22TO2I847Ka+HE+2waPUBfDHCaxD8vxPD7lbVI3J9 QlEgxhUobppvHfsKpWHHJaBd9EF+Q4r5dIAhQX3tt7xLi6bupm5X4byAljaFiw7+ JB3c+7xmu767+NFMe7/839fRfRnO8K15c2fkbheY0bHLH9qlwMoUr5DnvYchByXw SrcZRnYZMaB5zvmYV0Q8TmHCGb3i9R+PIDbvIUL9p95ega8lDi9a5qKA/wN1lEdy GgkJ1mY/WmhcSEVtYPnUk0ZnmM/ZBjcuZw4lS8h9bXr8LMfCOyp2z6WehrgRXPob GNu2e5geD9ARJ0Y0f8etjj29Fh79LD+U1zEdH0UyMqu+AVJObX9U/+tIwixz7sa4 DR2zsf+OPGcOXWwGyEhE/WwPOFOCwEatdyBGU9g2mPIhnbPrNUwddfCvTFXsdNcZ 9e4ULnN+/xqictSyyoCBLXazNnAcbjssLhEz/Mw3yTtlOQk0UKKYUvxydFcmxY6l bqheTm5a/3+VV1G6S4CflYeW4bWxKyiQSgo2N2+BSxrQFnzcFd3up3F3LhH6NYu8 f4MHTt0kpZp8MC2JAJjW =7/LJ -END PGP SIGNATURE-
Re: Content-free Last Call comments
On Jun 12, 2013, at 3:31 PM, Peter Saint-Andre stpe...@stpeter.im wrote: I think these messages are useless, not harmful. But perhaps I have more confidence in the inherent skepticism of your average IETF participant than Pete does... FWIW, until I read Pete's document on consensus, I thought that +1 statements were part of the consensus process. This was not a strongly held opinion—it was just my understanding of how consensus operated, from having watched other working group chairs run their working groups. I think the point Pete is making is very important, because the consensus process Pete describes is more in keeping with how I think the IETF ought to operate than the process in which +1 counts for something. (BTW, in case it wasn't obvious, I've been engaging in this discussion with my AD and working group chair experience in the back of my mind, but my AD hat off.)
Re: Content-free Last Call comments
'scuse front posting, but I'm going to outrageously summarise Pete's point as I want substance in all Last Call comments, or alternatively I will ignore +1 just as I will ignore -1. That isn't unreasonable, but personally I would interpret I've read it and I think it's good work as substantive, especially if it comes from a known expert. YMMV Regards Brian On 12/06/2013 08:31, Pete Resnick wrote: On 6/11/13 3:05 PM, Brian E Carpenter wrote: Pete, On 12/06/2013 07:45, Pete Resnick wrote: It's interesting to see that people are interpreting me to mean I want more text. I don't. I want less. Save your breath. There is no reason to send one line of support, and it only encourages the view that we're voting. Details below. Just to test what you are saying, let me ask the following. Oooo...I love a test. How would you react to one that says something like: I've read the draft, and I've considered Joe Blow's objection, but I still support publication of the draft (*not followed by a reasoned rebuttal of Joe Blow's argument)? I would be rather grumpy with such a message. If there's an outstanding (reasonable) objection to a document, I need to know why to consider that argument in the rough. I'd have to ask for more detail from the sender. If the response I get back is, I figured it was obvious why Joe Blow was full of crap, I'd ask, Then why did you bother posting? If the sender happens to be an expert (and Joe Blow is not), I'm still not going to take it at face value that Joe Blow is wrong. If I did, Joe would be well within rights to appeal because his argument got blown off. So, if you're saying something that is perfectly obvious, no need to say it. But if it's not perfectly obvious, I do want more text. pr
Re: Content-free Last Call comments
On 6/12/13 3:37 PM, Brian E Carpenter wrote: 'scuse front posting, but I'm going to outrageously summarise Pete's point as I want substance in all Last Call comments, or alternatively I will ignore +1 just as I will ignore -1. Maybe not outrageous, but certainly wrong because... ...personally I would interpret I've read it and I think it's good work as substantive I agree with that. Confused now? Go back and look at my message of an hour and a half ago and see if you can tell why I agree. :-) As I've said to other people, I like straw men. They make fine neighbors. They're quiet, and they don't smoke. But if you're going to summarize (or summarise), best to do so accurately. pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
RE: Content-free Last Call comments
Are the IESG people who disagree with you speaking for the IESG, or for themselves? I've noticed a tendency for IESG weight to be (inadvertently?) thrown around, lending more weight to comments than would otherwise be given. Lloyd Wood http://sat-net.com/L.Wood/ From: ietf-boun...@ietf.org [ietf-boun...@ietf.org] On Behalf Of Pete Resnick [presn...@qti.qualcomm.com] Sent: 12 June 2013 20:17 To: Bob Hinden Cc: ietf@ietf.org Subject: Re: Content-free Last Call comments On 6/12/13 10:33 AM, Bob Hinden wrote: Please describe the context of your email. Are you speaking for the IESG, yourself as an AD, or an individual? Oh, crap. And given that I'm usually the one giving people a hard time about *this* issue, I feel especially bad about not being clearer. This is Pete, hatless individual talking. There are (as you've seen) IESG people, past and present who disagree with me. Nothing I've said is anything we've come to an IESG consensus around. Insofar as I think that only saying I support this document is a bogus thing to say, I am revealing something about how I behave as an AD in the face of such statements, and my further explanations that I value reviews and take them seriously also indicates my views as an AD, but that doesn't indicate some policy that I'm instituting with WGs for which I am responsible. I am terribly sorry about not making this clear up front. My sincere apologies. I hope nobody took this as a other than my commentary. Sheepishly, pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: Content-free Last Call comments
On 06/12/2013 10:56 PM, l.w...@surrey.ac.uk wrote: Are the IESG people who disagree with you speaking for the IESG, or for themselves? That's really not clear already? In any case, I was disagreeing with Pete as an individual since he was wrong regardless of hats. He and I do that all the time and both find it amusing:-) I've noticed a tendency for IESG weight to be (inadvertently?) thrown around, lending more weight to comments than would otherwise be given. I've not noticed that. Maybe I perceive things differently of course, but people ought know not to take opinions any more or less seriously just because someone's on some I* thing at the moment. S. Lloyd Wood http://sat-net.com/L.Wood/ From: ietf-boun...@ietf.org [ietf-boun...@ietf.org] On Behalf Of Pete Resnick [presn...@qti.qualcomm.com] Sent: 12 June 2013 20:17 To: Bob Hinden Cc: ietf@ietf.org Subject: Re: Content-free Last Call comments On 6/12/13 10:33 AM, Bob Hinden wrote: Please describe the context of your email. Are you speaking for the IESG, yourself as an AD, or an individual? Oh, crap. And given that I'm usually the one giving people a hard time about *this* issue, I feel especially bad about not being clearer. This is Pete, hatless individual talking. There are (as you've seen) IESG people, past and present who disagree with me. Nothing I've said is anything we've come to an IESG consensus around. Insofar as I think that only saying I support this document is a bogus thing to say, I am revealing something about how I behave as an AD in the face of such statements, and my further explanations that I value reviews and take them seriously also indicates my views as an AD, but that doesn't indicate some policy that I'm instituting with WGs for which I am responsible. I am terribly sorry about not making this clear up front. My sincere apologies. I hope nobody took this as a other than my commentary. Sheepishly, pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: Content-free Last Call comments
On 6/12/13 5:10 PM, Stephen Farrell wrote: On 06/12/2013 10:56 PM, l.w...@surrey.ac.uk wrote: Are the IESG people who disagree with you speaking for the IESG, or for themselves? That's really not clear already? In any case, I was disagreeing with Pete as an individual since he was wrong regardless of hats. He and I do that all the time and both find it amusing:-) Indeed, Stephen's wrongness is clearly a personal wrongness and not because he is on the IESG. ;-) Seriously though, Bob and I chatted about this a bit offline, and I see how I could be interpreted as saying something IESG-ish, and that was absolutely not my intention. Importantly, I also have no interest in turning anything I've said here into some procedure or policy, even if I convince Stephen and others of the error of their ways. :-) This conversation is truly about a set of principles, that (in my very individual opinion that I hope others will share) we should be careful to not fall into the pattern of treating Last Calls as simple votes on a topic, that we should not use language that leads us (or more importantly newcomers) to believe that we are doing so, and that we should be more concerned about why we think a document is good or bad rather than the number (or reputation) of people who claim that it is. Sometimes we fall into the habit of assuming, I know what Pete is talking when he says he supports a document, so everybody else will too. I think that's a bad habit. But this is (hopefully) something for the Tao, not something for RFC 2026. I've noticed a tendency for IESG weight to be (inadvertently?) thrown around, lending more weight to comments than would otherwise be given. I've not noticed that. Maybe I perceive things differently of course, but people ought know not to take opinions any more or less seriously just because someone's on some I* thing at the moment. Much as I would like what Stephen says to be true, I think Lloyd's probably right: People give more weight to opinions coming from people with dots on their name badges. I try to be careful because of that, but I wasn't careful enough this time. pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: Content-free Last Call comments
On Wed, 12 Jun 2013, Pete Resnick wrote: Much as I would like what Stephen says to be true, I think Lloyd's probably right: People give more weight to opinions coming from people with dots on their name badges. One has to be able to see the dots for it to possibly matter. Out here at the end of the email connection, I have almost no clue who might have a dot. I give weight to folks I've observed over time to provide thoughful comments. Many have been ADs, etc. at some point. Unless a message explicitly says I'm writing in role as an AD, it doesn't matter. The exception would be the WG chairs and ADs associated with WGs I follow. I'm well aware that any review comment I'd make would need to logically explain my position because I'm relatively unknown. Dave Morris
Re: Content-free Last Call comments
Hi - From: Ted Lemon ted.le...@nominum.com Sent: Jun 12, 2013 12:42 PM To: Peter Saint-Andre stpe...@stpeter.im Cc: ned+i...@mauve.mrochek.com ned+i...@mauve.mrochek.com, Alexey Melnikov alexey.melni...@isode.com, Pete Resnick presn...@qti.qualcomm.com, ietf@ietf.org Discussion ietf@ietf.org Subject: Re: Content-free Last Call comments On Jun 12, 2013, at 3:31 PM, Peter Saint-Andre stpe...@stpeter.im wrote: I think these messages are useless, not harmful. But perhaps I have more confidence in the inherent skepticism of your average IETF participant than Pete does... FWIW, until I read Pete's document on consensus, I thought that +1 statements were part of the consensus process. This was not a strongly held opinion—it was just my understanding of how consensus operated, from having watched other working group chairs run their working groups. I think the point Pete is making is very important, because the consensus process Pete describes is more in keeping with how I think the IETF ought to operate than the process in which +1 counts for something. ... As a former WG chair who's had to deal with some very rough consensus calls... Not counting a +1 is more consistent with a classical definition of consensus. But, particularly at a WG level (less so, perhaps, at the IETF level) +1 is very helpful in determining whether the previously mentioned Abilene Paradox should be of concern. Randy
Re: Content-free Last Call comments
I'm seeing two things here. One is that you need some context of *why* something is supported, as per your examples. The other is that you need a level of detail that's more than one line. However, I'd note that *all* of those examples are (in my MUA) one line each. So, can you clarify? I.e., if Russ had included one of those lines, would that have been enough? I'd support asking for that level of detail. OTOH I'm not for making people show their work to a greater level of detail; the overhead of participating in an effort is high, and the most relevant people are often exhausted by this point in the process. We shouldn't pile more work onto them (he says, conscious of the work piled onto ADs as well). Regards, On 11/06/2013, at 5:37 AM, Pete Resnick presn...@qti.qualcomm.com wrote: Russ, our IAB chair and former IETF chair, just sent a message to the IETF list regarding a Last Call on draft-ietf-pkix-est. Here is the entire contents of his message, save quoting the whole Last Call request: On 6/10/13 1:45 PM, Russ Housley wrote: I have read the document, I a support publication on the standards track. Russ A month ago, we had another very senior member of the community post just such a message (in that case directly to the IESG) in response to a different Last Call. I took that senior member of the community to task for it. But apparently Russ either disagrees with my complaint or didn't notice that discussion on the IESG list, so I think it's worth airing here in public: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. We don't vote in the IETF, so a statement of support without a reason is meaningless. We should not be encouraging folks to send such things, and having the IAB chair do so is encouraging bad behavior. Had I not known Russ and his particular expertise, I would have no reason to take it into consideration *at all*. We should not have to determine the reputation of the poster to determine the weight of the message. Even given my background knowledge of who Russ is, I cannot tell from that message which one of the following Russ is saying: - This document precisely describes a protocol of which I have been an implementer, and I was able to independently develop an interoperable implementation from the document. - This document is about a technology with which I have familiarity and I have reviewed the technical details. It's fine. - I've seen objection X to the document and I think the objection is incorrect for such-and-so reasons. - My company has a vested interest in this technology becoming a standard, and even though I know nothing about it, I support it becoming a standards track document. - My Aunt Gertrude is the document editor and she said that she needs statements of support, so here I am. - I have a running wager on when we're going to reach RFC 7000 and I want to increase my odds of winning. I take it I am supposed to presume from my friendship and knowledge of Russ that one of the first three is true and that the last three are not. (Well, maybe the last one might be true.) But if instead of from Russ Housely, the message was from Foo Bar, I would have absolutely no way to distinguish among the above. I think we should stop with these one-line statements of support. They don't add anything to the consensus call. I'm disappointed that Russ contributed to this pattern. Other opinions? pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478 -- Mark Nottingham http://www.mnot.net/
Re: Content-free Last Call comments
so now i am expected to do a write-up of why i show simple support of a document i have read? may i use carbon paper for the triplicate, or will a copier suffice? surely we can find a way to waste more time and effort. randy
Re: Content-free Last Call comments
On Jun 11, 2013, at 4:51 AM, Randy Bush ra...@psg.com wrote: so now i am expected to do a write-up of why i show simple support of a document i have read? may i use carbon paper for the triplicate, or will a copier suffice? surely we can find a way to waste more time and effort. If you say I support publication, in IETF last call without saying why, you are arguably wasting other peoples' time. If you say I read the document and I think it's technically solid; furthermore, it addresses a real need, then you've spent a solid 5 seconds more typing, and we have something to go on. Keep your carbon paper in the drawer where it belongs.
RE: Content-free Last Call comments
We have to know, not that you have read the document, but that you have -understood- it. Lloyd Wood http://sat-net.com/L.Wood/ From: ietf-boun...@ietf.org [ietf-boun...@ietf.org] On Behalf Of Randy Bush [ra...@psg.com] Sent: 11 June 2013 09:51 To: Mark Nottingham Cc: Pete Resnick; ietf@ietf.org Subject: Re: Content-free Last Call comments so now i am expected to do a write-up of why i show simple support of a document i have read? may i use carbon paper for the triplicate, or will a copier suffice? surely we can find a way to waste more time and effort. randy
RE: Content-free Last Call comments
Ad-hominem arguments are not good arguments. Peer review depends on what the peer says, not who the peer is - something any academic should know. Lloyd Wood http://sat-net.com/L.Wood/ From: ietf-boun...@ietf.org [ietf-boun...@ietf.org] On Behalf Of Stephen Farrell [stephen.farr...@cs.tcd.ie] Sent: 11 June 2013 01:36 To: Pete Resnick Cc: ietf@ietf.org Subject: Re: Content-free Last Call comments Hi Pete, I think you err when you say this: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. In fact, you do know Russ. If you did not, then the above would be far closer to correct. But in reality you do know a lot more than you claim below. With overwhelming probability, your choice #2 applies and I'm very surprised you don't also think that. If it made a significant difference and I wasn't sure I'd ask Russ to clarify. For me, I read Russ' mail and I concluded he meant #2. And I'm confident in that conclusion. So I'm really bemused when you say that you don't know how to interpret Russ' mail. Declaring that mail problematic also seems quite purist to me. And insisting on purity seems to me worse than being slightly but harmlessly ambiguous. At the same time I agree we do not want a procession of +1 mails. But then we won't get that for this draft. And if we did get that for any draft, some IETF participants (probably incl. you and I) would notice that and query it or object. So perhaps you're also being a tad trigger-happy on jumping on this message. Lastly, I think evaluating IETF LC messages only in terms of how they help the IESG evaluate consensus or not is wrong. Those messages are for and from the IETF community. So if e.g. some renowned NFS person who's hardly known to the IESG were to have sent an equivalent message, that might have been quite good input that Martin or Spencer could have translated for you. And that's just fine. And it would be fine if Russ' mail had been directed not at the IESG but at some other part of the community. So bottom line, I think you're wrong, Russ' mail was not content-free. S. PS: Yes, this is a not-very-good academic accusing an employee of an industrial behemoth of excess purity. Go figure:-) On 06/10/2013 09:37 PM, Pete Resnick wrote: Russ, our IAB chair and former IETF chair, just sent a message to the IETF list regarding a Last Call on draft-ietf-pkix-est. Here is the entire contents of his message, save quoting the whole Last Call request: On 6/10/13 1:45 PM, Russ Housley wrote: I have read the document, I a support publication on the standards track. Russ A month ago, we had another very senior member of the community post just such a message (in that case directly to the IESG) in response to a different Last Call. I took that senior member of the community to task for it. But apparently Russ either disagrees with my complaint or didn't notice that discussion on the IESG list, so I think it's worth airing here in public: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. We don't vote in the IETF, so a statement of support without a reason is meaningless. We should not be encouraging folks to send such things, and having the IAB chair do so is encouraging bad behavior. Had I not known Russ and his particular expertise, I would have no reason to take it into consideration *at all*. We should not have to determine the reputation of the poster to determine the weight of the message. Even given my background knowledge of who Russ is, I cannot tell from that message which one of the following Russ is saying: - This document precisely describes a protocol of which I have been an implementer, and I was able to independently develop an interoperable implementation from the document. - This document is about a technology with which I have familiarity and I have reviewed the technical details. It's fine. - I've seen objection X to the document and I think the objection is incorrect for such-and-so reasons. - My company has a vested interest in this technology becoming a standard, and even though I know nothing about it, I support it becoming a standards track document. - My Aunt Gertrude is the document editor and she said that she needs statements of support, so here I am. - I have a running wager on when we're going to reach RFC 7000 and I want to increase my odds of winning. I take it I am supposed to presume from my friendship and knowledge of Russ that one of the first three is true and that the last three are not. (Well, maybe the last one might be true.) But if instead of from Russ Housely, the message was from Foo Bar, I would have absolutely no way to distinguish among the above. I think we should stop with these one-line statements of support. They don't add anything to the consensus call. I'm
Re: Content-free Last Call comments
A comment is a comment (important for discussing) which I want to see, no matter if content-free or not, the origin requester (IETF Last Call/WGLC) of such comments SHOULD specify which type of comment they want if necessary. As long as it is a comment-on-discuss-lists any can ask questions to the commentor to know the comment-reason if necessary. IMO, we don't want no-comment, ignorance, or no answers if requested, which will mean no discussions. AB
Re: Content-free Last Call comments
On Jun 11, 2013, at 13:17, l.w...@surrey.ac.uk wrote: We have to know, not that you have read the document, but that you have -understood- it. Process experiment: end all Internet-Drafts with a multiple-choice test. Grüße, Carsten
Re: Content-free Last Call comments
On Mon, Jun 10, 2013 at 9:37 PM, Pete Resnick presn...@qti.qualcomm.comwrote: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. I think this is, in part, due to the question asked. The IETF's Last Call announcement presumes much on the part of those reading it. You're aiming to solicit something that's not asked for. Compare and contrast with the XSF's Last Call announcements, in particular the questionnaire at the end. Note that in this thread, almost all respondents are actually filling it in, and further note that at least some of those are experienced IETF participants, suggesting that even the jaded IETF folk might join in. http://jabber.996255.n3.nabble.com/LAST-CALL-XEP-0308-Last-Message-Correction-td14079.html I'd suggest that putting together a set of five questions you're hoping to have answered would be sensible and useful. Perhaps: 1) Do you believe this document is needed? 2) Is the document ready for publication as-is? 3) Are you intending to, or have you already, implemented and/or deployed this specification? 4) Does the document adequately explain the risks involved in implementing and/or deploying this specification? 5) Is the document sufficiently clear to allow unambiguous understanding of how to implement and/or deploy the specification? Dave.
Re: Content-free Last Call comments
Subject: Re: Content-free Last Call comments Date: Mon, Jun 10, 2013 at 11:46:29PM + Quoting Ted Lemon (ted.le...@nominum.com): Determining consensus in an IETF last call is a bit more complicated than that. It's not a working group last call. If someone objects to publication during IETF last call, and their objection has already been discussed and addressed in the working group, the objection in IETF last call doesn't break that consensus. So, if wg discussion has been ordered mute by the wg chairs because some wg participants believe the group-think consensus is good enough, can those objections again be raised in IETF LC or are they set in stone? -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 NATHAN ... your PARENTS were in a CARCRASH!! They're VOIDED -- They COLLAPSED They had no CHAINSAWS ... They had no MONEY MACHINES ... They did PILLS in SKIMPY GRASS SKIRTS ... Nathan, I EMULATED them ... but they were OFF-KEY ... signature.asc Description: Digital signature
Re: Content-free Last Call comments
Subject: Re: Content-free Last Call comments Date: Tue, Jun 11, 2013 at 01:52:46PM +0200 Quoting Måns Nilsson (mansa...@besserwisser.org): So, if wg discussion has been ordered mute by the wg chairs because some wg participants believe the group-think consensus is good enough, can those objections again be raised in IETF LC or are they set in stone? s/they/the WG decisions/ My apologies. -- Måns Nilsson primary/secondary/besserwisser/machina MN-1334-RIPE +46 705 989668 Once, there was NO fun ... This was before MENU planning, FASHION statements or NAUTILUS equipment ... Then, in 1985 ... FUN was completely encoded in this tiny MICROCHIP ... It contain 14,768 vaguely amusing SIT-COM pilots!! We had to wait FOUR BILLION years but we finally got JERRY LEWIS, MTV and a large selection of creme-filled snack cakes! signature.asc Description: Digital signature
Re: Content-free Last Call comments
On 6/11/2013 6:36 AM, Dave Cridland wrote: I think this is, in part, due to the question asked. The IETF's Last Call announcement presumes much on the part of those reading it. You're aiming to solicit something that's not asked for. Compare and contrast with the XSF's Last Call announcements, in Re-formulating the LC text sounds like an excellent idea, to call for more substantive comments. I'd suggest that putting together a set of five questions you're hoping to have answered would be sensible and useful. Perhaps: If we want the statements of support to be meaningful, they need to have the creator of the statement do some real work -- more than mechanically checking boxes -- demonstrating the 'understanding' that Lloyd suggests. Multiple guess questions don't demonstrate understanding; worse, they are too easily plagiarized as part of a campaign. One of the unfortunate realities in the current IETF is periodically seeing patterns of support that have more to do with politicking for a draft than for commenting on a critical review of it. There is no perfect protection against this, but asking each statement of support to demonstrate the commenter's own understanding will help. We also sometimes have drafts that have had little working group activity. This is independent of the quality of the work, but it means that there's little sense of community need or interest. It's not supposed to happen, but it's become more common in the current IETF. Again, there's no perfect protection against that, but seeing public activity during IETF LC that demonstrates enough community interest to do the minimal work of offering a capsule commentary on the draft will help. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: Content-free Last Call comments
On Jun 11, 2013, at 7:52 AM, Måns Nilsson mansa...@besserwisser.org wrote: So, if wg discussion has been ordered mute by the wg chairs because some wg participants believe the group-think consensus is good enough, can those objections again be raised in IETF LC or are they set in stone? You can always challenge the WG chair's finding, but you don't need to hassle the whole IETF to do that—just talk to the responsible AD for the working group, or to the IESG as a whole if the responsible AD fails you. That's what the appeals process is supposed to be for. You can of course raise the point on the IETF mailing list, and that is likely to result in the responsible AD considering the question of whether the WG chairs made the right call—I certainly would do so if someone raised such a point on a document for which I was responsible AD. But if the responsible AD decides that the chair made the right call, the objection you raise in IETF last call doesn't count against the working group consensus.
Re: Content-free Last Call comments
On Tue, Jun 11, 2013 at 12:58 PM, Dave Crocker d...@dcrocker.net wrote: If we want the statements of support to be meaningful, they need to have the creator of the statement do some real work -- more than mechanically checking boxes -- demonstrating the 'understanding' that Lloyd suggests. Multiple guess questions don't demonstrate understanding; worse, they are too easily plagiarized as part of a campaign. We want understanding, of course, but I think requiring Russ to demonstrate that by writing a paragraph or six on the finer points of the proposal would be daft. One of the unfortunate realities in the current IETF is periodically seeing patterns of support that have more to do with politicking for a draft than for commenting on a critical review of it. There is no perfect protection against this, but asking each statement of support to demonstrate the commenter's own understanding will help. If the politicking is from multiple organizations who all want to implement and deploy, then I'm all in favour... If there's only one implementer willing to say as much, then even quite a slew of deployment wannabees would have me concerned for the viability of the protocol. I'd note that the XSF's questions are only concerned with implementation rather than deployment - maybe that helps, I'm not sure either way. We also sometimes have drafts that have had little working group activity. This is independent of the quality of the work, but it means that there's little sense of community need or interest. It's not supposed to happen, but it's become more common in the current IETF. Again, there's no perfect protection against that, but seeing public activity during IETF LC that demonstrates enough community interest to do the minimal work of offering a capsule commentary on the draft will help. Perhaps having a shepherd-style write up included in the last call announcement? (Or available via a URL there). Dave.
Re: Content-free Last Call comments
On 6/11/2013 5:25 AM, Dave Cridland wrote: We want understanding, of course, but I think requiring Russ to demonstrate that by writing a paragraph or six on the finer points of the proposal would be daft. That's the problem with special-case exceptions, such as requiring less work by an august personage. It reduces to a cult of personality and it doesn't scale. For an organizational culture of the type the IETF expresses, that doesn't fit. The opinions of people IETF management positions are not supposed to automatically have more weight in determining the specifics of our specifications; they are supposed to make their case, just like everyone else. We try to distinguish between comment when wearing a formal IETF hat versus without a hat. So it's not the IETF Chair making the comment, it's merely a well-known personage. It's easy to give special rights to such folk, such as not requiring them to offer the substance behind their statement, but it actually has a pretty insidious effect. It's gets us used to pro-forma postings; it gets us relying on a few folk to sway things; it gets us to count rather than think. If the politicking is from multiple organizations who all want to implement and deploy, then I'm all in favour... Pete Resnick has been working on a careful formulation of what the IETF means when it talks about 'rough consensus'. My own interpretation of what he's developing -- and I want to stress this is me speaking, not me speaking for Pete -- is that consensus is a combination of both numbers and substance. The mere fact that almost everyone is in favor of something can't be enough. What is also required is that the arguments of objectors must have inadequately persuasive substance. One voice with a really solid concern, which withstands independent review, needs to be able to upset an overwhelming agreement. So no, the fact that the politicking is from multiple organizations needs to be insufficient. We also sometimes have drafts that have had little working group activity. ... Perhaps having a shepherd-style write up included in the last call announcement? (Or available via a URL there). Perhaps something like that, yeah. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: Content-free Last Call comments
On Tue, Jun 11, 2013 at 1:45 PM, Dave Crocker d...@dcrocker.net wrote: On 6/11/2013 5:25 AM, Dave Cridland wrote: We want understanding, of course, but I think requiring Russ to demonstrate that by writing a paragraph or six on the finer points of the proposal would be daft. That's the problem with special-case exceptions, such as requiring less work by an august personage. It reduces to a cult of personality and it doesn't scale. For an organizational culture of the type the IETF expresses, that doesn't fit. The opinions of people IETF management positions are not supposed to automatically have more weight in determining the specifics of our specifications; they are supposed to make their case, just like everyone else. We try to distinguish between comment when wearing a formal IETF hat versus without a hat. So it's not the IETF Chair making the comment, it's merely a well-known personage. It's easy to give special rights to such folk, such as not requiring them to offer the substance behind their statement, but it actually has a pretty insidious effect. It's gets us used to pro-forma postings; it gets us relying on a few folk to sway things; it gets us to count rather than think. Ah, sorry, that's not what I meant - I included Russ's name purely because he was the original exemplar, not because he's special in any particular way. I meant that requiring anyone to demonstrate understand of the draft by jumping through hoops would, ipso facto, require them to jump through hoops. If the politicking is from multiple organizations who all want to implement and deploy, then I'm all in favour... Pete Resnick has been working on a careful formulation of what the IETF means when it talks about 'rough consensus'. My own interpretation of what he's developing -- and I want to stress this is me speaking, not me speaking for Pete -- is that consensus is a combination of both numbers and substance. The mere fact that almost everyone is in favor of something can't be enough. What is also required is that the arguments of objectors must have inadequately persuasive substance. One voice with a really solid concern, which withstands independent review, needs to be able to upset an overwhelming agreement. So no, the fact that the politicking is from multiple organizations needs to be insufficient. Again, I think you're misunderstanding me - I meant (somewhat facetiously) I'm in favour of the politicking, not that I think that it should carry the day automatically. If there are unanswered objections, that should indeed count against. More generally, in the case of the XSF's small set of questions, if people answer the last call with one-word answers to those and nothing further, this gives that community sufficient information to gauge whether to advance a proposal along the standards track there - in other words, given a fairly minimal bar, any engagement meeting that bar is valuable. That bar has to be high enough to carry more than the single bit of information Russ's note carried on its own, though, but it also needs to be low enough that it won't prove a barrier to response. I think the XSF's questions are close to the right level (I think the XSF could, if it wanted, tweak these and improve them after this number of years); I don't think it would be very hard to find some similarly reasonable start point for the IETF. Dave.
Re: Content-free Last Call comments
On 6/11/13 8:12 AM, Ted Lemon wrote: On Jun 11, 2013, at 7:52 AM, Måns Nilsson mansa...@besserwisser.org wrote: So, if wg discussion has been ordered mute by the wg chairs because some wg participants believe the group-think consensus is good enough, can those objections again be raised in IETF LC or are they set in stone? You can always challenge the WG chair's finding, but you don't need to hassle the whole IETF to do that—just talk to the responsible AD for the working group, or to the IESG as a whole if the responsible AD fails you. That's what the appeals process is supposed to be for. You can of course raise the point on the IETF mailing list, and that is likely to result in the responsible AD considering the question of whether the WG chairs made the right call—I certainly would do so if someone raised such a point on a document for which I was responsible AD. But if the responsible AD decides that the chair made the right call, the objection you raise in IETF last call doesn't count against the working group consensus. Engaging during IETF LC on a point made during WGLC or earlier that wasn't adopted is a-okay in my book. I think of it as raising awareness. I mean maybe the authors and WG chair didn't get it right. But, then what Ted said kicks in. spt
Re: Content-free Last Call comments
Re-formulating the LC text sounds like an excellent idea, to call for more substantive comments. perhaps we should go to the source of the problem and require a phd dissertation and defense from draft authors. how much process chaos can we create? randy
Re: Content-free Last Call comments
On Jun 11, 2013, at 9:41 AM, Randy Bush ra...@psg.com wrote: how much process chaos can we create? Don't ask questions you don't want answered! :)
Re: Content-free Last Call comments
On Tue, Jun 11, 2013 at 2:41 PM, Randy Bush ra...@psg.com wrote: Re-formulating the LC text sounds like an excellent idea, to call for more substantive comments. perhaps we should go to the source of the problem and require a phd dissertation and defense from draft authors. how much process chaos can we create? I think using a reductio ad absurdum is a little unfair when I've demonstrated that the proposed reformulation works (whilst being purely voluntary) in a similar organization with common participants. In particular, I'm not suggesting any change to process. Dave.
RE: Content-free Last Call comments
How many RFCs describe things that are implemented? How many RFCs describe things that are deployed? Lloyd Wood http://sat-net.com/L.Wood/ From: ietf-boun...@ietf.org [ietf-boun...@ietf.org] On Behalf Of Dave Cridland [d...@cridland.net] Sent: 11 June 2013 12:36 To: Pete Resnick Cc: ietf@ietf.org Discussion Subject: Re: Content-free Last Call comments On Mon, Jun 10, 2013 at 9:37 PM, Pete Resnick presn...@qti.qualcomm.commailto:presn...@qti.qualcomm.com wrote: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. I think this is, in part, due to the question asked. The IETF's Last Call announcement presumes much on the part of those reading it. You're aiming to solicit something that's not asked for. Compare and contrast with the XSF's Last Call announcements, in particular the questionnaire at the end. Note that in this thread, almost all respondents are actually filling it in, and further note that at least some of those are experienced IETF participants, suggesting that even the jaded IETF folk might join in. http://jabber.996255.n3.nabble.com/LAST-CALL-XEP-0308-Last-Message-Correction-td14079.html I'd suggest that putting together a set of five questions you're hoping to have answered would be sensible and useful. Perhaps: 1) Do you believe this document is needed? 2) Is the document ready for publication as-is? 3) Are you intending to, or have you already, implemented and/or deployed this specification? 4) Does the document adequately explain the risks involved in implementing and/or deploying this specification? 5) Is the document sufficiently clear to allow unambiguous understanding of how to implement and/or deploy the specification? Dave.
Re: Content-free Last Call comments
So, if wg discussion has been ordered mute by the wg chairs because some wg participants believe the group-think consensus is good enough, can those objections again be raised in IETF LC or are they set in stone? If that were ever to happen, I don't see why not. In the recent cases I've seen where someone claims this is an issue, what actually happned is that a person only tangentially involved in the WG is obsessing about some technical wart and refuses to (or worse, can't) understand the overall context that led the WG to decide what it did. It's hard to see any benefit to rehashing such arguments in front of the whole IETF. R's, John
Re: Content-free Last Call comments
On 06/11/2013 10:43 AM, John Levine wrote: So, if wg discussion has been ordered mute by the wg chairs because some wg participants believe the group-think consensus is good enough, can those objections again be raised in IETF LC or are they set in stone? If that were ever to happen, I don't see why not. In the recent cases I've seen where someone claims this is an issue, what actually happned is that a person only tangentially involved in the WG is obsessing about some technical wart and refuses to (or worse, can't) understand the overall context that led the WG to decide what it did. It's hard to see any benefit to rehashing such arguments in front of the whole IETF. The flip side of that argument is that we don't want to assume working groups are infallible, or more importantly not subject to the groupthink phenomenon. Otherwise what is IETF LC for? Doug
Re: Content-free Last Call comments
On Jun 11, 2013, at 1:52 PM, Doug Barton do...@dougbarton.us wrote: The flip side of that argument is that we don't want to assume working groups are infallible, or more importantly not subject to the groupthink phenomenon. Otherwise what is IETF LC for? The IETF last call is for catching things the working group missed, not for rehashing arguments that were beaten to death in the working group. It is certainly possible for a discussion in the working group to go one way, and then for the same discussion to come up in IETF last call and go the other way, because the experts on the topic were not included in the discussion, or because their advice was inappropriately ignored. This does happen; unfortunately, when it happens it often doesn't get caught in IETF last call anyway.
Re: Content-free Last Call comments
On 6/11/13 9:52 AM, Doug Barton wrote: The flip side of that argument is that we don't want to assume working groups are infallible, or more importantly not subject to the groupthink phenomenon. Otherwise what is IETF LC for? Right. We've had some issues with document quality, and I can think of several documents that sailed through WG last call and should not have. Melinda
Re: Content-free Last Call comments
On 6/11/13 10:02 AM, Ted Lemon wrote: The IETF last call is for catching things the working group missed, not for rehashing arguments that were beaten to death in the working group. I am not sure I fully understand why we're having this conversation, or rather why this aspect of the broader discussion requires attention. Sometimes working groups make mistakes, and I don't think that in practice there'd be general objection to having serious problems identified during IETF last call. Yes, there have been cases where cranks who can't let go of an idea that was rejected try to flog it to death, but that doesn't mean that identifying and dealing with real problems should be dismissed. Melinda
Re: Content-free Last Call comments
It's interesting to see that people are interpreting me to mean I want more text. I don't. I want less. Save your breath. There is no reason to send one line of support, and it only encourages the view that we're voting. Details below. Specifically on Stephen's message: On 6/10/13 7:36 PM, Stephen Farrell wrote: I think you err when you say this: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. In fact, you do know Russ. If you did not, then the above would be far closer to correct. But in reality you do know a lot more than you claim below. With overwhelming probability, your choice #2 applies and I'm very surprised you don't also think that. If it made a significant difference and I wasn't sure I'd ask Russ to clarify. For me, I read Russ' mail and I concluded he meant #2. And I'm confident in that conclusion. So I'm really bemused when you say that you don't know how to interpret Russ' mail. Let's separate out a few issues. There's what the message means itself, there's its usefulness and how much weight it should be given, and then there's what's problematic about it. I think you've conflated a few things. As you know, on the earlier message I called out on the IESG list, I guessed wrong that the sender was claiming choice #3 (that is, I thought, incorrectly, that the sender disagreed with a recent argument against publication, but simply didn't explain why). In fact in that case, not only wasn't it choice #4, it was also neither choice #1 (an implementer claiming implementation) nor choice #2 (an expert on the topic), my other two guesses, but rather a generally smart IETFer who thought the work seemed like a good idea. So I'm a little disinclined to guess on motives. In the case of Russ's message, I did guess it was choice #2, but I wasn't really sure. But let's separate what Russ meant from the weight it should be given. The message was that a person (let's presume with some expertise in the area) had read it. So what? Didn't this document go through a WG? Weren't a good bunch of experts already reading and reviewing this document? If this was during the WG discussion and the chairs solicited some final check reviews, *maybe* the message could have some use. But during IETF Last Call? Even if the message was sent because it was trying to say, I'm an expert on this stuff and it looks technically sound, is that going to change anything that the IESG should do about the document? I just think at Last Call, these statements of support are either harmless-but-useless or they are nefarious. Declaring that mail problematic also seems quite purist to me. I always find it amusing that when Stephen and I disagree, it's almost always because he presumes I want purity (and usually a new process), which I don't, and it seems to me that he wants no rules-of-thumb at all and that everything should be on a case-by-case basis. (I guess that means that I'm into purity, where Stephen has no principles. ;-) ) I think general principles are a fine thing, and we should try to stick to them, but I've got no interest in purity. The single piece of mail isn't problematic at all. As I said, probably harmless but useless. But the overall impression it gives *is* problematic. It encourages the view that we are voting, that a simple statement of support is important to our process. (What really set my hair on end was the I support publication bit. That always sounds like a vote for a new RFC. From Russ, I have a pretty good idea that he didn't mean it that way, but it's a poor formulation and not what we should be encouraging.) It also indicates that during Last Call, we want to hear that a document is getting reviewed. I would like to presume that documents get serious reviews in the WG, and that Last Call time is for people who haven't been participating in the WG to do a final check, letting us know if there's something *wrong*. If the document wasn't getting good review in the WG and needs statements of support at Last Call time, something has seriously failed earlier in the process. The chair or responsible AD should have been saying early on, Dear Expert, can you please have a read over this document and see if it's sane. Last Call is the wrong time for that to happen. This just encourages more of the tail-heavy process that we discussed earlier. At the same time I agree we do not want a procession of +1 mails. Right. And having senior members of the community doing so will encourage such behavior in the future. But then we won't get that for this draft. And if we did get that for any draft, some IETF participants (probably incl. you and I) would notice that and query it or object. I'd rather have all of us crusty folks model good behavior rather than having to complain about bad behavior later, especially when we'll get responses like,
Re: Content-free Last Call comments
Pete, On 12/06/2013 07:45, Pete Resnick wrote: It's interesting to see that people are interpreting me to mean I want more text. I don't. I want less. Save your breath. There is no reason to send one line of support, and it only encourages the view that we're voting. Details below. Just to test what you are saying, let me ask the following. I will stipulate that a message that can be summarised accurately as +1 doesn't serve much purpose. How would you react to one that says something like: I've read the draft, and I've considered Joe Blow's objection, but I still support publication of the draft (*not followed by a reasoned rebuttal of Joe Blow's argument)? Brian
Re: Content-free Last Call comments
On Tue, Jun 11, 2013 at 8:45 PM, Pete Resnick presn...@qti.qualcomm.comwrote: It's interesting to see that people are interpreting me to mean I want more text. I don't. I want less. Save your breath. Well, this thread is surely evidence that you don't always get what you want. But more seriously, what are you expecting Russ to do? What did you want him to write? If your answer is Nothing, then how do you read IETF consensus for a document that gets no response in its Last Call? The XSF's stance is often It got nothing in Last Call, it shouldn't advance, which seems reasonable to me - I don't think defaulting to publication is right - certainly not in every case. My suggestion was simply to ask for what you want in the Last Call. But I imagine you could also just start an endless thread on an already overloaded list until someone guesses the answer. Dave.
Re: Content-free Last Call comments
On Jun 11, 2013, at 4:24 PM, Dave Cridland d...@cridland.netmailto:d...@cridland.net wrote: But more seriously, what are you expecting Russ to do? What did you want him to write? If your answer is Nothing, then how do you read IETF consensus for a document that gets no response in its Last Call? The XSF's stance is often It got nothing in Last Call, it shouldn't advance, which seems reasonable to me - I don't think defaulting to publication is right - certainly not in every case. My suggestion was simply to ask for what you want in the Last Call. I have a modest suggestion for you: read the rest of the message from Pete that you just replied to, in which he answers each of the questions you asked in your reply.
Re: Content-free Last Call comments
On 6/11/13 3:05 PM, Brian E Carpenter wrote: Pete, On 12/06/2013 07:45, Pete Resnick wrote: It's interesting to see that people are interpreting me to mean I want more text. I don't. I want less. Save your breath. There is no reason to send one line of support, and it only encourages the view that we're voting. Details below. Just to test what you are saying, let me ask the following. Oooo...I love a test. How would you react to one that says something like: I've read the draft, and I've considered Joe Blow's objection, but I still support publication of the draft (*not followed by a reasoned rebuttal of Joe Blow's argument)? I would be rather grumpy with such a message. If there's an outstanding (reasonable) objection to a document, I need to know why to consider that argument in the rough. I'd have to ask for more detail from the sender. If the response I get back is, I figured it was obvious why Joe Blow was full of crap, I'd ask, Then why did you bother posting? If the sender happens to be an expert (and Joe Blow is not), I'm still not going to take it at face value that Joe Blow is wrong. If I did, Joe would be well within rights to appeal because his argument got blown off. So, if you're saying something that is perfectly obvious, no need to say it. But if it's not perfectly obvious, I do want more text. pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: Content-free Last Call comments
On 06/11/2013 11:02 AM, Ted Lemon wrote: On Jun 11, 2013, at 1:52 PM, Doug Barton do...@dougbarton.us wrote: The flip side of that argument is that we don't want to assume working groups are infallible, or more importantly not subject to the groupthink phenomenon. Otherwise what is IETF LC for? The IETF last call is for catching things the working group missed, not for rehashing arguments that were beaten to death in the working group. As I understand it cross-disciplinary review is also an important function of the IETF LC. It is certainly possible for a discussion in the working group to go one way, and then for the same discussion to come up in IETF last call and go the other way, because the experts on the topic were not included in the discussion, or because their advice was inappropriately ignored. Right ... for example the recent issue in regards to the potential deprecation of the SPF DNS RRtype. This does happen; unfortunately, when it happens it often doesn't get caught in IETF last call anyway. Then the process is faulty. :) I should also point out that when I mentioned groupthink in my message I was not doing so in a snarky and/or throwaway manner. The phenomenon is very real, and IETF WGs are an ideal breeding ground for it on several levels. It's _very_ easy for a group of humans to get into a mutual confirmation bias feedback loop, and although we generally do better on that point than a lot of groups would in similar circumstances, having people from outside the WG review documents provides (or should provide) a much-needed sanity check. In many ways having people from outside the group provide a well thought out review is _more_ valuable than the process of creating the document within the WG itself (although obviously we need to give proper credit for that creation). Doug
Re: Content-free Last Call comments
On Tue, Jun 11, 2013 at 9:33 PM, Ted Lemon ted.le...@nominum.com wrote: On Jun 11, 2013, at 4:24 PM, Dave Cridland d...@cridland.net wrote: But more seriously, what are you expecting Russ to do? What did you want him to write? If your answer is Nothing, then how do you read IETF consensus for a document that gets no response in its Last Call? The XSF's stance is often It got nothing in Last Call, it shouldn't advance, which seems reasonable to me - I don't think defaulting to publication is right - certainly not in every case. My suggestion was simply to ask for what you want in the Last Call. I have a modest suggestion for you: read the rest of the message from Pete that you just replied to, in which he answers each of the questions you asked in your reply. If he did, either I am sufficiently stupid I cannot make it out, or he did so sufficiently obliquely that I can't see - depending on where you want to place the blame. Either way, I've tried to parse out potential responses (again), and the closest I can come (again) is that Pete seems to want nothing from Russ in this instance, and indeed, in any instance from any person except objections. This is not clear to me, however, and therefore I'd appreciate some clarification, which is why I asked. That in turn presumes we are defaulting to publication in all cases, and that in turn seems problematic to me, because his answers become, in order: a) Russ, and by extension anyone who supports the document's publication for whatever reason, is expected to do nothing. b) Russ, and by extension anyone who supports the document's publication for whatever reason, should write nothing. c) IETF-wide consensus is not judged here. IETF-wide apathy is IETF-wide consensus. My conclusion from that are: 1) Pete strongly supports publish-by-default. 2) Pete has his RFC 7000 sweepstake money on a much earlier date than I do. Dave.
Re: Content-free Last Call comments
On 6/11/2013 1:50 PM, Doug Barton wrote: As I understand it cross-disciplinary review is also an important function of the IETF LC. This gets at the reality that the current IETF uses processing phases rather more robustly than we used to. It certainly used to be that IETF LC was essentially a last chance for people with objections, but now it's a full round of broad reviews. It is, therefore, a real vetting phase by the larger community. As such, positive comments can be as valuable as negative, but of course only if they are backed with substance. Possibly contrary to Pete's stated preference, I'll suggest that statements of pure support -- backed up with substantive commentary -- can be useful, given the low level of activity and involvement many working groups currently show. To the extent that the IETF cares whether anyone beyond the authors care about the document, this shows that they do. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: Content-free Last Call comments
On Jun 11, 2013, at 5:27 PM, Dave Cridland d...@cridland.net wrote: That in turn presumes we are defaulting to publication in all cases, and that in turn seems problematic to me, because his answers become, in order: a) Russ, and by extension anyone who supports the document's publication for whatever reason, is expected to do nothing. b) Russ, and by extension anyone who supports the document's publication for whatever reason, should write nothing. c) IETF-wide consensus is not judged here. IETF-wide apathy is IETF-wide consensus. It is presumed that some degree of consensus to do the work of a working group existed when that working group was chartered; otherwise it would not have been chartered. When the working group reaches consensus to publish, therefore, it is assumed that the IETF has consensus to publish the document, because the IETF tasked the working group to go off and do its work, and the working group did it. Therefore, silence during IETF last call is not interpreted as apathy, but rather a lack of objection to the completion of a process that the IETF chose to embark on and that the IETF has brought to completion, through the instrument of the working group that produced the document. This is in fact how consensus is evaluated during IETF last call. If you think it should be done differently, write up a document and get IETF consensus on it, and we can change the procedure to whatever you think it should be. Maybe it would be an improvement.
Re: Content-free Last Call comments
On 10/06/13 21:37, Pete Resnick wrote: Russ, our IAB chair and former IETF chair, just sent a message to the IETF list regarding a Last Call on draft-ietf-pkix-est. Here is the entire contents of his message, save quoting the whole Last Call request: On 6/10/13 1:45 PM, Russ Housley wrote: I have read the document, I a support publication on the standards track. Russ Pete, I write gen-art reviews. A proportion of them at IETF Last Call, give or take a bunch of boilerplate, consist of the word 'Ready'*. How do you distinguish the usefulness (or otherwise) of such a review from Russ' one liner? /Elwyn * and some of the others say 'Not ready' followed by some or many lines of comments.
Re: Content-free Last Call comments
On Tue, Jun 11, 2013 at 10:54 PM, Ted Lemon ted.le...@nominum.com wrote: It is presumed that some degree of consensus to do the work of a working group existed when that working group was chartered; otherwise it would not have been chartered. When the working group reaches consensus to publish, therefore, it is assumed that the IETF has consensus to publish the document, because the IETF tasked the working group to go off and do its work, and the working group did it. Therefore, silence during IETF last call is not interpreted as apathy, but rather a lack of objection to the completion of a process that the IETF chose to embark on and that the IETF has brought to completion, through the instrument of the working group that produced the document. This is in fact how consensus is evaluated during IETF last call. That's interesting - judging by the messages on this thread, there doesn't appear to be a strong consensus on this... If you think it should be done differently, write up a document and get IETF consensus on it, and we can change the procedure to whatever you think it should be. Maybe it would be an improvement. ... and how would we judge IETF consensus on a document that doesn't get done under a charter (which would in turn have been granted consensus without any IETF comments?)
Re: Content-free Last Call comments
On Jun 11, 2013, at 6:03 PM, Dave Cridland d...@cridland.netmailto:d...@cridland.net wrote: ... and how would we judge IETF consensus on a document that doesn't get done under a charter (which would in turn have been granted consensus without any IETF comments?) I would expect that you'd start with a mailing list, see if there is interest, come up with a proposal for a BOF, come up with a working group charter, get IESG review on the charter, then get IETF consensus on the charter, and then start working on the document. That's how it's usually done. BTW, the fact that a few people think the process ought to work differently does not mean there is consensus for it to work differently. Also, what there may not be consensus on among the people who have weighed in on the topic is whether positive statements in favor of a document are relevant in IETF last call, but I don't really know how to reduce that to practice, because in reality I think it is rare for a quorum of IETF participants to read a document as a consequence of a last call announcement. Without that, I don't see how you can have any other last call process than the one we currently have.
Re: Content-free Last Call comments
Right. We've had some issues with document quality, and I can think of several documents that sailed through WG last call and should not have. there was a doc with which i had a small, but non trivial, issue. the author and the wg did not think it worthwhile. i did not want to argue endlessly, so i waited and raised it to the iesg at last call, and they understood it instantly. the author made the small change to the doc. i said thank you. i am sure we can make a constitutional crisis over this if we have too much time on our hands. randy
Re: Content-free Last Call comments
better than saying I have not read the document but I support publication I do not see all that much help in having someone list reasons they support publication unless there is some particularly wonderful feature or the prose is particularly clear the reverse is not the case, I think there is real value in someone saying in detail why the do not support publication of a document Scott On Jun 10, 2013, at 4:37 PM, Pete Resnick presn...@qti.qualcomm.com wrote: Russ, our IAB chair and former IETF chair, just sent a message to the IETF list regarding a Last Call on draft-ietf-pkix-est. Here is the entire contents of his message, save quoting the whole Last Call request: On 6/10/13 1:45 PM, Russ Housley wrote: I have read the document, I a support publication on the standards track. Russ A month ago, we had another very senior member of the community post just such a message (in that case directly to the IESG) in response to a different Last Call. I took that senior member of the community to task for it. But apparently Russ either disagrees with my complaint or didn't notice that discussion on the IESG list, so I think it's worth airing here in public: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. We don't vote in the IETF, so a statement of support without a reason is meaningless. We should not be encouraging folks to send such things, and having the IAB chair do so is encouraging bad behavior. Had I not known Russ and his particular expertise, I would have no reason to take it into consideration *at all*. We should not have to determine the reputation of the poster to determine the weight of the message. Even given my background knowledge of who Russ is, I cannot tell from that message which one of the following Russ is saying: - This document precisely describes a protocol of which I have been an implementer, and I was able to independently develop an interoperable implementation from the document. - This document is about a technology with which I have familiarity and I have reviewed the technical details. It's fine. - I've seen objection X to the document and I think the objection is incorrect for such-and-so reasons. - My company has a vested interest in this technology becoming a standard, and even though I know nothing about it, I support it becoming a standards track document. - My Aunt Gertrude is the document editor and she said that she needs statements of support, so here I am. - I have a running wager on when we're going to reach RFC 7000 and I want to increase my odds of winning. I take it I am supposed to presume from my friendship and knowledge of Russ that one of the first three is true and that the last three are not. (Well, maybe the last one might be true.) But if instead of from Russ Housely, the message was from Foo Bar, I would have absolutely no way to distinguish among the above. I think we should stop with these one-line statements of support. They don't add anything to the consensus call. I'm disappointed that Russ contributed to this pattern. Other opinions? pr -- Pete Resnickhttp://www.qualcomm.com/~presnick/ Qualcomm Technologies, Inc. - +1 (858)651-4478
Re: Content-free Last Call comments
On 6/10/13 2:37 PM, Pete Resnick wrote: I think we should stop with these one-line statements of support. +1 ;-) /psa
Re: Content-free Last Call comments
On 6/10/2013 4:52 PM, Bradner, Scott wrote: etter than saying I have not read the document but I support publication I do not see all that much help in having someone list reasons they support publication unless there is some particularly wonderful feature or the prose is particularly clear the reverse is not the case, I think there is real value in someone saying in detail why the do not support publication of a document Content free statements of support turn the public query into a pure popularity contest and/or a game of personal authority. Is your support more important than Pete's non-support? Absent substance in the statements, we've no idea how to evaluate any disparity. Statements that include substance about the nature and technical adequacies are a form of showing one's work on an exam: they demonstrate the substance that is the basis the support, not just the fact of it. d/ -- Dave Crocker Brandenburg InternetWorking bbiw.net
Re: Content-free Last Call comments
On Jun 10, 2013, at 5:52 PM, Bradner, Scott s...@harvard.edu wrote: I do not see all that much help in having someone list reasons they support publication unless there is some particularly wonderful feature or the prose is particularly clear I don't really see any point in expressing support for a document during IETF last call unless it is as a rejoinder to some valid expression of non-support. A valid expression of non-support would point out some technical point that either was not considered by the working group, or was considered, but the working group wasn't competent to evaluate the question. Or it would point out some process failure that occurred in the working group. Even if you review the document during IETF last call, unless you find a problem with it there's not much point in saying anything.
Re: Content-free Last Call comments
On Mon, 10 Jun 2013, Ted Lemon wrote: On Jun 10, 2013, at 5:52 PM, Bradner, Scott s...@harvard.edu wrote: I do not see all that much help in having someone list reasons they support publication unless there is some particularly wonderful feature or the prose is particularly clear I don't really see any point in expressing support for a document during IETF last call unless it is as a rejoinder to some valid expression of non-support. A valid expression of non-support would point out some technical point that either was not considered by the working group, or was considered, but the working group wasn't competent to evaluate the question. Or it would point out some process failure that occurred in the working group. Even if you review the document during IETF last call, unless you find a problem with it there's not much point in saying anything. I don't think there is another way to indicate you've reviewed a draft and found no issues. Surely rough concensus must include confidence that that silence means more than ignorance and I'm not aware of any mechanism to evaluate the level of review exercised during last call.
Re: Content-free Last Call comments
On Jun 10, 2013, at 6:19 PM, David Morris d...@xpasc.com wrote: I don't think there is another way to indicate you've reviewed a draft and found no issues. Surely rough concensus must include confidence that that silence means more than ignorance and I'm not aware of any mechanism to evaluate the level of review exercised during last call. The expectation is that the working group gave the document sufficient review; the point of the IETF last call then is to backstop that review, not augment it. A document that receives no comment in IETF last call is assumed to have consensus.
Re: Content-free Last Call comments
Hi Pete, At 13:37 10-06-2013, Pete Resnick wrote: A month ago, we had another very senior member of the community post just such a message (in that case directly to the IESG) in response to a different Last Call. I took that senior member of the community to task for it. But apparently Russ either disagrees with my complaint or didn't notice that discussion on the IESG list, so I think it's worth airing here in public: I don't expect IAB or IESG members to be infallible, i.e. they are individuals after all. A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. Yes. We don't vote in the IETF, so a statement of support without a reason is meaningless. We should not be encouraging folks to send such things, and having the IAB chair do so is encouraging bad behavior. Had I not known Russ and his particular expertise, I would have no reason to take it into consideration *at all*. We should not have to determine the reputation of the poster to determine the weight of the message. Even given my background knowledge of who Russ is, I cannot tell from that message which one of the following Russ is saying: The comment was from an individual. The issue is that if you do a blind review of the messages you don't know who sent the message and the only weight you could give is to the content of the message. I think we should stop with these one-line statements of support. They don't add anything to the consensus call. I'm disappointed that Russ contributed to this pattern. I agree that one-line statements are not of much use. It's more tedious to write a statement to support a proposal than an objection to it. Non-silent Last Calls usually draw objections. It's going to be difficult to balance that if one-line statements of support (or objections) are not considered in a determination of consensus. Regards, -sm
Re: Content-free Last Call comments
On Jun 10, 2013, at 7:21 PM, SM s...@resistor.net wrote: I agree that one-line statements are not of much use. It's more tedious to write a statement to support a proposal than an objection to it. Non-silent Last Calls usually draw objections. It's going to be difficult to balance that if one-line statements of support (or objections) are not considered in a determination of consensus. Determining consensus in an IETF last call is a bit more complicated than that. It's not a working group last call. If someone objects to publication during IETF last call, and their objection has already been discussed and addressed in the working group, the objection in IETF last call doesn't break that consensus.
Re: Content-free Last Call comments
In message 8d23d4052abe7a4490e77b1a012b6307751cf...@mbx-01.win.nominum.com, T ed Lemon writes: On Jun 10, 2013, at 7:21 PM, SM s...@resistor.net wrote: I agree that one-line statements are not of much use. It's more tedious = to write a statement to support a proposal than an objection to it. Non-si= lent Last Calls usually draw objections. It's going to be difficult to bal= ance that if one-line statements of support (or objections) are not conside= red in a determination of consensus. Determining consensus in an IETF last call is a bit more complicated than t= hat. It's not a working group last call. If someone objects to publicat= ion during IETF last call, and their objection has already been discussed a= nd addressed in the working group, the objection in IETF last call doesn't = break that consensus. Which breaks some of the reasons why we do IETF last calls. WGs do get too focused on a problem and do fail to do a balance response to problems. Mark -- Mark Andrews, ISC 1 Seymour St., Dundas Valley, NSW 2117, Australia PHONE: +61 2 9871 4742 INTERNET: ma...@isc.org
Re: Content-free Last Call comments
Hi Pete, I think you err when you say this: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. In fact, you do know Russ. If you did not, then the above would be far closer to correct. But in reality you do know a lot more than you claim below. With overwhelming probability, your choice #2 applies and I'm very surprised you don't also think that. If it made a significant difference and I wasn't sure I'd ask Russ to clarify. For me, I read Russ' mail and I concluded he meant #2. And I'm confident in that conclusion. So I'm really bemused when you say that you don't know how to interpret Russ' mail. Declaring that mail problematic also seems quite purist to me. And insisting on purity seems to me worse than being slightly but harmlessly ambiguous. At the same time I agree we do not want a procession of +1 mails. But then we won't get that for this draft. And if we did get that for any draft, some IETF participants (probably incl. you and I) would notice that and query it or object. So perhaps you're also being a tad trigger-happy on jumping on this message. Lastly, I think evaluating IETF LC messages only in terms of how they help the IESG evaluate consensus or not is wrong. Those messages are for and from the IETF community. So if e.g. some renowned NFS person who's hardly known to the IESG were to have sent an equivalent message, that might have been quite good input that Martin or Spencer could have translated for you. And that's just fine. And it would be fine if Russ' mail had been directed not at the IESG but at some other part of the community. So bottom line, I think you're wrong, Russ' mail was not content-free. S. PS: Yes, this is a not-very-good academic accusing an employee of an industrial behemoth of excess purity. Go figure:-) On 06/10/2013 09:37 PM, Pete Resnick wrote: Russ, our IAB chair and former IETF chair, just sent a message to the IETF list regarding a Last Call on draft-ietf-pkix-est. Here is the entire contents of his message, save quoting the whole Last Call request: On 6/10/13 1:45 PM, Russ Housley wrote: I have read the document, I a support publication on the standards track. Russ A month ago, we had another very senior member of the community post just such a message (in that case directly to the IESG) in response to a different Last Call. I took that senior member of the community to task for it. But apparently Russ either disagrees with my complaint or didn't notice that discussion on the IESG list, so I think it's worth airing here in public: A statement such as the above is almost entirely useless to me as an IESG member trying to determine consensus. It is content-free. We don't vote in the IETF, so a statement of support without a reason is meaningless. We should not be encouraging folks to send such things, and having the IAB chair do so is encouraging bad behavior. Had I not known Russ and his particular expertise, I would have no reason to take it into consideration *at all*. We should not have to determine the reputation of the poster to determine the weight of the message. Even given my background knowledge of who Russ is, I cannot tell from that message which one of the following Russ is saying: - This document precisely describes a protocol of which I have been an implementer, and I was able to independently develop an interoperable implementation from the document. - This document is about a technology with which I have familiarity and I have reviewed the technical details. It's fine. - I've seen objection X to the document and I think the objection is incorrect for such-and-so reasons. - My company has a vested interest in this technology becoming a standard, and even though I know nothing about it, I support it becoming a standards track document. - My Aunt Gertrude is the document editor and she said that she needs statements of support, so here I am. - I have a running wager on when we're going to reach RFC 7000 and I want to increase my odds of winning. I take it I am supposed to presume from my friendship and knowledge of Russ that one of the first three is true and that the last three are not. (Well, maybe the last one might be true.) But if instead of from Russ Housely, the message was from Foo Bar, I would have absolutely no way to distinguish among the above. I think we should stop with these one-line statements of support. They don't add anything to the consensus call. I'm disappointed that Russ contributed to this pattern. Other opinions? pr
Re: Content-free Last Call comments
On Jun 10, 2013, at 8:31 PM, Mark Andrews ma...@isc.org wrote: Which breaks some of the reasons why we do IETF last calls. WGs do get too focused on a problem and do fail to do a balance response to problems. If enough IETF last call people agree that the working group made a mistake, that could change the consensus evaluation, but there would have to be a substantive argument made. Revisiting the arguments that happened during WGLC in IETF last call with the same participants can't possibly change the consensus.
