Hi all, Sorry I’m replying again in the middle of this thread but I wanted to clarify my statements and didn’t have time to reply earlier.
> Based on how broken our current errata handing is, I reach the opposite > conclusion: we really need to throw it out and start from scratch. Taking this statement as a staring point for my reply: I think Carsten made this point to some extend as well. I agree that the tooling is bad and we need to replace it entirely, however, I don’t think everything about the errata process is entirely broken and we should also take into account those things that we know are right. I think processing of straightforward, “real” errata that are clearly right or wrong and were an oversight at publication time actually mostly works; yes, it could be easier and if the reporting tooling would be different if would be easier to display them in a useful way to the reader but the process where the AD verifies them is basically fine. The problem comes into play for those things which are not clear errata or at least things that don’t have a clear fix or a full agreement if something needs fixing at all. In my own AD time I looked at all TSV errata at least once but then in some cases I couldn’t figure out easily if something was an errata or not, meaning if the thing reported was really an oversight at time of writing or even something that was wrong at all. The current errata states assume that while the resolution of the errata might not be clear, you can at least make an assessment of there is something wrong that needs some fixing (verified or HFDU) or if the RFC as it is fully correct (reject). Also reading the rest of the discussion I think we need one more state that says something like “I looked at it and won’t reject because it not obviously not an errata it but it also not clear if it is one or what to do with it - however, it not worth addressing it now, let’s keep it on record for later”. Not sure how to describe this in one word thought. This is actually another problem of the current system that ADs don’t interpret verified and HFDU in a common way. I personally don’t think that is a huge problem but having had multiple discussion about these states on the IESG over the years, I think the problem might actually be that there is one state missing. I could of course also be wrong and one more state means even more confusion… however, I wanted to note that report of time “reported” do not mean that nobody every looked at it but there might just not be an appropriate other state to change to. One more side note on HFDU: the verifier/AD can add a verifier note but can also change the errata report. I would do that and then say in the note that I change it. However, this part of the process is a) also not well defined (the tooling allows the change) and b) didn’t always help me in cases were the resolution is not clear. So what I’m trying to say in summary is that I think we should start working on entirely new, better tooling for the part of the errata process that we know work (verify or reject reports that are obvious errata or not) and then improve the other parts or create even new process for the other things over time. E.g. we could constrain the tooling such that it is way harder to report things that are not errata (see some more thought below) but then provide a better way to direct reporters to others places to open a discussion within the context of the IETF (for IETF RFCs). More concreate I think we should start with the following things: 1) integrate an errata verifier view into the AD dashboard - Similar as we have it for other things on the AD dashboard now, it would be useful to also add an action holder, so the AD could e.g. look at the errata and figure out it has an active wg and needs wg discussion and then assign the wg chairs (or some dedicated shepherd?) as action holder - Further, as such an idea came up implicitly in the discuss, the AD could even assign the errata processing to someone else and then only approve it at the end, but maybe let’s leave that already as a second step? 2) Provide in-line errata reporting in the datatracker RFC view, meaning I have a button to edit the text and then can only provide something similar to a PR in git. - Here the question is really what’s about errata that are no inline changes. Should we also provide a way to just mark some text and leave a comment? Or should that those be directed to an entirely different system? (Is it even possible to always direct reports correct or do we need a way to redirect reported errata manually?). To begin with, I guess we could just keep the old errata system in parallel for a while; that would not be great but would be an easy first step. 3) Make the RFC version with in-line errata more visible - Actually, I think we should not take this part as the very first thing to fix as we at least have already the current in-line viewing option and this part might also depend on the report interface we provide. - Further, we have discussed this many times and I think this part actually has a policy question. I personally think we should at least make the version with inline errata the default in the datatracker, however, this might not be the same for RFC editor. But we really need to decide what we want and write it down somewhere before we change the tooling. - Another reason why we also came not fully to conclusion here, is that there are verified errata that cannot easily be shown inline. I think we would need some manual processing to fix those and then new, better tooling for reporting that make automated processing easier in future as a precondition for a policy change (as proposed in point 2). However, the question if it is worth the time and effort to go rough these non-inlinable verified errata depends also on the policy I guess. Having said all that, I think point 1 and 2 can mostly be started by the RPC and tools team right away. Specially point 1 depends on the IESG and they should provide input to the RPC on their needs rather than having a community discussion. With point 2 the tools team can simply experiment with it and maybe even request community input on usage report, potentially before shutting down the old system entirely). But I don’t really see much of a policy question in providing in-line editing for reporting. I guess there is some policy question on new process to handle the not clear errata cases, especially before we shut down the old errata reporting system completely, but that seems more like a question for mainly the IETF stream for me. There might be some other smaller policy points but then RPC could come back to RSWG or RSAB (depending on the concrete question and impact) if they detect those during the process of redoing/improving the tooling. For point 3 there is a policy question but we discussed it many times and I think we need some tooling change as proposed in point 2 first in order to be able to make some progress on the policy part. Sorry for the long mail. I still hope it provides some useful input to the discussion. Mirja > On 18. Dec 2024, at 15:26, Stephen Farrell <stephen.farr...@cs.tcd.ie> wrote: > > > Hiya, > > On 18/12/2024 11:11, "Mirja Kühlewind (IETF)" wrote: >> Replying to this mail in this already quite long discussion as >> Stephen noted more input from the stream managers would be good. I >> think there was high level agreement between the stream managers but >> of course details may differ, so here is my take. >> For me there were two points of input I wanted to provide to the RPC >> based on their initial proposal for a new errata system that they >> shared with the stream managers as a starting point for discussion: >> 1) the proposal seemed to focus more on the general problem of how >> to better support RFC readers. While this is something we could >> potentially improve, I think we should a) not merge this together >> with errata processing or at least focus on improving errata >> handling first and maybe consider further improvement later, and b) >> I think that is mostly a question for the IETF in the first place or >> at least each streams directly and maybe less so for the whole RFC >> series. What I mean is that yes it would be good to have one unified >> way to support RFC readers however this is most important for >> protocol standards and the technical expertise of the IETF is >> clearly needed here, so it will interact in some way with other ietf >> processes. For errata the problem is that we sometimes get reports >> that go beyond just clear errata and require discussion or reaching >> back. I think these are also often the cases that don’t get >> processed because it’s not necessarily an errata in the sense of an >> oversight when the doc was published but still there is something >> clearly wrong. It is impossible to fully ensure that the reporter >> will always get this right but I think we could improve the >> situation by better tooling and we probably need a different process >> to do something with these kind of reports that are basically not >> errata. I think those things need to go back to the wg in some form >> or if not present need some kind of consensus process to move beyond >> the reported state. However telling these things apart from actually >> straight forward errata is often the challenge. > > IMO it is not sensible to focus on "improving errata first" - the > current system is so broken that setting that as a goal would only > lead to wasted effort and an outcome that cannot be significantly > better than the currently awful situation. > >> 2) the second piece of feedback we/I gave was on tooling. I think we >> all thought it might not be a good idea to get yet another own stand- >> alone tool where we can have get another side discussion. The ietf >> doesn’t have a lot of process requirement but we actually require >> every wg to have mailing list. So that’s the smallest common >> denominator that we could define process on that works for all >> working groups. I don’t think requiring GitHub for all wg to process >> errata will fly (yet). However I wouldn’t want to stop any wg to >> create a GitHub based errata process for their group. As an >> organization I would also not want to bind our formal processes to >> GitHub. Actually in this case of tooling I would encourage people >> actually to think small and evolve over time. We know what we don’t >> like about the current system (and having an entirely separate tool >> I think it one if the points). I think we should start integrating >> this into the datatracker both for the verifiers (ADs and stream >> manager) as well as the reporters. And as this came up, I don’t >> think this would require a datatracker login for the reporters but >> we could e.g. at a manual inspection step if reported without login >> or other difference in treating and handling the reported errata. > > I'm not sure where the "requiring github" thing above comes from. I > didn't understand that to be part of anything proposed (and would be > against us getting that dependent on github). > > The above also seems to ignore the fact that most RFCs do not map > to an active WG (or RG) and that that'll likely always be the case. > (That's also my main reaction to Colin's mail.) > > Where there is an active WG/RG, of course their mailing list needs > to be involved at some point before an erratum is approved, but > that's the wrong thing on which to focus. > >> I hope that helps. I think I agree with a lot of things that have >> been said in this discussion but again my main points are: let’s not >> design another big stand-alone tool from scratch and let’s >> concentrate on the things we know already needs changing (both >> process and tooling wise) first rather then redesign everything at >> once. > > Based on how broken our current errata handing is, I reach the opposite > conclusion: we really need to throw it out and start from scratch. > > Cheers, > S. > >> Mirja >>> Am 18.12.2024 um 00:39 schrieb Stephen Farrell >>> <stephen.farr...@cs.tcd.ie>: >>> >>>> On 17/12/2024 23:23, Brian E Carpenter wrote: So here are my >>>> notes on the draft: >>> Thanks! Will incorporate tomorrow my time. (Note, I'm also fine if >>> the process runs from Jean's list and not my I-D, but no harm updating the >>> draft too along these lines.) >>> S. >>>> Rewrite the Abstract and the Introduction to say "this is the >>>> new errata policy". >>>>> 2. Policy versus Implementation >>>>> Some of the details below are provided via indirection, using >>>>> the [RPCTBD], reference. >>>> I'd simply say that all technical details will be defined and >>>> documented by the RPC. >>>>> The RPC are expected to consult with the community as changes >>>>> are considered. >>>> Yes. >>>>> 3. The New System >>>>> Once an RFC is published, then, on the datatracker web page >>>>> for viewing that RFC, there will be a "comment/discuss" button >>>> Saying that it's on the datatracker is a technical detail - and >>>> in any case, the button must also be on the RFC Editor's info >>>> page (where the errata button is today). Altogether there are >>>> too many details here (and no mention of moderation apart from >>>> the spam comment in Section 2). Requiring a DT login is a big >>>> move. I think we need to discuss that. >>>>> Discussion threads are expected to be re-directed to an IETF >>>>> mailing list as warranted. >>>> That needs to be streamized. Something like Each RFC stream must >>>> define its own procedure; for example, discussion threads on >>>> IETF stream documents are expected to be redirected to an IETF >>>> mailing list as warranted, but that decision would be made by >>>> the IETF, not by the RFC Editor. By default, old documents not >>>> assigned to a particular stream will be treated as if they were >>>> IETF stream, unless another stream manager claims them. >>>> [Comment: a lot of legacy Informationals would naturally be >>>> Independent Stream today.] >>>>> Comments labelled as errata can be upvoted or downvoted. >>>>> Voting power... >>>> I don't like this bit. It looks like an end-run around rough >>>> consensus. I tend to think this whole part should be delegated >>>> to the streams anyway. Certainly there are too many details. >>>>> The default HTML view of RFCs will be that with errata >>>>> applied. >>>> But what does that mean? What about everyone who's downloaded >>>> the original RFC? What about the copy that ChatGTP holds? I >>>> fully agree that a rendering with errata applied should be >>>> available, with and without diffs displayed. But saying it's the >>>> default doesn't seem like a well- defined concept to me. >>>>> 4. Handing existing errata >>>> ... >>>>> No action is required with respect to current, posted but >>>>> unprocessed, errata. >>>> I disagree. The current load of unprocessed errata is the >>>> biggest issue that we have, and I think they need to be force- >>>> fed to the new system. Regards Brian >>>>> On 18-Dec-24 10:27, Stephen Farrell wrote: >>>>> Hiya, >>>>> On 17/12/2024 20:44, Brian E Carpenter wrote: >>>>>> Eliot, I would love some technical team to work out the >>>>>> details of an improved errata system this afternoon, but >>>>>> there are policy questions and I think they need to be >>>>>> distilled out of the draft and made into a two page document >>>>>> that we could rapidly send to the RSAB. >>>>> If you'd like to suggest changes to [1] to go more in that direction, >>>>> that'd be fine. From my POV, [1] already almost does that already, by >>>>> only setting high level direction and >>>>> deferring to the RPC for details. >>>>> I'd be happy to refresh the draft with such changes, e.g. to cast the >>>>> more descriptive stuff as an example of what could be >>>>> an implementation. Or that could be done after it was adopted >>>>> somewhere. >>>>> Cheers, S. >>>>> [1] https://www.ietf.org/archive/id/draft-farrell- >>>>> errata-00.html >>>>>> Regards Brian >>>>>> On 18-Dec-24 06:37, Eliot Lear wrote: >>>>>>> I certainly don't want *any* draft sequenced or held up or >>>>>>> anything. And so a *big fat plus 1* for adopting draft- >>>>>>> farrell-errata. >>>>>>> Onward! >>>>>>> Eliot >>>>>>> On 17.12.2024 18:16, Paul Hoffman wrote: >>>>>>>> Yep, we like talking about things that can be improved >>>>>>>> with the RFC series. The (thankfully short) WG charter >>>>>>>> says: >>>>>>>> The RFC Series Working Group (RSWG) is the primary venue >>>>>>>> in which members of the community collaborate regarding >>>>>>>> the policies that govern the RFC Series. >>>>>>>> There was a thread started by the WG chairs on >>>>>>>> 2024-10-30, but never concluded, about what is policy >>>>>>>> and what is operations. Most of the comments on this >>>>>>>> errata thread are about operations of the errata process, not the >>>>>>>> policy for how errata apply to RFCs. >>>>>>>> I propose that the WG chairs definitively close out the >>>>>>>> "Operational vs. policy" thread so that the WG can move >>>>>>>> forward on the many drafts we already have. If the >>>>>>>> conclusion of that thread makes it clear that errata >>>>>>>> operations is in scope for this WG, I propose the WG officially adopt >>>>>>>> draft-farrell-errata; otherwise, I >>>>>>>> propose that Someone create a mailing list to discuss >>>>>>>> errata operations and this discussion move there. >>>>>>>> --Paul Hoffman >>> -- rswg mailing list -- rswg@rfc-editor.org To unsubscribe send an >>> email to rswg-le...@rfc-editor.org <OpenPGP_signature.asc> > > -- > rswg mailing list -- rswg@rfc-editor.org > To unsubscribe send an email to rswg-le...@rfc-editor.org -- rswg mailing list -- rswg@rfc-editor.org To unsubscribe send an email to rswg-le...@rfc-editor.org
