I will caveat my comments with the fact that I am no longer taking on new documents for review. However, having been involved in the process for 10 years now on the gen-art side and having done the reviews for 3 IETF chairs, and having much experience as a chair and document editor, I do have some concerns as laid out below.
Mary. On Fri, May 24, 2013 at 1:11 AM, Jari Arkko <[email protected]> wrote: > >> I think time will tell whether it's small. Getting into the thick of >> WG discussion in an area one doesn't know well can be quite a shock >> to the system. That's not to say we shouldn't try it, but I would ask >> for it to be a time-limited experiment initially. > > Yes - or scope limited (only some WGs or areas at the beginning). [MB] Absolutely, it needs to have a limited scope initially. As Brian said, the concept is not new, it's just that few took advantage of it. I would prefer that this stage remain optimal and up to the WG chairs as to whether it's necessary. Without the tool support that is available for the regular LC and Telechat reviews, I don't think we want to burden Jean too much until we have a very good idea of the level of effort and until there is tool support (e.g., chairs enter the docs into a tool that gets the doc into Jean's incoming queue for assignment). I also believe this increases the burden on the reviewers significantly more than the burden when we added the LC reviews. For folks that haven't been around a while, the original Gen-ART process only did reviews prior to the telechats. Adding reviews at LC did increase the workload for reviewers, however, we did see a somewhat increase in documents ready at Telechat time. [/MB] > >> It seems important to inform WGs explicitly that outside review >> is taking place *and* has to be taken seriously (because if not, >> the same reviewer will be back at a later stage with the same >> questions).' > > Good point. [MB] Exactly. [/MB] [MB] My biggest concern is that many documents that go through WGLC in the WGs in which I am active are NOT ready at that stage. I also have seen in cases of Gen-ART reviews, that documents clearly weren't reviewed by the shepherd or ADs due to the level of editorial issues and technical (the p2psip reload document is a good example of this). There were concerns raised in the WG with regards to its readiness but it was still progressed. Certainly, I hope this is the exception. However, my guess is that many documents will change quite a bit after LC. When I chaired the SIPPING WG, we had multiple stages of review in the WG where we had assigned reviewers. We had an initial review when the doc was first accepted as a WG document, then we had a pre-WGLC review and then the real WGLC. Many of the documents went through a 2nd last call. If we could get more WGs to be more disciplined about ensuring that documents receive adequate review prior to WGLC, then this idea might work. At this point, I see very few WGs that have an effective process in place. One also has to consider that in some WGs (many that I participate in and write docs for), there are very often folks that don't review any version of a document in detail until it goes through WGLC. So, I am personally quite skeptical that this proposal adds significant value to the overall process, in particular based on the level of effort required by both the gen-art reviewers. I think this is just moving an existing problem around and asking the wrong set of folks to help solve the problem. I think a more effective change would be for ADs to expect WGs to ensure the docs are thoroughly reviewed earlier in the stage by the WG. [/MB] > >> Then there is a question: will Gen-ART re-review at the time of >> IETF Last Call as well as when the draft reaches the IESG agenda? > > My initial though was that we'd do the same as we do today for drafts that > change, i.e., re-review by the same person. The likelihood for draft changes > is of course bigger when starting from the WGLC. Another possible work load > increase to watch for. I'd be interested in your thoughts on other approaches > to deal with re-reviews though. > > Jari > > _______________________________________________ > Gen-art mailing list > [email protected] > https://www.ietf.org/mailman/listinfo/gen-art _______________________________________________ Gen-art mailing list [email protected] https://www.ietf.org/mailman/listinfo/gen-art
