I think you have a point about A + B, so we should have independent
review periods for each ballot as we did with version 1.6.4 (we had
independent ballots to be reviewed and then a combined/aggregated new
version of the BRs).
I also received some personal messages supporting the "aggregated"
practice as it seems to be much more efficient for a CA's compliance
review to check against one new BR rather than 2 or 3 separate ones
within a very short timeframe.
More answers inline.
On 2020-09-15 5:34 μ.μ., Ryan Sleevi wrote:
On Tue, Sep 15, 2020 at 4:33 AM Dimitris Zacharopoulos (HARICA)
<[email protected] <mailto:[email protected]>> wrote:
On 14/9/2020 8:08 μ.μ., Ryan Sleevi wrote:
Yes, I'm aware of the "what", but it's not clear the "why".
The act of combining ballots is relatively new, as you can see
from https://cabforum.org/baseline-requirements-documents/ .
Producing multiple versions of the Guidelines, linear based on
when the Ballot concluded, was something our GitHub flow
intentionally was to make easy. While that page stopped listing
dates of adoption around Ballot 189, you can see previous ballot
pairs (e.g. 171+164, 125+118+134+135) that did that.
It seems worth figuring out the challenges you're facing, since
it was meant to be very easy to create a new version of the
document for each ballot, even ballots that conclude closely, and
to have IP reviews as such.
The administrative overhead of updating public web sites, sending
additional emails, and the fact that we would have versions of the
Guidelines that would be valid for a few seconds (which seems
unreasonable for a public standards document), are some of the
reasons behind aggregated final guidelines. Version 1.6.4
aggregated 3 ballots, 1.6.7 and 1.7.1. had 2 and now 1.7.2.
This process was discussed with Dean and Wayne back in February
2019, and we all considered it compliant with our Bylaws. The
results of each IPR review period were sent to the public lists
without receiving any objections or concerns.
Although we have documented a GitHub workflow that supports the
most common case (one ballot, one IPR review, one Final
Maintenance Guideline), it should not prohibit aggregated ballots
to minimize administrative overhead or the production of
Guidelines that have some reasonable validity time.
If there are strong objections to this process, we can revise it
going forward.
I think we should understand the concerns, so we can make sure we have
solutions that work.
The concerns for aggregated ballots is that, from an IP perspective,
it makes multiple ballots share the same fate and can easily delay
adoption. For example, imagine there are Ballots A and B. A makes a
change to one section, B makes a change to another, and only through
their combined implementation does a Member raise an IP concern.
In the serialized approach, A then B, A can be introduced to the IP
review and not introduce any IP obligations. It can become effective.
When B is introduced and undergoes IP review, and an exclusion is
filed (on the B+A), then B is basically sent to the PAG to work
through what to do, while A remains effective.
In the combined approach, A and B, when A+B trigger the IP review and
result in the IP obligation. An exclusion is filed, and now A+B are
sent to a PAG. The act of understanding whether it is A introducing
the IP obligation or B introducing the IP obligation is left for the
PAG to sort, with neither A nor B being effective until the PAG
reaches recommendations.
This was a concern debated at length throughout our governance
process, and was indeed one of the core motivations for our IPR policy
update (which, you may recall, was motivated by the removal of the
"any other method" introducing obligations on the other enumerated
methods). Our IP policy was revised to try to make it easier to
distinguish whether A or B was introducing the obligation, and allow
modifications like A (and subsequent modifications) to continue, even
while B was addressed within the PAG.
Beyond that explicit reason, the distinct versioning also helps better
review changes and tie back to ballots. It treats Ballots as they are
- logical units of change - rather than aggregations of change. The
Forum first started with an aggregation-of-change approach (through
submitting amendments and errata) and quickly confounded the ability
to understand the reconciled state. The same issue applies to unified
ballots. Given that we have CAs asserting that it's difficult to find
changes in Ballots they explicitly voted for, after lengthy reviews,
and with policies incorporated in at least two root programs, I'm
incredibly sensitive to wanting to make sure CAs are set up to succeed
in following the Requirements, by making changes discrete and digestible.
So that's the context about "why" serialized was intended. I'm also
totally sympathetic to the difficulty involved in producing ballots,
especially as we haven't quite got automation up to speed, and so I
can understand why wanting to avoid that work is undesirable. What I
take from your reply is that there are several tasks that, as a
consequence of our current tooling, represent undesirable time
commitments. It's unclear if that's "5 minutes more" or "5 hours
more", and so your help in breaking down how much time it takes for
the following would be greatly useful, as it will help prioritize what
is most important to resolve. I suspect these can become priorities
for the Infrastructure WG.
1) Merge a single Ballot into GitHub
This greatly depends on whether the ballot proposer has used the
documented and recommended (not yet normative/required) process, using
GitHub to create a redline. Even with GitHub, I've seen different
"flavors" of redlines, for example with and without pull requests,
making it somewhat challenging to find the right steps to create a
cabforum/documents branch that will also create the artifacts (docx
document) from which I can create the redline.
If we are talking about the Baseline Requirements. this process takes
between 15 minutes to 1,5 hours in the more weird cases. For the other
Guidelines, it takes between 30 minutes to 1 hour.
2) Produce a Final Guideline and a Redlined Version
Once I have the review period redline ready, it usually takes between 30
minutes to 1 hour to create the final documents, upload them to the wiki
(the word versions), produce the final PDF versions and upload them to
the public web site.
3) Draft and send the e-mail for the IP review period announcement
This usually takes between 10 and 20 minutes because I am using
templates on Thunderbird and just replacing the links and attachments.
4) Upload the Final Guideline and Redlined Version to the website
5) Update the Website to link to these
10 to 20 minutes for both 4 and 5.
6) Draft and send the e-mail concluding the IP review period
Again 10 to 20 minutes.
As a number of these tasks are very automatable, figuring out what we
can do so that it represents <30 minutes of additional work seems... a
reasonable target, right? This only comes up when we have multiple
ballots concluding within a short period of eachother, which although
historically rare, certainly was more common over the summer as stuff
got batched to accommodate Covid disruptions.
The most challenging and time consuming tasks come with ballots that
update more than one Guideline, for which we don't have the same tools
ready to produce docx versions. This requires me to do the GitHub
redlines for the BRs and docx versions (manually) for EV Guidelines or NSRs.
Even for the BRs, I usually have to remove text after the ToC because it
repeats information that is supposed to be in the cover page. Some of
these minor issues need to be fixed, and I would really appreciate the
help of the Infrastructure Subcommittee.
Would your concerns be addressed if we had a separate/distinct IPR
review for each ballot and once each IPR review is cleared with no
essential claims filed, produce an aggregate document, or you still
prefer separate Final documents with distinct versioning to be created?
What do others think?
Best regards,
Dimitris.
_______________________________________________
Public mailing list
[email protected]
https://lists.cabforum.org/mailman/listinfo/public
