Re: Core Infrastructure Initiative (CII) compliance for the Jenkins core - status thread

[Oleg Nenashev]

Hi all,

Just a quick update, after submitting the security checklist and our 
current Jira metrics to 
https://bestpractices.coreinfrastructure.org/en/projects/3538, I am happy 
to announce that we have reached the 133% mark and hence the Jenkins 
project is now officially passing the Core Infrastructure Initiative 
certification. Thanks a lot to all contributors, and special thanks to the 
Jenkins Security team (esp. Daniel and Wadeck) for multiple cycles of 
reviews in the checklist!

Next steps would be to keep working on the CII certification towards silver 
(200%) and gold (300%) grades. There are much more strict requirements on 
these levels (e.g. strict license file requirements, infra authorization 
guidelines, etc. etc.). There will be a lot of work to get there, but I 
think we can keep working on requirements which we consider beneficial to 
the Jenkins project and the community

Best regards,
Oleg


On Monday, June 22, 2020 at 1:53:57 PM UTC+2, Oleg Nenashev wrote:
>
> Updates here:
>
>- Right now we are at the 80% mark w.r.t the compliance: 
>https://bestpractices.coreinfrastructure.org/en/projects/3538
>- We would be interested to pass Core Infrastructure Initiative 
>certification as a part of the CDF graduation process (see this thread 
>).
>- I started working on addressing the current issues in the 
>certification:
>   - Issue Triage: We need a formal process w.r.t providing initial 
>   feedback to bug reports and feature requests. I restarted a thread 
> about 
>   the Bug Triage team for the Jenkins core. See 
>   https://groups.google.com/d/msg/jenkinsci-dev/XToix3QpL_k/u6-7awD4AwAJ 
>   and further comments
>   - Security checklist: I started a Google Doc 
>   
> 
>  
>   for the Security checklist. It should help us to perform a joint review 
> of 
>   the requirements and to prepare a response.
>
> Any feedback about the wording and the security checklist would be 
> appreciated.
>
> Best regards,
> Oleg
>
>
> On Tuesday, February 18, 2020 at 9:00:44 PM UTC+1, Tracy Miranda wrote:
>>
>> Hi Oleg,
>>
>> Thanks for putting this together and establishing that baseline score!
>>
>> IMHO it is a great exercise to run through as proven by the issues you 
>> raised in the email. (Also nice to see the badge linked when I click on 
>> Jenkins on the CDF  and CNCF 
>> landscapes). 
>> I look forward to the follow on threads, plus also plan to take a more 
>> detailed look at the report. 
>>
>> Thanks,
>> Tracy
>>
>> On Tue, Feb 18, 2020 at 6:50 AM Oleg Nenashev  wrote:
>>
>>> Hi all,
>>>
>>> This is a follow-up to the Community Bridge funding thread 
>>>  
>>> and 
>>> to contributor summit discussions about CII. As discussed there, Linux 
>>> Foundation expects all projects on Community Bridge to be also a part of 
>>> the Core Infrastructure Initiative  
>>> which 
>>> is their program for strengthening security in open-source projects. In 
>>> particular, there is a badge program here 
>>> . All Community Bridge 
>>> projects are expected to eventually pass certification there.
>>>
>>> I believe that being compliant with CII is a net positive thing for us, 
>>> because it can help to promote the project and to address some 
>>> quality-related and certification queries from current and potential 
>>> Jenkins users (e.g. see this recent thread 
>>> ). 
>>> It also unlocks access to  targeted security project funding / engineering 
>>> time donations by CII corporate members (Assistance program 
>>> ) and 
>>> to tooling like Snyk.
>>>
>>> I started working on a CII checklist for the Jenkins core, plugins are 
>>> out of the scope for me at the moment. You can find the current status on 
>>> this 
>>> page . 
>>> We are currently at the *80%* completion state, and there are some open 
>>> topics which need to be clarified. I have summarized the topics below after 
>>> the email, and I will start follow-up threads for them so that they can be 
>>> discussed separately.
>>>
>>> CII is definitely a case when the remaining 20% for the work require 80% 
>>> of effort, but I hope to gradually get to the full certification checklist 
>>> for the Jenkins core. Even if we do not pass the certification criteria 
>>> there, it is nice to have a documented status for quality/security 
>>> expectations. I will appreciate any feedback about the CII compliance in 
>>> 

Re: Core Infrastructure Initiative (CII) compliance for the Jenkins core - status thread

[Oleg Nenashev]

Updates here:

   - Right now we are at the 80% mark w.r.t the compliance: 
   https://bestpractices.coreinfrastructure.org/en/projects/3538
   - We would be interested to pass Core Infrastructure Initiative 
   certification as a part of the CDF graduation process (see this thread 
   ).
   - I started working on addressing the current issues in the 
   certification:
  - Issue Triage: We need a formal process w.r.t providing initial 
  feedback to bug reports and feature requests. I restarted a thread about 
  the Bug Triage team for the Jenkins core. See 
  https://groups.google.com/d/msg/jenkinsci-dev/XToix3QpL_k/u6-7awD4AwAJ 
and 
  further comments
  - Security checklist: I started a Google Doc 
  

 
  for the Security checklist. It should help us to perform a joint review 
of 
  the requirements and to prepare a response.
   
Any feedback about the wording and the security checklist would be 
appreciated.

Best regards,
Oleg


On Tuesday, February 18, 2020 at 9:00:44 PM UTC+1, Tracy Miranda wrote:
>
> Hi Oleg,
>
> Thanks for putting this together and establishing that baseline score!
>
> IMHO it is a great exercise to run through as proven by the issues you 
> raised in the email. (Also nice to see the badge linked when I click on 
> Jenkins on the CDF  and CNCF 
> landscapes). 
> I look forward to the follow on threads, plus also plan to take a more 
> detailed look at the report. 
>
> Thanks,
> Tracy
>
> On Tue, Feb 18, 2020 at 6:50 AM Oleg Nenashev  > wrote:
>
>> Hi all,
>>
>> This is a follow-up to the Community Bridge funding thread 
>>  and 
>> to contributor summit discussions about CII. As discussed there, Linux 
>> Foundation expects all projects on Community Bridge to be also a part of 
>> the Core Infrastructure Initiative  
>> which 
>> is their program for strengthening security in open-source projects. In 
>> particular, there is a badge program here 
>> . All Community Bridge 
>> projects are expected to eventually pass certification there.
>>
>> I believe that being compliant with CII is a net positive thing for us, 
>> because it can help to promote the project and to address some 
>> quality-related and certification queries from current and potential 
>> Jenkins users (e.g. see this recent thread 
>> ). It 
>> also unlocks access to  targeted security project funding / engineering 
>> time donations by CII corporate members (Assistance program 
>> ) and 
>> to tooling like Snyk.
>>
>> I started working on a CII checklist for the Jenkins core, plugins are 
>> out of the scope for me at the moment. You can find the current status on 
>> this 
>> page . We 
>> are currently at the *80%* completion state, and there are some open 
>> topics which need to be clarified. I have summarized the topics below after 
>> the email, and I will start follow-up threads for them so that they can be 
>> discussed separately.
>>
>> CII is definitely a case when the remaining 20% for the work require 80% 
>> of effort, but I hope to gradually get to the full certification checklist 
>> for the Jenkins core. Even if we do not pass the certification criteria 
>> there, it is nice to have a documented status for quality/security 
>> expectations. I will appreciate any feedback about the CII compliance in 
>> general and about the self-certification page 
>> . 
>> Unfortunately documentation-as-code is not supported there, but I am happy 
>> to incorporate any suggested changes.
>>
>> Best regards,
>> Oleg
>>
>>  Open topics: 
>>
>> *Problem 1. Incoming issues triage *(section status 
>> ). 
>> We do not longer have an active triage team which would be regularly 
>> reviewing incoming issues in Jira. Alex Earl made a proposal to have an 
>> official triage team in 2017 (dev list thread 
>> ),
>>  
>> but it was not implemented at the moment. I was doing regular issue triage 
>> until Dec 2018 before I stepped down (see the same thread). Right now we 
>> regularly look at the Jenkins release community ratings and reported 
>> regressions, but I would not say we have a real triage process, especially 
>> for RFEs and bugs reported to non-core 

Re: Core Infrastructure Initiative (CII) compliance for the Jenkins core - status thread

[Tracy Miranda]

Hi Oleg,

Thanks for putting this together and establishing that baseline score!

IMHO it is a great exercise to run through as proven by the issues you
raised in the email. (Also nice to see the badge linked when I click on
Jenkins on the CDF  and CNCF
landscapes).
I look forward to the follow on threads, plus also plan to take a more
detailed look at the report.

Thanks,
Tracy

On Tue, Feb 18, 2020 at 6:50 AM Oleg Nenashev 
wrote:

> Hi all,
>
> This is a follow-up to the Community Bridge funding thread
>  and
> to contributor summit discussions about CII. As discussed there, Linux
> Foundation expects all projects on Community Bridge to be also a part of
> the Core Infrastructure Initiative  which
> is their program for strengthening security in open-source projects. In
> particular, there is a badge program here
> . All Community Bridge
> projects are expected to eventually pass certification there.
>
> I believe that being compliant with CII is a net positive thing for us,
> because it can help to promote the project and to address some
> quality-related and certification queries from current and potential
> Jenkins users (e.g. see this recent thread
> ). It
> also unlocks access to  targeted security project funding / engineering
> time donations by CII corporate members (Assistance program
> ) and to
> tooling like Snyk.
>
> I started working on a CII checklist for the Jenkins core, plugins are out
> of the scope for me at the moment. You can find the current status on this
> page . We
> are currently at the *80%* completion state, and there are some open
> topics which need to be clarified. I have summarized the topics below after
> the email, and I will start follow-up threads for them so that they can be
> discussed separately.
>
> CII is definitely a case when the remaining 20% for the work require 80%
> of effort, but I hope to gradually get to the full certification checklist
> for the Jenkins core. Even if we do not pass the certification criteria
> there, it is nice to have a documented status for quality/security
> expectations. I will appreciate any feedback about the CII compliance in
> general and about the self-certification page
> .
> Unfortunately documentation-as-code is not supported there, but I am happy
> to incorporate any suggested changes.
>
> Best regards,
> Oleg
>
>  Open topics:
>
> *Problem 1. Incoming issues triage *(section status
> ).
> We do not longer have an active triage team which would be regularly
> reviewing incoming issues in Jira. Alex Earl made a proposal to have an
> official triage team in 2017 (dev list thread
> ),
> but it was not implemented at the moment. I was doing regular issue triage
> until Dec 2018 before I stepped down (see the same thread). Right now we
> regularly look at the Jenkins release community ratings and reported
> regressions, but I would not say we have a real triage process, especially
> for RFEs and bugs reported to non-core components
>
>- CII Criteria:
>- " The project MUST acknowledge a majority of bug reports submitted
>   in the last 2-12 months (inclusive); the response need not include a 
> fix."
>   - " The project SHOULD respond to a majority (>50%) of enhancement
>   requests in the last 2-12 months (inclusive).  "
>   - My assumption is that we are below these criteria
>- *Potential solution*: Maybe we should revise this topic. Since we
>have more active core maintainers now, maybe we could have a rotation for
>the incoming issues in Jenkins Jira. To be discussed in a separate thread
>
> *Problem 2. Quality and Code analysis warnings* (section status
> ).
> The project MUST enable one or more compiler warning flags, a "safe"
> language mode, or use a separate "linter" tool to look for code quality
> errors or common simple mistakes, if there is at least one FLOSS tool that
> can implement this criterion in the selected language. Jenkins core
> addresses it, because we have a bunch of tools enabled like Spotbugs,
> Animal Sniffer or Maven Enforcer. But there are some downstream criteria
>
>- Problematic CII criteria:
>- The project should fix warnings or mark them in the source code as
>   false positives. Ideally there would be no warnings, but a 

Core Infrastructure Initiative (CII) compliance for the Jenkins core - status thread

[Oleg Nenashev]

Hi all,

This is a follow-up to the Community Bridge funding thread
 and
to contributor summit discussions about CII. As discussed there, Linux
Foundation expects all projects on Community Bridge to be also a part of
the Core Infrastructure Initiative  which
is their program for strengthening security in open-source projects. In
particular, there is a badge program here
. All Community Bridge
projects are expected to eventually pass certification there.

I believe that being compliant with CII is a net positive thing for us,
because it can help to promote the project and to address some
quality-related and certification queries from current and potential
Jenkins users (e.g. see this recent thread
). It
also unlocks access to  targeted security project funding / engineering
time donations by CII corporate members (Assistance program
) and to
tooling like Snyk.

I started working on a CII checklist for the Jenkins core, plugins are out
of the scope for me at the moment. You can find the current status on this
page . We
are currently at the *80%* completion state, and there are some open topics
which need to be clarified. I have summarized the topics below after the
email, and I will start follow-up threads for them so that they can be
discussed separately.

CII is definitely a case when the remaining 20% for the work require 80% of
effort, but I hope to gradually get to the full certification checklist for
the Jenkins core. Even if we do not pass the certification criteria there,
it is nice to have a documented status for quality/security expectations. I
will appreciate any feedback about the CII compliance in general and about
the self-certification page
.
Unfortunately documentation-as-code is not supported there, but I am happy
to incorporate any suggested changes.

Best regards,
Oleg

 Open topics:

*Problem 1. Incoming issues triage *(section status
).
We do not longer have an active triage team which would be regularly
reviewing incoming issues in Jira. Alex Earl made a proposal to have an
official triage team in 2017 (dev list thread
),
but it was not implemented at the moment. I was doing regular issue triage
until Dec 2018 before I stepped down (see the same thread). Right now we
regularly look at the Jenkins release community ratings and reported
regressions, but I would not say we have a real triage process, especially
for RFEs and bugs reported to non-core components

   - CII Criteria:
   - " The project MUST acknowledge a majority of bug reports submitted in
  the last 2-12 months (inclusive); the response need not include a fix."
  - " The project SHOULD respond to a majority (>50%) of enhancement
  requests in the last 2-12 months (inclusive).  "
  - My assumption is that we are below these criteria
   - *Potential solution*: Maybe we should revise this topic. Since we have
   more active core maintainers now, maybe we could have a rotation for the
   incoming issues in Jenkins Jira. To be discussed in a separate thread

*Problem 2. Quality and Code analysis warnings* (section status
).
The project MUST enable one or more compiler warning flags, a "safe"
language mode, or use a separate "linter" tool to look for code quality
errors or common simple mistakes, if there is at least one FLOSS tool that
can implement this criterion in the selected language. Jenkins core
addresses it, because we have a bunch of tools enabled like Spotbugs,
Animal Sniffer or Maven Enforcer. But there are some downstream criteria

   - Problematic CII criteria:
   - The project should fix warnings or mark them in the source code as
  false positives. Ideally there would be no warnings, but a project MAY
  accept some warnings (typically less than 1 warning per 100 lines or less
  than 10 warnings).
  - It is SUGGESTED that projects be maximally strict with warnings in
  the software produced by the project, where practical.
   - *Problem*: We ignore some warnings without explicitly supressing them
   (Javadoc and other minor things). And we definitely do not set maximally
   strict requirements, our SpotBugs runs on the High threshold by default.
   Stefan Spieker is doing a great job with the issues cleanup, for "Medium",
   but there are still a lot of issues left
   - *Potential solution:* Fail the Suggested