On Tue, Feb 13, 2018 at 12:17 AM, SULLIVAN, BRYAN L (BRYAN L) <
bryan.sulli...@research.att.com> wrote:

> Comments etc inline
> Thanks,
> Bryan Sullivan | AT&T
> *From:* Luke Hinds [mailto:lhi...@redhat.com]
> *Sent:* Monday, February 12, 2018 9:04 AM
> *To:* SULLIVAN, BRYAN L (BRYAN L) <bryan.sulli...@research.att.com>
> *Cc:* opnfv-tech-discuss@lists.opnfv.org; degirmenci, fatih <
> fatih.degirme...@ericsson.com>; Raymond Paik <rp...@linuxfoundation.org>
> *Subject:* Re: [opnfv-tech-discuss] Anteater status and link issue
> On Tue, Feb 6, 2018 at 2:32 PM, SULLIVAN, BRYAN L (BRYAN L) <
> bryan.sulli...@research.att.com> wrote:
> Hi all,
> I’m wondering where the Anteater program is – and want to note a broken
> link: build jobs with Anteater violations reference “Please visit:
> https://wiki.opnfv.org/x/5oey
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.opnfv.org_x_5oey&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=s4zQHMsxrgVhlTs-Sw4-uGIsKYDMsnIQuvx0TehUoSk&e=>
> ”, which is the wiki page https://wiki.opnfv.org/pages/v
> iewpage.action?pageId=11700198
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.opnfv.org_pages_viewpage.action-3FpageId-3D11700198&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=burTDZjfgUSG9lAKW4MjRDZULxleQEsKGknHvhdqzbA&e=>,
> which says “Project specific exceptions can be added for file_name,
> file_contents and binaries, by using the name of the repository within the
> anteater/exceptions/ directory of the releng-anteater
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.opnfv.org_gerrit.opnfv.org-3A29418_releng-2Danteater.git&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=LrWykp0HOa_RUbxOEJDo7sojbPgNgsVsrlV6jmwMVx4&e=>
>  repository.” – but that link (releng-anteater
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.opnfv.org_gerrit.opnfv.org-3A29418_releng-2Danteater.git&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=LrWykp0HOa_RUbxOEJDo7sojbPgNgsVsrlV6jmwMVx4&e=>)
> is broken.
> I want to start adding the exceptions for Models etc as an example for the
> LF IT team that is setting up the Acumos project gerrit/CI/CD process, and
> in general to help optimize the Anteater overhead for projects. I think we
> need to get some analysis of the types of exceptions that are typical, and
> establish a process for vetting those exceptions that goes beyond a simple
> review by a releng committer.
> Further, we need to bring in other scan tools (e.g. security
> vulnerability, virus, or malicious code scans) into the Anteater process.
> This is in response to concerns about the security of the governance
> process for open source (e.g. upstream, but also direct contribution in
> projects) that is used to build production-oriented systems. We need to
> demonstrate that OPNFV and other LF projects are addressing these concerns
> through their infra toolsets.
> Sorry Bryan, I missed a few of these emails thanks (or rather no thanks)
> to a bad mail filter rule.
> I am working on the following now which we will see soon:
> Much better documentation: http://anteater.readthedocs.io/en/latest/
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__anteater.readthedocs.io_en_latest_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=AdeEyIqajKWRGD1zz3MXcKrWoAWYR6mXmQDgVVzp1Zo&e=>
> [bryan] Are you going to start hosting these docs at docs.opnfv.org?
We can do yes, although I guess it make sense to have the main body of the
documentation around the tool upstream (once the github re-homing happens),
and then have everything OPNFV developers need to know about how anteater
is used in OPNFV at docs.opnfv.org - this way there won't be materials in
docs.opnfv.org around using Travis CI (which would confuse people).

> Virus total integration:
>    * Any binaries will be scanned using the virus total API, unless a
> sha256 waiver is already present e.g. https://github.com/opnfv/relen
> g-anteater/blob/master/exceptions/calipso.yaml#L9
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_opnfv_releng-2Danteater_blob_master_exceptions_calipso.yaml-23L9&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=WNetEYMktH0pxwVzSJXZyDFVnJr6lIDBhM6laGBrbjs&e=>
>    * Any IP addresses / domain name / URL will be scanned (again using the
> Virus Total API) for known malware and other nastiness.
> [bryan] VirusTotal looks like a useful service. Are there any stats for
> its effectiveness at detecting threats, including new threats and delay in
> supporting them?
Its pretty much the epicentre of community based threat collaboration .  It
aggregates 40 virus / malware scanners to asses files, and domains / IP
addresses are assessed against 70 URL/domain blacklisting services:


It also fits well into anteater:

Currently anteater will generate a sha256sum of any blobs it finds, and
will report them, *unless* a sha256 is entered into the exception files. I
will extend this, so that if a blob is found an *no* sha256sum exception
exists, we send the file hash to Virustotal to see if its registered as
nefarious. If it is we will fail the job and alarm the finding. If not, an
exception can be entered and it will be ignored from there on in and we
won't trouble the VT API again for that particular file - unless someone at
a later point changes the file (which would change the checkum) and then a
scan is made again - this way we can be sure that an infected file is not
checked into a project and we are not aware as it has the same name as

> I also have a load of new strings to add to dig out and report anything of
> a more recent finding (for example a javascript based bitcoin miner).
> [bryan] I would like to see how we can improve the contextual
> effectiveness of the pattern matching approach. Any bar (or port in a
> storm) may seem to be better than none, and can at least catch newbie
> mistakes and anti-patterns, but most of the strings I’ve included in
> https://github.com/opnfv/models/blob/master/tools/anteater-exceptions.yaml
> relate to IMO innocuous (if admittedly sometimes cheap or anti-patterned)
> use of prohibited words. Others, I clearly need to fix.

So I am very open to switching off the more noisy regexs that emit false
positives and also open to new approaches. I am sure I can fine tune them
much better as well.

Likewise open to any feature recommendations etc.

> The project is also hopefully going to move into github (once agreed with
> LF) to encourage wider contributions and allow it to be more easily
> consumed else.
> [bryan] Anything that broadens contribution and consumption makes sense to
> me. Are there any other open source projects in this same space that you
> are considering leveraging, to avoid re-developing features unnecessarily?
We plan to discuss wider LFN adoption , one example being OpenDayLight
where I manage security. I also plan to get more eyes on the tool for
smaller projects to utilise.  An OpenStack project is also considering the
tool, but more for finding depreciated key directives and release tags.

> Once the above is in place, docs will be clearer to follow, project will
> be more presentable, with more coverage in finding vulns will be wider.
> [bryan] We probably need more docs re the process for getting exceptions
> approved, and how the community can track its effectiveness in the mission
> represented by this toolset, through the types of approved exception
> patterns, as they grow (or shrink… it would be good to see the community
> improving through reduction in the need to maintain exceptions, and partly
> because the tool is getting smarter).

Very much agree, it would be great to see people add to the master
exception / ignore list and feedback on where the tool works well / is
annoying etc.

I also agree on the docs and enlarging upon the process for getting
exceptions approved. I plan to have all this done before ONS so we can see
it in place for then.

> Bryan Sullivan | AT&T
> _______________________________________________
> opnfv-tech-discuss mailing list
> opnfv-tech-discuss@lists.opnfv.org
> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.opnfv.org_mailman_listinfo_opnfv-2Dtech-2Ddiscuss&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=8NPFgQFDZsv688HirOlM8HW1u0X9QVVgUfsN6B5PP_s&e=>
opnfv-tech-discuss mailing list

Reply via email to