Top post with an example using the Virus Total API:

> anteater --bincheck --project testproject --path
/home/luke/repos/personal/anteater/tests/testproject

2018-02-13 14:49:18,349 - anteater.src.get_lists - INFO - Loaded
testproject specific file_audits entries
2018-02-13 14:49:18,352 - anteater.src.get_lists - INFO - Loaded
testproject specific file_contents entries
2018-02-13 14:49:18,375 - anteater.src.project_scan - INFO - Non
Whitelisted Binary file:
/home/luke/repos/personal/anteater/tests/testproject/images/pal.png
2018-02-13 14:49:18,376 - anteater.src.project_scan - INFO - Performing
Scan: /home/luke/repos/personal/anteater/tests/testproject/images/pal.png
2018-02-13 14:49:18,824 - anteater.src.project_scan - INFO - File last
scanned and shown as clean on:, 2018-02-13 13:44:11
2018-02-13 14:49:18,825 - anteater.src.project_scan - INFO - Full report
here:
https://www.virustotal.com/file/a71e13ebeb2500ed20781ab3ae8a9b306cf69a6c8be9a31e96d4e04f1657b4d8/analysis/1518529451

2018-02-13 14:49:18,825 - anteater.src.project_scan - INFO - The following
sha256 hash can be used in your testproject.yaml file:
a71e13ebeb2500ed20781ab3ae8a9b306cf69a6c8be9a31e96d4e04f1657b4d8

Should have the URL / Domain / IP stuff working later in the week.


On Tue, Feb 13, 2018 at 9:41 AM, Luke Hinds <lhi...@redhat.com> wrote:

>
>
> On Tue, Feb 13, 2018 at 12:17 AM, SULLIVAN, BRYAN L (BRYAN L) <
> bryan.sulli...@research.att.com> wrote:
>
>> Comments etc inline
>>
>>
>>
>> Thanks,
>>
>> Bryan Sullivan | AT&T
>>
>>
>>
>> *From:* Luke Hinds [mailto:lhi...@redhat.com]
>> *Sent:* Monday, February 12, 2018 9:04 AM
>> *To:* SULLIVAN, BRYAN L (BRYAN L) <bryan.sulli...@research.att.com>
>> *Cc:* opnfv-tech-discuss@lists.opnfv.org; degirmenci, fatih <
>> fatih.degirme...@ericsson.com>; Raymond Paik <rp...@linuxfoundation.org>
>> *Subject:* Re: [opnfv-tech-discuss] Anteater status and link issue
>>
>>
>>
>>
>>
>>
>>
>> On Tue, Feb 6, 2018 at 2:32 PM, SULLIVAN, BRYAN L (BRYAN L) <
>> bryan.sulli...@research.att.com> wrote:
>>
>> Hi all,
>>
>> I’m wondering where the Anteater program is – and want to note a broken
>> link: build jobs with Anteater violations reference “Please visit:
>> https://wiki.opnfv.org/x/5oey
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.opnfv.org_x_5oey&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=s4zQHMsxrgVhlTs-Sw4-uGIsKYDMsnIQuvx0TehUoSk&e=>
>> ”, which is the wiki page https://wiki.opnfv.org/pages/v
>> iewpage.action?pageId=11700198
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.opnfv.org_pages_viewpage.action-3FpageId-3D11700198&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=burTDZjfgUSG9lAKW4MjRDZULxleQEsKGknHvhdqzbA&e=>,
>> which says “Project specific exceptions can be added for file_name,
>> file_contents and binaries, by using the name of the repository within the
>> anteater/exceptions/ directory of the releng-anteater
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.opnfv.org_gerrit.opnfv.org-3A29418_releng-2Danteater.git&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=LrWykp0HOa_RUbxOEJDo7sojbPgNgsVsrlV6jmwMVx4&e=>
>>  repository.” – but that link (releng-anteater
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wiki.opnfv.org_gerrit.opnfv.org-3A29418_releng-2Danteater.git&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=LrWykp0HOa_RUbxOEJDo7sojbPgNgsVsrlV6jmwMVx4&e=>)
>> is broken.
>>
>> I want to start adding the exceptions for Models etc as an example for
>> the LF IT team that is setting up the Acumos project gerrit/CI/CD process,
>> and in general to help optimize the Anteater overhead for projects. I think
>> we need to get some analysis of the types of exceptions that are typical,
>> and establish a process for vetting those exceptions that goes beyond a
>> simple review by a releng committer.
>>
>> Further, we need to bring in other scan tools (e.g. security
>> vulnerability, virus, or malicious code scans) into the Anteater process.
>> This is in response to concerns about the security of the governance
>> process for open source (e.g. upstream, but also direct contribution in
>> projects) that is used to build production-oriented systems. We need to
>> demonstrate that OPNFV and other LF projects are addressing these concerns
>> through their infra toolsets.
>>
>>
>>
>> Sorry Bryan, I missed a few of these emails thanks (or rather no thanks)
>> to a bad mail filter rule.
>>
>> I am working on the following now which we will see soon:
>>
>> Much better documentation: http://anteater.readthedocs.io/en/latest/
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__anteater.readthedocs.io_en_latest_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=AdeEyIqajKWRGD1zz3MXcKrWoAWYR6mXmQDgVVzp1Zo&e=>
>>
>> [bryan] Are you going to start hosting these docs at docs.opnfv.org?
>>
> We can do yes, although I guess it make sense to have the main body of the
> documentation around the tool upstream (once the github re-homing happens),
> and then have everything OPNFV developers need to know about how anteater
> is used in OPNFV at docs.opnfv.org - this way there won't be materials in
> docs.opnfv.org around using Travis CI (which would confuse people).
>
>> Virus total integration:
>>
>>    * Any binaries will be scanned using the virus total API, unless a
>> sha256 waiver is already present e.g. https://github.com/opnfv/relen
>> g-anteater/blob/master/exceptions/calipso.yaml#L9
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_opnfv_releng-2Danteater_blob_master_exceptions_calipso.yaml-23L9&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=WNetEYMktH0pxwVzSJXZyDFVnJr6lIDBhM6laGBrbjs&e=>
>>
>>    * Any IP addresses / domain name / URL will be scanned (again using
>> the Virus Total API) for known malware and other nastiness.
>>
>> [bryan] VirusTotal looks like a useful service. Are there any stats for
>> its effectiveness at detecting threats, including new threats and delay in
>> supporting them?
>>
> Its pretty much the epicentre of community based threat collaboration .
> It aggregates 40 virus / malware scanners to asses files, and domains / IP
> addresses are assessed against 70 URL/domain blacklisting services:
>
> https://support.virustotal.com/hc/en-us/articles/115002126889-How-it-works
>
> It also fits well into anteater:
>
> Currently anteater will generate a sha256sum of any blobs it finds, and
> will report them, *unless* a sha256 is entered into the exception files. I
> will extend this, so that if a blob is found an *no* sha256sum exception
> exists, we send the file hash to Virustotal to see if its registered as
> nefarious. If it is we will fail the job and alarm the finding. If not, an
> exception can be entered and it will be ignored from there on in and we
> won't trouble the VT API again for that particular file - unless someone at
> a later point changes the file (which would change the checkum) and then a
> scan is made again - this way we can be sure that an infected file is not
> checked into a project and we are not aware as it has the same name as
> before.
>
>> I also have a load of new strings to add to dig out and report anything
>> of a more recent finding (for example a javascript based bitcoin miner).
>>
>> [bryan] I would like to see how we can improve the contextual
>> effectiveness of the pattern matching approach. Any bar (or port in a
>> storm) may seem to be better than none, and can at least catch newbie
>> mistakes and anti-patterns, but most of the strings I’ve included in
>> https://github.com/opnfv/models/blob/master/tools/anteater-e
>> xceptions.yaml relate to IMO innocuous (if admittedly sometimes cheap or
>> anti-patterned) use of prohibited words. Others, I clearly need to fix.
>>
>
> So I am very open to switching off the more noisy regexs that emit false
> positives and also open to new approaches. I am sure I can fine tune them
> much better as well.
>
> Likewise open to any feature recommendations etc.
>
>> The project is also hopefully going to move into github (once agreed with
>> LF) to encourage wider contributions and allow it to be more easily
>> consumed else.
>>
>> [bryan] Anything that broadens contribution and consumption makes sense
>> to me. Are there any other open source projects in this same space that you
>> are considering leveraging, to avoid re-developing features unnecessarily?
>>
> We plan to discuss wider LFN adoption , one example being OpenDayLight
> where I manage security. I also plan to get more eyes on the tool for
> smaller projects to utilise.  An OpenStack project is also considering the
> tool, but more for finding depreciated key directives and release tags.
>
>> Once the above is in place, docs will be clearer to follow, project will
>> be more presentable, with more coverage in finding vulns will be wider.
>>
>>
>>
>> [bryan] We probably need more docs re the process for getting exceptions
>> approved, and how the community can track its effectiveness in the mission
>> represented by this toolset, through the types of approved exception
>> patterns, as they grow (or shrink… it would be good to see the community
>> improving through reduction in the need to maintain exceptions, and partly
>> because the tool is getting smarter).
>>
>
> Very much agree, it would be great to see people add to the master
> exception / ignore list and feedback on where the tool works well / is
> annoying etc.
>
> I also agree on the docs and enlarging upon the process for getting
> exceptions approved. I plan to have all this done before ONS so we can see
> it in place for then.
>
> Thanks,
>>
>> Bryan Sullivan | AT&T
>>
>>
>>
>>
>> _______________________________________________
>> opnfv-tech-discuss mailing list
>> opnfv-tech-discuss@lists.opnfv.org
>> https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.opnfv.org_mailman_listinfo_opnfv-2Dtech-2Ddiscuss&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=rxS4DJ0v-VW97BzYjY1_yRCmM1znHlxObcXiTIP6RBI&s=8NPFgQFDZsv688HirOlM8HW1u0X9QVVgUfsN6B5PP_s&e=>
>>
>>


-- 
Luke Hinds | NFV Partner Engineering | CTO Office | Red Hat
e: lhi...@redhat.com | irc: lhinds @freenode | t: +44 12 52 36 2483
_______________________________________________
opnfv-tech-discuss mailing list
opnfv-tech-discuss@lists.opnfv.org
https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss

Reply via email to