I wonder is this might be a good topic for the next infra-wg. One thing we may be able to do is fix up the regex so stuff such as 'yum install curl' or 'apt-get install wget' don't cause false alarms.
The good thing with the framework is the regexs are easy to get at, so its easy for anyone to push a patch with a better fine tuned regex. On Thu, Mar 8, 2018 at 3:07 PM, SULLIVAN, BRYAN L (BRYAN L) < [email protected]> wrote: > Fatih, > > > > I think the problem is that it’s very difficult to differentiate between > use of wget/curl etc for: > > - Pulling in components/data from external sources, as part of a > deploy or test process, or even by design of the component being developed > (e.g. a config file is pulled from some repo/place and used in component > configuration) > - Use of APIs exposed by the NFV platform or applications running > under it > > > > Requiring all the latter uses to be allowed through regexp rules would be > the onerous part. > > > > Thanks, > > Bryan Sullivan | AT&T > > > > *From:* Fatih Degirmenci [mailto:[email protected]] > *Sent:* Thursday, March 08, 2018 7:01 AM > *To:* SULLIVAN, BRYAN L (BRYAN L) <[email protected]>; Luke > Hinds <[email protected]>; opnfv-tech-discuss <opnfv-tech-discuss@lists. > opnfv.org> > > *Subject:* Re: [opnfv-tech-discuss] [releng][security][infra] Anteater > Improvements > > > > Hi Brian, > > > > My comment wasn’t about the tools themselves but what they are used for > and to be honest the suggestion is nothing sort of heavy-handed approach. > > > > If someone includes something in an artifact that is consumed by someone > else for different purposes, we have responsibility to them that we will > always be able to recreate that artifact for different purposes such as > troubleshooting. > > If people hook their own machines to store artifacts that go into final > OPNFV release and if these machines disappear, we will have no possibility > to recreate what we released. > > Apart from that, if people want to rebuild something for some reason, they > will be unable to do that since the dependencies will not be available > anymore. > > These concerns are based on what we found while looking at our earlier > releases so I’m not talking about something that’s possible to happen but a > real issue that happened and we had to intervene. > > > > If projects guarantee the long term availability of the artifacts on the > original location (ie until the EOL of that specific release) and they ask > for exception, there will be no blocking. > > > > As you said, you are free to the promote tools and approaches like how I > am doing here since I believe they are important to be shared. > > > > /Fatih > > > > *From: *"SULLIVAN, BRYAN L (BRYAN L)" <[email protected]> > *Date: *Thursday, 8 March 2018 at 15:31 > *To: *Fatih Degirmenci <[email protected]>, Luke Hinds < > [email protected]>, "[email protected]" < > [email protected]> > *Subject: *RE: [opnfv-tech-discuss] [releng][security][infra] Anteater > Improvements > > > > I do recommend that we rely upon tools that can focus on the trust of > specific sources, and not the use of platform capabilities such as curl, > wget, etc. These (curl, wget, etc) are tools that can be used for many > purposes inside an application like an OPNFV platform, or its > deployment/testing tools. The flagging/blocking of patches just because the > code contains use of wget/curl/etc is an onerous and heavy-handed approach > to the goal. Having to create regexp rules for each valid use of wget/curl > is a non-starter for me and likely many others in this project. Thus I > support Luke’s plan, and intend to promote these same tools and approaches > for use in the Acumos and other LFN projects. > > > > Thanks, > > Bryan Sullivan | AT&T > > > > *From:* [email protected] [ > mailto:[email protected] > <[email protected]>] *On Behalf Of *Fatih > Degirmenci > *Sent:* Thursday, March 08, 2018 6:12 AM > *To:* Luke Hinds <[email protected]>; opnfv-tech-discuss < > [email protected]> > *Subject:* Re: [opnfv-tech-discuss] [releng][security][infra] Anteater > Improvements > > > > Hi Luke, > > > > I have few comments and followup questions regarding this: > > “This in turn means we won't raise alarms over curl, git clone and wget > and will instead check the IP addresses or URLS that those commands query. > This should make anteater a lot less chatty at gate.” > > > > You might remember that one of the reasons we have checks for curl/wget is > to find out if projects pull artifacts from unknown IPs during > build/deployment/testing. > > These are not malicious but we have seen that few of the IPs where the > projects fetch the artifacts belong to non-production/personal devices that > tend to disappear over time. > > As you know, this is an important issue from reproducibility and > traceability perspectives. > > > > Now the questions are; > > Assuming the IPs are not explicitly added to exception list for the > corresponding project, do you mean that we will stop flagging changes/files > that contain wget/curl against unknown IPs if they are not marked as > malicious on VirusTotal? > > We also had plans to make anteater checks voting/blocking. Will we discard > this plan since wget/curl against IPs are not even planned to be flagged? > > > > /Fatih > > > > *From: *<[email protected]> on behalf of Luke > Hinds <[email protected]> > *Date: *Thursday, 8 March 2018 at 14:02 > *To: *"[email protected]" <opnfv-tech-discuss@lists. > opnfv.org> > *Subject: *[opnfv-tech-discuss] [releng][security][infra] Anteater > Improvements > > > > Hello, > > I have some changes to improve the reporting ability and hopefully tone > down the false positives. > > Aneater will now interface with the VirusTotal public API: > > 1. If anteater finds a public IP address, the DNS history will be quiered > to see if the IP has past or present associations with malicious domains. > > > > 2. If a URL is found, it is checked against the VirusTotal API to see if > its marked as malicous. > > 3. Binaries will be sent to VirusTotal for a scan by the aggregation of > scanners hosted there. > > For anyone wanting a demo, please see the following: > > https://asciinema.org/a/JfzUPWpBGm0wDKPCN3KlK2DK0 > <https://urldefense.proofpoint.com/v2/url?u=https-3A__asciinema.org_a_JfzUPWpBGm0wDKPCN3KlK2DK0&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=ML-JPRZQOfToJjMwlJLPlcWimAEwMA5DZGNIrk-cgy0&m=Phto5YquFG94SB6m2dgJ83G2mmGj0cNr14cgRF9LHwA&s=2OjBdIMqqEr-gCgFDNZLOLPPdiF3zOemzOq73XIqiSU&e=> > > I will work with various people to get this rigged into CI. > > This in turn means we won't raise alarms over curl, git clone and wget and > will instead check the IP addresses or URLS that those commands query. This > should make anteater a lot less chatty at gate. > > Cheers, > > Luke > -- Luke Hinds | NFV Partner Engineering | CTO Office | Red Hat e: [email protected] | irc: lhinds @freenode | t: +44 12 52 36 2483
_______________________________________________ opnfv-tech-discuss mailing list [email protected] https://lists.opnfv.org/mailman/listinfo/opnfv-tech-discuss
