Re: [openstack-dev] [Security]Would people see a value in the cve-check-tool?

[Jeremy Stanley]

On 2015-08-11 20:42:26 + (+), Bhandaru, Malini K wrote:
[...]
> Another place I see value is running periodically against past
> releases – Icehouse, Juno etc to catch any vulnerabilities in
> production systems. When we issue security notes we typically
> specify any past releases that carry the vulnerability and this
> would be on par with that.
[...]

I don't see how this would help. We cap the versions of libraries we
support from PyPI solely for the benefit of our stable branch
testing. We can't support changing those upper bounds in already
existing stable releases since the vast majority of them don't have
similar stable backport policies for security fixes. So while this
tool might be able to *detect* that our prior releases only work
with vulnerable versions of dependencies, we could never *fix* those
ourselves so they'd be forever alerting on every run thereafter.

This is the sort of work which downstream package maintainers and
distributors are well equipped to take care of, and something which
we really can't control upstream at all.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security]Would people see a value in the cve-check-tool?

[Bhandaru, Malini K]

Rob, Timur, Travis, and Victor, thank you for your input! We are excited about 
the feedback.

Added [Security] in subject per Rob’s suggestion. Copied all the security 
interested parties who responded.

Another place I see value is running periodically against past releases – 
Icehouse, Juno etc
to catch any vulnerabilities in production systems. When we issue security 
notes we typically specify any past releases that carry the vulnerability and 
this would be on par with that.

A developer could introduce a vulnerability in any edit, which bandit would 
catch. However CVE check would not be such an active threat, running it once a 
day may  be adequate.

Regards
Malini

From: Clark, Robert Graham [mailto:robert.cl...@hp.com]
Sent: Tuesday, August 04, 2015 11:17 AM
To: OpenStack Development Mailing List (not for usage questions)
Cc: Heath, Constanza M; Ding, Jian-feng; Demeter, Michael; Bhandaru, Malini K
Subject: RE: [openstack-dev] Would people see a value in the cve-check-tool?

Hi Elena,

This is interesting work, thanks for posting it (and for posting it here on 
openstack-dev, we are trying to wind down the security ML) though maybe use the 
[Security] tag in the subject line next time.

I think this is a very interesting project, though it’s unclear to me who might 
be the targeted users for this? It seems like it would make the most sense for 
this to be in the gate. Now this could be the standard build gates (Jenkins 
etc) but I’m not sure how much sense that makes on its own, after all most 
production consumers (those who care about CVEs) of OpenStack are probably not 
consuming it vanilla from source but are more likely to be consuming it from a 
vendor who’s already packaged it up.

In the latter case, I’m sure vendors would find this tool very useful, we do 
something similar at HP today but I’m sure a tool like this would add value and 
it’s probably something we could contribute to.

As I write this I’ve realised that there would be an interesting possibility in 
the former case (putting this in the upstream OpenStack gates). It would be 
interesting to see something running that regularly checks for CVE’s in the 
libraries that _could_ be included in OpenStack, (library requirements within 
OpenStack often include more than one version) and bumps the version to the 
next safest and submits a change request for manual verification etc.

-Rob







From: Adam Heczko [mailto:ahec...@mirantis.com]
Sent: 03 August 2015 23:18
To: OpenStack Development Mailing List (not for usage questions)
Cc: Heath, Constanza M; Ding, Jian-feng; Demeter, Michael; Bhandaru, Malini K
Subject: Re: [openstack-dev] Would people see a value in the cve-check-tool?

Hi Elena, the tool looks very interesting.
Maybe try to spread out this proposal also through openstack-security@ ML.
BTW, I can't find the wrapper mentioned - am I missing something?

Regards,

Adam

On Mon, Aug 3, 2015 at 11:08 PM, Reshetova, Elena 
mailto:elena.reshet...@intel.com>> wrote:
Hi,

We would like to ask opinions if people find it valuable to include a 
cve-check-tool into the OpenStack continuous integration process?
A tool can be run against the package and module dependencies of OpenStack 
components and detect any CVEs (in future there are also plans to integrate 
more functionality to the tool, such as scanning of other vulnerability 
databases and etc.). It would not only provide fast detection of new 
vulnerabilities that are being released for existing dependencies, but also 
control that people are not introducing new vulnerable dependencies.

The tool is located here: https://github.com/ikeydoherty/cve-check-tool

I am attaching an example of a very simple Python wrapper for the tool, which 
is able to process formats like: 
http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.txt
and an example of html output if you would be running it for the python module 
requests 2.2.1 version (which is vulnerable to 3 CVEs).

Best Regards,
Elena.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Adam Heczko
Security Engineer @ Mirantis Inc.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Reshetova, Elena]

> I guess it depends on whether the tool needs to read the entire database
to perform its queries (in which case using AFS would be basically the same
as downloading).

I am including below the reply from Michael, cve-check-tool maintainer, and
also including him in this conversation.

" Right now we force-update the database every 4 hours as this is roughly
how often the NVD DB is centrally updated with new/modified entries.

This behaviour can be disabled, and cve-check-update can be run to manually
update. We download all of the NVD XML feeds and convert them into a local
sqlite3 database for faster usage.

What you're proposing sounds more like making cve-check-tool run as a
Security As A Service setup, which is feasible. Would they be hosting
internal copies of the XML feeds or prefer a central remote DB to be used
here?"

-Original Message-
From: Jeremy Stanley [mailto:fu...@yuggoth.org] 
Sent: Wednesday, August 5, 2015 10:16 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Security] Would people see a value in the
cve-check-tool? (Reshetova, Elena)

On 2015-08-05 09:54:52 -0700 (-0700), Clint Byrum wrote:
> Doesn't this feel like a job for AFS? Maintain the db there, and let 
> the nodes access it as-needed?

I guess it depends on whether the tool needs to read the entire database to
perform its queries (in which case using AFS would be basically the same as
downloading).
--
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


smime.p7s
Description: S/MIME cryptographic signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Jeremy Stanley]

On 2015-08-05 09:54:52 -0700 (-0700), Clint Byrum wrote:
> Doesn't this feel like a job for AFS? Maintain the db there, and let the
> nodes access it as-needed?

I guess it depends on whether the tool needs to read the entire
database to perform its queries (in which case using AFS would be
basically the same as downloading).
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Jeremy Stanley]

On 2015-08-05 15:22:29 + (+), Jeremy Stanley wrote:
[...]
> Now that we've dissolved more of those arbitrary distinctions, this
> seems like a great opportunity for tracking with a governance tag.
> I'll go ahead and propose one later today if I get a spare moment.

Actually, I take that back. Now that the TC has decreed tags don't
apply to individual source repositories (they apply to either
project-teams or to "deliverables" which are higher-level
collections of repos) I'm no longer sure how we would go about
documenting repo-specific details like this anyway.

To Clark's point, "tracking requirements" is an emergent property
from the combination of requirements sync proposals and requirements
enforcement jobs. If we can find a way to force one of those to
depend on the other (perhaps the requirements sync can stop using a
flat file and instead operate on parsed output from our zuul
layout?) then that would be a cleaner means of identifying this
repo-specific detail.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Clint Byrum]

Excerpts from Reshetova, Elena's message of 2015-08-05 09:08:16 -0700:
> > The only concern that I have is the requisite database.  Downloading a
> 500MB + CVE database for the jobs could become painful.  We could either
> keep the CVE database on each node in the test pool or download it at the
> start of each cve-check job.  I¹d >be curious what the infra wizards have to
> say.
> 
> Actually the database is downloaded only once ( thefirst time) and then only
> database diffs are downloaded, which is much faster. I don't know enough
> about your node setup (do you fully clean up each node between the builds?)
> and etc., so the best way to test this would be if somebody can try it out
> and tell if it is a problem. If it is a problem, then we can discuss with
> the tool maintainer on how to address it. 
> 

Doesn't this feel like a job for AFS? Maintain the db there, and let the
nodes access it as-needed?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Jeremy Stanley]

On 2015-08-05 16:08:16 + (+), Reshetova, Elena wrote:
[...]
> Actually the database is downloaded only once ( thefirst time) and
> then only database diffs are downloaded, which is much faster. I
> don't know enough about your node setup (do you fully clean up
> each node between the builds?) and etc., so the best way to test
> this would be if somebody can try it out and tell if it is a
> problem. If it is a problem, then we can discuss with the tool
> maintainer on how to address it.
[...]

Yes, we actually don't reuse job workers. We run each job in a fresh
virtual machine, delete it when complete and launch a new VM to take
its place. Thus the database would need to be downloaded as part of
the job, or pre-cached in the worker images we boot, or part of some
remote query service so that we don't have to have the entire
database local to the job runner.

> I am actually a bit confused why this file for example called
> "upper-constraints"? The name would indicate an upper border, but
> that doesn't make that much sense with packaging systems.
[...]

It is the maximum available version of each of our dependencies,
calculated by the Python packaging tool "pip" when attempting to
resolve mutually coinstallable versions of declared dependency
version ranges within the transitive set (which is nontrivial to
identify without recursively downloading and installing those
packages since they embed their dependency information, can
sometimes only determine it at runtime, and vary dependencies by
interpreter version as well).

That is to say, the upper-constraints.txt file lists the maximum
versions of dependencies you can possibly run our software with when
installed with the normal Python packaging toolchain. I don't see
where it would make sense to test for vulnerabilities in older
versions of dependencies anyway since it's not up to us to notify
end users of bugs in software we don't distribute unless it actually
prevents our software from running.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Jeremy Stanley]

On 2015-08-05 08:28:27 -0700 (-0700), Clark Boylan wrote:
> We already track it in the requirements repo itself [0]. Not sure if we
> need an additional tracking method.
> 
> [0]
> https://git.openstack.org/cgit/openstack/requirements/tree/projects.txt

That tracks repos which get reqs sync proposals (and possibly also
those for which hard requirements rewriting is performed in
DevStack-based jobs), but does not reflect repos which have reqs
enforcement jobs gating on them.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Reshetova, Elena]

> The only concern that I have is the requisite database.  Downloading a
500MB + CVE database for the jobs could become painful.  We could either
keep the CVE database on each node in the test pool or download it at the
start of each cve-check job.  I¹d >be curious what the infra wizards have to
say.

Actually the database is downloaded only once ( thefirst time) and then only
database diffs are downloaded, which is much faster. I don't know enough
about your node setup (do you fully clean up each node between the builds?)
and etc., so the best way to test this would be if somebody can try it out
and tell if it is a problem. If it is a problem, then we can discuss with
the tool maintainer on how to address it. 

>I¹d also really like to see what the baseline results look like.  If you
run it against current global requirements does it find legitimate issues?
Does it find false positives.  In any case it seems worth exploring as
vulnerabilities in upstream dependencies are a >key weakness in our current
system.

I have so far only run the tool against this file:
http://git.openstack.org/cgit/openstack/requirements/tree/upper-constraints.
txt and it didn't find any issues in it, given that the version there is
taken as minimal supported version.  I am myself not actually working in the
OpenStack project (so please excuse my ignorance on some even basic
matters), but I am actually a bit confused why this file for example called
"upper-constraints"? The name would indicate an upper border, but that
doesn't make that much sense with packaging systems. 

If you can point me to a full set of files that should be analyser with
cve-check-tool, I can do the runs and post results here. 

Best Regards,
Elena.

-Original Message-
From: McPeak, Travis [mailto:travis.mcp...@hp.com] 
Sent: Wednesday, August 5, 2015 6:15 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Security] Would people see a value in the
cve-check-tool? (Reshetova, Elena)

(Merging thread from security ML)

Bandit probably isn¹t the correct integration point for this - cve-check has
its own analysis procedures while Bandit uses Python AST.  Also I see the
use workflows being different.
For Bandit a developer/gate wants to
check a specific code snippet whereas for cve-check to be effective it
really needs to examine the entire dependency chain.

As Rob and I mentioned earlier a gate process on ³openstack-requirements²
seems like an ideal target for this.
The idea would be anytime a requirement is added (for example to enable a
newer version or an entirely new library to be used) we could run a
cve-check job that ensures the new library (or version) doesn¹t have any
known CVE¹s against it.  This way we can be covered across OpenStack (since
OpenStack projects can¹t use Libraries that aren¹t in global requirements).
The gate processing time is minimal since it doesn¹t have to run for each
project.

The only concern that I have is the requisite database.  Downloading a 500MB
+ CVE database for the jobs could become painful.  We could either keep the
CVE database on each node in the test pool or download it at the start of
each cve-check job.  I¹d be curious what the infra wizards have to say.

I¹d also really like to see what the baseline results look like.  If you run
it against current global requirements does it find legitimate issues?  Does
it find false positives.  In any case it seems worth exploring as
vulnerabilities in upstream dependencies are a key weakness in our current
system.

>Hi folks!
>
>Idea really looks good.
>
>I am attaching an example of a very simple Python wrapper for the tool
>
>
>Looks like this wrapper is lightweight. But maybe try to integrate it 
>with Bandit and not to create a new tool?
>
>--?
>Victor Ryzhenkin
>freerunner on #freenode
>
>


smime.p7s
Description: S/MIME cryptographic signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Clark Boylan]

On Wed, Aug 5, 2015, at 08:22 AM, Jeremy Stanley wrote:
> On 2015-08-05 15:04:15 + (+), Ian Cordasco wrote:
> > One point of clarification. Not every project has to opt into
> > global-requirements so this isn't necessarily true. Also with the
> > merging of the stackforge and openstack namespaces, it'll be
> > harder to distinguish when a project is or isn't using g-r since
> > in the past it was fairly safe to assume that stackforge/ projects
> > were more likely to not use g-r.
> 
> Agreed, this used to be a (perhaps not well-documented) necessity
> for repos which were in or dependencies of the integrated release.
> Now that we've dissolved more of those arbitrary distinctions, this
> seems like a great opportunity for tracking with a governance tag.
> I'll go ahead and propose one later today if I get a spare moment.
>
We already track it in the requirements repo itself [0]. Not sure if we
need an additional tracking method.

[0]
https://git.openstack.org/cgit/openstack/requirements/tree/projects.txt

Clark

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Jeremy Stanley]

On 2015-08-05 15:04:15 + (+), Ian Cordasco wrote:
> One point of clarification. Not every project has to opt into
> global-requirements so this isn't necessarily true. Also with the
> merging of the stackforge and openstack namespaces, it'll be
> harder to distinguish when a project is or isn't using g-r since
> in the past it was fairly safe to assume that stackforge/ projects
> were more likely to not use g-r.

Agreed, this used to be a (perhaps not well-documented) necessity
for repos which were in or dependencies of the integrated release.
Now that we've dissolved more of those arbitrary distinctions, this
seems like a great opportunity for tracking with a governance tag.
I'll go ahead and propose one later today if I get a spare moment.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Ian Cordasco]

On 8/5/15, 08:14, "McPeak, Travis"  wrote:

>(Merging thread from security ML)
>
>Bandit probably isn¹t the correct integration point for this - cve-check
>has its own analysis procedures while
>Bandit uses Python AST.  Also I see the use workflows being different.
>For Bandit a developer/gate wants to
>check a specific code snippet whereas for cve-check to be effective it
>really needs to examine the entire
>dependency chain.
>
>As Rob and I mentioned earlier a gate process on ³openstack-requirements²
>seems like an ideal target for this.
>The idea would be anytime a requirement is added (for example to enable a
>newer version or an entirely new
>library to be used) we could run a cve-check job that ensures the new
>library (or version) doesn¹t have any
>known CVE¹s against it.  This way we can be covered across OpenStack
>(since OpenStack projects can¹t use
>Libraries that aren¹t in global requirements).  The gate processing time
>is minimal since it doesn¹t have to
>run for each project.

One point of clarification. Not every project has to opt into
global-requirements so this isn't necessarily true. Also with the merging
of the stackforge and openstack namespaces, it'll be harder to distinguish
when a project is or isn't using g-r since in the past it was fairly safe
to assume that stackforge/ projects were more likely to not use g-r.

>
>The only concern that I have is the requisite database.  Downloading a
>500MB + CVE database for the jobs could
>become painful.  We could either keep the CVE database on each node in the
>test pool or download it at the
>start of each cve-check job.  I¹d be curious what the infra wizards have
>to say.
>
>I¹d also really like to see what the baseline results look like.  If you
>run it against current global
>requirements does it find legitimate issues?  Does it find false
>positives.  In any case it seems worth
>exploring as vulnerabilities in upstream dependencies are a key weakness
>in our current system.
>
>>Hi folks!
>>
>>Idea really looks good.
>>
>>I am attaching an example of a very simple Python wrapper for the tool
>>
>>
>>Looks like this wrapper is lightweight. But maybe try to integrate it
>>with Bandit and not to create a new tool?
>>
>>--?
>>Victor Ryzhenkin
>>freerunner on #freenode
>>
>>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[Jeremy Stanley]

On 2015-08-05 13:14:40 + (+), McPeak, Travis wrote:
[...]
> The only concern that I have is the requisite database.
> Downloading a 500MB + CVE database for the jobs could become
> painful.  We could either keep the CVE database on each node in
> the test pool or download it at the start of each cve-check job.
[...]

Oh, yep that's a whopper. Downloading that during the job is very
likely to make it slow and unreliable. Baking it into our worker
base images is also questionable since we need to be able to boot
them in cloud providers who may give us as little as a 20 GiB root
filesystem device. If it can be compressed or filtered to an order
of magnitude smaller, then that seems more reasonable to work with.
Otherwise we'd need some separate online query service to hold the
database and handle the lookups (either hosted in our infrastructure
or elsewhere).
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Would people see a value in the cve-check-tool? (Reshetova, Elena)

[McPeak, Travis]

(Merging thread from security ML)

Bandit probably isn¹t the correct integration point for this - cve-check
has its own analysis procedures while
Bandit uses Python AST.  Also I see the use workflows being different.
For Bandit a developer/gate wants to
check a specific code snippet whereas for cve-check to be effective it
really needs to examine the entire
dependency chain.

As Rob and I mentioned earlier a gate process on ³openstack-requirements²
seems like an ideal target for this.
The idea would be anytime a requirement is added (for example to enable a
newer version or an entirely new
library to be used) we could run a cve-check job that ensures the new
library (or version) doesn¹t have any
known CVE¹s against it.  This way we can be covered across OpenStack
(since OpenStack projects can¹t use
Libraries that aren¹t in global requirements).  The gate processing time
is minimal since it doesn¹t have to
run for each project.

The only concern that I have is the requisite database.  Downloading a
500MB + CVE database for the jobs could
become painful.  We could either keep the CVE database on each node in the
test pool or download it at the
start of each cve-check job.  I¹d be curious what the infra wizards have
to say.

I¹d also really like to see what the baseline results look like.  If you
run it against current global
requirements does it find legitimate issues?  Does it find false
positives.  In any case it seems worth
exploring as vulnerabilities in upstream dependencies are a key weakness
in our current system.

>Hi folks!
>
>Idea really looks good.
>
>I am attaching an example of a very simple Python wrapper for the tool
>
>
>Looks like this wrapper is lightweight. But maybe try to integrate it
>with Bandit and not to create a new tool?
>
>--?
>Victor Ryzhenkin
>freerunner on #freenode
>
>


smime.p7s
Description: S/MIME cryptographic signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev