[
https://issues.apache.org/jira/browse/CASSANDRA-14612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17424339#comment-17424339
]
Stefan Miklosovic commented on CASSANDRA-14612:
-----------------------------------------------
Hi [~e.dimitrova],
I am trying to wrap my head around this. The output from my build is this:
{code}
08:39:59 dependency-check-download:
08:39:59 [echo] Downloading OWASP Dependency checks ...
08:39:59 [mkdir] Created dir: /tmp/dependency-check
08:39:59 [get] Getting:
https://github.com/jeremylong/DependencyCheck/releases/download/v6.3.1/dependency-check-ant-6.3.1-release.zip
08:39:59 [get] To:
/tmp/dependency-check/dependency-check-ant-6.3.1-release.zip
08:40:00 [get]
https://github.com/jeremylong/DependencyCheck/releases/download/v6.3.1/dependency-check-ant-6.3.1-release.zip
moved to
https://github-releases.githubusercontent.com/5663857/82a68585-2e0b-46c0-a561-09298a19a2bf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211001%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211001T064000Z&X-Amz-Expires=300&X-Amz-Signature=36fbcbef4872994d5f87c03437e5b94a3c3e3370528d42a7532eee77de42b176&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5663857&response-content-disposition=attachment%3B%20filename%3Ddependency-check-ant-6.3.1-release.zip&response-content-type=application%2Foctet-stream
08:40:01 [unzip] Expanding:
/tmp/dependency-check/dependency-check-ant-6.3.1-release.zip into
/tmp/dependency-check
{code}
but your output is this:
{code}
00:50:56 dependency-check-download:
00:50:56 [echo] Downloading OWASP Dependency checks ...
00:50:56 [get] Getting:
https://github.com/jeremylong/DependencyCheck/releases/download/v6.3.1/dependency-check-ant-6.3.1-release.zip
00:50:56 [get] To:
/tmp/dependency-check/dependency-check-ant-6.3.1-release.zip
00:50:57 [get]
https://github.com/jeremylong/DependencyCheck/releases/download/v6.3.1/dependency-check-ant-6.3.1-release.zip
moved to
https://github-releases.githubusercontent.com/5663857/82a68585-2e0b-46c0-a561-09298a19a2bf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20211003%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20211003T225055Z&X-Amz-Expires=300&X-Amz-Signature=0f54fbc8b4034062b8c1e7a31b40668177d47330d37c3d815fc7b6b9795e4547&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=5663857&response-content-disposition=attachment%3B%20filename%3Ddependency-check-ant-6.3.1-release.zip&response-content-type=application%2Foctet-stream
00:50:57 [get] Not modified - so not downloaded
00:50:57 [unzip] Expanding:
/tmp/dependency-check/dependency-check-ant-6.3.1-release.zip into
/tmp/dependency-check
{code}
So the only difference is that your output contains "[get] Not modified - so
not downloaded" string.
Now the very fact this target proceeded with its execution is based on the fact
whether that owasp zip is already downloaded or not (1) so here it just didt
find the jar and it proceeded with the downloading.
However, I see that they released version 6.3.2 recently in the meanwhile so
get task probably evaluated this as "not modified" or whatever which resulted
in empty zip downloaded (or no zip at all) which then failed to extract it.
This is what Get Ant task writes (2)
Lets put that logic here and elaborate on it a little bit:
{code}
if (responseCode == HttpURLConnection.HTTP_NOT_MODIFIED
|| (lastModified != 0 && hasTimestamp && timestamp >= lastModified)) {
// not modified so no file download. just return
// instead and trace out something so the user
// doesn't think that the download happened when it
// didn't
log("Not modified - so not downloaded", logLevel);
return null;
}
{code}
So either response code is HTTP_NOT_MODIFIED or the other part is true. If the
second part is true, lastModified can not be 0 in the first place, the logic
dealing with this is also in Get task:
{code}
//set the timestamp to the file date.
long timestamp = 0;
boolean hasTimestamp = false;
if (useTimestamp && dest.exists()) {
timestamp = dest.lastModified();
if (verbose) {
final Date t = new Date(timestamp);
log("local file date : " + t.toString(), logLevel);
}
hasTimestamp = true;
}
{code}
We do use "useTimestamp" but the whole idea here is that dest does exist but
our target assumes that it does not so I consider this code to not be invoked
hence I bet that the reason it was skipped was that HTTP_NOT_MODIFIED header
was present.
If that is the case, I am not completely sure how we can force the dowload if
response code is HTTP_NOT_MODIFIED which probably happens as they do a new
release every now and then which would result in this error.
We might bundle this zip directly into image which Jenkins uses for builds so
we do not need to download anything, then we would just bypass this completely.
(1) https://github.com/apache/cassandra/blob/trunk/.build/build-owasp.xml#L30
(2)
https://github.com/apache/ant/blob/master/src/main/org/apache/tools/ant/taskdefs/Get.java#L802-L810
(1)
> Please add OWASP Dependency Check to the build (pom.xml)
> --------------------------------------------------------
>
> Key: CASSANDRA-14612
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14612
> Project: Cassandra
> Issue Type: New Feature
> Components: Build
> Environment: All development, build, test, environments.
> Reporter: Albert Baker
> Assignee: Stefan Miklosovic
> Priority: Normal
> Labels: build, security
> Fix For: 3.0.26, 3.11.12, 4.0.2, 4.1
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Please add OWASP Dependency Check to the build (pom.xml). OWASP DC makes an
> outbound REST call to MITRE Common Vulnerabilities & Exposures (CVE) to
> perform a lookup for each dependant .jar to list any/all known
> vulnerabilities for each jar. This step is needed because a manual MITRE CVE
> lookup/check on the main component does not include checking for
> vulnerabilities in components or in dependant libraries.
> OWASP Dependency check :
> https://www.owasp.org/index.php/OWASP_Dependency_Check has plug-ins for most
> Java build/make types (ant, maven, ivy, gradle).
> Also, add the appropriate command to the nightly build to generate a report
> of all known vulnerabilities in any/all third party libraries/dependencies
> that get pulled in. example : mvn -Powasp -Dtest=false -DfailIfNoTests=false
> clean aggregate
> Generating this report nightly/weekly will help inform the project's
> development team if any dependant libraries have a reported known
> vulnerailities. Project teams that keep up with removing vulnerabilities on a
> weekly basis will help protect businesses that rely on these open source
> componets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
