Bug#1001451: Candidate script updates
On Wed, 26 Jan 2022 10:10:04 +0100 Salvatore Bonaccorso wrote: > Hi Neil, > > I think, if there are no objections from other, that we can look > forward into merging the grab-cve-fixes and merge-cve-list updates. https://salsa.debian.org/security-tracker-team/security-tracker/-/merge_requests/100 I haven't set a reviewer or assignee for the MR. > While testing there was one minor glitch which might be checked later > if we want to have it fixed, please double check if my claim is > correct. If a list to be merged contains a listing of a reserved CVE, > say something like > > CVE [temporary description] > RESERVED > - source fixedversion > NOTE: note > > then merge-cve-list will stumple over the RESERVED keyword. Bu this > can easily be workrounded by cleaning up the list one want to merge by > the RESERVED keywords. I can look at that as an extension of the existing bug report, I've also got the REJECTED support to add to update-vuln for Moritz. > Maybe you can put both in a MR and give others say until end of week > time to object on both scripts updates and then go ahead with the > merging. > > I have not looked at the third script (update-vuln), I had so far > little usecases to directly work with it. > > Regards and thanks a lot for your work on this part as well. > Salvatore -- Neil Williams = https://linux.codehelp.co.uk/ pgpHbmAsNX69d.pgp Description: OpenPGP digital signature
Bug#1001451: Candidate script updates
Hi Neil, I think, if there are no objections from other, that we can look forward into merging the grab-cve-fixes and merge-cve-list updates. While testing there was one minor glitch which might be checked later if we want to have it fixed, please double check if my claim is correct. If a list to be merged contains a listing of a reserved CVE, say something like CVE [temporary description] RESERVED - source fixedversion NOTE: note then merge-cve-list will stumple over the RESERVED keyword. Bu this can easily be workrounded by cleaning up the list one want to merge by the RESERVED keywords. Maybe you can put both in a MR and give others say until end of week time to object on both scripts updates and then go ahead with the merging. I have not looked at the third script (update-vuln), I had so far little usecases to directly work with it. Regards and thanks a lot for your work on this part as well. Salvatore
Bug#1001451: Candidate script updates
On Wed, 12 Jan 2022 11:11:42 +0100 Salvatore Bonaccorso wrote: > Hi, > > On Wed, Jan 12, 2022 at 09:22:45AM +, Neil Williams wrote: > > On Wed, 12 Jan 2022 12:44:14 +0800 > > Paul Wise wrote: > > > > > On Tue, 2022-01-11 at 11:20 +, Neil Williams wrote: > > > > > > > I might need to brush up on my Perl and make a patch for lintian > > > > which downloads the sec tracker JSON and checks the CVE list in > > > > the .changes file - warnings from lintian are more likely to > > > > get fixed prior to upload. Depends if you think this happens > > > > sufficiently often that it is a problem worth solving. > > > > (Considering how long it's been since I did that amount of code > > > > in Perl, maybe I'm better filing the bug against lintian and > > > > seeing if someone else can come up with a patch... - again, > > > > only if it happens sufficiently often.) > > > > > > FTR, lintian does not do any network requests, so this approach > > > won't be accepted. The best option you can get is a script to do > > > the download at the lintian release time. Unfortunately this > > > means the data will get outdated quickly and make the check less > > > useful. > > > > > > This check could be added to devscripts, debsecan or duck. > > > > debsecan looks promising. It already has support for reporting a > > list of CVEs by source_package name, directly from > > https://security-tracker.debian.org/tracker/ and it's Python3. I'll > > have a look at a patch which accepts a .changes file or d.changelog > > entry and verifies if all listed CVEs actually exist for the source > > package of that change. > > To jump in in that part of the discussion: debsecan's scope is > different what we want to tackle here. Its a tool which is used on > user systems to generate a list of vulnerabilities which affect the > installed Debian system, gathering data from the security-tracker and > comparing on what is installed. True, so a patch directly to debsecan isn't the best option. However, debsecan does have an API for the network operation needed for a developer to download specific security tracker data without needing to get the entire JSON. This is the bit that can't go into lintian & debsecan's API is fast. This API allows a new script that uses the some of that debsecan code and some of the grab-cve-in-fix code to parse a .changes file on STDIN, extract the CVEs which the maintainer claims will be closed by that upload and verify that the CVE numbers exist in the current security tracker, at the time of running the script, for the source package and new version string that are specified in the changes file. It can do this needing only python3-apt and python3-debian, without needing to import Python code or local data files from the security tracker git tree. i.e. the code in debsecan gets me to a proof-of-concept for such a script that works, as below (using a modified .changes file copied from https://tracker.debian.org/news/1291853/accepted-vim-2823995-1-source-into-unstable/ that deliberately includes the typo from that epiphany test we've used before). initial PoC output: CVE-2021-3984 is already fixed in vim CVE-2022-0158 in vim will be closed by this upload CVE-2022-0156 in vim will be closed by this upload CVE-2021-4508 is not an open CVE in vim So I can then remove the unwanted parts of this PoC and create a new script, derived from debsecan, that could go into devscripts or similar. We should discuss this "abuse"/re-use of the debsecan support in https://security-tracker.debian.org/tracker/ and whether to clone that support for a new script - no changes would be needed in the security tracker code other than possibly a change in User-Agent & the debsecan part of the URL. Let me know your preference for: 1. re-use the debsecan API without changes 2. clone the debsecan API in security-tracker with minimal changes (UserAgent, URL & Version only) - the code would use precisely the same DB tables as debsecan. Tested locally with http://127.0.0.1:10605/tracker/debsecan/release/1/ - the script works in under half a second. I can then file a new bug in devscripts for a new script (check-cve-upload? or cvecheck?), complete with an initial PoC. It's possible that some of this common code in cvecheck & grab-cve-in-fix could go into python3-debian but the PoC only needs to add 43 lines to debsecan, so it might not be worth it. The guts of it is just comparing two lists that are already available from the current scripts. What's in the changes list but not in the debsecan list is not a CVE that should be mentioned by the upload. These are the typos we're trying to find, so report an error & exit non-zero. What's in the debsecan list but not in the changes list is a CVE that will remain open after the upload. These the maintainer should know about in case something has been omitted, so report a warning & exit non-zero. What is in the changes list and in debsecan list *but as already fixed* according to
Bug#1001451: Candidate script updates
Hi, On Wed, Jan 12, 2022 at 09:22:45AM +, Neil Williams wrote: > On Wed, 12 Jan 2022 12:44:14 +0800 > Paul Wise wrote: > > > On Tue, 2022-01-11 at 11:20 +, Neil Williams wrote: > > > > > I might need to brush up on my Perl and make a patch for lintian > > > which downloads the sec tracker JSON and checks the CVE list in the > > > .changes file - warnings from lintian are more likely to get fixed > > > prior to upload. Depends if you think this happens sufficiently > > > often that it is a problem worth solving. (Considering how long > > > it's been since I did that amount of code in Perl, maybe I'm better > > > filing the bug against lintian and seeing if someone else can come > > > up with a patch... - again, only if it happens sufficiently often.) > > > > > > > FTR, lintian does not do any network requests, so this approach won't > > be accepted. The best option you can get is a script to do the > > download at the lintian release time. Unfortunately this means the > > data will get outdated quickly and make the check less useful. > > > > This check could be added to devscripts, debsecan or duck. > > debsecan looks promising. It already has support for reporting a list > of CVEs by source_package name, directly from > https://security-tracker.debian.org/tracker/ and it's Python3. I'll > have a look at a patch which accepts a .changes file or d.changelog > entry and verifies if all listed CVEs actually exist for the source > package of that change. To jump in in that part of the discussion: debsecan's scope is different what we want to tackle here. Its a tool which is used on user systems to generate a list of vulnerabilities which affect the installed Debian system, gathering data from the security-tracker and comparing on what is installed. Regards, Salvatore
Bug#1001451: Candidate script updates
On Wed, 12 Jan 2022 12:44:14 +0800 Paul Wise wrote: > On Tue, 2022-01-11 at 11:20 +, Neil Williams wrote: > > > I might need to brush up on my Perl and make a patch for lintian > > which downloads the sec tracker JSON and checks the CVE list in the > > .changes file - warnings from lintian are more likely to get fixed > > prior to upload. Depends if you think this happens sufficiently > > often that it is a problem worth solving. (Considering how long > > it's been since I did that amount of code in Perl, maybe I'm better > > filing the bug against lintian and seeing if someone else can come > > up with a patch... - again, only if it happens sufficiently often.) > > > > FTR, lintian does not do any network requests, so this approach won't > be accepted. The best option you can get is a script to do the > download at the lintian release time. Unfortunately this means the > data will get outdated quickly and make the check less useful. > > This check could be added to devscripts, debsecan or duck. debsecan looks promising. It already has support for reporting a list of CVEs by source_package name, directly from https://security-tracker.debian.org/tracker/ and it's Python3. I'll have a look at a patch which accepts a .changes file or d.changelog entry and verifies if all listed CVEs actually exist for the source package of that change. > > The sectracker JSON is very large, so I think that a new API will be > needed for any tool that implements such a check. :-) -- Neil Williams = https://linux.codehelp.co.uk/ pgprFFmbUYNSx.pgp Description: OpenPGP digital signature
Bug#1001451: Candidate script updates
On Tue, 2022-01-11 at 11:20 +, Neil Williams wrote: > I might need to brush up on my Perl and make a patch for lintian which > downloads the sec tracker JSON and checks the CVE list in the .changes > file - warnings from lintian are more likely to get fixed prior to > upload. Depends if you think this happens sufficiently often that it is > a problem worth solving. (Considering how long it's been since I did > that amount of code in Perl, maybe I'm better filing the bug against > lintian and seeing if someone else can come up with a patch... - again, > only if it happens sufficiently often.) FTR, lintian does not do any network requests, so this approach won't be accepted. The best option you can get is a script to do the download at the lintian release time. Unfortunately this means the data will get outdated quickly and make the check less useful. This check could be added to devscripts, debsecan or duck. The sectracker JSON is very large, so I think that a new API will be needed for any tool that implements such a check. -- bye, pabs https://wiki.debian.org/PaulWise signature.asc Description: This is a digitally signed message part
Bug#1001451: Candidate script updates
Hi, On Tue, Jan 11, 2022 at 05:21:40PM +0100, Salvatore Bonaccorso wrote: > Hi Neil, > > On Tue, Jan 11, 2022 at 11:20:16AM +, Neil Williams wrote: > > Hi Salvatore, > > > > On Mon, 10 Jan 2022 17:10:15 +0100 > > Salvatore Bonaccorso wrote: > > > > > > So for that epiphany tracker, there is a typo in the d.changelog - > > > > the automated link for CVE-2021-4508 is a 404. > > > > > > > > I've updated the script to catch this and report the error. From the > > > > security-tracker source-package page for epiphany, it looks like the > > > > d.changelog entry should be CVE-2021-45088 - a simple typo to omit > > > > the final repeated digit. > > > > > > > > Currently, I'm handling this by advising that the script is re-run > > > > using the offline support and a corrected list of CVE IDs. > > > > > > > > This also adds a --force-version option to the offline support, in > > > > case sid has moved ahead of the fixed version by the time the CVE > > > > list is updated. > > > > > > Ack, this works indeed in this case as the CVE-2021-4508 does not > > > exist (yet). But in other cases typos are e.g. just wapping a number > > > or misstype the year, or other typos, which lead to an existing CVE. > > > So basically this all really boils down to, people working on > > > security-tracker trying as much as possible, in the human limits :), > > > to do a diligent work as possible. > > > > The script also prints out the CVE description and then tries to > > identify any existing package link by checking the PackageAnnotation of > > each CVE. It prints a warning if the source package of the CVE from the > > changes input doesn't match the source package of the changes itself or > > the source package(s) of other CVEs in the list. It can't always be an > > error as the situation with embedded copies & removed|reintroduced > > packages makes it hard for the script. However, it should be obvious > > from the output if a typo has been made. > > > > I've also added output if a typo matches a CVE that is an NFU as > > there are lot more of those in data/CVE/list e.g. > > > > $ cat test.changes| ./bin/grab-cve-in-fix --input > > grab-cve-in-fix - INFO - Retrieving data STDIN ... > > grab-cve-in-fix - INFO - CVE-2021-45085: (XSS can occur in GNOME Web > > (aka Epiphany) before 40.4 and 41.x before ...) > > grab-cve-in-fix - INFO - CVE-2021-45086: (XSS can occur in GNOME Web > > (aka Epiphany) before 40.4 and 41.x before ...) > > grab-cve-in-fix - INFO - CVE-2021-45087: (XSS can occur in GNOME Web > > (aka Epiphany) before 40.4 and 41.x before ...) > > grab-cve-in-fix - INFO - CVE-2021-3757: (immer is vulnerable to > > Improperly Controlled Modification of Object Pr ...) > > grab-cve-in-fix - ERROR - CVE CVE-2021-3757 is not attributed to a > > Debian package: ['NOT-FOR-US', 'NOTE'] > > > > (test.changes in this case was edited to change the typo to > > CVE-2021-3757 to demonstrate the output). > > Thanks! > > I noted another case which triggers an error but in this case should > not (I think the same issue might be present in merge-cve-list, but > need to double check): > > ../gragcvefix/bin/grab-cve-in-fix --tracker > https://tracker.debian.org/news/1294112/accepted-systemd-2502-1-source-into-unstable/ > grab-cve-in-fix - INFO - Retrieving data from distro-tracker... > grab-cve-in-fix - INFO - CVE-2021-3997: > grab-cve-in-fix - ERROR - CVE CVE-2021-3997 is not attributed to a Debian > package: ['RESERVED'] nevermind, the problem here was PEBKAC. merge-cve-list will later on stuble still over RESERVED, but I need to respin my tests with the scripts first before I can give sensible feedback. The above was nonsense. Regards, Salvatore
Bug#1001451: Candidate script updates
Hi Neil, On Tue, Jan 11, 2022 at 11:20:16AM +, Neil Williams wrote: > Hi Salvatore, > > On Mon, 10 Jan 2022 17:10:15 +0100 > Salvatore Bonaccorso wrote: > > > > So for that epiphany tracker, there is a typo in the d.changelog - > > > the automated link for CVE-2021-4508 is a 404. > > > > > > I've updated the script to catch this and report the error. From the > > > security-tracker source-package page for epiphany, it looks like the > > > d.changelog entry should be CVE-2021-45088 - a simple typo to omit > > > the final repeated digit. > > > > > > Currently, I'm handling this by advising that the script is re-run > > > using the offline support and a corrected list of CVE IDs. > > > > > > This also adds a --force-version option to the offline support, in > > > case sid has moved ahead of the fixed version by the time the CVE > > > list is updated. > > > > Ack, this works indeed in this case as the CVE-2021-4508 does not > > exist (yet). But in other cases typos are e.g. just wapping a number > > or misstype the year, or other typos, which lead to an existing CVE. > > So basically this all really boils down to, people working on > > security-tracker trying as much as possible, in the human limits :), > > to do a diligent work as possible. > > The script also prints out the CVE description and then tries to > identify any existing package link by checking the PackageAnnotation of > each CVE. It prints a warning if the source package of the CVE from the > changes input doesn't match the source package of the changes itself or > the source package(s) of other CVEs in the list. It can't always be an > error as the situation with embedded copies & removed|reintroduced > packages makes it hard for the script. However, it should be obvious > from the output if a typo has been made. > > I've also added output if a typo matches a CVE that is an NFU as > there are lot more of those in data/CVE/list e.g. > > $ cat test.changes| ./bin/grab-cve-in-fix --input > grab-cve-in-fix - INFO - Retrieving data STDIN ... > grab-cve-in-fix - INFO - CVE-2021-45085: (XSS can occur in GNOME Web > (aka Epiphany) before 40.4 and 41.x before ...) > grab-cve-in-fix - INFO - CVE-2021-45086: (XSS can occur in GNOME Web > (aka Epiphany) before 40.4 and 41.x before ...) > grab-cve-in-fix - INFO - CVE-2021-45087: (XSS can occur in GNOME Web > (aka Epiphany) before 40.4 and 41.x before ...) > grab-cve-in-fix - INFO - CVE-2021-3757: (immer is vulnerable to > Improperly Controlled Modification of Object Pr ...) > grab-cve-in-fix - ERROR - CVE CVE-2021-3757 is not attributed to a > Debian package: ['NOT-FOR-US', 'NOTE'] > > (test.changes in this case was edited to change the typo to > CVE-2021-3757 to demonstrate the output). Thanks! I noted another case which triggers an error but in this case should not (I think the same issue might be present in merge-cve-list, but need to double check): ../gragcvefix/bin/grab-cve-in-fix --tracker https://tracker.debian.org/news/1294112/accepted-systemd-2502-1-source-into-unstable/ grab-cve-in-fix - INFO - Retrieving data from distro-tracker... grab-cve-in-fix - INFO - CVE-2021-3997: grab-cve-in-fix - ERROR - CVE CVE-2021-3997 is not attributed to a Debian package: ['RESERVED'] (note I used your worktree teqnique to access the current versions, thanks again for this tip). In the above case MITRE has not yet populated the CVE entry and so the entry looks like: CVE-2021-3997 [Uncontrolled recursion in systemd's systemd-tmpfiles] RESERVED - systemd (bug #1003467) [bullseye] - systemd (Minor issue; can be fixed via point release) [buster] - systemd (Minor issue; not exploitable before upstream commit e535840) NOTE: https://bugzilla.redhat.com/show_bug.cgi?id=2024639 NOTE: https://github.com/systemd/systemd/pull/22070 NOTE: https://www.openwall.com/lists/oss-security/2022/01/10/2 NOTE: Exploitable after (but present before): https://github.com/systemd/systemd/commit/e5358401b5df8d395e99815b7a69b8424887472c (v242-rc1) NOTE: Prerequisite/Preparation: https://github.com/systemd/systemd/commit/3bac86abfa1b1720180840ffb9d06b3d54841c11 NOTE: Prerequisite/Preparation: https://github.com/systemd/systemd/commit/84ced330020c0bae57bd4628f1f44eec91304e69 NOTE: Fixed by: https://github.com/systemd/systemd/commit/5b1cf7a9be37e20133c0208005274ce4a5b5c6a1 > I might need to brush up on my Perl and make a patch for lintian which > downloads the sec tracker JSON and checks the CVE list in the .changes > file - warnings from lintian are more likely to get fixed prior to > upload. Depends if you think this happens sufficiently often that it is > a problem worth solving. (Considering how long it's been since I did > that amount of code in Perl, maybe I'm better filing the bug against > lintian and seeing if someone else can come up with a patch... - again, > only if it happens sufficiently often.) I'm not
Bug#1001451: Candidate script updates
Hi Salvatore, On Mon, 10 Jan 2022 17:10:15 +0100 Salvatore Bonaccorso wrote: > > So for that epiphany tracker, there is a typo in the d.changelog - > > the automated link for CVE-2021-4508 is a 404. > > > > I've updated the script to catch this and report the error. From the > > security-tracker source-package page for epiphany, it looks like the > > d.changelog entry should be CVE-2021-45088 - a simple typo to omit > > the final repeated digit. > > > > Currently, I'm handling this by advising that the script is re-run > > using the offline support and a corrected list of CVE IDs. > > > > This also adds a --force-version option to the offline support, in > > case sid has moved ahead of the fixed version by the time the CVE > > list is updated. > > Ack, this works indeed in this case as the CVE-2021-4508 does not > exist (yet). But in other cases typos are e.g. just wapping a number > or misstype the year, or other typos, which lead to an existing CVE. > So basically this all really boils down to, people working on > security-tracker trying as much as possible, in the human limits :), > to do a diligent work as possible. The script also prints out the CVE description and then tries to identify any existing package link by checking the PackageAnnotation of each CVE. It prints a warning if the source package of the CVE from the changes input doesn't match the source package of the changes itself or the source package(s) of other CVEs in the list. It can't always be an error as the situation with embedded copies & removed|reintroduced packages makes it hard for the script. However, it should be obvious from the output if a typo has been made. I've also added output if a typo matches a CVE that is an NFU as there are lot more of those in data/CVE/list e.g. $ cat test.changes| ./bin/grab-cve-in-fix --input grab-cve-in-fix - INFO - Retrieving data STDIN ... grab-cve-in-fix - INFO - CVE-2021-45085: (XSS can occur in GNOME Web (aka Epiphany) before 40.4 and 41.x before ...) grab-cve-in-fix - INFO - CVE-2021-45086: (XSS can occur in GNOME Web (aka Epiphany) before 40.4 and 41.x before ...) grab-cve-in-fix - INFO - CVE-2021-45087: (XSS can occur in GNOME Web (aka Epiphany) before 40.4 and 41.x before ...) grab-cve-in-fix - INFO - CVE-2021-3757: (immer is vulnerable to Improperly Controlled Modification of Object Pr ...) grab-cve-in-fix - ERROR - CVE CVE-2021-3757 is not attributed to a Debian package: ['NOT-FOR-US', 'NOTE'] (test.changes in this case was edited to change the typo to CVE-2021-3757 to demonstrate the output). I might need to brush up on my Perl and make a patch for lintian which downloads the sec tracker JSON and checks the CVE list in the .changes file - warnings from lintian are more likely to get fixed prior to upload. Depends if you think this happens sufficiently often that it is a problem worth solving. (Considering how long it's been since I did that amount of code in Perl, maybe I'm better filing the bug against lintian and seeing if someone else can come up with a patch... - again, only if it happens sufficiently often.) > But if you spot an easy or obvious to detect case which the script can > warn of, then this can be helpful! After all you are trying to write > up scripts which should help the workflow. > > > It's hard to see how to best test these scripts properly on an > > ongoing basis. The "correct" output of each script depends entirely > > on the current state of data/CVE/list and the setup_paths support > > gets in the way of using a static CVE list in a test directory. > > > > I yet may push all of the non-argparse code into a > > lib.python.sectracker module to at least have some way to test > > individual functions with carefully mangled static copies of > > data/CVE/list content. > > In general I would find it perfect if we can have a sort of testsuite > or CI for the code part indeed of the security-tracker. In fact the > situation now without any tests has potential for many breackages when > one want to fix something, as the code has grown over decades almost > :). If you have a solution for that, and if splittiong out code into > libs can help, then that would be a welcome improvmenet. OK, I had looked at the lib/python/sectracker_test directory but the tests failed after trying to find lenny apt sources & then failed again when I changed that to something current I'll work on CI that can run in Salsa & hopefully will be easier to keep up to date. > (Note: I expect that the security-tracker will need some refactoring > in the 2022 year, as MITRE will change e.g. how data is exported and > which formats are supported, in particular this will affect our > automatic update scripts currently relying on downloading the > allitmes.html file, where it's planned now that this will change and > there will be only a JSON export of the data, we will need to adapt > the security-tracker on that regard, and possibly you will be > interested here :)) > > >
Bug#1001451: Candidate script updates
Hi Neil, On Thu, Jan 06, 2022 at 02:13:38PM +, Neil Williams wrote: > Hi Salvatore, > > On Mon, 20 Dec 2021 12:30:36 +0100 Salvatore Bonaccorso > wrote: > > Hi Neil, > > > > Btw, if you want "real" cases why not to do automatic merges: > > https://tracker.debian.org/news/1287752/accepted-epiphany-browser-412-1-source-into-unstable/ > > typo in CVE, and > > https://tracker.debian.org/news/1287534/accepted-bluez-562-1-source-into-unstable/ > > where one CVE was actually not really fixed. > > So for that epiphany tracker, there is a typo in the d.changelog - > the automated link for CVE-2021-4508 is a 404. > > I've updated the script to catch this and report the error. From the > security-tracker source-package page for epiphany, it looks like the > d.changelog entry should be CVE-2021-45088 - a simple typo to omit the > final repeated digit. > > Currently, I'm handling this by advising that the script is re-run > using the offline support and a corrected list of CVE IDs. > > This also adds a --force-version option to the offline support, in case > sid has moved ahead of the fixed version by the time the CVE list is > updated. Ack, this works indeed in this case as the CVE-2021-4508 does not exist (yet). But in other cases typos are e.g. just wapping a number or misstype the year, or other typos, which lead to an existing CVE. So basically this all really boils down to, people working on security-tracker trying as much as possible, in the human limits :), to do a diligent work as possible. But if you spot an easy or obvious to detect case which the script can warn of, then this can be helpful! After all you are trying to write up scripts which should help the workflow. > Should the script also advise that a (normal|minor) bug is filed > against the source-package to fix the d.changelog (retrospectively or > in the next changelog entry?) or is it sufficient that the > security-tracker has the correct information? it is suficient thath the tracker will have correct information, IMHO. Basically there are as well different policies in teams on "touching old changelog entries". For instance for the kernel-team, now switching hat, we almost never touch an old entry. What has been written, has been written (we e.g. do not add retrospecively for instance a CVE id, or change a CVE id which was rejected, and assigned a new CVE). > For the bluez case, the 5.62-1 upload would update both CVEs. My > expectation is that finding out that CVE-2021-41229 was not fixed would > result in a separate change to data/CVE/list to remove the fixed > version of 5.62-1. Then when 5.62-2 as uploaded, CVE-2021-41229 would > be marked as fixed in 5.62-2 as normal. In case a separate change is > not made to data/CVE/list, I've adjusted the script so that if the fix > version is older than the version from the retrieved changes entry, the > CVE is updated to show the newer version. i.e. 5.62-1 would be changed > to 5.62-2. This introduces a dependency on python3-apt to be able to > get the correct version comparison support, hopefully that is OK. > (python3-debian doesn't have dpkg --compare-versions type support.) Hmm not sure. In if we can say that in general. In the particular bluez case I spotted that while reviewing the entry, because I had in the back of my mind still some information about the triage of the bluez issues, knowing that one was fixed in a new upstream version and the other not yet, which puzzled me then that both were mentioned, and so i double checked the source again, confirming that no cherry-picked patch was applied. So probably yes, if I would not have had this information then *probably* i would have initially marked wrongly both CVEs as fixed in 5.62-1. But again this boils down in people doing this review work to be as carefull as possible, still acknowledging we do a lot of mistakes which hopefully can be spotted by peer review of commits (at least jmm/Moritz reviews most of the commits done by another team member, so we have some backup of each other). What does that mean for the above: having that support to adjust the version would be ok, but i expect here that hopefully the 5.62-1 wrong marking would live in the tracker only for a very short time after a peer review has spotted a possible mistake. > It's hard to see how to best test these scripts properly on an ongoing > basis. The "correct" output of each script depends entirely on the > current state of data/CVE/list and the setup_paths support gets in the > way of using a static CVE list in a test directory. > > I yet may push all of the non-argparse code into a lib.python.sectracker > module to at least have some way to test individual functions with > carefully mangled static copies of data/CVE/list content. In general I would find it perfect if we can have a sort of testsuite or CI for the code part indeed of the security-tracker. In fact the situation now without any tests has potential for many breackages when one want to fix
Bug#1001451: Candidate script updates
Hi Salvatore, On Mon, 20 Dec 2021 12:30:36 +0100 Salvatore Bonaccorso wrote: > Hi Neil, > > Btw, if you want "real" cases why not to do automatic merges: > https://tracker.debian.org/news/1287752/accepted-epiphany-browser-412-1-source-into-unstable/ > typo in CVE, and > https://tracker.debian.org/news/1287534/accepted-bluez-562-1-source-into-unstable/ > where one CVE was actually not really fixed. So for that epiphany tracker, there is a typo in the d.changelog - the automated link for CVE-2021-4508 is a 404. I've updated the script to catch this and report the error. From the security-tracker source-package page for epiphany, it looks like the d.changelog entry should be CVE-2021-45088 - a simple typo to omit the final repeated digit. Currently, I'm handling this by advising that the script is re-run using the offline support and a corrected list of CVE IDs. This also adds a --force-version option to the offline support, in case sid has moved ahead of the fixed version by the time the CVE list is updated. Should the script also advise that a (normal|minor) bug is filed against the source-package to fix the d.changelog (retrospectively or in the next changelog entry?) or is it sufficient that the security-tracker has the correct information? For the bluez case, the 5.62-1 upload would update both CVEs. My expectation is that finding out that CVE-2021-41229 was not fixed would result in a separate change to data/CVE/list to remove the fixed version of 5.62-1. Then when 5.62-2 as uploaded, CVE-2021-41229 would be marked as fixed in 5.62-2 as normal. In case a separate change is not made to data/CVE/list, I've adjusted the script so that if the fix version is older than the version from the retrieved changes entry, the CVE is updated to show the newer version. i.e. 5.62-1 would be changed to 5.62-2. This introduces a dependency on python3-apt to be able to get the correct version comparison support, hopefully that is OK. (python3-debian doesn't have dpkg --compare-versions type support.) It's hard to see how to best test these scripts properly on an ongoing basis. The "correct" output of each script depends entirely on the current state of data/CVE/list and the setup_paths support gets in the way of using a static CVE list in a test directory. I yet may push all of the non-argparse code into a lib.python.sectracker module to at least have some way to test individual functions with carefully mangled static copies of data/CVE/list content. For now, I'll mirror the real changes in data/CVE/list, trying to use the scripts to make some of the same changes. (Not all operations on data/CVE/list are to be covered.) Should bin/merge-cve-files also clean up the .xpck files that are created? On success only? https://salsa.debian.org/codehelp/security-tracker/-/tree/grabcvefix https://salsa.debian.org/codehelp/security-tracker/-/blob/grabcvefix/bin/grab-cve-in-fix https://salsa.debian.org/codehelp/security-tracker/-/raw/grabcvefix/bin/grab-cve-in-fix https://salsa.debian.org/codehelp/security-tracker/-/blob/grabcvefix/bin/merge-cve-files -- Neil Williams = https://linux.codehelp.co.uk/ pgpbxNC8aquUR.pgp Description: OpenPGP digital signature
