Re: [OE-core] [PATCH 2/2] cve-update-db-native: Remove hash column from database.

2019-07-18 Thread Pierre Le Magourou
eu. 18 juil. 2019 à 15:10, Burton, Ross a écrit : > > > Ross > > On Thu, 18 Jul 2019 at 13:41, Pierre Le Magourou wrote: > > > > From: Pierre Le Magourou > > > > djb2 hash algorithm was found to do collisions, so the database was > > somet

[OE-core] [PATCH 1/2] cve-check: Replace CVE_CHECK_CVE_WHITELIST by CVE_CHECK_WHITELIST

2019-07-18 Thread Pierre Le Magourou
From: Pierre Le Magourou CVE_CHECK_WHITELIST does not contain version anymore, as it was not used. This variable should be set per recipe. Signed-off-by: Pierre Le Magourou --- meta/classes/cve-check.bbclass | 22 +++--- 1 file changed, 11 insertions(+), 11 deletions(-) diff

[OE-core] [PATCH 2/2] cve-update-db-native: Remove hash column from database.

2019-07-18 Thread Pierre Le Magourou
From: Pierre Le Magourou djb2 hash algorithm was found to do collisions, so the database was sometime missing data. Remove this hash mechanism, clear and populate elements from scratch in PRODUCTS table if the current year needs an update. Signed-off-by: Pierre Le Magourou --- meta/classes

Re: [OE-core] [PATCH 1/4] cve-update-db: New recipe to update CVE database

2019-07-10 Thread Pierre Le Magourou
also drop hash function, remove everything from the database and recreate all entries at each update but it will increase database update time. I don't have the same hash as you for CVE-2018-1000873 and CVE-2018-18338, do you use my latest patches from master ? I did several changes recently. Pie

Re: [OE-core] [PATCH] cve-update-db: Skip recipe when cve-check class is not loaded.

2019-07-08 Thread Pierre Le Magourou
> > Great, the "bitbake universe --runall=fetch" works now. > I just forgot to rebase this patch on the previous ones that are in master-next, I sent a rebased v2. Pierre -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org

[OE-core] [PATCH v2] cve-update-db-native: Skip recipe when cve-check class is not loaded.

2019-07-08 Thread Pierre Le Magourou
From: Pierre Le Magourou Signed-off-by: Pierre Le Magourou --- meta/recipes-core/meta/cve-update-db-native.bb | 5 + 1 file changed, 5 insertions(+) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index d658c7bfea..e16c41a72f

Re: [OE-core] [meta-oe][PATCH v5] cve-update-db: do_populate_cve_db depends on do_fetch

2019-07-08 Thread Pierre Le Magourou
Hello, > Did you try > > $ bitbake universe --runall=fetch > > Please? It still doesn't work on latest master branch. > I sent a patch to skip cve-update-db recipe if the cve-check class is not loaded. This should fix the problem. Pierre -- ___

[OE-core] [PATCH] cve-update-db: Skip recipe when cve-check class is not loaded.

2019-07-08 Thread Pierre Le Magourou
From: Pierre Le Magourou Signed-off-by: Pierre Le Magourou --- meta/recipes-core/meta/cve-update-db.bb | 5 + 1 file changed, 5 insertions(+) diff --git a/meta/recipes-core/meta/cve-update-db.bb b/meta/recipes-core/meta/cve-update-db.bb index ae8f1a958b..cfeee91c2e 100644 --- a/meta

[OE-core] [PATCH v2 2/3] cve-update-db: Use NVD CPE data to populate PRODUCTS table

2019-07-05 Thread Pierre Le Magourou
From: Pierre Le Magourou Instead of using expanded list of affected versions that is not reliable, use the 'cpe_match' node in the 'configurations' json node. For cve-check to correctly match affected CVE, the sqlite database need to contain operator_start, operator_end and the corresponding

[OE-core] [PATCH v2 1/3] cve-check: Depends on cve-update-db-native

2019-07-05 Thread Pierre Le Magourou
From: Pierre Le Magourou do_populate_cve_db is a native task. Signed-off-by: Pierre Le Magourou --- meta/classes/cve-check.bbclass | 2 +- meta/recipes-core/meta/{cve-update-db.bb => cve-update-db-native.bb} | 0 2 files changed, 1 insertion(+)

[OE-core] [PATCH v2 3/3] cve-check: Update unpatched CVE matching

2019-07-05 Thread Pierre Le Magourou
From: Pierre Le Magourou Now that cve-update-db added CPE information to NVD database. We can check for unpatched versions with operators '<', '<=', '>', and '>='. Signed-off-by: Pierre Le Magourou --- meta/classes/cve-check.bbclass | 54 +++---

Re: [OE-core] [meta-oe][PATCH 1/3] cve-check: Depends on cve-update-db-native

2019-07-05 Thread Pierre Le Magourou
Hello, > > From: Pierre Le Magourou > > > > do_populate_cve_db is a native task. > > > > Signed-off-by: Pierre Le Magourou < > > pierre.lemagou...@softbankrobotics.com> > > --- > > meta/classes/cve-check.bbclass | 2 +- > >

[OE-core] [meta-oe][PATCH 1/3] cve-check: Depends on cve-update-db-native

2019-07-04 Thread Pierre Le Magourou
From: Pierre Le Magourou do_populate_cve_db is a native task. Signed-off-by: Pierre Le Magourou --- meta/classes/cve-check.bbclass | 2 +- meta/recipes-core/meta/cve-update-db.bb | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/meta/classes/cve-check.bbclass b

[OE-core] [meta-oe][PATCH 2/3] cve-update-db: Use NVD CPE data to populate PRODUCTS table

2019-07-04 Thread Pierre Le Magourou
From: Pierre Le Magourou Instead of using expanded list of affected versions that is not reliable, use the 'cpe_match' node in the 'configurations' json node. For cve-check to correctly match affected CVE, the sqlite database need to contain operator_start, operator_end and the corresponding

[OE-core] [meta-oe][PATCH 3/3] cve-check: Update unpatched CVE matching

2019-07-04 Thread Pierre Le Magourou
From: Pierre Le Magourou Now that cve-update-db added CPE information to NVD database. We can check for unpatched versions with operators '<', '<=', '>', and '>='. Signed-off-by: Pierre Le Magourou --- meta/classes/cve-check.bbclass | 54 +++---

Re: [OE-core] [meta-oe][PATCH v5] cve-update-db: do_populate_cve_db depends on do_fetch

2019-07-03 Thread Pierre Le Magourou
Hello, > This patch broke "bitbake universe --runall=fetch" totally: > > The line which caused the problem is: > addtask do_populate_cve_db before do_fetch > > Would you please fix it? Or I can help if needed. > This seems related to the NVD website down problem. I sent a patch to manage

[OE-core] [meta-oe][PATCH] cve-update-db: Catch request.urlopen errors.

2019-07-03 Thread Pierre Le Magourou
From: Pierre Le Magourou If the NVD url is not accessible, print a warning on top of the CVE report, and continue. The database will not be fully updated, but cve_check can still run on the previous database. Signed-off-by: Pierre Le Magourou --- meta/classes/cve-check.bbclass | 5

Re: [OE-core] [PATCH] cve-update-db: Use std library instead of urllib3

2019-07-02 Thread Pierre Le Magourou
Hi, > > On 07/02/2019 03:39 PM, ChenQi wrote: > > > A recent do_populate_cve_db failure: > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/23/builds/1005/steps/7/logs/errors > > > > > > > > > Could you please help look at what's going on? > On Tue, 2019-07-02 at 16:08 +0800, ChenQi

[OE-core] [meta-oe][PATCH v5] cve-update-db: do_populate_cve_db depends on do_fetch

2019-06-27 Thread Pierre Le Magourou
From: Pierre Le Magourou To be able to populate NVD database on a fetchall (bitbake --run-all=fetch), set the do_populate_cve_db task to be executed before do_fetch. Do not get CVE_CHECK_DB_DIR, CVE_CHECK_DB_FILE and CVE_CHECK_TMP_FILE variable because do_populate_cve_db can be called

[OE-core] [meta-oe][PATCH v4] cve-update-db: do_populate_cve_db depends on do_fetch

2019-06-27 Thread Pierre Le Magourou
From: Pierre Le Magourou To be able to populate NVD database on a fetchall (bitbake --run-all=fetch), set the do_populate_cve_db task to be executed before do_fetch. Do not get CVE_CHECK_DB_DIR, CVE_CHECK_DB_FILE and CVE_CHECK_TMP_FILE variable because do_populate_cve_db can be called

[OE-core] [meta-oe][PATCH v3] cve-update-db: do_populate_cve_db depends on do_fetch

2019-06-27 Thread Pierre Le Magourou
From: Pierre Le Magourou To be able to populate NVD database on a fetchall (bitbake --run-all=fetch), set the do_populate_cve_db task to be executed before do_fetch. Do not get CVE_CHECK_DB_DIR, CVE_CHECK_DB_FILE and CVE_CHECK_TMP_FILE variable because do_populate_cve_db can be called

Re: [OE-core] [meta-oe][PATCH v2] cve-update-db: do_populate_cve_db depends on do_fetch

2019-06-27 Thread Pierre Le Magourou
> Thanks for the fast turnaround on the patch. This seems to move the > problem to: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/23/builds/990/steps/7/logs/step1b > > :( > > Perhaps we need to skip the recipe if the class isn't enabled? Is that > the reason its failing like this?

[OE-core] [meta-oe][PATCH v2] cve-update-db: do_populate_cve_db depends on do_fetch

2019-06-27 Thread Pierre Le Magourou
From: Pierre Le Magourou To be able to populate NVD database on a fetchall (bitbake --run-all=fetch), set the do_populate_cve_db task to be executed before do_fetch. Signed-off-by: Pierre Le Magourou --- meta/recipes-core/meta/cve-update-db.bb | 7 +-- 1 file changed, 5 insertions(+), 2

Re: [OE-core] [PATCH 1/4] cve-update-db: New recipe to update CVE database

2019-06-27 Thread Pierre Le Magourou
Hi, > It looks like CVE_CHECK_DB_DIR has no default value which resulted in: > > https://autobuilder.yoctoproject.org/typhoon/#/builders/23/builds/988/steps/7/logs/step1b > > We only started seeing that error after your later patch to add back > the do_fetch task. build-appliance is trying to

[OE-core] [PATCH] cve-update-db: Manage proxy if needed.

2019-06-26 Thread Pierre Le Magourou
From: Pierre Le Magourou If https_proxy environment variable is defined, manage proxy to be able to download meta and json data feeds from https://nvd.nist.gov Signed-off-by: Pierre Le Magourou --- meta/recipes-core/meta/cve-update-db.bb | 11 +-- 1 file changed, 9 insertions(+), 2

[OE-core] [PATCH] cve-update-db: do_populate_cve_db depends on do_fetch

2019-06-26 Thread Pierre Le Magourou
From: Pierre Le Magourou To be able to populate NVD database on a fetchall (bitbake --run-all=fetch), set the do_populate_cve_db task to be executed before do_fetch. Signed-off-by: Pierre Le Magourou --- meta/recipes-core/meta/cve-update-db.bb | 3 +-- 1 file changed, 1 insertion(+), 2

Re: [OE-core] [PATCH 1/4] cve-update-db: New recipe to update CVE database

2019-06-25 Thread Pierre Le Magourou
Hi, > Also, the CVE db is updated using this custom task without link to > do_fetch, which means a fetchall task would not update the database for > off line NO_NETWORK builds. > > Could the task be added as dependency to do_fetch() or are there some other > side effects? > Yes I can do that, I

[OE-core] [PATCH] cve-update-db: Use std library instead of urllib3

2019-06-24 Thread Pierre Le Magourou
From: Pierre Le Magourou urllib3 was used in this recipe but it was not set as a dependency. As it is not specifically needed, rewrite the recipe with urllib from the standard library. Signed-off-by: Pierre Le Magourou --- meta/recipes-core/meta/cve-update-db.bb | 10 -- 1 file

Re: [OE-core] [PATCH 1/4] cve-update-db: New recipe to update CVE database

2019-06-24 Thread Pierre Le Magourou
Hi, > > This adds python3 urllib3 (python3-urllib3 in Debian) to build environment > > dependencies. It's the first user of urllib3 in poky, AFAIK. Maybe > > documentation could be updated too, e.g. > >

Re: [OE-core] [PATCH 1/4] cve-update-db: New recipe to update CVE database

2019-06-20 Thread Pierre Le Magourou
> Not sure which of the changes is responsible, but this is new: > WARNING: flex-native-2.6.0-r0 do_cve_check: Found unpatched CVE > (CVE-2015-1773) > > https://nvd.nist.gov/vuln/detail/CVE-2015-1773 > > Note that the flex tool is completely unrelated to Apache Flex. > > I see, the 4/4 patch is

Re: [OE-core] [PATCH 2/4] cve-check: Remove dependency to cve-check-tool-native

2019-06-19 Thread Pierre Le Magourou
> Does this mean we can delete the cve-check-tool recipe itself too? > Yes, with this patch cve-check class does not need cve-check-tool anymore. Pierre -- ___ Openembedded-core mailing list Openembedded-core@lists.openembedded.org

[OE-core] [PATCH 4/4] cve-check: Consider CVE that affects versions with less than operator

2019-06-19 Thread Pierre Le Magourou
From: Pierre Le Magourou In the NVD json CVE feed, affected versions can be strictly matched to a version, but they can also be matched with the operator '<='. Add a new condition in the sqlite query to match affected versions that are defined with the operator '<='. Then use LooseV

[OE-core] [PATCH 3/4] cve-check: Manage CVE_PRODUCT with more than one name

2019-06-19 Thread Pierre Le Magourou
From: Pierre Le Magourou In some rare cases (eg. curl recipe) the CVE_PRODUCT contains more than one name. Signed-off-by: Pierre Le Magourou --- meta/classes/cve-check.bbclass | 25 ++--- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/meta/classes/cve

[OE-core] [PATCH 1/4] cve-update-db: New recipe to update CVE database

2019-06-19 Thread Pierre Le Magourou
From: Pierre Le Magourou cve-check-tool-native do_populate_cve_db task was using deprecated NVD xml data feeds, cve-update-db uses NVD json data feeds. Sqlite database schema was updated to take into account CVSSv3 CVE scores and operator in affected product versions. A new META table was added

[OE-core] [PATCH 2/4] cve-check: Remove dependency to cve-check-tool-native

2019-06-19 Thread Pierre Le Magourou
From: Pierre Le Magourou Use the new update-cve-db recipe to update database. Signed-off-by: Pierre Le Magourou --- meta/classes/cve-check.bbclass | 71 -- 1 file changed, 26 insertions(+), 45 deletions(-) diff --git a/meta/classes/cve-check.bbclass b