From: Paul Barker <[email protected]> When moving the updated CVE database file to the downloads directory, ensure that it has a different inode number to the previous version of this file.
We have seen "sqlite3.DatabaseError: database disk image is malformed" exceptions on our autobuilder when trying to read the CVE database in do_cve_check tasks. The context here is that the downloads directory (where the updated database file is copied to) is shared between workers as an NFS mount. Different autobuilder workers were seeing different checksums for the database file, which indicates that a mix of both new and stale data was being read. Forcing each new version of the database file to have a different inode number will prevent stale data from being read from local caches. This should fix [YOCTO #16086]. Signed-off-by: Paul Barker <[email protected]> Signed-off-by: Richard Purdie <[email protected]> (cherry picked from commit f63622bbec1cfaca6d0b3e05e11466e4c10fa86e) [YC: backported to also fix [YOCTO #15660] ] Signed-off-by: Yoann Congal <[email protected]> --- meta/recipes-core/meta/cve-update-db-native.bb | 9 +++++++-- meta/recipes-core/meta/cve-update-nvd2-native.bb | 9 +++++++-- 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-db-native.bb b/meta/recipes-core/meta/cve-update-db-native.bb index e042e67b09a..d6a3cf0dd66 100644 --- a/meta/recipes-core/meta/cve-update-db-native.bb +++ b/meta/recipes-core/meta/cve-update-db-native.bb @@ -64,8 +64,13 @@ python do_fetch() { shutil.copy2(db_file, db_tmp_file) if update_db_file(db_tmp_file, d) == True: - # Update downloaded correctly, can swap files - shutil.move(db_tmp_file, db_file) + # Update downloaded correctly, we can swap files. To avoid potential + # NFS caching issues, ensure that the destination file has a new inode + # number. We do this in two steps as the downloads directory may be on + # a different filesystem to tmpdir we're working in. + new_file = "%s.new" % (db_file) + shutil.move(db_tmp_file, new_file) + os.rename(new_file, db_file) else: # Update failed, do not modify the database bb.note("CVE database update failed") diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index d50d9a2ceaf..f9d265479d9 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -83,8 +83,13 @@ python do_fetch() { shutil.copy2(db_file, db_tmp_file) if update_db_file(db_tmp_file, d, database_time) == True: - # Update downloaded correctly, can swap files - shutil.move(db_tmp_file, db_file) + # Update downloaded correctly, we can swap files. To avoid potential + # NFS caching issues, ensure that the destination file has a new inode + # number. We do this in two steps as the downloads directory may be on + # a different filesystem to tmpdir we're working in. + new_file = "%s.new" % (db_file) + shutil.move(db_tmp_file, new_file) + os.rename(new_file, db_file) else: # Update failed, do not modify the database bb.warn("CVE database update failed")
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#242386): https://lists.openembedded.org/g/openembedded-core/message/242386 Mute This Topic: https://lists.openembedded.org/mt/120527633/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
