Instead of iterating through a file by hand to hash it, call into the
hashing code in bitbake's utils.py. This uses mmap() instead of plain
file operations which is ~30% faster in benchmarks[1], and will speed
up hash calculations when writing sstate.
[1] On my build machine, hashing a 300MB kernel image 100 times takes
34s with open() and 26s with mmap()
Signed-off-by: Ross Burton <[email protected]>
---
meta/lib/oe/sstatesig.py | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index af6df60271d..f016fc67704 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -684,10 +684,10 @@ def OEOuthashBasic(path, sigfile, task, d):
update_hash(" " * 10)
update_hash(" ")
- fh = hashlib.sha256()
if stat.S_ISREG(s.st_mode):
# Hash file contents
if filterfile:
+ fh = hashlib.sha256()
# Need to ignore paths in crossscripts and
postinst-useradd files.
with open(path, 'rb') as d:
chunk = d.read()
@@ -701,13 +701,13 @@ def OEOuthashBasic(path, sigfile, task, d):
else:
chunk = chunk.replace(bytes(r,
encoding='utf8'), b'')
fh.update(chunk)
+ update_hash(fh.hexdigest())
else:
- with open(path, 'rb') as d:
- for chunk in iter(lambda: d.read(4096), b""):
- fh.update(chunk)
- update_hash(fh.hexdigest())
+ # Plain file that we're not filtering, use the
fastpath in bb.utils
+ update_hash(bb.utils.sha256_file(path))
else:
- update_hash(" " * len(fh.hexdigest()))
+ # SHA256 has a 64 character hex digest
+ update_hash(" " * 64)
update_hash(" %s" % path)
--
2.43.0
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#239632):
https://lists.openembedded.org/g/openembedded-core/message/239632
Mute This Topic: https://lists.openembedded.org/mt/119989585/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
