From: Yoann Congal <yoann.con...@smile.fr>

Printed logs during the reproducible test look like this in case of
failure:
2025-01-12 00:39:00,807 - oe-selftest - INFO - Building reproducibleA (sstate 
allowed)...
2025-01-12 00:40:08,323 - oe-selftest - ERROR - Bitbake failed! but keep 
going... Log:
2025-01-12 00:40:08,323 - oe-selftest - INFO - Found ERROR (line 126):
2025-01-12 00:40:08,323 - oe-selftest - INFO -       NOTE: recipe 
opkg-utils-native-0.7.0-r0: task do_populate_sysroot_setscene: Succeeded
2025-01-12 00:40:08,323 - oe-selftest - INFO -       NOTE: Running setscene 
task 74 of 333 
(virtual:native:.../poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.7.0.bb:do_recipe_qa_setscene)
2025-01-12 00:40:08,323 - oe-selftest - INFO -       NOTE: recipe 
bash-5.2.37-r0: task do_populate_sysroot_setscene: Succeeded
2025-01-12 00:40:08,323 - oe-selftest - INFO -       NOTE: recipe 
cve-update-nvd2-native-1.0-r0: task do_unpack: Succeeded
2025-01-12 00:40:08,323 - oe-selftest - INFO -       NOTE: Running task 4 of 
974 (.../poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_cve_check)
2025-01-12 00:40:08,323 - oe-selftest - INFO -       NOTE: recipe 
bash-5.2.37-r0: task do_packagedata_setscene: Succeeded
2025-01-12 00:40:08,323 - oe-selftest - INFO -       NOTE: recipe 
hello-fail-1.0-r0: task do_packagedata_setscene: Started
2025-01-12 00:40:08,323 - oe-selftest - INFO -       NOTE: Running setscene 
task 77 of 333 
(.../poky/meta/recipes-extended/bash/bash_5.2.37.bb:do_recipe_qa_setscene)
2025-01-12 00:40:08,323 - oe-selftest - INFO -       NOTE: Running setscene 
task 78 of 333 
(.../poky/meta/recipes-core/ncurses/ncurses_6.5.bb:do_populate_sysroot_setscene)
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: recipe 
hello-fail-1.0-r0: task do_populate_sysroot_setscene: Started
2025-01-12 00:40:08,324 - oe-selftest - INFO -       ERROR: hello-fail-1.0-r0 
do_populate_sysroot_setscene: this is an error
2025-01-12 00:40:08,324 - oe-selftest - INFO -       ERROR: hello-fail-1.0-r0 
do_populate_sysroot_setscene: Error executing a python function in 
exec_func_python() autogenerated:
2025-01-12 00:40:08,324 - oe-selftest - INFO -
2025-01-12 00:40:08,324 - oe-selftest - INFO -       The stack trace of python 
calls that resulted in this exception/failure was:
2025-01-12 00:40:08,324 - oe-selftest - INFO -       File: 'exec_func_python() 
autogenerated', lineno: 2, function: <module>
2025-01-12 00:40:08,324 - oe-selftest - INFO -            0001:
2025-01-12 00:40:08,324 - oe-selftest - INFO -        *** 
0002:do_populate_sysroot_setscene(d)
2025-01-12 00:40:08,324 - oe-selftest - INFO -            0003:
2025-01-12 00:40:08,324 - oe-selftest - INFO -       File: 
'.../poky/meta/recipes-core/hello-single/hello-fail_1.0.bb', lineno: 17, 
function: do_populate_sysroot_setscene
2025-01-12 00:40:08,324 - oe-selftest - INFO -            0013:}
2025-01-12 00:40:08,324 - oe-selftest - INFO - Found ERROR (line 127):
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: Running setscene 
task 74 of 333 
(virtual:native:.../poky/meta/recipes-devtools/opkg-utils/opkg-utils_0.7.0.bb:do_recipe_qa_setscene)
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: recipe 
bash-5.2.37-r0: task do_populate_sysroot_setscene: Succeeded
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: recipe 
cve-update-nvd2-native-1.0-r0: task do_unpack: Succeeded
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: Running task 4 of 
974 (.../poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_cve_check)
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: recipe 
bash-5.2.37-r0: task do_packagedata_setscene: Succeeded
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: recipe 
hello-fail-1.0-r0: task do_packagedata_setscene: Started
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: Running setscene 
task 77 of 333 
(.../poky/meta/recipes-extended/bash/bash_5.2.37.bb:do_recipe_qa_setscene)
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: Running setscene 
task 78 of 333 
(.../poky/meta/recipes-core/ncurses/ncurses_6.5.bb:do_populate_sysroot_setscene)
2025-01-12 00:40:08,324 - oe-selftest - INFO -       NOTE: recipe 
hello-fail-1.0-r0: task do_populate_sysroot_setscene: Started
2025-01-12 00:40:08,324 - oe-selftest - INFO -       ERROR: hello-fail-1.0-r0 
do_populate_sysroot_setscene: this is an error
2025-01-12 00:40:08,324 - oe-selftest - INFO -       ERROR: hello-fail-1.0-r0 
do_populate_sysroot_setscene: Error executing a python function in 
exec_func_python() autogenerated:
2025-01-12 00:40:08,325 - oe-selftest - INFO -
2025-01-12 00:40:08,325 - oe-selftest - INFO -       The stack trace of python 
calls that resulted in this exception/failure was:
2025-01-12 00:40:08,325 - oe-selftest - INFO -       File: 'exec_func_python() 
autogenerated', lineno: 2, function: <module>
2025-01-12 00:40:08,325 - oe-selftest - INFO -            0001:
2025-01-12 00:40:08,325 - oe-selftest - INFO -        *** 
0002:do_populate_sysroot_setscene(d)
2025-01-12 00:40:08,325 - oe-selftest - INFO -            0003:
2025-01-12 00:40:08,325 - oe-selftest - INFO -       File: 
'.../poky/meta/recipes-core/hello-single/hello-fail_1.0.bb', lineno: 17, 
function: do_populate_sysroot_setscene
2025-01-12 00:40:08,325 - oe-selftest - INFO -            0013:}
2025-01-12 00:40:08,325 - oe-selftest - INFO -            0014:
2025-01-12 00:40:08,325 - oe-selftest - INFO - End of log:
2025-01-12 00:40:08,325 - oe-selftest - INFO -       NOTE: Running task 955 of 
974 
(.../poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_populate_sysroot)
2025-01-12 00:40:08,325 - oe-selftest - INFO -       NOTE: recipe 
hello-fail-1.0-r0: task do_populate_sysroot: Started
2025-01-12 00:40:08,325 - oe-selftest - INFO -       NOTE: recipe 
hello-fail-1.0-r0: task do_populate_sysroot: Succeeded
2025-01-12 00:40:08,325 - oe-selftest - INFO -       NOTE: Running noexec task 
973 of 974 (.../poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_build)
2025-01-12 00:40:08,325 - oe-selftest - INFO -       NOTE: Running noexec task 
974 of 974 
(.../poky/meta/recipes-core/hello-single/hello-norepro_1.0.bb:do_build)
2025-01-12 00:40:08,325 - oe-selftest - INFO -       NOTE: Tasks Summary: 
Attempted 974 tasks of which 958 didn't need to be rerun and all succeeded.
2025-01-12 00:40:08,325 - oe-selftest - INFO -       NOTE: Generating JSON CVE 
summary
2025-01-12 00:40:08,325 - oe-selftest - INFO -       Complete CVE JSON report 
summary created at: 
.../poky/build-master-st/reproducibleA/tmp/log/cve/cve-summary.json
2025-01-12 00:40:08,325 - oe-selftest - INFO -
2025-01-12 00:40:08,325 - oe-selftest - INFO -       Summary: There were 2 
ERROR messages, returning a non-zero exit code.
2025-01-12 00:40:08,326 - oe-selftest - ERROR - reproducibleA build failed. 
Trying to compute built packages differences but the test will fail.

Fixes [YOCTO #15655] (second half).

Signed-off-by: Yoann Congal <yoann.con...@smile.fr>
---
RFC: As you can see, some lines are displayed multiple times if they are
in multiple errors contexts. I started implementing merging contexts
blocks but the code I wrote is really not good. If someone has a clever
idea on how to do this, I'll take it.
In the meantime, I think this simpler patch can be merged and the
print_condensed_error_log function improved later if that is worth it.
---
 meta/lib/oeqa/selftest/cases/reproducible.py | 25 ++++++++++++++------
 1 file changed, 18 insertions(+), 7 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py 
b/meta/lib/oeqa/selftest/cases/reproducible.py
index 9597d76697..3fb0e5ee5b 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -239,18 +239,31 @@ class ReproducibleTests(OESelftestTestCase):
         # may reuse the previous log file so restart the bitbake server.
         bitbake("--kill-server")
 
+        def print_condensed_error_log(logs, context_lines=10, tail_lines=20):
+            """Prints errors with context and the end of the log."""
+
+            logs = logs.split("\n")
+            for i, line in enumerate(logs):
+                if line.startswith("ERROR"):
+                    self.logger.info("Found ERROR (line %d):" % (i + 1))
+                    for l in logs[i-context_lines:i+context_lines]:
+                        self.logger.info("      " + l)
+
+            self.logger.info("End of log:")
+            for l in logs[-tail_lines:]:
+                self.logger.info("      " + l)
+
         bitbake_failure_count = 0
         if not use_sstate:
             if self.sstate_targets:
                self.logger.info("Building prebuild for %s (sstate allowed)..." 
% (name))
                self.write_config(config)
                try:
-                   bitbake("--continue "+' '.join(self.sstate_targets), 
limit_exc_output=20)
+                   bitbake("--continue "+' '.join(self.sstate_targets))
                except AssertionError as e:
                    bitbake_failure_count += 1
                    self.logger.error("Bitbake failed! but keep going... Log:")
-                   for line in str(e).split("\n"):
-                       self.logger.info("    "+line)
+                   print_condensed_error_log(str(e))
 
             # This config fragment will disable using shared and the sstate
             # mirror, forcing a complete build from scratch
@@ -264,13 +277,11 @@ class ReproducibleTests(OESelftestTestCase):
         d = get_bb_vars(capture_vars)
         # targets used to be called images
         try:
-            bitbake("--continue "+' '.join(getattr(self, 'images', 
self.targets)), limit_exc_output=20)
+            bitbake("--continue "+' '.join(getattr(self, 'images', 
self.targets)))
         except AssertionError as e:
             bitbake_failure_count += 1
-
             self.logger.error("Bitbake failed! but keep going... Log:")
-            for line in str(e).split("\n"):
-                self.logger.info("    "+line)
+            print_condensed_error_log(str(e))
 
             # The calling function expects the existence of the deploy
             # directories containing the packages.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#209675): 
https://lists.openembedded.org/g/openembedded-core/message/209675
Mute This Topic: https://lists.openembedded.org/mt/110562801/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Reply via email to