Hello Alex, I had this failure on the oe-selftest-armhost builder:
https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/2290/steps/14/logs/stdio Task texinfo-dummy-native:do_configure couldn't be used from the cache because: We need hash 76c87e43650c854836e1c9a95db39c16de49bf0271535a4da3f030056fcc3fb2, most recent matching task was 8e7ba727bc84dd547f1a52515f25189fddcbcb01634b52521741e389cae9103e Hash for task dependency texinfo-dummy-native:do_patch changed from 19ba8d4ae1894b1725dd9b7e90afb71139de2d72c73208c1bea4094fe1ce259d to 11fdb7aa5419cb36b11f0fa281f6e6ca5dcbcc2da515c4d1782b7c0b61ccc447 Hash for task dependency texinfo-dummy-native:do_unpack changed from 8864a86b4edb8cf635c7b597596fb4bab900afa8648d486f7dfb00c9c8ce03c3 to 8165664451ef563a6c2c84a93abfa4cdf4d617ce1afe55a850fb71cf67b81a01 basehash changed from 935b6ffebc04d72ae1d16476f82aee22c18c7aaeee340f1ab34d571c3e121147 to 1367421b45cdb92b9994acadd6503c4a0874535e1d175e9c39001998a1c8b862 Variable ERROR_QA value changed: @@ -1,3 +1,2 @@ dev-so debug-deps dev-deps debug-files arch pkgconfig la perms dep-cmp pkgvarcheck perm-config perm-line perm-link split-strip packages-list pkgv-undefined var-undefined version-going-backwards expanded-d invalid-chars license-checksum dev-elf file-rdeps configure-unsafe configure-gettext perllocalpod shebang-size already-stripped installed-vs-shipped ldflags compile-host-path install-host-path pn-overrides unknown-configure-option useless-rpaths rpaths staticdev empty-dirs patch-fuzz ${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', ' usrmerge', '', d)} patch-status DISTRO_FEATURES{usrmerge} = Unset -_remove of packages-list ERROR: Can't find a task we're supposed to have written out? (hash: e79d70b9c2cc72030c1ce822525510699a1eeb1ddf5986271d3217422244366a)? Task gettext-minimal-native:do_configure couldn't be used from the cache because: We need hash ee11593c797a4579bfb48c61940f50ebd2c6da4883ca265d3231352fd5e248c6, most recent matching task was 140c1b528103db5f34ecd8db962b1ccc715a9da95573b9cb9d0441d7c604317c basehash changed from ade121cb5b2caa1f610daf254c4d19d28959630e468700c6a5dee6fc44ee8869 to 5190156579dda094eafc1f1df780def5d749972885a93ed7d952833d286f8177 Variable base_do_configure value changed: @@ -13,3 +13,4 @@ mkdir -p `dirname ${CONFIGURESTAMPFILE}` echo ${BB_TASKHASH} > ${CONFIGURESTAMPFILE} fi + echo "this changes base_do_configure() definiton" Task perlcross-native:do_configure couldn't be used from the cache because: We need hash f557deda75fdaf738574f41d517b54338c609677a9a0dbb1774d811714f3b7e1, most recent matching task was 7838e1dc635c14048dee89663af3378861d33e779edb609e7a4c5f00fc5af70c Task dependencies changed from: ['AR', 'AS', 'B', 'BUILD_AR', 'BUILD_AS', 'BUILD_AS_ARCH', 'BUILD_CC', 'BUILD_CCLD', 'BUILD_CC_ARCH', 'BUILD_CFLAGS', 'BUILD_CPP', 'BUILD_CPPFLAGS', 'BUILD_CXX', 'BUILD_CXXFLAGS', 'BUILD_FC', 'BUILD_LD', 'BUILD_LDFLAGS', 'BUILD_LD_ARCH', 'BUILD_NM', 'BUILD_OPTIMIZATION', 'BUILD_PREFIX', 'BUILD_RANLIB', 'BUILD_STRIP', 'CC', 'CCLD', 'CFLAGS', 'CONFIG_SITE', 'CPP', 'CPPFLAGS', 'CXX', 'CXXFLAGS', 'DEBUG_BUILD', 'DEPENDS', 'ERROR_QA', 'EXTRA_NATIVE_PKGCONFIG_PATH', 'EXTRA_OECONF', 'FC', 'HOST_PREFIX', 'INSANE_SKIP', 'LC_ALL', 'LD', 'LDFLAGS', 'MAKE', 'MLPREFIX', 'NATIVE_PACKAGE_PATH_SUFFIX', 'NM', 'OBJCOPY', 'OBJDUMP', 'P', 'PACKAGECONFIG', 'PERL_HASH_SEED', 'PKG_CONFIG_DIR', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'PKG_CONFIG_LIBDIR', 'PKG_CONFIG_PATH', 'PKG_CONFIG_SYSROOT_DIR', 'PN', 'PSEUDO_DISABLED', 'PV', 'PYTHONHASHSEED', 'QA_ERRORS_FOUND', 'QA_LOGFILE', 'RANLIB', 'READELF', 'RECIPE_SYSROOT_NATIVE', 'S', 'STAGING_BASE_LIBDIR_NATIVE', 'STAGING_BINDIR_NATIVE', 'STAGING_DATADIR', 'STAGING_DATADIR_NATIVE', 'STAGING_DIR_NATIVE', 'STAGING_ETCDIR_NATIVE', 'STAGING_INCDIR_NATIVE', 'STAGING_LIBDIR_NATIVE', 'STAGING_SBINDIR_NATIVE', 'STRINGS', 'STRIP', 'TZ', 'UNKNOWN_CONFIGURE_OPT_IGNORE', 'base_bindir', 'base_libdir', 'base_libdir_native', 'base_prefix', 'base_sbindir', 'bindir', 'bindir_native', 'datadir', 'datadir_native', 'do_qa_configure', 'docdir', 'exec_prefix', 'includedir', 'includedir_native', 'infodir', 'libdir', 'libdir_native', 'libexecdir', 'localstatedir', 'lt_cv_sys_lib_dlsearch_path_spec', 'mandir', 'nonarch_base_libdir', 'nonarch_libdir', 'oe.qa.exit_if_errors', 'oe.qa.exit_with_message_if_errors', 'oe.qa.handle_error', 'oe.qa.write_error', 'oe.types.boolean', 'oe.utils.vartrue', 'oldincludedir', 'prefix', 'prefix_native', 'root_prefix', 'sbindir', 'sbindir_native', 'servicedir', 'sharedstatedir', 'sysconfdir', 'sysconfdir_native', 'systemd_system_unitdir', 'systemd_unitdir', 'systemd_user_unitdir'] to: ['AR', 'AS', 'B', 'BUILD_AR', 'BUILD_AS', 'BUILD_AS_ARCH', 'BUILD_CC', 'BUILD_CCLD', 'BUILD_CC_ARCH', 'BUILD_CFLAGS', 'BUILD_CPP', 'BUILD_CPPFLAGS', 'BUILD_CXX', 'BUILD_CXXFLAGS', 'BUILD_FC', 'BUILD_LD', 'BUILD_LDFLAGS', 'BUILD_LD_ARCH', 'BUILD_NM', 'BUILD_OPTIMIZATION', 'BUILD_PREFIX', 'BUILD_RANLIB', 'BUILD_STRIP', 'CC', 'CCLD', 'CFLAGS', 'CONFIG_SITE', 'CPP', 'CPPFLAGS', 'CXX', 'CXXFLAGS', 'DEBUG_BUILD', 'DEPENDS', 'DISABLE_STATIC', 'ERROR_QA', 'EXTRA_NATIVE_PKGCONFIG_PATH', 'EXTRA_OECONF', 'FC', 'HOST_PREFIX', 'INSANE_SKIP', 'LC_ALL', 'LD', 'LDFLAGS', 'MAKE', 'MLPREFIX', 'NATIVE_PACKAGE_PATH_SUFFIX', 'NM', 'OBJCOPY', 'OBJDUMP', 'P', 'PACKAGECONFIG', 'PERL_HASH_SEED', 'PKG_CONFIG_DIR', 'PKG_CONFIG_DISABLE_UNINSTALLED', 'PKG_CONFIG_LIBDIR', 'PKG_CONFIG_PATH', 'PKG_CONFIG_SYSROOT_DIR', 'PN', 'PSEUDO_DISABLED', 'PV', 'PYTHONHASHSEED', 'QA_ERRORS_FOUND', 'QA_LOGFILE', 'RANLIB', 'READELF', 'RECIPE_SYSROOT_NATIVE', 'S', 'STAGING_BASE_LIBDIR_NATIVE', 'STAGING_BINDIR_NATIVE', 'STAGING_DATADIR', 'STAGING_DATADIR_NATIVE', 'STAGING_DIR_NATIVE', 'STAGING_ETCDIR_NATIVE', 'STAGING_INCDIR_NATIVE', 'STAGING_LIBDIR_NATIVE', 'STAGING_SBINDIR_NATIVE', 'STRINGS', 'STRIP', 'TZ', 'UNKNOWN_CONFIGURE_OPT_IGNORE', 'base_bindir', 'base_libdir', 'base_libdir_native', 'base_prefix', 'base_sbindir', 'bindir', 'bindir_native', 'datadir', 'datadir_native', 'do_qa_configure', 'docdir', 'exec_prefix', 'includedir', 'includedir_native', 'infodir', 'libdir', 'libdir_native', 'libexecdir', 'localstatedir', 'lt_cv_sys_lib_dlsearch_path_spec', 'mandir', 'nonarch_base_libdir', 'nonarch_libdir', 'oe.qa.exit_if_errors', 'oe.qa.exit_with_message_if_errors', 'oe.qa.handle_error', 'oe.qa.write_error', 'oe.types.boolean', 'oe.utils.vartrue', 'oldincludedir', 'prefix', 'prefix_native', 'root_prefix', 'sbindir', 'sbindir_native', 'servicedir', 'sharedstatedir', 'sysconfdir', 'sysconfdir_native', 'systemd_system_unitdir', 'systemd_unitdir', 'systemd_user_unitdir'] basehash changed from 4866ba7810881112d6ce0cb85301756c02eb4b282604bae69516ec23d7b20acf to 297e5c7592ebd6378c8fc49fe98ffd9e675a04324346c5b50811c6a5eebbe78e List of dependencies for variable EXTRA_OECONF changed from 'frozenset()' to 'frozenset({'DISABLE_STATIC'})' Dependency on variable DISABLE_STATIC was added Variable EXTRA_OECONF value changed from '' to '${DISABLE_STATIC}' ERROR: Can't find a task we're supposed to have written out? (hash: 999250461834bf74545962a8577bc26854a3d08bb2a96140d5efe314e0ef2d63)? ERROR: Can't find a task we're supposed to have written out? (hash: e79d70b9c2cc72030c1ce822525510699a1eeb1ddf5986271d3217422244366a)? On 18/10/2023 10:46:52+0200, Alexander Kanavin wrote: > 'bitbake -S printdiff' is a useful diagnostic facility for finding out > why sstate is not being reused, but until now it had no tests that would > ensure it works. This commit adds three basic scenarios: > > 1. make a change in a really basic, common recipe that is at the very root > of dependency trees (quilt-native), and ensure that change is correctly > discovered when > building an image. > > 2. make a change in gcc-source recipe, which is somewhat special > (operates in work-shared), and ensure that gcc-runtime builds track > that down as well. > > 3. make a change in base_do_configure() definition from base.bbclass, > which is not recipe-specific, but affects many basic recipes, and ensure that > is correctly reported as well. > > The test itself actually runs twice: > - first against a fully populated build directory, where > the printdiff code is guaranteed to find the correct previous > stamp that can be compared with in a predictable manner. > > - then in an empty build directory where the printdiff code > goes to look in the sstate cache, and so the existence of the > previous signature can be tested, but not the difference with it > (what the exact difference would be is unpredictable as the > sstate cache is indeed shared between many builds). > > Signed-off-by: Alexander Kanavin <a...@linutronix.de> > --- > .../base-do-configure-modified.bbclass | 3 + > .../gcc-source/gcc-source_%.bbappend | 2 + > .../quilt-native/quilt-native_%.bbappend | 2 + > meta/lib/oeqa/selftest/cases/sstatetests.py | 106 ++++++++++++++++++ > 4 files changed, 113 insertions(+) > create mode 100644 meta-selftest/classes/base-do-configure-modified.bbclass > create mode 100644 > meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend > create mode 100644 > meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend > > diff --git a/meta-selftest/classes/base-do-configure-modified.bbclass > b/meta-selftest/classes/base-do-configure-modified.bbclass > new file mode 100644 > index 00000000000..3f96827a428 > --- /dev/null > +++ b/meta-selftest/classes/base-do-configure-modified.bbclass > @@ -0,0 +1,3 @@ > +base_do_configure:append () { > + echo "this changes base_do_configure() definiton" > +} > diff --git a/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend > b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend > new file mode 100644 > index 00000000000..205720982cb > --- /dev/null > +++ b/meta-selftest/recipes-test/gcc-source/gcc-source_%.bbappend > @@ -0,0 +1,2 @@ > +# This bbappend is used to alter the recipe using the test_recipe.inc file > created by tests. > +include test_recipe.inc > diff --git a/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend > b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend > new file mode 100644 > index 00000000000..205720982cb > --- /dev/null > +++ b/meta-selftest/recipes-test/quilt-native/quilt-native_%.bbappend > @@ -0,0 +1,2 @@ > +# This bbappend is used to alter the recipe using the test_recipe.inc file > created by tests. > +include test_recipe.inc > diff --git a/meta/lib/oeqa/selftest/cases/sstatetests.py > b/meta/lib/oeqa/selftest/cases/sstatetests.py > index bdad9088d37..6ef339897bf 100644 > --- a/meta/lib/oeqa/selftest/cases/sstatetests.py > +++ b/meta/lib/oeqa/selftest/cases/sstatetests.py > @@ -773,3 +773,109 @@ addtask tmptask2 before do_tmptask1 > latestfiles = sorted(filedates.keys(), key=lambda f: > filedates[f])[-2:] > bb.siggen.compare_sigfiles(latestfiles[-2], latestfiles[-1], > recursecb) > self.assertEqual(recursecb_count,1) > + > +class SStatePrintdiff(SStateBase): > + def run_test_printdiff_changerecipe(self, target, change_recipe, > change_bbtask, change_content, expected_sametmp_output, > expected_difftmp_output): > + self.write_config(""" > +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff" > +""") > + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff") > + bitbake(target) > + bitbake("-S none {}".format(target)) > + bitbake(change_bbtask) > + self.write_recipeinc(change_recipe, change_content) > + result_sametmp = bitbake("-S printdiff {}".format(target)) > + > + self.write_config(""" > +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2" > +""") > + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2") > + result_difftmp = bitbake("-S printdiff {}".format(target)) > + > + self.delete_recipeinc(change_recipe) > + for item in expected_sametmp_output: > + self.assertIn(item, result_sametmp.output) > + for item in expected_difftmp_output: > + self.assertIn(item, result_difftmp.output) > + > + def run_test_printdiff_changeconfig(self, target, change_content, > expected_sametmp_output, expected_difftmp_output): > + self.write_config(""" > +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff" > +""") > + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff") > + bitbake(target) > + bitbake("-S none {}".format(target)) > + self.append_config(change_content) > + result_sametmp = bitbake("-S printdiff {}".format(target)) > + > + self.write_config(""" > +TMPDIR = "${TOPDIR}/tmp-sstateprintdiff-2" > +""") > + self.append_config(change_content) > + self.track_for_cleanup(self.topdir + "/tmp-sstateprintdiff-2") > + result_difftmp = bitbake("-S printdiff {}".format(target)) > + > + for item in expected_sametmp_output: > + self.assertIn(item, result_sametmp.output) > + for item in expected_difftmp_output: > + self.assertIn(item, result_difftmp.output) > + > + > + # Check if printdiff walks the full dependency chain from the image > target to where the change is in a specific recipe > + def test_image_minimal_vs_quilt(self): > + expected_output = ("Task quilt-native:do_install couldn't be used > from the cache because:", > +"We need hash", > +"most recent matching task was") > + expected_sametmp_output = expected_output + ("Variable do_install > value changed",'+ echo "this changes the task signature"') > + expected_difftmp_output = expected_output > + > + self.run_test_printdiff_changerecipe("core-image-minimal", > "quilt-native", "-c do_install quilt-native", > +""" > +do_install:append() { > + echo "this changes the task signature" > +} > +""", > +expected_sametmp_output, expected_difftmp_output) > + > + # Check if changes to gcc-source (which uses tmp/work-shared) are > correctly discovered > + def test_gcc_runtime_vs_gcc_source(self): > + gcc_source_pn = 'gcc-source-%s' % get_bb_vars(['PV'], 'gcc')['PV'] > + > + expected_output = ("Task {}:do_preconfigure couldn't be used from > the cache because:".format(gcc_source_pn), > +"We need hash", > +"most recent matching task was") > + expected_sametmp_output = expected_output + ("Variable > do_preconfigure value changed",'+ print("this changes the task > signature")') > + #FIXME: printdiff is supposed to find at least one preconfigure task > signature in the sstate cache, but isn't able to > + #expected_difftmp_output = expected_output > + expected_difftmp_output = () > + > + self.run_test_printdiff_changerecipe("gcc-runtime", "gcc-source", > "-c do_preconfigure {}".format(gcc_source_pn), > +""" > +python do_preconfigure:append() { > + print("this changes the task signature") > +} > +""", > +expected_sametmp_output, expected_difftmp_output) > + > + # Check if changing a really base task definiton is reported against > multiple core recipes using it > + def test_image_minimal_vs_base_do_configure(self): > + expected_output = ("Task zstd-native:do_configure couldn't be used > from the cache because:", > +"Task texinfo-dummy-native:do_configure couldn't be used from the cache > because:", > +"Task ldconfig-native:do_configure couldn't be used from the cache because:", > +"Task gettext-minimal-native:do_configure couldn't be used from the cache > because:", > +"Task tzcode-native:do_configure couldn't be used from the cache because:", > +"Task makedevs-native:do_configure couldn't be used from the cache because:", > +"Task pigz-native:do_configure couldn't be used from the cache because:", > +"Task update-rc.d-native:do_configure couldn't be used from the cache > because:", > +"Task unzip-native:do_configure couldn't be used from the cache because:", > +"Task gnu-config-native:do_configure couldn't be used from the cache > because:", > +"We need hash", > +"most recent matching task was") > + expected_sametmp_output = expected_output + ("Variable > base_do_configure value changed",'+ echo "this changes base_do_configure() > definiton"') > + expected_difftmp_output = expected_output > + > + self.run_test_printdiff_changeconfig("core-image-minimal", > +""" > +INHERIT += "base-do-configure-modified" > +""", > +expected_sametmp_output, expected_difftmp_output) > -- > 2.39.2 > > > > -- Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189421): https://lists.openembedded.org/g/openembedded-core/message/189421 Mute This Topic: https://lists.openembedded.org/mt/102035446/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
