On Mon, 13 May 2024 at 22:35, Alexander Kanavin via lists.openembedded.org <[email protected]> wrote: > The incomplete patch for this issue is here on top: > https://git.yoctoproject.org/poky-contrib/log/?h=akanavin/package-version-updates > > Unfortunately, that is not all, as file names and file sizes are now > the same in libstd-rs, but content still isn't for some of them > (different hashes reported by sha256sum). And in target rust both > names and content differs. > > I'm out of time today to look further into it, but you are welcome to > pick it up.
This is the worst rabbit hole of this year so far, made worse by having to confirm every point with a 30 minute build or test execution. The good news is I have a fix. The bad news is that the fix breaks rust selftest, and fixing the rust selftest breaks rust reproducibility. Let's review: - pre-built cargo from upstream snapshots writes output of 'rust -vV' into hashes used to form filenames of build artefacts, and that output includes the build host architecture. - we can patch this out in cargo-native and I have the patch. - to use cargo-native we need to tweak rust's build config so that it uses that and not cargo from upstream binary snapshot for bootstrapping. At the same time we need to tweak location of rustc in the same config, as they would otherwise mismatch and build will break. I have the patch for that too, and I confirmed the target filenames start to match with it. - this breaks the rust selftest, as it is now looking into native sysroot instead of rust-snapshot/ and can no longer find rustdoc there that it needs for the tests - rustdoc is present in binary rust snapshots but not in rust-native installations. Why? Because it used to cause reproducibility problems (and still does), and the purported reproducibility fix simply prevented rustdoc from being installed: https://git.yoctoproject.org/poky/commit/?id=321aebfa281bd28e368c684ece57867f6bd0cbe7 (the culprit is the rust-demangler line which replaces rust's default installation set (that includes rustdoc) with just that utility: https://github.com/rust-lang/rust/blob/master/config.example.toml#L320 ) - I don't know if pgo tweak in the above indeed makes rustdoc reproducible, as that tweak was later reverted due to causing issues with chromium: https://git.yoctoproject.org/poky/commit/?id=694b85a217209787fe8c503a4b697dc6c2bfbe84 I would strongly suggest that fixing random autobuilder breakage is far more important than having a passing test suite for a specific component or having that component pass reproducibility, so the proposal is: - merge my fixes - disable rust selftest - re-enable rustdoc installation, as that is the default and is expected in typical rust setups - put rust back into reproducibility exclusion due to rustdoc - update rust to latest upstream release (we're now three releases behind) - and only then look again, properly, into re-enabling both reproducibility and selftests, keeping all of the above in mind Cheers, Alex
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#199288): https://lists.openembedded.org/g/openembedded-core/message/199288 Mute This Topic: https://lists.openembedded.org/mt/106113200/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
