On Fri, Sep 27, 2019 at 7:50 PM Joshua Watt <jpewhac...@gmail.com> wrote:
>
>
> On 9/27/19 1:03 PM, Alex Kiernan wrote:
> > On Thu, Sep 26, 2019 at 6:58 PM Joshua Watt <jpewhac...@gmail.com> wrote:
> >> The deployment of the source date epoch file had a race condition where
> >> any task attempting to read from the file would race with creation of
> >> the sstate archive for the do_deploy_source_date_epoch task. The
> >> creation of the sstate archive requires moving the directory to a
> >> temporary location, then moving it back. This means that the file
> >> disappears for a short period of time, which will cause a failure if any
> >> other task is running and trying to open the file to get the current
> >> source date epoch.
> >>
> >> The solution is to copy the source date epoch file to a separate
> >> directory when deploying so the file never disappears. When the file is
> >> restored from sstate, it is moved to the correct location after being
> >> extracted.
> >>
> >> [YOCTO #13501]
> >>
> >> Signed-off-by: Joshua Watt <jpewhac...@gmail.com>
> >> ---
> >> meta/classes/reproducible_build.bbclass | 12 ++++++++++--
> >> 1 file changed, 10 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/meta/classes/reproducible_build.bbclass
> >> b/meta/classes/reproducible_build.bbclass
> >> index 8788ad7145c..99b749a9ee2 100644
> >> --- a/meta/classes/reproducible_build.bbclass
> >> +++ b/meta/classes/reproducible_build.bbclass
> >> @@ -39,19 +39,27 @@ inherit
> >> ${@oe.utils.ifelse(d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1', 'repr
> >>
> >> SDE_DIR ="${WORKDIR}/source-date-epoch"
> >> SDE_FILE = "${SDE_DIR}/__source_date_epoch.txt"
> >> +SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch"
> >>
> >> SSTATETASKS += "do_deploy_source_date_epoch"
> >>
> >> do_deploy_source_date_epoch () {
> >> echo "Deploying SDE to ${SDE_DIR}."
> >> + mkdir -p ${SDE_DEPLOYDIR}
> >> + if [ -e ${SDE_FILE} ]; then
> >> + cp -p ${SDE_FILE} ${SDE_DEPLOYDIR}/__source_date_epoch.txt
> >> + fi
> >> }
> >>
> >> python do_deploy_source_date_epoch_setscene () {
> >> sstate_setscene(d)
> >> + sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'),
> >> '__source_date_epoch.txt')
> >> + if os.path.exists(sde_file):
> >> + os.rename(sde_file, d.getVar('SDE_FILE'))
> >> }
> >>
> >> -do_deploy_source_date_epoch[dirs] = "${SDE_DIR}"
> >> -do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DIR}"
> >> +do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
> >> +do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
> >> addtask do_deploy_source_date_epoch_setscene
> >> addtask do_deploy_source_date_epoch before do_configure after do_patch
> >>
> > This seems to have broken builds with reproducible builds enabled and
> > sstate downloaded from a mirror - I just get a blizzard of:
> >
> > ERROR: Logfile of failure stored in:
> > /home/akiernan/nanohub/build/tmp/work/x86_64-linux/automake-native/1.16.1-r0/temp/log.do_deploy_source_date_epoch_setscene.113823
> > WARNING: Setscene task
> > (virtual:native:/home/akiernan/nanohub/build/../poky/meta/recipes-devtools/automake/automake_1.16.1.bb:do_deploy_source_date_epoch_setscene)
> > failed with exit code '1' - real task will be run instead
> > ERROR: autoconf-native-2.69-r11 do_deploy_source_date_epoch_setscene:
> > Error executing a python function in exec_python_func() autogenerated:
> >
> > The stack trace of python calls that resulted in this exception/failure was:
> > File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
> > 0001:
> > *** 0002:do_deploy_source_date_epoch_setscene(d)
> > 0003:
> > File:
> > '/home/akiernan/nanohub/build/../poky/meta/classes/reproducible_build.bbclass',
> > lineno: 58, function: do_deploy_source_date_epoch_setscene
> > 0054:python do_deploy_source_date_epoch_setscene () {
> > 0055: sstate_setscene(d)
> > 0056: sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'),
> > '__source_date_epoch.txt')
> > 0057: if os.path.exists(sde_file):
> > *** 0058: os.rename(sde_file, d.getVar('SDE_FILE'))
> > 0059:}
> > 0060:
> > 0061:do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
> > 0062:do_deploy_source_date_epoch[sstate-plaindirs] =
> > "${SDE_DEPLOYDIR}"
> > Exception: FileNotFoundError: [Errno 2] No such file or directory:
> > '/home/akiernan/nanohub/build/tmp/work/x86_64-linux/autoconf-native/2.69-r11/deploy-source-date-epoch/__source_date_epoch.txt'
> > ->
> > '/home/akiernan/nanohub/build/tmp/work/x86_64-linux/autoconf-native/2.69-r11/source-date-epoch/__source_date_epoch.txt'
>
> Sorry about that. It looks like destination directory isn't getting
> created. You can try giving
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=jpew/hash-equivalence&id=7177bb4c4c174f0852b984b9b2d2d23de92e70cd
> a try to see if it resolves the issue.
>
LGTM.
--
Alex Kiernan
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
