Re: svn commit: r365846 - head
On Wed, Sep 23, 2020 at 9:30 AM Warner Losh wrote:
> On Wed, Sep 23, 2020, 7:21 AM Kyle Evans wrote:
>>
>> On Mon, Sep 21, 2020 at 7:23 PM Ravi Pokala wrote:
>> >
>> > -Original Message-
>> > From: on behalf of Ed Maste
>> >
>> > Date: 2020-09-17, Thursday at 11:47
>> > To: , ,
>> >
>> > Subject: svn commit: r365846 - head
>> >
>> > Author: emaste
>> > Date: Thu Sep 17 18:47:23 2020
>> > New Revision: 365846
>> > URL: https://svnweb.freebsd.org/changeset/base/365846
>> >
>> > Log:
>> > Cirrus-CI: build as an unprivileged user
>> >
>> > The Cirrus-CI-provided working tree is owned by root. Leave that as
>> > is
>> > for simplicity but build as an unprivileged user; this tests building
>> > with an unmodifiable source tree as a side effect.
>> >
>> > Hi Ed,
>> >
>> > We're still generating the LINT kernconfs into the src tree though, right?
>> > Moving that to allow for universe/tinderboxing a r/o src tree seems like
>> > an obvious idea. The fact that we don't already do that implies that
>> > there's a non-obvious complication with that idea; does anyone know why
>> > that is?
>> >
>> > Thanks,
>> >
>> > Ravi (rpokala@)
>>
>> So, the main limiting factor here is config(8) logistics. You've got
>> two questions to sort out:
>>
>> 1. Where in .OBJDIR do you put LINT?
>> 2. How do you tell config(8) to find that LINT?
>>
>> I think both of these are actually pretty easy to solve. For the
>> former, sys/conf/makeLINT.mk would need to be directed to put them in
>> ${OBJTOP}/${.CURDIR} instead of ${.CURDIR} so that it ends up in
>> ${OBJTOP}/sys/${MACHINE}/conf -- this is kind of unusual since nothing
>> else will get stored there, but ultimately not a big deal. There's an
>> inline patch that gets us probably the most of the way, but universe
>> would still need a little bit of love. I don't think that part would
>> suck much either, and it probably would clean things up a little bit
>> too- for MAKE_LINT_KERNELS you wouldn't need to bother searching the
>> in-tree confdir.
>>
>> Thanks,
>>
>> Kyle Evans
>>
>> (as an aside, I re-read Ed's commit message after this and realized I
>> had asked something almost verbatim as the commit message explained
>> it... sorry)
>>
>> [... patch omitted ...]
>
> Won't this break non LINT builds? Maybe we should fix the crazy way we
> generate lint?
>
We've discussed this out-of-band a little bit, but to bring it back to
the list: there really isn't a reason to keep generating these LINT
files, config(8) supports include these days, so we think we can just
make individual LINTs include the global and per-arch NOTES and do
whatever nodevice/nooptions it needs to do.
To address the first question; this particular patch just taught
buildkernel to look in two different places for any KERNCONF and
recorded where it found the KERNCONF so that LINT/non-LINT both worked
in any combination (GENERIC LINT, LINT, or just GENERIC). It's subpar
compared to discontinue generation of these files, especially when you
look at comments in makeLINT like:
# cat is available, not sure if cp is?
This is one less place to care about what's available in the build
environment, especially when it's ultimately something trivial like
`cat NOTES OTHERNOTES` and a series of echo.
Thanks,
Kyle Evans
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
Re: svn commit: r365846 - head
Won't this break non LINT builds? Maybe we should fix the crazy way we
generate lint?
On Wed, Sep 23, 2020, 7:21 AM Kyle Evans wrote:
> On Mon, Sep 21, 2020 at 7:23 PM Ravi Pokala wrote:
> >
> > -Original Message-
> > From: on behalf of Ed Maste
>
> > Date: 2020-09-17, Thursday at 11:47
> > To: , , <
> svn-src-h...@freebsd.org>
> > Subject: svn commit: r365846 - head
> >
> > Author: emaste
> > Date: Thu Sep 17 18:47:23 2020
> > New Revision: 365846
> > URL: https://svnweb.freebsd.org/changeset/base/365846
> >
> > Log:
> > Cirrus-CI: build as an unprivileged user
> >
> > The Cirrus-CI-provided working tree is owned by root. Leave that
> as is
> > for simplicity but build as an unprivileged user; this tests
> building
> > with an unmodifiable source tree as a side effect.
> >
> > Hi Ed,
> >
> > We're still generating the LINT kernconfs into the src tree though,
> right? Moving that to allow for universe/tinderboxing a r/o src tree seems
> like an obvious idea. The fact that we don't already do that implies that
> there's a non-obvious complication with that idea; does anyone know why
> that is?
> >
> > Thanks,
> >
> > Ravi (rpokala@)
>
> So, the main limiting factor here is config(8) logistics. You've got
> two questions to sort out:
>
> 1. Where in .OBJDIR do you put LINT?
> 2. How do you tell config(8) to find that LINT?
>
> I think both of these are actually pretty easy to solve. For the
> former, sys/conf/makeLINT.mk would need to be directed to put them in
> ${OBJTOP}/${.CURDIR} instead of ${.CURDIR} so that it ends up in
> ${OBJTOP}/sys/${MACHINE}/conf -- this is kind of unusual since nothing
> else will get stored there, but ultimately not a big deal. There's an
> inline patch that gets us probably the most of the way, but universe
> would still need a little bit of love. I don't think that part would
> suck much either, and it probably would clean things up a little bit
> too- for MAKE_LINT_KERNELS you wouldn't need to bother searching the
> in-tree confdir.
>
> Thanks,
>
> Kyle Evans
>
> (as an aside, I re-read Ed's commit message after this and realized I
> had asked something almost verbatim as the commit message explained
> it... sorry)
>
> root@viper:/usr/src# git diff Makefile.inc1 sys/conf/makeLINT.mk
> diff --git a/Makefile.inc1 b/Makefile.inc1
> index de4970efeef..2a095f2f903 100644
> --- a/Makefile.inc1
> +++ b/Makefile.inc1
> @@ -1591,9 +1591,11 @@ KERNCONF?= GENERIC
> INSTKERNNAME?= kernel
>
> KERNSRCDIR?= ${.CURDIR}/sys
> -KRNLCONFDIR= ${KERNSRCDIR}/${TARGET}/conf
> +KRNLRELCONFDIR=${TARGET}/conf
> +KRNLCONFDIR= ${KERNSRCDIR}/${KRNLRELCONFDIR}
> KRNLOBJDIR=${OBJTOP}${KERNSRCDIR:C,^${.CURDIR},,}
> KERNCONFDIR?= ${KRNLCONFDIR}
> +KERNCONFOBJDIR?= ${OBJTOP}/sys/${KRNLRELCONFDIR}
>
> BUILDKERNELS=
> INSTALLKERNEL=
> @@ -1602,8 +1604,14 @@ INSTALLKERNEL=
> BUILDKERNELS+= dummy
> .endif
> .for _kernel in ${KERNCONF}
> -.if !defined(_MKSHOWCONFIG) && exists(${KERNCONFDIR}/${_kernel})
> +.if !defined(_MKSHOWCONFIG) && \
> +(exists(${KERNCONFDIR}/${_kernel}) ||
> exists(${KERNCONFOBJDIR}/${_kernel}))
> BUILDKERNELS+= ${_kernel}
> +.if exists(${KERNCONFDIR}/${_kernel})
> +KERNCONFDIR_${TARGET}_${_kernel}=${KERNCONFDIR}
> +.else
> +KERNCONFDIR_${TARGET}_${_kernel}=${KERNCONFOBJDIR}
> +.endif
> .if empty(INSTALLKERNEL) && !defined(NO_INSTALLKERNEL)
> INSTALLKERNEL= ${_kernel}
> .endif
> @@ -1661,7 +1669,7 @@ buildkernel: .MAKE .PHONY
> PATH=${TMPPATH} \
> config ${CONFIGARGS} -d ${KRNLOBJDIR}/${_kernel} \
> -I '${KERNCONFDIR}' -I '${KRNLCONFDIR}' \
> - '${KERNCONFDIR}/${_kernel}'
> + '${KERNCONFDIR_${TARGET}_${_kernel}}/${_kernel}'
> .endif
> .if ${MK_CLEAN} == "yes" && !defined(NO_KERNELCLEAN)
> @echo
> diff --git a/sys/conf/makeLINT.mk b/sys/conf/makeLINT.mk
> index b2fc256621b..edaea99750a 100644
> --- a/sys/conf/makeLINT.mk
> +++ b/sys/conf/makeLINT.mk
> @@ -1,8 +1,5 @@
> # $FreeBSD$
>
> -# The LINT files need to end up in the kernel source directory.
> -.OBJDIR: ${.CURDIR}
> -
> all:
> @echo "make LINT only"
>
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
Re: svn commit: r365846 - head
On Mon, Sep 21, 2020 at 7:23 PM Ravi Pokala wrote:
>
> -Original Message-
> From: on behalf of Ed Maste
>
> Date: 2020-09-17, Thursday at 11:47
> To: , ,
>
> Subject: svn commit: r365846 - head
>
> Author: emaste
> Date: Thu Sep 17 18:47:23 2020
> New Revision: 365846
> URL: https://svnweb.freebsd.org/changeset/base/365846
>
> Log:
> Cirrus-CI: build as an unprivileged user
>
> The Cirrus-CI-provided working tree is owned by root. Leave that as is
> for simplicity but build as an unprivileged user; this tests building
> with an unmodifiable source tree as a side effect.
>
> Hi Ed,
>
> We're still generating the LINT kernconfs into the src tree though, right?
> Moving that to allow for universe/tinderboxing a r/o src tree seems like an
> obvious idea. The fact that we don't already do that implies that there's a
> non-obvious complication with that idea; does anyone know why that is?
>
> Thanks,
>
> Ravi (rpokala@)
So, the main limiting factor here is config(8) logistics. You've got
two questions to sort out:
1. Where in .OBJDIR do you put LINT?
2. How do you tell config(8) to find that LINT?
I think both of these are actually pretty easy to solve. For the
former, sys/conf/makeLINT.mk would need to be directed to put them in
${OBJTOP}/${.CURDIR} instead of ${.CURDIR} so that it ends up in
${OBJTOP}/sys/${MACHINE}/conf -- this is kind of unusual since nothing
else will get stored there, but ultimately not a big deal. There's an
inline patch that gets us probably the most of the way, but universe
would still need a little bit of love. I don't think that part would
suck much either, and it probably would clean things up a little bit
too- for MAKE_LINT_KERNELS you wouldn't need to bother searching the
in-tree confdir.
Thanks,
Kyle Evans
(as an aside, I re-read Ed's commit message after this and realized I
had asked something almost verbatim as the commit message explained
it... sorry)
root@viper:/usr/src# git diff Makefile.inc1 sys/conf/makeLINT.mk
diff --git a/Makefile.inc1 b/Makefile.inc1
index de4970efeef..2a095f2f903 100644
--- a/Makefile.inc1
+++ b/Makefile.inc1
@@ -1591,9 +1591,11 @@ KERNCONF?= GENERIC
INSTKERNNAME?= kernel
KERNSRCDIR?= ${.CURDIR}/sys
-KRNLCONFDIR= ${KERNSRCDIR}/${TARGET}/conf
+KRNLRELCONFDIR=${TARGET}/conf
+KRNLCONFDIR= ${KERNSRCDIR}/${KRNLRELCONFDIR}
KRNLOBJDIR=${OBJTOP}${KERNSRCDIR:C,^${.CURDIR},,}
KERNCONFDIR?= ${KRNLCONFDIR}
+KERNCONFOBJDIR?= ${OBJTOP}/sys/${KRNLRELCONFDIR}
BUILDKERNELS=
INSTALLKERNEL=
@@ -1602,8 +1604,14 @@ INSTALLKERNEL=
BUILDKERNELS+= dummy
.endif
.for _kernel in ${KERNCONF}
-.if !defined(_MKSHOWCONFIG) && exists(${KERNCONFDIR}/${_kernel})
+.if !defined(_MKSHOWCONFIG) && \
+(exists(${KERNCONFDIR}/${_kernel}) || exists(${KERNCONFOBJDIR}/${_kernel}))
BUILDKERNELS+= ${_kernel}
+.if exists(${KERNCONFDIR}/${_kernel})
+KERNCONFDIR_${TARGET}_${_kernel}=${KERNCONFDIR}
+.else
+KERNCONFDIR_${TARGET}_${_kernel}=${KERNCONFOBJDIR}
+.endif
.if empty(INSTALLKERNEL) && !defined(NO_INSTALLKERNEL)
INSTALLKERNEL= ${_kernel}
.endif
@@ -1661,7 +1669,7 @@ buildkernel: .MAKE .PHONY
PATH=${TMPPATH} \
config ${CONFIGARGS} -d ${KRNLOBJDIR}/${_kernel} \
-I '${KERNCONFDIR}' -I '${KRNLCONFDIR}' \
- '${KERNCONFDIR}/${_kernel}'
+ '${KERNCONFDIR_${TARGET}_${_kernel}}/${_kernel}'
.endif
.if ${MK_CLEAN} == "yes" && !defined(NO_KERNELCLEAN)
@echo
diff --git a/sys/conf/makeLINT.mk b/sys/conf/makeLINT.mk
index b2fc256621b..edaea99750a 100644
--- a/sys/conf/makeLINT.mk
+++ b/sys/conf/makeLINT.mk
@@ -1,8 +1,5 @@
# $FreeBSD$
-# The LINT files need to end up in the kernel source directory.
-.OBJDIR: ${.CURDIR}
-
all:
@echo "make LINT only"
___
svn-src-all@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
Re: svn commit: r365846 - head
-Original Message- From: on behalf of Ed Maste Date: 2020-09-17, Thursday at 11:47 To: , , Subject: svn commit: r365846 - head Author: emaste Date: Thu Sep 17 18:47:23 2020 New Revision: 365846 URL: https://svnweb.freebsd.org/changeset/base/365846 Log: Cirrus-CI: build as an unprivileged user The Cirrus-CI-provided working tree is owned by root. Leave that as is for simplicity but build as an unprivileged user; this tests building with an unmodifiable source tree as a side effect. Hi Ed, We're still generating the LINT kernconfs into the src tree though, right? Moving that to allow for universe/tinderboxing a r/o src tree seems like an obvious idea. The fact that we don't already do that implies that there's a non-obvious complication with that idea; does anyone know why that is? Thanks, Ravi (rpokala@) Continue running the smoke test as root for now, as it failed when run as an unprivileged user - pkg reported "Fail to chmod /usr/bin/.pkgtemp.lpq.dUHpEqPGJ9pq:Operation not permitted" Sponsored by: The FreeBSD Foundation Modified: head/.cirrus.yml Modified: head/.cirrus.yml == --- head/.cirrus.ymlThu Sep 17 18:24:51 2020(r365845) +++ head/.cirrus.ymlThu Sep 17 18:47:23 2020(r365846) @@ -12,9 +12,13 @@ task: timeout_in: 120m install_script: - pkg install -y qemu-devel uefi-edk2-qemu-x86_64 + setup_user_script: + - pw useradd user + - mkdir -p /usr/obj/$(pwd -P) + - chown user:user /usr/obj/$(pwd -P) script: - - make -j$(sysctl -n hw.ncpu) WITHOUT_TOOLCHAIN=yes buildworld buildkernel + - su user -c "make -j$(sysctl -n hw.ncpu) WITHOUT_TOOLCHAIN=yes buildworld buildkernel" package_script: - - make WITHOUT_TOOLCHAIN=yes PKG_FORMAT=tar packages + - su user -c "make WITHOUT_TOOLCHAIN=yes PKG_FORMAT=tar packages" test_script: - sh tools/boot/ci-qemu-test.sh ___ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
Re: svn commit: r365846 - head
On Thu, 17 Sep 2020 at 18:35, Ed Maste wrote: > > On Thu, 17 Sep 2020 at 15:07, Kyle Evans wrote: > > > > Is there any way we can sneak a r/o src tree build in here? I've had a > > PR open to do this in our Jenkins builds for a couple years, but > > without any ability to test it myself I suspect it'll continue > > unmerged. > > Cirrus-CI does the git clone for us and it is done as root, so we > should in effect have a RO src tree here. I'll confirm sometime later > on. Indeed, Cirrus checks out the tree under /tmp/cirrus-ci-build as root, and there are no files/directories writable by the unprivileged user added by this change. ___ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
Re: svn commit: r365846 - head
On Thu, 17 Sep 2020 at 15:07, Kyle Evans wrote: > > Is there any way we can sneak a r/o src tree build in here? I've had a > PR open to do this in our Jenkins builds for a couple years, but > without any ability to test it myself I suspect it'll continue > unmerged. Cirrus-CI does the git clone for us and it is done as root, so we should in effect have a RO src tree here. I'll confirm sometime later on. ___ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
Re: svn commit: r365846 - head
On Thu, Sep 17, 2020 at 1:47 PM Ed Maste wrote: > > Author: emaste > Date: Thu Sep 17 18:47:23 2020 > New Revision: 365846 > URL: https://svnweb.freebsd.org/changeset/base/365846 > > Log: > Cirrus-CI: build as an unprivileged user > > The Cirrus-CI-provided working tree is owned by root. Leave that as is > for simplicity but build as an unprivileged user; this tests building > with an unmodifiable source tree as a side effect. > > Continue running the smoke test as root for now, as it failed when run > as an unprivileged user - pkg reported "Fail to chmod > /usr/bin/.pkgtemp.lpq.dUHpEqPGJ9pq:Operation not permitted" > > Sponsored by: The FreeBSD Foundation > Is there any way we can sneak a r/o src tree build in here? I've had a PR open to do this in our Jenkins builds for a couple years, but without any ability to test it myself I suspect it'll continue unmerged. Thanks, Kyle Evans ___ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
svn commit: r365846 - head
Author: emaste Date: Thu Sep 17 18:47:23 2020 New Revision: 365846 URL: https://svnweb.freebsd.org/changeset/base/365846 Log: Cirrus-CI: build as an unprivileged user The Cirrus-CI-provided working tree is owned by root. Leave that as is for simplicity but build as an unprivileged user; this tests building with an unmodifiable source tree as a side effect. Continue running the smoke test as root for now, as it failed when run as an unprivileged user - pkg reported "Fail to chmod /usr/bin/.pkgtemp.lpq.dUHpEqPGJ9pq:Operation not permitted" Sponsored by: The FreeBSD Foundation Modified: head/.cirrus.yml Modified: head/.cirrus.yml == --- head/.cirrus.ymlThu Sep 17 18:24:51 2020(r365845) +++ head/.cirrus.ymlThu Sep 17 18:47:23 2020(r365846) @@ -12,9 +12,13 @@ task: timeout_in: 120m install_script: - pkg install -y qemu-devel uefi-edk2-qemu-x86_64 + setup_user_script: + - pw useradd user + - mkdir -p /usr/obj/$(pwd -P) + - chown user:user /usr/obj/$(pwd -P) script: - - make -j$(sysctl -n hw.ncpu) WITHOUT_TOOLCHAIN=yes buildworld buildkernel + - su user -c "make -j$(sysctl -n hw.ncpu) WITHOUT_TOOLCHAIN=yes buildworld buildkernel" package_script: - - make WITHOUT_TOOLCHAIN=yes PKG_FORMAT=tar packages + - su user -c "make WITHOUT_TOOLCHAIN=yes PKG_FORMAT=tar packages" test_script: - sh tools/boot/ci-qemu-test.sh ___ svn-src-all@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/svn-src-all To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
