Re: GSoC: Separation of Ports Build Process from Local Installation
On 2019-05-29 09:49, Matthew Seaman wrote:
On 29/05/2019 10:56, Peter Pentchev wrote:
Hmm, I could be wrong, but isn't ${LOCALBASE} supposed to be where
ports find stuff*during the build*, and ${PREFIX} where they
install the built files? Of course, I haven't actually touched
a FreeBSD ports build in years, so I might very likely be wrong.
I also seem to remember a series of test port builds done a long
time ago with a different value for LOCALBASE, specifically to catch
ports that do not honor the policy in this regard.
No, you are correct. The LOCALBASE setting is where the ports will
look for build dependencies. However, the problem here is there is no
guarrantee that the LOCALBASE value will not somehow be compiled into
a resulting package (eg. as RPATH for dynamically loading a shlib) so
would also be needed for runtime dependencies.
LOCALBASE does control where ports find their dependencies at build
time, but it is not the only purpose. There is, as best I can tell, no
kind of separation between LOCALBASE as location to find files at
compile time vs. at run time. LOCALBASE serves the same effective
purpose as PREFIX, except that the separation is maintained so that a
port can use some other PREFIX in special cases. Therefore it seems
reasonable to me to leave the established meaning of LOCALBASE as-is and
to use a different mechanism for accessing the dependency install
location at build time.
Besides, hijacking LOCALBASE is not really an option, since there are
existing tests such as:
.if (${PREFIX} != ${LOCALBASE} && ${PREFIX} != ${LINUXBASE} && \
which seem reasonable to me.
On 2019-05-29 05:56, Peter Pentchev wrote:
I also seem to remember a series of test port builds done a long
time ago with a different value for LOCALBASE, specifically to catch
ports that do not honor the policy in this regard.
I will be doing a similar test, using syscall tracing, to check that
nothing reads from the real ${LOCALBASE} during building.
On 2019-05-29 17:42, Lorenzo Salvadore via freebsd-ports wrote:
Indeed, I maintain a port where LOCALBASE is a value that appears in the
resulting package's files (not as RPATH): math/maxima. See its Makefile at
line 65.
Indeed, many other ports as well make this kind of assumption.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: GSoC: Separation of Ports Build Process from Local Installation
> On 29/05/2019 10:56, Peter Pentchev wrote:
>
> > Hmm, I could be wrong, but isn't ${LOCALBASE} supposed to be where
> > ports find stuffduring the build, and ${PREFIX} where they
> > install the built files? Of course, I haven't actually touched
> > a FreeBSD ports build in years, so I might very likely be wrong.
> > I also seem to remember a series of test port builds done a long
> > time ago with a different value for LOCALBASE, specifically to catch
> > ports that do not honor the policy in this regard.
>
> No, you are correct. The LOCALBASE setting is where the ports will look
> for build dependencies. However, the problem here is there is no
> guarrantee that the LOCALBASE value will not somehow be compiled into a
> resulting package (eg. as RPATH for dynamically loading a shlib) so
> would also be needed for runtime dependencies.
Indeed, I maintain a port where LOCALBASE is a value that appears in the
resulting package's files (not as RPATH): math/maxima. See its Makefile at
line 65.
> Being able to build with a custom value of LOCALBASE and then install
> and run the resulting package onto a system using different value for
> PREFIX would be an interesting piece of work. The applicability would
> be people that custom build their own ports, but who are for some reason
> unable or unwilling to set up some sort of clean-build environment.
> This does strike me as an awful lot of work to solve a perceived problem
> where we do already have some pretty good solutions in place, but not to
> everybodies' taste.
Here again I can give an example of someone who does not want any
clean-build environment: myself. My machine is pretty old, cpu is slow,
ram is low and poudriere gets annoying on it: it needs to install and deinstall
plenty of dependencies for each single new package it builds. My solution
at the moment is using ports keeping all the build depenencies always installed.
However, if I understood the project correctly, I fear it would not offer a
useful alternative in my case.
Lorenzo Salvadore.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: GSoC: Separation of Ports Build Process from Local Installation
On May 28, 2019 3:51:08 PM PDT, Theron wrote: >Hello All, > >For Google Summer of Code 2019 I am working on FreeBSD's ports tree >makefiles towards eliminating the dependency of the ports building >process on the local system's installed packages. Currently this level > >of separation can only be accomplished in practice through chroot or >Jail. The project will eliminate the need for cooperation of the root >user since /usr/local will not need to be touched. > >The major technical obstacle to be overcome is that ports expect to >find >files of their dependencies installed in /usr/local. To support this >without touching that location on the installed system, file accesses >will be redirected to a location controlled by the ports build process >through use of a library to intercept file accesses. > >Once I have that working (well enough to build one port at a time) I >will move on to modify bsd.port.mk itself (and related files) to >utilize >this mechanism for virtual installation of port dependencies during >builds. > >The full project proposal can be seen at >https://docs.google.com/document/d/1B30U9csgY299W59tNraSX1LYjzsba2i04OrYAUpdIZs/edit > >. > >My goal is that this work can be integrated well enough into >/usr/ports/Mk so that unlike Jail, no set up work should be required >for >using ports tree to build a set of installable packages. > >Please let me know if you are interested in this project; feedback is >appreciated. If someone would like to provide ongoing feedback or >mentorship that would be especially helpful. Bakul Shah is my mentor >officially for GSoC but I would be happy to have additional support >from >someone who is experienced with internals of the port infrastructure >makefiles. > >Theron Tarigo >___ >freebsd-hack...@freebsd.org mailing list >https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >To unsubscribe, send any mail to >"freebsd-hackers-unsubscr...@freebsd.org" How is this different from what poudriiere already does? -- Pardon the typos and autocorrect, small keyboard in use. Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: GSoC: Separation of Ports Build Process from Local Installation
Am 29.05.19 um 00:51 schrieb Theron: > Hello All, > > For Google Summer of Code 2019 I am working on FreeBSD's ports tree makefiles > towards eliminating the dependency of the ports building process on the local > system's installed packages. Currently this level of separation can only be > accomplished in practice through chroot or Jail. The project will eliminate > the need for cooperation of the root user since /usr/local will not need to be > touched. > > The major technical obstacle to be overcome is that ports expect to find files > of their dependencies installed in /usr/local. To support this without > touching that location on the installed system, file accesses will be > redirected to a location controlled by the ports build process through use of > a library to intercept file accesses. > > Once I have that working (well enough to build one port at a time) I will move > on to modify bsd.port.mk itself (and related files) to utilize this mechanism > for virtual installation of port dependencies during builds. > > The full project proposal can be seen at > https://docs.google.com/document/d/1B30U9csgY299W59tNraSX1LYjzsba2i04OrYAUpdIZs/edit > . What's wrong with using chroot to provide a clean build environment? That is what synth does, and I have been using my re-implementation of portmaster for this purpose for some time, which uses a chroot jail with read-only null-mounts of all relevant file systems and a clean copy of some files and directories in /etc and /var that can be written without root privileges. The jail is set up in not measurable time (irrelevant compared to the time required to build the port). The only problem with this approach is that it requires extra disk space for the build environment (e.g., the specific C compiler required by some port) plus the work space for the actual port build process. I'm using tmpfs file systems within the jail for the work directory and the copies of parts of /etc and /var that need to be written to. Is there a risk of mis-use of the interception library to attack the system, BTW? [Its use is not restricted to root and it might be used to re-map file system paths for commands that check e.g. policy files to decide whether some operation is authorized ... SUID programs should not be vulnerable to such an attack (since they do not allow the library pre-load required to intercept the file operations), but there might be application programs that are restricted by non-writable files in hard-coded directories that could be subverted this way ... (such a command would be ill-designed, since any user could compile her own interception library, but providing such a library with the system and possibly having hooks for it in libc might simplify such an attack, especially if there is no compiler and easy way to install such a library on a host).] > My goal is that this work can be integrated well enough into /usr/ports/Mk so > that unlike Jail, no set up work should be required for using ports tree to > build a set of installable packages. Yes, this might be beneficial. But there will be huge differences compared to the current build process. And in the end you'll probably have to put the logic used by, e.g., portmaster to track dependencies and determine the availability of up-to-date packages (to use as build dependencies) into the ports system. > Please let me know if you are interested in this project; feedback is > appreciated. If someone would like to provide ongoing feedback or mentorship > that would be especially helpful. Bakul Shah is my mentor officially for GSoC > but I would be happy to have additional support from someone who is > experienced with internals of the port infrastructure makefiles. I'd be interested to get further information about your approach and the progress you make and my experience working on a somewhat similar project with portmaster might allow me to answer questions or provide some help ... Regards, STefan ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: GSoC: Separation of Ports Build Process from Local Installation
On 29/05/2019 10:56, Peter Pentchev wrote:
Hmm, I could be wrong, but isn't ${LOCALBASE} supposed to be where
ports find stuff*during the build*, and ${PREFIX} where they
install the built files? Of course, I haven't actually touched
a FreeBSD ports build in years, so I might very likely be wrong.
I also seem to remember a series of test port builds done a long
time ago with a different value for LOCALBASE, specifically to catch
ports that do not honor the policy in this regard.
No, you are correct. The LOCALBASE setting is where the ports will look
for build dependencies. However, the problem here is there is no
guarrantee that the LOCALBASE value will not somehow be compiled into a
resulting package (eg. as RPATH for dynamically loading a shlib) so
would also be needed for runtime dependencies.
Being able to build with a custom value of LOCALBASE and then install
and run the resulting package onto a system using different value for
PREFIX would be an interesting piece of work. The applicability would
be people that custom build their own ports, but who are for some reason
unable or unwilling to set up some sort of clean-build environment.
This does strike me as an awful lot of work to solve a perceived problem
where we do already have some pretty good solutions in place, but not to
everybodies' taste.
Cheers,
Matthew
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: GSoC: Separation of Ports Build Process from Local Installation
On Tue, May 28, 2019 at 08:28:41PM -0700, Rodney W. Grimes wrote:
> [ Charset UTF-8 unsupported, converting... ]
> > Hello All,
> >
> > For Google Summer of Code 2019 I am working on FreeBSD's ports tree
> > makefiles towards eliminating the dependency of the ports building
> > process on the local system's installed packages.? Currently this level
> > of separation can only be accomplished in practice through chroot or
> > Jail.? The project will eliminate the need for cooperation of the root
> > user since /usr/local will not need to be touched.
> >
> > The major technical obstacle to be overcome is that ports expect to find
> > files of their dependencies installed in /usr/local.? To support this
> > without touching that location on the installed system, file accesses
> > will be redirected to a location controlled by the ports build process
> > through use of a library to intercept file accesses.
>
> Assumption of /usr/local was considered wrong long long ago and it
> should always be ${PREFIX}. Any place that actually assumes this
> value to be /usr/local should be fixed.
>
> Had this policy been properly maintained it would simply be a mater
> of changing ${PREFIX} to a new and empty place before starting
> a ports build and things should of just worked.
>
> Restoration to this ancient and functional behavior is desirable
> at least on my part.
Hmm, I could be wrong, but isn't ${LOCALBASE} supposed to be where
ports find stuff *during the build*, and ${PREFIX} where they
install the built files? Of course, I haven't actually touched
a FreeBSD ports build in years, so I might very likely be wrong.
I also seem to remember a series of test port builds done a long
time ago with a different value for LOCALBASE, specifically to catch
ports that do not honor the policy in this regard.
> > Once I have that working (well enough to build one port at a time) I
> > will move on to modify bsd.port.mk itself (and related files) to utilize
> > this mechanism for virtual installation of port dependencies during builds.
> >
> > The full project proposal can be seen at
> > https://docs.google.com/document/d/1B30U9csgY299W59tNraSX1LYjzsba2i04OrYAUpdIZs/edit
> >
> > .
> >
> > My goal is that this work can be integrated well enough into
> > /usr/ports/Mk so that unlike Jail, no set up work should be required for
> > using ports tree to build a set of installable packages.
> >
> > Please let me know if you are interested in this project; feedback is
> > appreciated.? If someone would like to provide ongoing feedback or
> > mentorship that would be especially helpful.? Bakul Shah is my mentor
> > officially for GSoC but I would be happy to have additional support from
> > someone who is experienced with internals of the port infrastructure
> > makefiles.
G'luck,
Peter
--
Peter Pentchev roam@{ringlet.net,debian.org,FreeBSD.org} p...@storpool.com
PGP key:http://people.FreeBSD.org/~roam/roam.key.asc
Key fingerprint 2EE7 A7A5 17FC 124C F115 C354 651E EFB0 2527 DF13
signature.asc
Description: PGP signature
GSoC: Separation of Ports Build Process from Local Installation
Hello All, For Google Summer of Code 2019 I am working on FreeBSD's ports tree makefiles towards eliminating the dependency of the ports building process on the local system's installed packages. Currently this level of separation can only be accomplished in practice through chroot or Jail. The project will eliminate the need for cooperation of the root user since /usr/local will not need to be touched. The major technical obstacle to be overcome is that ports expect to find files of their dependencies installed in /usr/local. To support this without touching that location on the installed system, file accesses will be redirected to a location controlled by the ports build process through use of a library to intercept file accesses. Once I have that working (well enough to build one port at a time) I will move on to modify bsd.port.mk itself (and related files) to utilize this mechanism for virtual installation of port dependencies during builds. The full project proposal can be seen at https://docs.google.com/document/d/1B30U9csgY299W59tNraSX1LYjzsba2i04OrYAUpdIZs/edit . My goal is that this work can be integrated well enough into /usr/ports/Mk so that unlike Jail, no set up work should be required for using ports tree to build a set of installable packages. Please let me know if you are interested in this project; feedback is appreciated. If someone would like to provide ongoing feedback or mentorship that would be especially helpful. Bakul Shah is my mentor officially for GSoC but I would be happy to have additional support from someone who is experienced with internals of the port infrastructure makefiles. Theron Tarigo ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
