Re: RUN_DEPENDS and portmaster
Am 19.09.18 um 14:25 schrieb Kurt Jaeger: > Hi! > >> Why is Ada only available on i386/amd64? > > Because nobody provided fixes for the build on other platforms up to now. And given that the only use of Ada in FreeBSD ports seems to be synth, it seems a lot easier to implement the synth functionality in some more portable (or at least readily available) language. There is nothing in Ada, that makes it specifically suitable for synth. The synth code consists mostly of string processing and program invocations. It uses Ada exception handling, but I did not spot anything that could not easily be implemented in any other language. In fact, due to the many invocations of external binaries, porting of synth to a shell seems sensible. I do not consider the time-outs on all the build phases a strict requirement, but even those could be implemented with shell mechanisms, The setup of the clean environment for the package builds is easy to extract and I have considered adding that feature to portmaster, to support building of ports that currently fail if a previous version is installed (generally caused by include paths that prefer installed headers over those in the sources of the new version). Regards, STefan ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
On Wed, 19 Sep 2018, the wise RW via freebsd-ports wrote: I presume that's because you use a restricted set of packages. Without flavor support portupgrade can't always get the port directory from the origin. The are currently 782 ports installed on this system. Didn't have much difficulties with flavors yet. That said "in the next few weeks" last year. I know but developers time is limited. Important thing is that portupgrade generally still works. -- Cats are intended to teach us that not everything in nature has a function. -- Garrison Keillor ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
On Wed, 19 Sep 2018 14:14:42 +0200 (CEST) Marco Beishuizen wrote: > On Wed, 19 Sep 2018, the wise Thomas Mueller wrote: > > > This discussion of portmaster prompts me to ask, what is the status > > of portupgrade? > > > > I used portupgrade at first but subsequently switched to > > portmaster. > > I still use portupgrade daily and it works fine. I presume that's because you use a restricted set of packages. Without flavor support portupgrade can't always get the port directory from the origin. > Afaik the maintainer > is adding support for flavors into portupgrade but I don't know the > status of it: > > https://lists.freebsd.org/pipermail/freebsd-ports/2017-December/111445.html That said "in the next few weeks" last year. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
Hi! > Why is Ada only available on i386/amd64? Because nobody provided fixes for the build on other platforms up to now. -- p...@freebsd.org +49 171 3101372 2 years to go ! ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
On Wed, 19 Sep 2018, the wise Thomas Mueller wrote: This discussion of portmaster prompts me to ask, what is the status of portupgrade? I used portupgrade at first but subsequently switched to portmaster. I still use portupgrade daily and it works fine. Afaik the maintainer is adding support for flavors into portupgrade but I don't know the status of it: https://lists.freebsd.org/pipermail/freebsd-ports/2017-December/111445.html Regards, Marco -- There are three reasons for becoming a writer: the first is that you need the money; the second that you have something to say that you think the world should know; the third is that you can't think what to do with the long winter evenings. -- Quentin Crisp ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
Excerpt from STefan Esser: > I have been using a portmaster-rewrite for many months, which is ready > for release except for some performance tuning. (The portmaster in ports > is not un-maintainable, but it's hard to modify a monolithic 4000 line > shell script that uses global variables to pass state and recursive > invocation of itself to provide local state when required.) > The performance problems are caused by bad design of the FLAVOR feature, > which ignored the requirements of tools like portmaster (I've written > about this at length when FLAVOR support had been committed). > Synth is a non-starter for me, since it is written in Ada and only > available on i386/amd64. I have plans to implement the functionality > of synth in portmaster (not really hard, since the complex parts are > the logic that deals with moved ports and conflicts, while the actual > port building is simple). Portmaster can already create packages > without installing them (unless they are BUILD_DEPENDS of some later > port, of course) and you can populate your local repository with > portmaster. > Different from poudriere or synth, portmaster adapts to the preferences > of the user (and e.g. upgrades samba48 used by some port that specifies > a dependency on samba46, if the system already has an outdated samba48 > installed). > Portmaster will use what's available on a system and does allow selective > upgrades (keeping some ports at a back-level on purpose, but still upgrade > other ports that depend on them), while a poudirere/pkg based upgrade will > typically require all dependencies to exactly match what was present at > the time the package was built (in a clean environment, not resembling the > system the packages are going to be installed on). Why is Ada only available on i386/amd64? I don't think gcc is so limited. Or is it an idiosyncrasy of Dragonlace? This discussion of portmaster prompts me to ask, what is the status of portupgrade? I used portupgrade at first but subsequently switched to portmaster. On this computer, synth crashes, sometimes making the system crash. But I am in NetBSD as I type this. Tom ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
Am 18.09.18 um 14:05 schrieb Matthias Fechner: > Dear Stefan, > > Am 17.09.2018 um 14:31 schrieb Stefan Esser: >> But the behavior of portmaster will not be changed. >> RUN_DEPENDS are dependencies required to run a port, not dependencies >> required to install a port. >> >> >> And I do not care whether bsd.port.mk treats RUN_DEPENDS as if they >> were INSTALL_DEPENDS (which do not exist). The fact that bsd.port.mk >> works in that way is due to the fact, that it generally executes sub >> processes in a depth first manner. Portmaster distinguishes build and >> run dependencies and makes sure, that build dependencies not only exist, >> but are updated before the ports they depend on, while bsd.port.mk will >> use any build dependency that satisfies the range requirements (if any) >> and does not upgrade existing but outdated (in the sense that an upgrade >> is available) dependencies. Portmaster will then upgrade any out-dated >> run dependencies (again if an upgrade is available, not only if it is >> strictly required). Thus portmaster guarantees, that a port is built >> with the latest available build tools, and that run dependency upgrades >> see the upgraded port that requires them, in case they depend on it. > > I fully understand you. > > Maybe it will be a good idea to phase portmaster out as it seems to be a > unmaintable beast? > > Maybe synth can replace it for users that are not used to poudriere? I have been using a portmaster-rewrite for many months, which is ready for release except for some performance tuning. (The portmaster in ports is not un-maintainable, but it's hard to modify a monolithic 4000 line shell script that uses global variables to pass state and recursive invocation of itself to provide local state when required.) The performance problems are caused by bad design of the FLAVOR feature, which ignored the requirements of tools like portmaster (I've written about this at length when FLAVOR support had been committed). Synth is a non-starter for me, since it is written in Ada and only available on i386/amd64. I have plans to implement the functionality of synth in portmaster (not really hard, since the complex parts are the logic that deals with moved ports and conflicts, while the actual port building is simple). Portmaster can already create packages without installing them (unless they are BUILD_DEPENDS of some later port, of course) and you can populate your local repository with portmaster. Different from poudriere or synth, portmaster adapts to the preferences of the user (and e.g. upgrades samba48 used by some port that specifies a dependency on samba46, if the system already has an outdated samba48 installed). Portmaster will use what's available on a system and does allow selective upgrades (keeping some ports at a back-level on purpose, but still upgrade other ports that depend on them), while a poudirere/pkg based upgrade will typically require all dependencies to exactly match what was present at the time the package was built (in a clean environment, not resembling the system the packages are going to be installed on). Both, portmaster and poudriere/pkg have their use-cases, and there is only a partial overlap. There are quite a number of portmaster users, and they use it since their use-cases are not well covered by other tools. The way the ports system handles installs (in that it installs RUN_DEPENDS before the port that needs them) is an artifact of the way Makefiles naturally work, i.e. of the tool the ports system is based on. I do not understand why you intend on having RUN_DEPENDS cause installation of dependencies before your port. If you need those dependencies before the port is installed, then they are not really run dependencies, but dependencies of your particular build process. Portmaster has worked for far more than a decade with >20,000 ports. I do not see that your single port that expects run dependencies to be available before the installation has completed is reasonable cause to change the way portmaster works with dependencies. It pre-dates the "new" ports framework by far, and it used to be common practice, that changes respect established practices. BTW: Your port-install target will not be run when installing from a package (or building a package) anyway. Portmaster will take care of providing the required dependencies, as will pkg. So what's the reasoning that this test in do-install is required at all? You already specify exact versions in the RUN_DEPENDS, which are checked by the ports framework. Portmaster will take care, that all these ports are re-built to the latest level (hopefully satisfying the version test) after gitlab-ce has been installed. If you use pkg, the test is performed at install time, too. Are there any PRs due to lack of that test? Regards, STefan ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To
Re: RUN_DEPENDS and portmaster
Dear Stefan, Am 17.09.2018 um 14:31 schrieb Stefan Esser: > But the behavior of portmaster will not be changed. > RUN_DEPENDS are dependencies required to run a port, not dependencies > required to install a port. > > > And I do not care whether bsd.port.mk treats RUN_DEPENDS as if they > were INSTALL_DEPENDS (which do not exist). The fact that bsd.port.mk > works in that way is due to the fact, that it generally executes sub > processes in a depth first manner. Portmaster distinguishes build and > run dependencies and makes sure, that build dependencies not only exist, > but are updated before the ports they depend on, while bsd.port.mk will > use any build dependency that satisfies the range requirements (if any) > and does not upgrade existing but outdated (in the sense that an upgrade > is available) dependencies. Portmaster will then upgrade any out-dated > run dependencies (again if an upgrade is available, not only if it is > strictly required). Thus portmaster guarantees, that a port is built > with the latest available build tools, and that run dependency upgrades > see the upgraded port that requires them, in case they depend on it. I fully understand you. Maybe it will be a good idea to phase portmaster out as it seems to be a unmaintable beast? Maybe synth can replace it for users that are not used to poudriere? Gruß Matthias -- "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning." -- Rich Cook ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
Am 17.09.18 um 07:47 schrieb Matthias Fechner: > Am 10.09.18 um 12:16 schrieb Mathieu Arnold: >> Reading Mk/bsd.port.mk at line 5274, run-depends are installed before >> do-install runs. > > thanks, I see it the same way and created a PR for it, to get this fixed > in portmaster. You are of course free to create a PR against portmaster. But the behavior of portmaster will not be changed. RUN_DEPENDS are dependencies required to run a port, not dependencies required to install a port. And I do not care whether bsd.port.mk treats RUN_DEPENDS as if they were INSTALL_DEPENDS (which do not exist). The fact that bsd.port.mk works in that way is due to the fact, that it generally executes sub processes in a depth first manner. Portmaster distinguishes build and run dependencies and makes sure, that build dependencies not only exist, but are updated before the ports they depend on, while bsd.port.mk will use any build dependency that satisfies the range requirements (if any) and does not upgrade existing but outdated (in the sense that an upgrade is available) dependencies. Portmaster will then upgrade any out-dated run dependencies (again if an upgrade is available, not only if it is strictly required). Thus portmaster guarantees, that a port is built with the latest available build tools, and that run dependency upgrades see the upgraded port that requires them, in case they depend on it. I have spent hundreds of hours to work around the bad design of the FLAVOR support, which ignored the requirements of tools like portmaster or portupgrade. Changes to the port infrastructure tend to ignore the existence and requirements of build tools that have a decade long history and use cases not covered by the port infrastructure alone. I'm not going to spend any time on a change that made portmaster install RUN_DEPENDS before executing "make install" for a port. You are free to create a patch to that effect, but be aware that it is extremely likely to break lots of upgrade scenarios, and I'll make you responsible for fixing them (or back-out your assumed patch that treats run dependencies as if they were build dependencies). Regards, STefan signature.asc Description: OpenPGP digital signature
Re: RUN_DEPENDS and portmaster
Am 10.09.18 um 12:16 schrieb Mathieu Arnold: > Reading Mk/bsd.port.mk at line 5274, run-depends are installed before > do-install runs. thanks, I see it the same way and created a PR for it, to get this fixed in portmaster. Gruß, Matthias -- "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning." -- Rich Cook signature.asc Description: OpenPGP digital signature
Re: RUN_DEPENDS and portmaster
Am 12.09.18 um 07:58 schrieb Matthias Fechner: > Dear Stefan, Dear Mathieu, > > Am 10.09.18 um 14:10 schrieb Stefan Esser: >> This is a design decision in portmaster that has existed for at least >> a decade. >> >> Use INSTALL_DEPENDS if you depend on a port being available and upgraded >> before your port's do-install is invoked. >> >> Changing the current portmaster version in ports is no option, since it >> does not offer to recursively upgrade or install any other port while >> working on some port and it cannot easily be made to support such a >> sequence of actions. > > thanks a lot for your explanation, so it seems to be a problem with > portmaster. > But as I do not want to block all users from using gitlab-ce that are > using portmaster I think it is ok to define all RUN_DEPENDS also as > BUILD_DEPENDS? Yes, adding them to BUILD_DEPENDS will cause all those ports to be built and installed by portmaster before the port that executes those tests. I had thought there also was INSTALL_DEPENDS, but now I see that in fact there only is INSTALLS_DEPENDS, which is used internally in bsd.port.mk. So, BUILD_DEPENDS is the variable to use in that case. Regards, STefan ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
Dear Stefan, Dear Mathieu, Am 10.09.18 um 14:10 schrieb Stefan Esser: > This is a design decision in portmaster that has existed for at least > a decade. > > Use INSTALL_DEPENDS if you depend on a port being available and upgraded > before your port's do-install is invoked. > > Changing the current portmaster version in ports is no option, since it > does not offer to recursively upgrade or install any other port while > working on some port and it cannot easily be made to support such a > sequence of actions. thanks a lot for your explanation, so it seems to be a problem with portmaster. But as I do not want to block all users from using gitlab-ce that are using portmaster I think it is ok to define all RUN_DEPENDS also as BUILD_DEPENDS? I think this approach is already used by many ports... Gruß, Matthias -- "Programming today is a race between software engineers striving to build bigger and better idiot-proof programs, and the universe trying to produce bigger and better idiots. So far, the universe is winning." -- Rich Cook ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
Am 10.09.18 um 06:54 schrieb Matthias Fechner:
> Dear all,
>
> I have a questions reagarding the RUN_DEPENDS and at which step
> dependencies should be installed.
> I'm the maintainer of gitlab-ce port and I added a feature that checks
> in the install target:
> do-install:
> (cd ${WRKSRC} && ${RM} Gemfile.lock && bundle install --local)
> ${FIND} ${WRKSRC} -name '*.orig' -delete
> ...
>
> that all gems available in the system do match the version required by
> the Gemfile.
> Poudriere works fine with this, but portmaster fails.
> Regarding the documentation RUN_DEPENDS packages should be installed
> before the install is happening.
Hi Matthias,
this is the description of the sequence of actions performed by the
ports framework alone.
INSTALLS_DEPENDS covers the case of dependencies that are required
to be available when a port is being installed.
Portmaster installs RUN_DEPENDS only after the port that depends on
them, since it is assumed, that they are actually only required to
execute it after it has been completely installed.
The reason is, that portmaster is typically used to upgrade multiple
ports in such a way, that all BUILD_DEPENDS are up to date (not only
available) when some some dependent port is compiled.
In case that some upgraded port actually is a build dependency of
some other port, it will need to have its run dependencies installed
and upgraded, and portmaster will assure this is the case.
The sequence of upgrade actions was chosen to follow this scheme to
prevent dependency loops (which typically will consist of a mix of
build and run dependencies).
> Is this install related to the do-install target or the installation
> of the package itself?
I have re-implemented portmaster and have been using my version to
maintain my ports since May. Due to several bad design decisions (or
rather the lack of thorough design) of the FLAVOR feature, it took me
quite some effort and time to get performance of that version to an
acceptable level.
Currently I'm building and installing ports in the same order as the
current official portmaster version, but that could easily be changed.
I have considered following the same concept as synth does (i.e. build
ports in a clean environment), at least as an option. But I have not
started to implement such a feature, yet.
> Is this maybe a problem with portmaster as poudiere handles this correctly?
This is a design decision in portmaster that has existed for at least
a decade.
Use INSTALL_DEPENDS if you depend on a port being available and upgraded
before your port's do-install is invoked.
Changing the current portmaster version in ports is no option, since it
does not offer to recursively upgrade or install any other port while
working on some port and it cannot easily be made to support such a
sequence of actions.
Regards, STefan
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: RUN_DEPENDS and portmaster
On Mon, Sep 10, 2018 at 06:54:19AM +0200, Matthias Fechner wrote:
> Dear all,
>
> I have a questions reagarding the RUN_DEPENDS and at which step
> dependencies should be installed.
Reading Mk/bsd.port.mk at line 5274, run-depends are installed before
do-install runs.
> I'm the maintainer of gitlab-ce port and I added a feature that checks
> in the install target:
> do-install:
> (cd ${WRKSRC} && ${RM} Gemfile.lock && bundle install --local)
> ${FIND} ${WRKSRC} -name '*.orig' -delete
> ...
>
> that all gems available in the system do match the version required by
> the Gemfile.
> Poudriere works fine with this, but portmaster fails.
> Regarding the documentation RUN_DEPENDS packages should be installed
> before the install is happening.
> Is this install related to the do-install target or the installation of
> the package itself?
>
> Is this maybe a problem with portmaster as poudiere handles this correctly?
It's probably a problem with portmaster not doing clean builds.
--
Mathieu Arnold
signature.asc
Description: PGP signature
RUN_DEPENDS and portmaster
Dear all,
I have a questions reagarding the RUN_DEPENDS and at which step
dependencies should be installed.
I'm the maintainer of gitlab-ce port and I added a feature that checks
in the install target:
do-install:
(cd ${WRKSRC} && ${RM} Gemfile.lock && bundle install --local)
${FIND} ${WRKSRC} -name '*.orig' -delete
...
that all gems available in the system do match the version required by
the Gemfile.
Poudriere works fine with this, but portmaster fails.
Regarding the documentation RUN_DEPENDS packages should be installed
before the install is happening.
Is this install related to the do-install target or the installation of
the package itself?
Is this maybe a problem with portmaster as poudiere handles this correctly?
Thanks.
Gruß,
Matthias
--
"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the universe trying to
produce bigger and better idiots. So far, the universe is winning." --
Rich Cook
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
