Re: portmaster new development
Guido Falsi wrote on 2020/12/29 08:49: > This is intentional behaviour, hand there are good reasons for this. > > Anyway poudriere has the CHECK_CHANGED_DEPS option which can be disabled and > should restrict this behavior. I have never tested it though, I don't think > the risk of getting and incoherent repo is worth it. > Oh, thank you. However, CHECK_CHANGED_DEPS does not seem to change it's behavior. For example, if we have the following line [00:01:22] [Dry Run] Deleting harfbuzz-2.7.2_1.txz: new version: 2.7.4 gtk3 will be involved and removed. [00:01:26] [Dry Run] Deleting gtk3-3.24.24.txz: missing dependency: harfbuzz-2.7.2_1 When gtk3 is removed, this will also be removed. [00:01:26] [Dry Run] Deleting gtkspell3-3.0.10.txz: missing dependency: gtk3-3.24.24 [00:01:26] [Dry Run] Deleting webkit2-gtk3-2.30.2.txz: missing dependency: gtk3-3.24.24 webkit2-gtk3 also requires harfbuzz, so it must be built. orz But gtkspell3 doesn't seem to have anything to do with harfbuzz, so if we rebuild gtk3 by poudriere bulk -j jail -J 2 x11-toolkits/gtk30, we won't have to rebuild it. It's such a repetition. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
On 29/12/20 00:07, Tatsuki Makino wrote: Where should I hang out to reply? :) poudriere has weaknesses in updating packages such as libxml and glib. When run all at once, all packages that depend on the package being updated and all packages that depend on the package being removed will be removed. The text is not clear :), but qt5-webkit and webkit2-gtk3 are removed, and the time-consuming build begins. This is intentional behaviour, hand there are good reasons for this. Anyway poudriere has the CHECK_CHANGED_DEPS option which can be disabled and should restrict this behavior. I have never tested it though, I don't think the risk of getting and incoherent repo is worth it. -- Guido Falsi ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
To add to the extremes, it is enough to be able to topologically sort in some
way.
#!/bin/tcsh
foreach origin (`pkg version -vPL= | cut -f 1 -w | xargs pkg
some-topological-sort-command-can-someone-make-this -o`)
make -C /usr/ports/${origin:q} clean
make -C /usr/ports/${origin:q} build deinstall install clean
end
That should be enough. :)
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
Where should I hang out to reply? :) poudriere has weaknesses in updating packages such as libxml and glib. When run all at once, all packages that depend on the package being updated and all packages that depend on the package being removed will be removed. The text is not clear :), but qt5-webkit and webkit2-gtk3 are removed, and the time-consuming build begins. To prevent packages that do not need to be removed from being removed, I manually do the following. poudriere ports -u -v poudriere bulk -j jail -J 2 -S -f pkglist.txt ||: poudriere bulk -j jail -J 2 -n -f pkglist.txt # visually look for packages that have broken dependencies with the updated package here. poudriere bulk -j jail -J 2 originfor/dependencymissingpackage1 originfor/dependencymissingpackage2 ... :|| ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
Am 28.12.20 um 22:07 schrieb Michael Grimm:>> On 28. Dec 2020, at 21:41, Stefan Esser wrote: Poudriere works best on sufficiently powerful build servers and it often requires rebuilding dependencies over hours when I just want to test a new port before committing it. Excuse me, but that is not true in this generality. I do run poudriere on ZFS in a cloud instance with 7G of memory and a CPU of 2.1 MHz. Never had to wait for more then one hour for recompiling *all* of my 240+ ports for STABLE-12. YMMV, yes, but stating that in that totality is nonsense, at least IMHO. My build host is much bigger and I often have to build dependencies over night before I can test-build a new port with poudriere. Maybe your ports do not have as many big dependencies, but building LLVM and GCC as a dependency for 3 release versions takes its time and whenever these ports have been updated I cannot run "poudriere test" for my new port before the compilers are updated. If there are no complex dependencies, you are right, but MESA, KDE (or even Qt5) and all ports that need a specific compiler version that is receiving updates cause excessive delays before a port can be test built with poudriere. So yes, YMMV, but it depends on the complexity of the dependencies. And I test with different options in the different jails (e.g. with/without DOCS, one with non-standard PREFIX, etc.) for a better test coverage. Therefore I cannot fetch pre-compiled compilers and other dependencies to speed up my port tests. And this might be typical for poudriere users. If you do not want to build with non-default options you are better served by using pre-compiled packages. Regards, STefan OpenPGP_signature Description: OpenPGP digital signature
Re: portmaster new development
Hi > On 28. Dec 2020, at 21:41, Stefan Esser wrote: > > Poudriere works best on sufficiently powerful build servers and it > often requires rebuilding dependencies over hours when I just want to > test a new port before committing it. Excuse me, but that is not true in this generality. I do run poudriere on ZFS in a cloud instance with 7G of memory and a CPU of 2.1 MHz. Never had to wait for more then one hour for recompiling *all* of my 240+ ports for STABLE-12. YMMV, yes, but stating that in that totality is nonsense, at least IMHO. Regards, Michael ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
Am 28.12.20 um 15:04 schrieb abi via freebsd-ports: On 28.12.2020 16:16, Stefan Esser wrote: Am 28.12.20 um 11:11 schrieb abi via freebsd-ports:> I build my ports in poudriere in VM without zfs or ssd on pre-Sandy Bridge CPU. I don't have enough memory or disk space, so I don't use tmpfs or ccache either. I migrated from portmaster when it was abandoned several years ago and don't think I'll come back, especially if new portmaster will be written on bash. The idea behind portmaster was zero dependencies, so it doesn't brake after major upgrades. You are free to use poudriere and it definitely is the official tool for FreeBSD package building (and I have to use it myself and it has cost me a lot of time rebuilding broken poudriere jails and keeping them in state that I can use them to test new ports on a number of different releases as well as i386 plus amd64). And while you are free to never again use portmaster, telling people that it has been abandoned is just a _lie_ and I'd want to ask you to stop telling it. It has been continuously maintained for decades. I remember portmaster marked as deprecated in 2016. I've switched to poudriere because of that. So, it _was_ abandoned when I migrated. It is good that it is not, the more options - the better. But some people here telling that poudriere requires ZFS and powerful dedicated hardware, I just pointed that they are wrong. The portmaster port had been marked deprecated by the author of synth and this commit has been reverted by him on request of portmgr on the next day. The portmaster port has never been abandoned - it does not suffice that some unrelated committer considers his port management tool superior and decides to deprecate a "competing" one. Poudriere works best on sufficiently powerful build servers and it often requires rebuilding dependencies over hours when I just want to test a new port before committing it. My goal is to have portmaster build ports in a clean jail as synth does (restricted to i386 and amd64 due to the dependency on Ada) and poudriere do, but without giving up the ease of use of portmaster for other use cases. Regards, STefan OpenPGP_signature Description: OpenPGP digital signature
Re: portmaster new development
On 12/27/20 4:00 AM, LuMiWa via freebsd-ports wrote: > The subject is 'portmaster new development' but again start pushing > poudriere to FreeBSD users. I do not use zfs file system and I do not > use poudriere and I do not want to use on my computer for building some > ports and then spending hours and hours with poudriere with not enough > machine. For me is portmaster perfect as is now. More tools and improvements to tools can be a good thing. There are some things that portmaster did better and others that portupgrade did better. Can't remember which for each task but I think I used portmaster for configure dialogs and fetching as it would perform those steps before builds started. I found myself using portmaster to create a list of ports I had installed (portmaster --list-origins > ~/installed-port-list) to then feed into poudriere as a list to build though pkg prime-origins completes in a small fraction of a second instead of more than 5 seconds. I've dealt with a number of ports that fail to build because of a conflict (usually not noted). May have been avahi or a different port sounding similar in my head but one would regularly fail to upgrade for me if it was currently installed because the build found and used tools from the old version during the build of the new even after it spent time building the new tools in the work directory. People probably think those issues are rare in the ports tree but back around 2009-2011 I upgraded a x86 to x64 by upgrading in place. Running portupgrade lead to many failures due to ports using locally installed 32bit tools and libraries even though in theory everything should have been rebuilt/reinstalled in an orderly fashion where it would be replaced before it gets used. I've thought about repeating that to document findings but unless other developers want to look into the issues then there is no point as I have not been able to fix those type of build issues when I've tried. Deleting any installed package before its own port rebuild would work around many of those issues I saw but you really need to uninstall all nondependent ports to guarantee an individual port build succeeds. If uninstalling a port does not remove everything (bad pkg-plist) it created then there is still a chance of a contaminated build. Some builds will break if you have nondependencies installed and other times it may just modify what you get from building it as the program detects its surroundings and makes findings the port maintainer didn't account for when the build autodetects surroundings with things like autotools which didn't have the maintainer override or add a dependency for every possible optional manual detection. Remove port(s), rebuild, and reinstall once built is higher downtime than a pkg upgrade of already built ports. That is made worse when portmaster aborts on an unrelated dependency tree branch causing a complete build halt mid-upgrade. Portupgrade's "continue building all others on fail" leaves less programs broken without interaction but still higher than if it rolled back the entire dependency tree of a failed port. Building in a jail gets around the issue of needing a separate build machine, removing installed packages, and having program downtime while waiting for the chain of it+dependencies to all be rebuilt (incomplete build of program+dependencies leads to broken programs if I cannot resolve). That time would be extended when I would have days worth of building to do and portmaster would stop building within hours or minutes of starting and I won't see results for hours later when I get home from work or wake up. I completed very little with portmaster before going back to portupgrade back in the day because abort on failure rarely errored out on the first or last of a dependency chain and stopping in the middle without rolling back the entire chain would mean the update attempt generally broke things other than the one failed port. Installing from packages after all building is done offers the shortest downtime of programs on the system. A poudriere build stays in its own area and keeps it clean as it goes so those problems go away. Other solutions including a portmaster/portupgrade building in a jail should be able to do so too. Poudriere can be configured to not use excessive resources by controlling number of different ports to build at once, how many threads to use per port build, and how much to use memory disks vs real disks for workdirs, ccache, etc. Poudriere will take longer due to always starting with a clean environment, being more aggressive about deciding when a port needs a rebuild, and time spent building/updating the base system to be used inside the jail. Other jailed build systems will likely perform faster due to less cleanup per package. Poudriere defaults performing a full build for my machine while I use my machine is a bad experience but had controls to adjust settings to all be more san
Re: portmaster new development
On 2020-12-28 05:16, Stefan Esser wrote: Am 28.12.20 um 11:11 schrieb abi via freebsd-ports:> I build my ports in poudriere in VM without zfs or ssd on pre-Sandy Bridge CPU. I don't have enough memory or disk space, so I don't use tmpfs or ccache either. I migrated from portmaster when it was abandoned several years ago and don't think I'll come back, especially if new portmaster will be written on bash. The idea behind portmaster was zero dependencies, so it doesn't brake after major upgrades. You are free to use poudriere and it definitely is the official tool for FreeBSD package building (and I have to use it myself and it has cost me a lot of time rebuilding broken poudriere jails and keeping them in state that I can use them to test new ports on a number of different releases as well as i386 plus amd64). And while you are free to never again use portmaster, telling people that it has been abandoned is just a _lie_ and I'd want to ask you to stop telling it. It has been continuously maintained for decades. The next version will not be using bash but LUA, which is highly portable and does not have problematic dependencies. I'm well aware that a pure shell script has its advantages, but bringing down the time to scan for updates on my system from 300 to less than 10 seconds (for > 2000 installed packages) combined with the ability to build ports in a clean jail might make it an attractive choice for current users of the /bin/sh based version. The lovely thing about options, is that *everyone* gets to have one. The more the merrier! :-) @Stefan Really excited to hear about your LUA version of portmaster. Thanks! :-) --Chris Regards, STefan ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
On Mon, Dec 28, 2020 at 6:04 AM abi via freebsd-ports < freebsd-ports@freebsd.org> wrote: > On 28.12.2020 16:16, Stefan Esser wrote: > > Am 28.12.20 um 11:11 schrieb abi via freebsd-ports:> I build my ports > > in poudriere in VM without zfs or ssd on pre-Sandy > >> Bridge CPU. I don't have enough memory or disk space, so I don't use > >> tmpfs or ccache either. I migrated from portmaster when it was > >> abandoned several years ago and don't think I'll come back, > >> especially if new portmaster will be written on bash. The idea behind > >> portmaster was zero dependencies, so it doesn't brake after major > >> upgrades. > > > > You are free to use poudriere and it definitely is the official tool > > for FreeBSD package building (and I have to use it myself and it has > > cost me a lot of time rebuilding broken poudriere jails and keeping > > them in state that I can use them to test new ports on a number of > > different releases as well as i386 plus amd64). > > > > And while you are free to never again use portmaster, telling people > > that it has been abandoned is just a _lie_ and I'd want to ask you to > > stop telling it. It has been continuously maintained for decades. > > > I remember portmaster marked as deprecated in 2016. I've switched to > poudriere because of that. So, it _was_ abandoned when I migrated. It is > good that it is not, the more options - the better. But some people here > telling that poudriere requires ZFS and powerful dedicated hardware, I > just pointed that they are wrong. > I'm afraid your recollection is in error. While there was a lot of talk of deprecating it after Doug stopped supporting it. there was only discussion. Check the history in svn and it simply never happened. Getting rid of it was heavily pushed by a few people of the "poudreire is the solution to all port updating issues and all other should be removed" stripe, but it is and was never the solution for many even though it is a great tool and should be considered the "standard" method of ports management. If Stefan had not stepped up, deprecation would have had to happen as bitrot would have required it. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkober...@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
On 28.12.2020 16:16, Stefan Esser wrote: Am 28.12.20 um 11:11 schrieb abi via freebsd-ports:> I build my ports in poudriere in VM without zfs or ssd on pre-Sandy Bridge CPU. I don't have enough memory or disk space, so I don't use tmpfs or ccache either. I migrated from portmaster when it was abandoned several years ago and don't think I'll come back, especially if new portmaster will be written on bash. The idea behind portmaster was zero dependencies, so it doesn't brake after major upgrades. You are free to use poudriere and it definitely is the official tool for FreeBSD package building (and I have to use it myself and it has cost me a lot of time rebuilding broken poudriere jails and keeping them in state that I can use them to test new ports on a number of different releases as well as i386 plus amd64). And while you are free to never again use portmaster, telling people that it has been abandoned is just a _lie_ and I'd want to ask you to stop telling it. It has been continuously maintained for decades. I remember portmaster marked as deprecated in 2016. I've switched to poudriere because of that. So, it _was_ abandoned when I migrated. It is good that it is not, the more options - the better. But some people here telling that poudriere requires ZFS and powerful dedicated hardware, I just pointed that they are wrong. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
Am 28.12.20 um 11:11 schrieb abi via freebsd-ports:> I build my ports in poudriere in VM without zfs or ssd on pre-Sandy Bridge CPU. I don't have enough memory or disk space, so I don't use tmpfs or ccache either. I migrated from portmaster when it was abandoned several years ago and don't think I'll come back, especially if new portmaster will be written on bash. The idea behind portmaster was zero dependencies, so it doesn't brake after major upgrades. You are free to use poudriere and it definitely is the official tool for FreeBSD package building (and I have to use it myself and it has cost me a lot of time rebuilding broken poudriere jails and keeping them in state that I can use them to test new ports on a number of different releases as well as i386 plus amd64). And while you are free to never again use portmaster, telling people that it has been abandoned is just a _lie_ and I'd want to ask you to stop telling it. It has been continuously maintained for decades. The next version will not be using bash but LUA, which is highly portable and does not have problematic dependencies. I'm well aware that a pure shell script has its advantages, but bringing down the time to scan for updates on my system from 300 to less than 10 seconds (for > 2000 installed packages) combined with the ability to build ports in a clean jail might make it an attractive choice for current users of the /bin/sh based version. Regards, STefan OpenPGP_signature Description: OpenPGP digital signature
Re: portmaster new development
On 28.12.2020 12:44, David Gessel wrote: Original Message Subject: Re: portmaster new development From: LuMiWa via freebsd-ports To: freebsd-ports@freebsd.org Date: 2020-12-27 02:00+0300 On Sun, 27 Dec 2020 11:16:23 +0100 Michael Grimm wrote: Matthias Apitz wrote: El día domingo, diciembre 27, 2020 a las 09:22:42a. m. +0100, Kurt Jaeger escribió: That works as well. I have a checkout of the ports tree, use make config to define non-default port options. This stores the selected OPTIONs in /var/db/ports/, and poudriere uses those options just fine. Re/ the options, I copy them into the jail with something like this procedure: # cd /usr/ports/mail/mutt # make config # mkdir -p /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt # cp /var/db/ports/mail_mutt/options /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt 'freebsd-head' is the name of the poudriere jail (I have some of them) and the ports options stay there, as well the make.conf options in /usr/local/etc/poudriere.d/freebsd-head-make.conf I am following stable, and my jail's name has been set to stable. All of poudriere's settings/configs are kept in: /usr/local/etc/poudriere.d The subject is 'portmaster new development' but again start pushing poudriere to FreeBSD users. I do not use zfs file system and I do not use poudriere and I do not want to use on my computer for building some ports and then spending hours and hours with poudriere with not enough machine. For me is portmaster perfect as is now. I have to agree, portmaster works for certain user cases where poudriere doesn't, like mine. The answer seems to be just (buy) a high end machine and dedicate it to build with lots of RAM, high end CPU's, and a big ZFS array with the right combination of SSDs etc and it is fast and stable! I build my ports in poudriere in VM without zfs or ssd on pre-Sandy Bridge CPU. I don't have enough memory or disk space, so I don't use tmpfs or ccache either. I migrated from portmaster when it was abandoned several years ago and don't think I'll come back, especially if new portmaster will be written on bash. The idea behind portmaster was zero dependencies, so it doesn't brake after major upgrades. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
Original Message Subject: Re: portmaster new development From: LuMiWa via freebsd-ports To: freebsd-ports@freebsd.org Date: 2020-12-27 02:00+0300 On Sun, 27 Dec 2020 11:16:23 +0100 Michael Grimm wrote: Matthias Apitz wrote: El día domingo, diciembre 27, 2020 a las 09:22:42a. m. +0100, Kurt Jaeger escribió: That works as well. I have a checkout of the ports tree, use make config to define non-default port options. This stores the selected OPTIONs in /var/db/ports/, and poudriere uses those options just fine. Re/ the options, I copy them into the jail with something like this procedure: # cd /usr/ports/mail/mutt # make config # mkdir -p /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt # cp /var/db/ports/mail_mutt/options /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt 'freebsd-head' is the name of the poudriere jail (I have some of them) and the ports options stay there, as well the make.conf options in /usr/local/etc/poudriere.d/freebsd-head-make.conf I am following stable, and my jail's name has been set to stable. All of poudriere's settings/configs are kept in: /usr/local/etc/poudriere.d The subject is 'portmaster new development' but again start pushing poudriere to FreeBSD users. I do not use zfs file system and I do not use poudriere and I do not want to use on my computer for building some ports and then spending hours and hours with poudriere with not enough machine. For me is portmaster perfect as is now. I have to agree, portmaster works for certain user cases where poudriere doesn't, like mine. The answer seems to be just (buy) a high end machine and dedicate it to build with lots of RAM, high end CPU's, and a big ZFS array with the right combination of SSDs etc and it is fast and stable! While I'm sure that's true, that's not consistent with everyone's environment. I'm reminded of many client-server applications that are developed by people on gigabit fiber and seem to consider the "edge" case of the rest of the world on spotty internet not worthy of consideration, complaints merely whiny carping by people who should just lift their internet up by the bootstraps. I've run a server with a set of jails providing services for about 20 years. Maintaining them with portsnap and portmaster works and is efficient and functional and an efficient and practical use of the compute resources I have. Adding new and potentially better tools has been a pleasure of the community, but abandoning users always going to create friction and dismissing another's use case as "doing it wrong" is a great way to create animosity and dysfunction. The first wave of poudriereism was very annoying and offputting, but in the last year I've been delighted by the excellent and very responsive work of port maintainers to resolve issues quickly and cleanly and those of us still doing it the "old way" can still do so successfully. It'll be annoying and a little disruptive to lose the excellent tool that portsnap has been all these years, truly one of the brilliant, focused, and tremendously useful tools in the FreeBSD kit, but we'll figure out how to keep things working. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
On Sat, Dec 26, 2020 at 07:23:59PM -0800, Thomas Mueller wrote: > ... An improved portmaster arouses my interest. Maybe modify the name so it > can be added to the ports tree and coexist with the "official" portmaster. > Desired features/options would be to keep going rather than stop when one > port fails to build, and the ability to install build dependencies, which may > be useful for building other software. > > With synth, I had a difficult time getting everything that was built to > install, some packages like bison are needed in building other software. > > How is poudriere in that regard? I never used poudriere, have been > intimidated by not wanting to use zfs or dialog4ports, or such an elaborate > setup just to update one or a few ports. I'm a dinosaur, so I was a fan of old portmaster for quite some time. I'd say it's downfall was incompatible build dependencies, since it didn't build them in a jail. As packages got more complex and port-count went up, lots of breakage (at least outside of poudriere). I tried poudriere, didn't quite get my head around it and found synth. The original synth worked for me for quite some time, but it's downfall was the dependency on gcc6-aux (I think for ADA support?) when I got off the beaten path into ARM and didn't want to cross-compile. I had a raspberry pi and I really wanted it to be able to recompile itself natively. I think there were plans for a csynth (rewritten in C, as I understood it) to fix that "issue" but by then I'd taken a second shot at figuring out poudriere and moved on. I think the new portmaster was in the works by then, but I'd reported enough build issues at that point to be wary of non-poudriere build issues. The author seemed to be hacking away at my original portmaster issues, I just didn't want to wait. My $.02? Don't worry about build dependencies, make them a target and you'll be fine. I've got devel/gmake, devel/m4 and devel/bison in mine. I'm a fan of ZFS, so that isn't a showstopper for me. Do what works for you, so long as it handles incompatible build dependencies right. Using jail(s) is one way to do that, but I'm pretty sure others have used just plain old chroot() to pull it off. Poudriere (and synth) tended to rebuild EVERYTHING that depended on some- thing that changed. That was the safe behavior, but boy did that hurt when people kept on making more and more compilers a dependency. The original portmaster didn't, but occasionally had problems when it didn't. I got in the habit of periodically recompiling everything. I've said "incompatible build dependency" a few times. I don't have a real, old example but if A depends on B and C depends on D and you want A & B but C can't co-exist with D, you've got issues. I'd run into that with programs that ran on top of X having incompatible build deps, but not an issue if built in jails where they didn't have to be on the disk at the same time as with original portmaster. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
Am 27.12.20 um 06:04 schrieb Thomas Mueller: And as portsnap user I have a question: Do they planning deprecation of portmaster too? No, I'm actively working on portmaster and have rewritten it from scratch for better performance (and additional features, e.g. building in a clean chroot jail, similar to synth or poudriere). I have been using that version for more than one year, but the functionality is not complete, yet. On a test system with > 2200 installed ports it takes less than 10 seconds to identify the ~600 out-of-date ports (that I keep in this state for testing of the upgrade strategy function), which is more than 30 times faster than the same operation with the "official" portmaster. Until completion of that version, I'll continue to maintain and update the current portmaster port ... Regards, STefan Question about the relation of portsnap and portmaster reminds me of Java and Javascript, or potato and sweet potato (not closely related). Yes, portsnap and portmaster do not share anything beyond the first 4 letters of their names ... Since my question is about a new portmaster, I rename the subject to "portmaster" or "portmaster new development", rather than hijack the "portsnap" thread. Which portmaster do I get if I build and install what is currently in the ports tree? This is a version that I became maintainer of when it lacked flavor support and the original developer (Doug Barton) had left the project. amelia2# ls -l ports-mgmt/portmaster total 16 -rw-r--r-- 1 root wheel 1479 Dec 27 02:01 Makefile -rw-r--r-- 1 root wheel 184 Feb 28 2018 distinfo drwxr-xr-x 2 root wheel 512 Dec 27 02:01 files -rw-r--r-- 1 root wheel 1189 May 6 2019 pkg-descr from a fresh svn up of the ports tree. Yes, I added a feature requested by Kevin Obermann, yesterday, who had noticed that a locked port could be built but not installed, if the user answers "yes" to the question whether the lock should be ignored. An improved portmaster arouses my interest. Maybe modify the name so it can be added to the ports tree and coexist with the "official" portmaster. After taking over the current portmaster I noticed that it is ancient in its structure. It used global variables throughout and forked a lot if sub-processes to allow for local state. Since it is extremely cumbersom to work on a linear 4000 line shell script, I re-wrote it from scratch using bash associative arrays and used that version since the summer of 2019. That version did already support clean builds in a chroot jail as an option. But I noticed that better data structures would allow to substantially speed-up the scanning phase and decided to port that bash based version to LUA. It can be found in my GH repo, but I'm currently reworking the planning phase to allow for multiple pre-decessors of a port (e.g. if A-v1 is integrated into B-v2 and B-v1 is already installed). My current (not pushed) development version locks up due to such a case (deskutils/kdepim-apps-libs has been merged into net/akonadi-contacts and akonadi-contacts is downlevel, leading two an attempt to get an exclusive lock on the target package name twice, since the two update tasks are not currently merged into one). But other than that it has been usable for simple updates for a long time already. Desired features/options would be to keep going rather than stop when one port fails to build, and the ability to install build dependencies, which may be useful for building other software. Yes, I mark failed updates as such and cancel planned updates of dependencies, but unrelated updates go on. With synth, I had a difficult time getting everything that was built to install, some packages like bison are needed in building other software. The bash version supported 4 modes: traditional portmaster, delayed installation of packages not depended on in the update run, jailed builds with installation of packages after completion of all builds and repository mode to just create packages in a jail. I ran into issues when I tried to optimize the disk space used for the jail by de-installing no longer required build dependencies from the jail as soon as possible. I needed better data structures than offered by bash to efficiently express these dependencies. That was the point where I decided to migrate the code to LUA. How is poudriere in that regard? I never used poudriere, have been intimidated by not wanting to use zfs or dialog4ports, or such an elaborate setup just to update one or a few ports. I have to use poudriere to test ports before commit, but I do not like it. The build jails are quickly becoming stale and have to be rebuilt (causing all previously built packages to be compiled again) and I often run into issues where a build dependency fails for reasons that ar enot obvious (e.g. gmake) and I have found that the easiest method of recov
Re: portmaster new development
On Sun, 27 Dec 2020 11:16:23 +0100 Michael Grimm wrote: > Matthias Apitz wrote: > > El día domingo, diciembre 27, 2020 a las 09:22:42a. m. +0100, Kurt > > Jaeger escribió: > > >> That works as well. I have a checkout of the ports tree, use > >> make config to define non-default port options. This stores the > >> selected OPTIONs in /var/db/ports/, and poudriere uses those > >> options just fine. > > > Re/ the options, I copy them into the jail with something like this > > procedure: > > > > # cd /usr/ports/mail/mutt > > # make config > > > > # mkdir -p /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt > > # cp /var/db/ports/mail_mutt/options > > /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt > > > > 'freebsd-head' is the name of the poudriere jail (I have some of > > them) and the ports options stay there, as well the make.conf > > options in /usr/local/etc/poudriere.d/freebsd-head-make.conf > > > I am following stable, and my jail's name has been set to stable. > > All of poudriere's settings/configs are kept in: > > /usr/local/etc/poudriere.d > The subject is 'portmaster new development' but again start pushing poudriere to FreeBSD users. I do not use zfs file system and I do not use poudriere and I do not want to use on my computer for building some ports and then spending hours and hours with poudriere with not enough machine. For me is portmaster perfect as is now. -- “Waiter! A cup of coffee without cream, please! I’m sorry, sir, we have no cream, only milk, so can it be a coffee without milk?” ― Ernst Lubitsch’s Ninotchka ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
Matthias Apitz wrote: > El día domingo, diciembre 27, 2020 a las 09:22:42a. m. +0100, Kurt Jaeger > escribió: >> That works as well. I have a checkout of the ports tree, use >> make config to define non-default port options. This stores the >> selected OPTIONs in /var/db/ports/, and poudriere uses those options >> just fine. > Re/ the options, I copy them into the jail with something like this > procedure: > > # cd /usr/ports/mail/mutt > # make config > > # mkdir -p /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt > # cp /var/db/ports/mail_mutt/options > /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt > > 'freebsd-head' is the name of the poudriere jail (I have some of them) > and the ports options stay there, as well the make.conf options in > /usr/local/etc/poudriere.d/freebsd-head-make.conf I am following stable, and my jail's name has been set to stable. All of poudriere's settings/configs are kept in: /usr/local/etc/poudriere.d Whenever I do feel the need for non-default port options, I do use poudriere's functionality to define and store those options: poudriere options -n -j stable mail/postfix These non-default options are then put into: /usr/local/etc/poudriere.d/stable-options/mail_postfix This way one can keep separate options for different jails in one space. But, there are obviously different approaches possible ;-) Regards, Michael ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
El día domingo, diciembre 27, 2020 a las 09:22:42a. m. +0100, Kurt Jaeger escribió: > Hi! > > > How is poudriere in that regard? I never used poudriere, have been > > intimidated by not wanting to use zfs or dialog4ports, or such an elaborate > > setup just to update one or a few ports. > > poudriere is really, really useful. Because it delivers a complete, > consistent package repo of all the ports (with individual options). > And it does so repeatable, down to one or several seperate os versions > to build for or a specific point in time for the ports tree, > with some individual options etc. > > > I found that poudriere uses dialog4ports; I much prefer to save options in > > a file such as Gentoo Linux does with make.conf and (NetBSD) pkgsrc does > > with mk.conf . > > That works as well. I have a checkout of the ports tree, use > make config to define non-default port options. This stores the > selected OPTIONs in /var/db/ports/, and poudriere uses those options > just fine. > > Combined with lots of RAM, ZFS on SSDs and ccache, compilation is very fast. Hi, For some years I do use poudriere too on a very fast rack unit with lots of RAM, CPUs and ZFS. Re/ the options, I copy them into the jail with something like this procedure: # cd /usr/ports/mail/mutt # make config # mkdir -p /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt # cp /var/db/ports/mail_mutt/options /usr/local/etc/poudriere.d/freebsd-head-options/mail_mutt 'freebsd-head' is the name of the poudriere jail (I have some of them) and the ports options stay there, as well the make.conf options in /usr/local/etc/poudriere.d/freebsd-head-make.conf matthias -- Matthias Apitz, ✉ g...@unixarea.de, http://www.unixarea.de/ +49-176-38902045 Public GnuPG key: http://www.unixarea.de/key.pub ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
On 27 Dec, Kurt Jaeger wrote: > Hi! > >> How is poudriere in that regard? I never used poudriere, have been >> intimidated by not wanting to use zfs or dialog4ports, or such an >> elaborate setup just to update one or a few ports. > > poudriere is really, really useful. Because it delivers a complete, > consistent package repo of all the ports (with individual options). > And it does so repeatable, down to one or several seperate os versions > to build for or a specific point in time for the ports tree, > with some individual options etc. > >> I found that poudriere uses dialog4ports; I much prefer to save >> options in a file such as Gentoo Linux does with make.conf and >> (NetBSD) pkgsrc does with mk.conf . > > That works as well. I have a checkout of the ports tree, use > make config to define non-default port options. This stores the > selected OPTIONs in /var/db/ports/, and poudriere uses those options > just fine. I set the options in /usr/local/etc/poudriere.d/make.conf. It is easier to set options consistently across ports, for instance enabling CUPS globally, than with dialog4ports, while also allowing per-port option setting. It is also possible to have multiple option settings for different OS versions, arches, or option sets contained in different make.conf files. ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
Re: portmaster new development
Hi! > How is poudriere in that regard? I never used poudriere, have been > intimidated by not wanting to use zfs or dialog4ports, or such an elaborate > setup just to update one or a few ports. poudriere is really, really useful. Because it delivers a complete, consistent package repo of all the ports (with individual options). And it does so repeatable, down to one or several seperate os versions to build for or a specific point in time for the ports tree, with some individual options etc. > I found that poudriere uses dialog4ports; I much prefer to save options in a > file such as Gentoo Linux does with make.conf and (NetBSD) pkgsrc does with > mk.conf . That works as well. I have a checkout of the ports tree, use make config to define non-default port options. This stores the selected OPTIONs in /var/db/ports/, and poudriere uses those options just fine. Combined with lots of RAM, ZFS on SSDs and ccache, compilation is very fast. -- p...@opsec.eu+49 171 3101372Now what ? ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
portmaster new development
> > And as portsnap user I have a question: Do they planning deprecation of > > portmaster too? > No, I'm actively working on portmaster and have rewritten it from > scratch for better performance (and additional features, e.g. building > in a clean chroot jail, similar to synth or poudriere). > I have been using that version for more than one year, but the > functionality is not complete, yet. > On a test system with > 2200 installed ports it takes less than 10 > seconds to identify the ~600 out-of-date ports (that I keep in this > state for testing of the upgrade strategy function), which is more > than 30 times faster than the same operation with the "official" > portmaster. > Until completion of that version, I'll continue to maintain and > update the current portmaster port ... > Regards, STefan Question about the relation of portsnap and portmaster reminds me of Java and Javascript, or potato and sweet potato (not closely related). Since my question is about a new portmaster, I rename the subject to "portmaster" or "portmaster new development", rather than hijack the "portsnap" thread. Which portmaster do I get if I build and install what is currently in the ports tree? amelia2# ls -l ports-mgmt/portmaster total 16 -rw-r--r-- 1 root wheel 1479 Dec 27 02:01 Makefile -rw-r--r-- 1 root wheel 184 Feb 28 2018 distinfo drwxr-xr-x 2 root wheel 512 Dec 27 02:01 files -rw-r--r-- 1 root wheel 1189 May 6 2019 pkg-descr from a fresh svn up of the ports tree. An improved portmaster arouses my interest. Maybe modify the name so it can be added to the ports tree and coexist with the "official" portmaster. Desired features/options would be to keep going rather than stop when one port fails to build, and the ability to install build dependencies, which may be useful for building other software. With synth, I had a difficult time getting everything that was built to install, some packages like bison are needed in building other software. How is poudriere in that regard? I never used poudriere, have been intimidated by not wanting to use zfs or dialog4ports, or such an elaborate setup just to update one or a few ports. Gentoo Linux with portage has "--with-bdeps=y" which installs build dependencies when desired. I found that poudriere uses dialog4ports; I much prefer to save options in a file such as Gentoo Linux does with make.conf and (NetBSD) pkgsrc does with mk.conf . I once got a royal mess of circular/jumbled dependencies with dialog4ports; cleaning was a major nuisance, nothing simple like editing /etc/mk.conf or /etc/make.conf . I would like to be free of dialog4ports; the older dialog was worse and messed up my screen. Tom ___ freebsd-ports@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-ports To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"
