Re: Starting with poudriere

[Miroslav Lachman]

Jose Quinteiro wrote on 2020/02/19 20:35:

On 2020-02-15 20:43, Dan McGrath wrote:



Probably not ideal since you generally want to disable the FreeBSD
repository, and use only your poudriere repo, instead. You would need to
build everything you intend to install in the jail, however. While I
believe that you can enable multiple repositories (FreeBSD's, and your 
own
poudriere one), I am not sure about repo priorities, or how you would 
deal

with conflicts with build options that pull in common ports. It is
something I have been meaning to look into, sorry! Perhaps someone else
here can give some advice?

You can set priorities for remote package repositories in the repo's 
conf file. The default priority is 0, which is lowest. I plan to set up 
a repo with the ports I build with poudriere at a higher priority. Are 
there concerns with this approach?


I am using 2 or 3 repositories (all built with our poudriere) for some 
time. It works but the problem can be if some repo has different default 
versions. For example one repo has PostgreSQL 9.6 as default version for 
all dependencies and another repo has PostgreSQL 10. Then you need to 
carefully choose what to install from what repo.


Miroslav Lachman


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[Jose Quinteiro]

On 2020-02-15 20:43, Dan McGrath wrote:

On Sat, Feb 15, 2020 at 11:03 PM @lbutlr  wrote:


Let’s say I want to build and install a single port via poudrier. For the
same of argument some port that has configuration options I want to change.



Probably not ideal since you generally want to disable the FreeBSD
repository, and use only your poudriere repo, instead. You would need to
build everything you intend to install in the jail, however. While I
believe that you can enable multiple repositories (FreeBSD's, and your own
poudriere one), I am not sure about repo priorities, or how you would deal
with conflicts with build options that pull in common ports. It is
something I have been meaning to look into, sorry! Perhaps someone else
here can give some advice?

You can set priorities for remote package repositories in the repo's 
conf file. The default priority is 0, which is lowest. I plan to set up 
a repo with the ports I build with poudriere at a higher priority. Are 
there concerns with this approach?


Thanks,
Jose
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[Baptiste Daroussin]

On Mon, Feb 17, 2020 at 07:55:25AM -0800, George Hartzell wrote:
> Baptiste Daroussin writes:
>  > 
>  > You should really have a look at overlays which are supported in
>  > poudriere-devel, it will allow you to get rid of portshaker with your use 
> case:
>  > 
>  > https://reviews.freebsd.org/rP510950
>  > 
>  > in poudriere an overlay is just a "regular ports tree" appended via -O 
> during
>  > the bulk.
> 
> Neat, thanks for pointing that out!
> 
> How much of a "regular ports tree" does it need to be?  Can it just
> include my local ports or does it need other elements of the tree
> (e.g. it's own index files or )?
> 
It can be your actual ports tree :D

plus the SUBDIR

Best regards,
Bapt


signature.asc
Description: PGP signature

Re: Starting with poudriere

[George Hartzell]

Baptiste Daroussin writes:
 > 
 > You should really have a look at overlays which are supported in
 > poudriere-devel, it will allow you to get rid of portshaker with your use 
 > case:
 > 
 > https://reviews.freebsd.org/rP510950
 > 
 > in poudriere an overlay is just a "regular ports tree" appended via -O during
 > the bulk.

Neat, thanks for pointing that out!

How much of a "regular ports tree" does it need to be?  Can it just
include my local ports or does it need other elements of the tree
(e.g. it's own index files or )?

g.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[Baptiste Daroussin]

On Sun, Feb 16, 2020 at 10:18:33AM -0800, George Hartzell wrote:
> Dan McGrath writes:
>  > [...] I am not sure about repo priorities, or how you would deal
>  > with conflicts with build options that pull in common ports. It is
>  > something I have been meaning to look into, sorry! Perhaps someone else
>  > here can give some advice?
>  >
> 
> One way to solve this is via "portshaker", which can layer a "thin"
> ports tree on top of the standard tree.
> 
> Here's a [perhaps not entirely graceful, but It Works For Me] example
> where I layer a couple of ports onto the standard tree.
> 
> https://github.com/hartzell/freebsd-ports
> 
> I use the resulting tree for poudriere builds and populate jails with
> e.g. my LMS audio system.
> 

You should really have a look at overlays which are supported in
poudriere-devel, it will allow you to get rid of portshaker with your use case:

https://reviews.freebsd.org/rP510950

in poudriere an overlay is just a "regular ports tree" appended via -O during
the bulk.

Best regards,
Bapt


signature.asc
Description: PGP signature

Re: Starting with poudriere

[George Hartzell]

@lbutlr writes:
 > [...]
 > Not sure I quite get how the Webserver lets your other machines get
 > the packages in such a way that they can be dropped in place, [...]

Nicely described in the DigitalOcean Tutorial[do] 

But basically, you put your tree full of built packages somewhere
where they're accessible via http[s] and the you configure the pkg
system on the client machines to use that url instead of
pkg.freebsd.org (or whatever the standard is).

Other things are possible, you could put them on a file server, mount
them via NFS onto the client and configure the pkg system to use
file:// URLS

g.


[do]: 
https://www.digitalocean.com/community/tutorials/how-to-set-up-a-poudriere-build-system-to-create-packages-for-your-freebsd-servers
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[George Hartzell]

Dan McGrath writes:
 > [...] I am not sure about repo priorities, or how you would deal
 > with conflicts with build options that pull in common ports. It is
 > something I have been meaning to look into, sorry! Perhaps someone else
 > here can give some advice?
 >

One way to solve this is via "portshaker", which can layer a "thin"
ports tree on top of the standard tree.

Here's a [perhaps not entirely graceful, but It Works For Me] example
where I layer a couple of ports onto the standard tree.

https://github.com/hartzell/freebsd-ports

I use the resulting tree for poudriere builds and populate jails with
e.g. my LMS audio system.

g.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[George Hartzell]

John Kennedy writes:
 > On Sat, Feb 15, 2020 at 09:02:39PM -0700, @lbutlr wrote:
 > > [...]
 > > Am I writing a config file for this every port I want to build?
 >
 > Personally, I have a single, custom make.conf that I maintain and shove into
 > /usr/local/etc/poudriere.d (default location I believe).
 >
 > Inside the make.conf, you can bracket non-default options like this:
 >
 >  .if ${.CURDIR:M*/ftp/curl}
 >  OPTIONS_FILE_UNSET +=   TLS_SRP
 >  .endif
 >

I managed these bits via ansible, using

https://github.com/reallyenglish/ansible-role-poudriere

g.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[George Hartzell]

@lbutlr writes:
 > On 15 Feb 2020, at 21:43, Dan McGrath  wrote:
 > > You would run "poudriere bulk", then sit back sipping coffee while it
 > > churns through all of the packages.
 >
 > Hang on a second, so the intended use for poudriere is to build ALL
 > packages?

That's not a poudriere constraint, it's a packages vs. ports thing.
And it's not ALL PACKAGES, it's all the packages you intend to use.

You can run a system using pkgs, because the packages are built from a
single consistent ports tree.  If you're going to build and install
things from ports, you should build and install *all the things* from
ports so that you can guarantee that everything's using consistent
[versions/configurations of] dependencies.  If/when you update the
ports tree, you should rebuild *all the things* [that have changed].

You can sometimes get away with installing one or two things from
ports, but eventually, sadness happens.

Populating your own package repository with pkgs that you build from
ports gives you the best of both worlds.  You can manage your system
using the `pkg` tool and/but you get to choose the versions and
configuration of the ports you want to use.

 > Right now I have two jails setup, one for 12.1 amd64 and one for
 > 11.3 i386. That seems like a *LOT* of compiling/building.

I used to be able to build everything I needed to build a
postfix/dovecot/etc... mailserver (loosely this:
https://www.c0ffee.net/blog/mail-server-guide/) in under an hour on a
2-core 4GB server at ARP networks.  Sadly I learned to love ripgrep,
so now I need to build rust, which takes ages and ages.

 > And once I’ve build, say, ImageMagick or
 > postfix/doveco/.=mariadb/apache/etc how do I then deploy them to
 > the 11.3 server (as in, a different machine)?

The digital ocean tutorial I pointed you at earlier explains this
nicely, in the "Configuring Package Clients" section.  Rather than
bother with nginx I just use:

s3cmd sync -F --delete-removed  
/usr/local/poudriere/data/packages/12_1-ports/.latest/ 
s3:///pkg-builder/packages/FreeBSD:12:amd64-ports/

and then perhaps an `s3cmd setacl ...` to set permissions as required.

g.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[Dan McGrath]

Hi,

On Sun, Feb 16, 2020 at 10:51 AM Grzegorz Junka  wrote:

> Just a note that this is not a strict requirement. I have been upgrading
> from FreeBSD 9 to 12 currently and was always building on the same
> system that I am deploying to. Yes, poudriere will complain that the
> jail is newer than the base system, but that did not create any major
> practical problem for me yet.
>
> I think only on one occasion I got a build error due to missing symbols.
> Then the solution for me was to upgrade the base system. This of course
> broke the applications that were installed for the older base, but
> thanks to the FreeBSD's separation of base from ports, it's still
> possible to start FreeBSD with just the command line. Then I finished
> building the ports and reinstalled them.
>
> Not that I encourage this approach, it might create additional issues to
> solve, but it is possible/manageable and shouldn't be held against using
> poudriere.
>

Ah, good to know. It's been a long time now since I ran into that, so I was
a little hazy on the details of the error.

I love the separation of FreeBSD from ports, and indeed, being able to
recover from broken userland is nice, although I hate the days where I
remove libs first, then try run sudo, and have to go in via BMC. Worse is
if you have a critical website down. Especially if you have to stop and
spend a bunch of time compiling ports before you go live! :D Also, tex and
llvm, wtf is with those build times?! heh

Anyway, thanks for clarifying!


Cheers,

Dan McGrath
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[Grzegorz Junka]

On 16/02/2020 00:14, Dan McGrath wrote:

Hi,

Just a bit of a heads up that poudriere will require you to be on the new
version of FreeBSD before you can build for it on the current system. For
example, if you are running 12.1, and you upgrade poudriere's jail to 13.0,
it will complain that you have to be running that version on the host you
are using it on. Ideally, poudriere should be running on it's own dedicated
system, not the one you intend to deploy to.


Just a note that this is not a strict requirement. I have been upgrading 
from FreeBSD 9 to 12 currently and was always building on the same 
system that I am deploying to. Yes, poudriere will complain that the 
jail is newer than the base system, but that did not create any major 
practical problem for me yet.


I think only on one occasion I got a build error due to missing symbols. 
Then the solution for me was to upgrade the base system. This of course 
broke the applications that were installed for the older base, but 
thanks to the FreeBSD's separation of base from ports, it's still 
possible to start FreeBSD with just the command line. Then I finished 
building the ports and reinstalled them.


Not that I encourage this approach, it might create additional issues to 
solve, but it is possible/manageable and shouldn't be held against using 
poudriere.


GrzegorzJ

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[Kurt Jaeger]

Hi!

> On 16 Feb 2020, at 02:02, Kurt Jaeger  wrote:
> > /usr/local/etc/poudriere.d/121-default-make.conf
> 
> And this conf file applies when you use the jail named exactly '121-default'

The 121-default is the name of the jail and the name of the ports tree
used.

You can have several ports trees in parallel:

$ poudriere ports -l
PORTSTREE METHOD   TIMESTAMP   PATH
default   portsnap 2020-02-16 10:02:15 /pou/ports/default
snmp  portsnap 2019-11-01 16:21:32 /pou/ports/snmp

I have one to test net/net-snmp 5.8 as well.

> So for my jail named 121x64 the conf file would be 
> 
> "121x64-make.conf"
> 
> And for 113x86
> 
> "113x86-make.conf"

Not quite, see above.

> Ok, this is starting to make some sense. Now I just need to get
> a list of the ports on the 11.3 system currently in the form
> / so I can at least set that up with the current
> ports.

The man page for pkg-query shows many options to ask for all this.

pkg query '%o'

gives you this list.

> Not sure I quite get how the Webserver lets your other machines
> get the packages in such a way that they can be dropped in place,
> but I don't need to do that right now anyway (that will be a task
> for when I have migrated that machine to 12.1 amd64 anyway), hopefully
> in the next month.

pkg accesses the repo, downloads some meta data (like meta.txz), and
compares that data with the packages you already have.

-- 
p...@opsec.eu+49 171 3101372Now what ?
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[@lbutlr]

On 16 Feb 2020, at 02:02, Kurt Jaeger  wrote:
> /usr/local/etc/poudriere.d/121-default-make.conf

And this conf file applies when you use the jail named exactly “121-default”? 
So for my jail named 121x64 the conf file would be 

"121x64-make.conf"

And for 113x86

"113x86-make.conf"

Ok, this is starting to make some sense. Now I just need to get a list of the 
ports pon the 11.3 system currently in the form / so I can 
at least set that up with the current ports.

Not sure I quite get how the Webserver lets your other machines get the 
packages in such a way that they can be dropped in place, but I don’t need to 
do that right now anyway (that will be a task for when I have migrated that 
machine to 12.1 amd64 anyway), hopefully in the next month.


-- 
"He has no enemies, but is intensely disliked by his friends.." Oscar
Wilde

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[Kurt Jaeger]

Hi!

> On 15 Feb 2020, at 21:43, Dan McGrath  wrote:
> > You would run "poudriere bulk", then sit back sipping coffee while it
> > churns through all of the packages.
> 
> Hang on a second, so the intended use for poudriere is to build ALL packages?

Only those you want, and automatically those, that are required to
build those.

For example, I have a file 'server', which is used to build
all the ports I need for servers.

---
archivers/arc
archivers/arj
archivers/cabextract
archivers/deb2targz
archivers/gtar
archivers/lha
archivers/libmspack
archivers/libzip
[...]
---

I use a small script:

---
#!/usr/local/bin/bash

poudriere ports -u

stmp=`date +%s`

cd ~/pkg
cat php74 server > all

poudriere bulk -T -f ~/pkg/all -j cur > ~/logs/cur-all-$stmp
---

to build those. My build host runs current, and it can build for
current and older versions:

poudriere jail -l lists my build systems:

JAILNAME VERSION  ARCH  METHOD  TIMESTAMP   
PATH
112  11.2-RELEASE amd64 ftp 2018-06-28 07:36:14 
/pou/jails/112
113  11.3-RELEASE amd64 ftp 2019-07-11 06:42:40 
/pou/jails/113
120  12.0-RELEASE-p9  amd64 ftp 2019-08-10 15:06:39 
/pou/jails/120
121  12.1-RELEASE amd64 ftp 2019-11-04 20:37:14 
/pou/jails/121
12i  12.1-RELEASE i386  ftp 2019-11-04 21:46:18 
/pou/jails/12i
arm  13.0-CURRENT 1300013 r344710 arm64.aarch64 svn+ssh 2019-03-02 14:35:50 
/pou/jails/arm
cur  13.0-CURRENT 1300054 r354278 amd64 svn+ssh 2019-11-03 13:17:47 
/pou/jails/cur
curi 13.0-CURRENT 1300054 r354278 i386  svn+ssh 2019-11-03 12:12:11 
/pou/jails/curi

> Right now I have two jails setup, one for 12.1 amd64 and one for 11.3 i386. 
> That seems like a *LOT* of compiling/building.

Yes, it's a lot of compiling. I only build our selection of ports
(approx. 2000 ports), and all it requirements, in sum it's approx. 3800
ports.

To be fair, I could probably strip a awful lot of the 2000 ports,
that number is that high because I did not find the time to prune
that list.

And: I do not build it every day, only one every week or so.

And: If I want to build a new port, I have a different script, to build
exactly that and its requirements:

---
#!/usr/local/bin/bash

echo $1 > ~/pkg/one
poudriere ports -u
poudriere bulk -T -f ~/pkg/one -j cur

---

Called like this:

bulkcurone shells/bash

it just builds bash, and updates bash and its requirements.

If that means to rebuild gmake, it will also rebuild gmake.

So after such a 'bulkone' action, if I update the target system using

pkg upgrade

it might upgrade only bash, or a few other packages, or a lot,
if I did not upgrade that target system for a long time.

Which is fine, because, for almost all cases, nothing breaks if
all that stuff is updated.

I can even test complex dependencies. I do this using jail-specific
make.conf files, for example:

/usr/local/etc/poudriere.d/121-default-make.conf

WARNING_WAIT=0
WITH_MPM=event
DEFAULT_VERSIONS= perl5=5.30 python=3.6 python3=3.6 ruby=2.6 pgsql=12 php=7.3 
mysql=10.4m gcc=9 samba=4.10

If I want to experiment with php 7.4, I change that field, rebuild
all (bulk121), wait a bit and everything is set.

> And once I've build, say, ImageMagick or
> postfix/dovecot/mariadb/apache/etc how do I then deploy them to the
> 11.3 server (as in, a different machine)?

I have a webserver running on the box, https://repo.opsec.eu/ (not
public) or on the company build host, https://repo.nepustil.net/ (public).

There, we have the ready-to-use package repos available, just as symlinks
into the poudriere directory tree.

On my target systems, I use:

mkdir -p /usr/local/etc/pkg/repos/
cat > nepustil.conf 

Re: Starting with poudriere

[@lbutlr]

On 15 Feb 2020, at 21:43, Dan McGrath  wrote:
> You would run "poudriere bulk", then sit back sipping coffee while it
> churns through all of the packages.

Hang on a second, so the intended use for poudriere is to build ALL packages?

Right now I have two jails setup, one for 12.1 amd64 and one for 11.3 i386. 
That seems like a *LOT* of compiling/building.

And once I’ve build, say, ImageMagick or postfix/doveco/.=mariadb/apache/etc 
how do I then deploy them to the 11.3 server (as in, a different machine)?


-- 
Cause love's such an old fashioned word
And love dares you to care for the people on the
Edge of the night, and love dares you to change
Our way of caring about ourselves, 
This is our last dance This is ourselves.
Under Pressure.
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[John Kennedy]

On Sat, Feb 15, 2020 at 09:02:39PM -0700, @lbutlr wrote:
> On 15 Feb 2020, at 16:32, @lbutlr  wrote:
> > Sorry for the rather basic questions.
> 
> Thanks everyone for your comments. One more dumb question I can???t find the 
> answer to.
> 
> Let???s say I want to build and install a single port via poudrier. For the 
> same of argument some port that has configuration options I want to change.
> 
> I have the ports tree setup, have my jail setup, and I want to build it.
> 
> And then I want to deploy it to the target machine once it???s built.
> 
> Or I want to deploy to to the local machine.
> 
> Let???s say, for fun, it???s something like ImageMafick that has a lot that 
> goes with it.
> 
> Am I writing a config file for this every port I want to build?

Personally, I have a single, custom make.conf that I maintain and shove into
/usr/local/etc/poudriere.d (default location I believe).

Inside the make.conf, you can bracket non-default options like this:

.if ${.CURDIR:M*/ftp/curl}
OPTIONS_FILE_UNSET +=   TLS_SRP
.endif

I try to leave things as default, generally, but libressl isn't the default and
things go a bit sideways from there some days.

Those options I pull from /var/db/ports after I do a "make config" (be sure to
clean out /var/db/ports when you're done).  Pay attention to the packages where
you change the defaults and put the minimum number of options in (for example,
you may need to set your new value and unset the old, default value).

Before that, I was trying to sync /var/db/ports across various machines and
that was way more messy (I think that was pre-poudriere, in the synth or even
portmaster days where things wanted to have config files, even if they were
only filled with the default values).
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[Dan McGrath]

Hi,

On Sat, Feb 15, 2020 at 11:03 PM @lbutlr  wrote:

> Let’s say I want to build and install a single port via poudrier. For the
> same of argument some port that has configuration options I want to change.
>

Probably not ideal since you generally want to disable the FreeBSD
repository, and use only your poudriere repo, instead. You would need to
build everything you intend to install in the jail, however. While I
believe that you can enable multiple repositories (FreeBSD's, and your own
poudriere one), I am not sure about repo priorities, or how you would deal
with conflicts with build options that pull in common ports. It is
something I have been meaning to look into, sorry! Perhaps someone else
here can give some advice?


> I have the ports tree setup, have my jail setup, and I want to build it.
>

You would run "poudriere bulk", then sit back sipping coffee while it
churns through all of the packages.


> And then I want to deploy it to the target machine once it’s built.
>

I personally make a web server on the jail to serve up the path to the
packages that poudriere built, generally available under:

/usr/local/poudriere/data/packages



> Or I want to deploy to to the local machine.
>

I suppose you could "bind mount" (see: man nullfs) the path into the jail,
or just specify the file path as a URI to the poudriere.conf pkg config.


> Let’s say, for fun, it’s something like ImageMafick that has a lot that
> goes with it.
>

Poudriere takes care of the dependencies. For example, if the _only_ thing
that you want in your jail is ImageMagick, as if you literally boot
strapped a blank jail, then ran "pkg install ImageMagick", then you would
only need to built that, and pkg, in poudriere. Your build set file would
probably look like this:

ports-mgmt/pkg
graphics/ImageMagick7-nox11


I would have to actually double check if pkg is even needed since I believe
it may be implicitly depended on, in which case it would already be built,
but I appear to have it specified in all of my sets anyway.


> Am I writing a config file for this every port I want to build?


You would create a file for all the leaf ports for the repository that you
want to create for your machines to use. I generally will do something like
install portmaster in a jail, then install all of the stuff I want, then
use something like "portmaster -L" (requires /usr/ports to be available) to
identify the root and leaf ports. Then I will place all of those files into
the file I pass (via -f) to poudriere to build. Poudriere will take care of
all the dependencies, and will leave you with a repo that has exactly
everything you need to install.

Anyway, hope this maybe helps a bit? Since flavours have came out, I have
been slowly trying to avoid using custom repos, and try migrate things to
stock upstream packages. While it is fun to tinker with, the amount of time
I have wasted over the years doing buildworld and building ports by hand
has made me start to look at more practical approaches, such as pkg's and
freebsd-update. Unless you have trust issues with upstream binaries, or
require unusual build options, I think it may not be worth using poudriere,
and perhaps better to convince upstream to add flavours? Just a thought!


Cheers,

Dan McGrath
___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[@lbutlr]

On 15 Feb 2020, at 16:32, @lbutlr  wrote:
> Sorry for the rather basic questions.

Thanks everyone for your comments. One more dumb question I can’t find the 
answer to.

Let’s say I want to build and install a single port via poudrier. For the same 
of argument some port that has configuration options I want to change.

I have the ports tree setup, have my jail setup, and I want to build it.

And then I want to deploy it to the target machine once it’s built.

Or I want to deploy to to the local machine.

Let’s say, for fun, it’s something like ImageMafick that has a lot that goes 
with it.

Am I writing a config file for this every port I want to build?


-- 
The fact is that camels are far more intelligent than dolphins.
Footnote: Never trust a species that grins all the time. It's up
to something. —Pyramids


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[George Hartzell]

@lbutlr writes:
 > [...]
 > Sorry for the rather basic questions.
 > 

I'm also a poudriere beginner and/but I found this DigitalOcean
tutorial very useful:

https://www.digitalocean.com/community/tutorials/how-to-set-up-a-poudriere-build-system-to-create-packages-for-your-freebsd-servers

g.

___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"

Re: Starting with poudriere

[Dan McGrath]

Hi,

Just a bit of a heads up that poudriere will require you to be on the new
version of FreeBSD before you can build for it on the current system. For
example, if you are running 12.1, and you upgrade poudriere's jail to 13.0,
it will complain that you have to be running that version on the host you
are using it on. Ideally, poudriere should be running on it's own dedicated
system, not the one you intend to deploy to.

As for how I use it here, I use salt to deploy to the poudriere machine,
with one repo per host/jail. It's been good, but things get very hairy with
this approach since all of the ports need to be recompiled whenever you
update the jail (say from 12.1-RELEASE-p1 to 12.1-RELEASE-p2). When you
start to have multiple port trees, multiple jails and multiple sets, the
permutations of this start to really kick you in the face :) I built on a
tiny old quad core VM, which makes matters worse.

In terms of how I actually use things, originally I started off doing
"poudriere options ...", and would go through pressing enter for every menu
config option, and for each repo (host/jail) that I have. This was a pain,
especially during quarterly changes where the on disk files in
/usr/local/etc/poudriere.d/ for each -- directory would
have to be copied to a new filename in order to start from the previous
settings, and then change what is new etc. Eventually I learned that I
could just erase all that, skip the "poudriere options", and specify the
defaults that I want/need to override in the
/usr/local/etc/poudriere.d/make.conf file. That I could specify options per
port, as needed with something like this:

OPTIONS_SET+= GSSAPI_NONE
OPTIONS_UNSET+= CUPS DBUS X11 DOCBOOK EXAMPLES KERBEROS PERFSCHEMA PERFSCHM
GSSAPI_BASE GSSAPI_HEIMDAL GSSAPI_MIT WAYLAND OPENGL XCB
DEFAULT_VERSIONS+= ssl=openssl
# Python
DEFAULT_VERSIONS+= python=3.7 python3=3.7
# Ensure apache uses event MPM or ZTS will disable and php extensions will
fail
# to build inside poduriere!
WITH_MPM=event
# Apache 2.4
www_apache24_UNSET+= MPM_PREFORK MPM_WORKER
www_apache24_SET+= MPM_EVENT
# Ensure ZTS
www_mod_php72_SET+= ZTS
lang_php72_SET+= ZTS


That should get you an idea how to setup the make.conf, at least. With this
approach, I don't have to deal with the headaches of going through millions
of menu options every time stuff changes. While I still need to migrate to
more of a "single repo" approach, it is nice having dedicated repos for
each jail, but it does mean that you are compiling the same stuff over and
over, so be warned! Interesting note was that I originally had ccache
installed in the system, and even with SSD's over network storage, was
finding that the I/O was actually adding to my compile times, so I just
disable it now.

As for your question of the file to use to build a bunch of ports, the
approach I use is a layout like this:

/poudriere/
/poudriere/113amd64
/poudriere/113amd64/2019Q4
/poudriere/113amd64/2019Q4/host_a
/poudriere/113amd64/2019Q4/host_b
/poudriere/113amd64/2020Q1
/poudriere/113amd64/2020Q1/host_a
/poudriere/113amd64/2020Q1/host_b
/poudriere/113amd64/bulk-2019Q4.sh
/poudriere/113amd64/bulk-2020Q1.sh
/poudriere/113amd64/clean-2019Q4.sh
/poudriere/113amd64/clean-2020Q1.sh
/poudriere/121amd64
/poudriere/121amd64/2019Q4
/poudriere/121amd64/2019Q4/host_a
/poudriere/121amd64/2019Q4/host_b
/poudriere/121amd64/2020Q1
/poudriere/121amd64/2020Q1/host_a
/poudriere/121amd64/2020Q1/host_b
/poudriere/121amd64/bulk-2019Q4.sh
/poudriere/121amd64/bulk-2020Q1.sh
/poudriere/121amd64/clean-2019Q4.sh
/poudriere/121amd64/clean-2020Q1.sh
/poudriere/bulk-all.sh
/poudriere/update-2019Q4.sh
/poudriere/update-2020Q1.sh


Where the bulk-all.sh script just runs the bulk-.sh and clean
scripts. As sample of one of them would be:

#!/bin/sh

#
# Because variables
#
JAIL="113amd64"
TREE="2020Q1"
LIST_PREFIX="/poudriere/${JAIL}/${TREE}"

# Bulk compile
for lcv in ${LIST_PREFIX}/*; do
echo "Bulk for $lcv..."
#read -p "Press ENTER to continue. " INPUT
nice poudriere bulk -j $JAIL -p $TREE -z $(basename ${lcv}) -f
"${lcv}"
done


Of course, the clean script would replace s/bulk/clean/, but you get the
idea. More of a prototype script than anything, but it "works", although
could most certainly be done better!

Hopefully this gives you a bit of insight and understanding into the
system, and possible approaches to setting things up. Best of luck! o/


Cheers,

Danny McGrath

On Sat, Feb 15, 2020 at 6:33 PM @lbutlr  wrote:

> I’ve setup a new FreeBSD 12.1 system and am going to give poudriere a
> whirl on it.
>
> The machine is currently setup as a two drive zfs mirror and I have used
> pkg to install a few basic things (sudo, zsh, poudriere itself, etc).
>
> I am reading through <
> https://wiki.freebsd.org/VladimirKrstulja/Guides/Poudriere#A3._Create_the_list_of_ports_wanted_in_the_repo>
> and I have some questions.
>
> First, the example of says that if we don’t change any options, the
> defaults will be used. OK, but it 

Starting with poudriere

[@lbutlr]

I’ve setup a new FreeBSD 12.1 system and am going to give poudriere a whirl on 
it.

The machine is currently setup as a two drive zfs mirror and I have used pkg to 
install a few basic things (sudo, zsh, poudriere itself, etc).

I am reading through 

 and I have some questions.

First, the example of says that if we don’t change any options, the defaults 
will be used. OK, but it makes no mention on how we might change or set 
options? Is this simply talking about the build options panels that sometimes 
come up (Build examples, support ssl, use sqlite or MySQL, etc), and when I 
first go to build a “persistent list” those option screens will come up and I 
can make changes and they will be saved? Or is there something else that I need 
to do?

Second, let's say that I want to setup the machine to build postfix, dovecot, 
apache, pigeonhole, marisadb, postfixadmin, rpoundcube, and horde all in on 
“persistent list” since all of these pacajkges are need to do mail and webmail, 
do I simply list the packages in a single file the way the page lists 
www/firefox?

Once I have poudriere setup and running and I’m comfortable with it, do I need 
to do anything to have it “take over” sudo and anything else that I installed 
via pkg?

Sorry for the rather basic questions.


-- 
"Are you pondering what I'm pondering?"
"I think so, Brain, but why would anyone want to see Snow White and
the Seven Samurai?”


___
freebsd-ports@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-ports
To unsubscribe, send any mail to "freebsd-ports-unsubscr...@freebsd.org"