Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Guido Günther]

Hi,
On Wed, Dec 08, 2021 at 02:45:24PM -0800, Quanah Gibson-Mount wrote:
> 
> 
> --On Wednesday, December 8, 2021 12:15 PM -0800 Quanah Gibson-Mount
>  wrote:
> 
> > --On Wednesday, December 8, 2021 11:03 AM -0800 Quanah Gibson-Mount
> >  wrote:
> > 
> > > That got me farther, but then I ran into a known bug that OTHERMIRROR is
> > > broken.
> > > 
> > >  > > l>
> > 
> > 
> > So override config has an annoying behavior of resetting everything,
> > rather than overriding just what you specify.  I've been able to get this
> > to work now using the repo hook as noted in the bug description.  I'll
> > need a config per branch + distribution, which is annoying, but at least
> > I can get it down to a single system. :)
> 
> 
> So the config override seems to caused some sort of bug in gbp buildpackage
> (or lower?).  Builds now no longer include the source orig.tar.gz file.  I
> workaround this by passing the "-sa" argument to gbp buildpackage and can
> work around it by adding DEBBUILDOPTS=-sa to the profile.  However, the man
> page says nothing about the deb build opts being reset by pointing at a
> different profile.

gbp itself is pretty much out of the equation it mostly just passed on
to pbuilder.

What you can always do (and what I've done in the past to keep chroots
apart) is create a chroot with

 DIST=focal gbp-pbuilder 

and then keep that chroot for a special purpose 'purpose':

 mv /var/cache/pbuilder/base-focal.cow/ 
/var/cache/pbuilder/base-focal-purpose.cow/

and from there on use

 gbp buildpackage --git-dist=focal-purpose

to build in that chroot. This might be simpler than messing with
different pbuilder configurations. Updating the chroot etc still works
if you pass DIST=focal-purpose from there on. 

As a last resort you can even make all kinds of modifications in
/var/cache/pbuilder/base-focal-puprpose.cow/ like adding files to
/var/cache/pbuilder/base-focal-purpose.cow/etc/apt/sources.lists.d - i
wouldn't do that - just mentioning in so there's always a way to fix
things up.

As a related note: gbp is not tied to pbuilder you can also use sbuild
as --git-builder which is more maintained these days.

Cheers,
 -- Guido
___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage

Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Quanah Gibson-Mount]

--On Wednesday, December 8, 2021 12:15 PM -0800 Quanah Gibson-Mount 
 wrote:



--On Wednesday, December 8, 2021 11:03 AM -0800 Quanah Gibson-Mount
 wrote:


That got me farther, but then I ran into a known bug that OTHERMIRROR is
broken.





So override config has an annoying behavior of resetting everything,
rather than overriding just what you specify.  I've been able to get this
to work now using the repo hook as noted in the bug description.  I'll
need a config per branch + distribution, which is annoying, but at least
I can get it down to a single system. :)



So the config override seems to caused some sort of bug in gbp buildpackage 
(or lower?).  Builds now no longer include the source orig.tar.gz file.  I 
workaround this by passing the "-sa" argument to gbp buildpackage and can 
work around it by adding DEBBUILDOPTS=-sa to the profile.  However, the man 
page says nothing about the deb build opts being reset by pointing at a 
different profile.



From the man page:


  --override-config
 Specify to use different apt set up inside the chroot than it 
was used for creating the base.tgz.  Specify this when you want to do 
pbuilder update with a

 different distribution target set up.

 --distribution , --components , --mirror , --othermirror 
options are only valid when --override-config option is  specified  in 
update  target,  or  when

 pbuilder create is being called.




Regards,
Quanah



--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage

Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Quanah Gibson-Mount]

--On Wednesday, December 8, 2021 11:03 AM -0800 Quanah Gibson-Mount 
 wrote:



That got me farther, but then I ran into a known bug that OTHERMIRROR is
broken.





So override config has an annoying behavior of resetting everything, rather 
than overriding just what you specify.  I've been able to get this to work 
now using the repo hook as noted in the bug description.  I'll need a 
config per branch + distribution, which is annoying, but at least I can get 
it down to a single system. :)


--Quanah

--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage

Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Quanah Gibson-Mount]

--On Wednesday, December 8, 2021 10:05 AM -0800 Quanah Gibson-Mount 
 wrote:





--On Wednesday, December 8, 2021 9:10 AM -0800 Quanah Gibson-Mount
 wrote:


The following packages have unmet dependencies:
 pbuilder-satisfydepends-dummy : Depends: symas-buildtools which is a
virtual package and is not provided by any available package


Ah, I just saw this in the output:

W: --override-config is not set; not updating apt.conf Read the manpage
for details.


That got me farther, but then I ran into a known bug that OTHERMIRROR is 
broken.




--Quanah



--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage

Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Quanah Gibson-Mount]

--On Wednesday, December 8, 2021 9:10 AM -0800 Quanah Gibson-Mount 
 wrote:



The following packages have unmet dependencies:
 pbuilder-satisfydepends-dummy : Depends: symas-buildtools which is a
virtual package and is not provided by any available package


Ah, I just saw this in the output:

W: --override-config is not set; not updating apt.conf Read the manpage for 
details.


let me see about that.

--Quanah



--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage

Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Quanah Gibson-Mount]

--On Wednesday, December 8, 2021 9:06 AM -0800 Quanah Gibson-Mount 
 wrote:



So it appears that while the option is passed in, it doesn't actually do
anything in regards to it. :/


Since "othermirror" is also a valid option, I tried that in gbp.conf, but 
again it's not actually honored:


I: Invoking pbuilder
I: forking: pbuilder build --debbuildopts  --debbuildopts  --othermirror 
'deb http://apito1.rb.symas.net/repo/deb/main/ol25 buster main' 
--buildplace /var/cache/pbuilder/bui
ld/cow.81788 --buildresult /home/build/git/soldap/25/d10 --mirror 
http://us.archive.ubuntu.com/ubuntu --architecture amd64 --distribution 
devel --extrapackages ccache --no-t
argz --internal-chrootexec 'chroot /var/cache/pbuilder/build/cow.81788 
cow-shell' /home/build/git/soldap/25/d10/symas-openssl_1.1.1l-2buster1.dsc




The following packages have unmet dependencies:
pbuilder-satisfydepends-dummy : Depends: symas-buildtools which is a 
virtual package and is not provided by any available package


--Quanah



--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage

Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Quanah Gibson-Mount]

--On Wednesday, December 8, 2021 5:38 PM +0100 Guido Günther 
 wrote:



Hi Quanah,

I think you want

--git-pbuilder-options=

then which allow you to pass options through to pbuilder. There's an
example in the manpage that show how to use a different pbuilderrc. This
option can also be specified as `pbuilder-options` in gbp.conf.


Ok, I see the option get passed in:

I: forking: pbuilder build --debbuildopts  --debbuildopts  --configfile 
/home/build/.pbuilderrc.ol25 --buildplace 
/var/cache/pbuilder/build/cow.72185 --buildresult /home/bui
ld/git/soldap/25/d10 --mirror http://us.archive.ubuntu.com/ubuntu 
--architecture amd64 --distribution devel --extrapackages ccache --no-targz 
--internal-chrootexec 'chroot /
var/cache/pbuilder/build/cow.72185 cow-shell' 
/home/build/git/soldap/25/d10/symas-openssl_1.1.1l-2buster1.dsc


But it doesn't appear to actually honor the OTHERMIRROR setting:

cat ~/.pbuilderrc.ol25
OTHERMIRROR="deb [trusted=yes] 
http://apito1.rb.symas.net/repo/deb/main/ol25 buster main"



because it's unable to find the package:

The following packages have unmet dependencies:
pbuilder-satisfydepends-dummy : Depends: symas-buildtools which is a 
virtual package and is not provided by any available package



If I log in directly:
DIST=buster ARCH=amd64 git-pbuilder login

add it to the sources list:

root@ub20build:/# echo "deb [trusted=yes] 
http://apito1.rb.symas.net/repo/deb/main/ol25 buster main" >> 
/etc/apt/sources.list

root@ub20build:/# apt update
Get:1 http://apito1.rb.symas.net/repo/deb/main/ol25 buster InRelease [5717 
B]

Hit:2 http://http.us.debian.org/debian buster InRelease
Get:3 http://apito1.rb.symas.net/repo/deb/main/ol25 buster/main amd64 
Packages [484 B]

root@ub20build:/# apt install symas-buildtools
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
 symas-buildtools
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.


So it appears that while the option is passed in, it doesn't actually do 
anything in regards to it. :/


--Quanah





--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage

Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Guido Günther]

Hi Quanah,

On Wed, Dec 08, 2021 at 08:22:19AM -0800, Quanah Gibson-Mount wrote:
> That doesn't appear to do what I need.  What I need is:
> 
> pbuilder image buster1 -> has othermirror "REPOA"
> pbuilder image buster2 -> has othermirror "REPOB"
> 
> Currently I can set up a single system that has pbuilder's for buster,
> bullseye, bionic, and focal that all point at REPOA.  Then I can have a
> second system that has pbuilder's for buster, bullseye, bionic, and focal
> that all point at REPOB.
> 
> Alternatively, what would be best, is if I could pass (from gbp) an
> "othermirror" option like:
> 
> gbp buildpackage --othermirror="repoa"
>
> Even better, is if I could set the othermirror in gbp.conf, since I have a
> branch for REPOA builds and a branch for REPOB builds.  Then I could just
> run:

I think you want 

--git-pbuilder-options=

then which allow you to pass options through to pbuilder. There's an
example in the manpage that show how to use a different pbuilderrc. This
option can also be specified as `pbuilder-options` in gbp.conf.

Cheers,
 -- Guido

> 
> gbp buildpackage
> 
> from the appropriate branch, and the correct mirror would be added to the
> image just for use during the build, which is all I really need it do do.
> 
> I'm able to do something like this with git buildpackage-rpm, where I can
> point mock at custom images in the [buildpackage-rpm] section in .gbp.conf
> via the "dist" parameter.  This allows me to have one "dist" that uses repoA
> and another "dist" that uses repoB, meaning I can use a single server with
> mock installed to do all the builds.
> 
> --Quanah
> 
> --
> 
> Quanah Gibson-Mount
> Product Architect
> Symas Corporation
> Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
> 
> 
___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage

Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Quanah Gibson-Mount]

--On Wednesday, December 8, 2021 10:27 AM +0100 Guido Günther 
 wrote:



Hi Quanah,
On Tue, Dec 07, 2021 at 03:43:31PM -0800, Quanah Gibson-Mount wrote:

I have two versions of a piece of software I build for each distribution
I build on.  Currently I do this by having two different VMs with
git-buildpackage and pbuilder, and then create an image for each dist
(buster, bullseye, bionic, and focal) on each of them, but with a
different OTHERMIRRORS setting.

I'd like to get this down to a single server, but I can't find a way to
get pbuilder to either:

a) Use different OTHERMIRRORS at build time with the existing dist image

b) Have multiple images of a given dist with different OTHERMIRROR values

Anyone have any hints? :)


Given recent deboostrap (e.g. from Debian bookwork) you should be able
to use all of them from Debian:

For bullseye, buster (all Debian based):

DIST=buster git-pbuilder create
DIST=bullseye git-pbuilder create

For bionic, focal, ... (ubuntu based):

sudo apt install ubuntu-archive-keyring
DIST=bionic git-pbuilder create
--mirror=http://us.archive.ubuntu.com/ubuntu --components="main universe"
DIST=focal git-pbuilder create
--mirror=http://us.archive.ubuntu.com/ubuntu --components="main universe"

then e.g.:

gbp buildpackage --git-dist=bionic


Hi Guido,

That doesn't appear to do what I need.  What I need is:

pbuilder image buster1 -> has othermirror "REPOA"
pbuilder image buster2 -> has othermirror "REPOB"

Currently I can set up a single system that has pbuilder's for buster, 
bullseye, bionic, and focal that all point at REPOA.  Then I can have a 
second system that has pbuilder's for buster, bullseye, bionic, and focal 
that all point at REPOB.


Alternatively, what would be best, is if I could pass (from gbp) an 
"othermirror" option like:


gbp buildpackage --othermirror="repoa"

Even better, is if I could set the othermirror in gbp.conf, since I have a 
branch for REPOA builds and a branch for REPOB builds.  Then I could just 
run:


gbp buildpackage

from the appropriate branch, and the correct mirror would be added to the 
image just for use during the build, which is all I really need it do do.


I'm able to do something like this with git buildpackage-rpm, where I can 
point mock at custom images in the [buildpackage-rpm] section in .gbp.conf 
via the "dist" parameter.  This allows me to have one "dist" that uses 
repoA and another "dist" that uses repoB, meaning I can use a single server 
with mock installed to do all the builds.


--Quanah

--

Quanah Gibson-Mount
Product Architect
Symas Corporation
Packaged, certified, and supported LDAP solutions powered by OpenLDAP:

___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage

Re: [git-buildpackage] Different mirrors with pbuilder and dist?

[Guido Günther]

Hi Quanah,
On Tue, Dec 07, 2021 at 03:43:31PM -0800, Quanah Gibson-Mount wrote:
> I have two versions of a piece of software I build for each distribution I
> build on.  Currently I do this by having two different VMs with
> git-buildpackage and pbuilder, and then create an image for each dist
> (buster, bullseye, bionic, and focal) on each of them, but with a different
> OTHERMIRRORS setting.
> 
> I'd like to get this down to a single server, but I can't find a way to get
> pbuilder to either:
> 
> a) Use different OTHERMIRRORS at build time with the existing dist image
> 
> b) Have multiple images of a given dist with different OTHERMIRROR values
> 
> Anyone have any hints? :)

Given recent deboostrap (e.g. from Debian bookwork) you should be able
to use all of them from Debian:

For bullseye, buster (all Debian based):

DIST=buster git-pbuilder create
DIST=bullseye git-pbuilder create

For bionic, focal, ... (ubuntu based):

sudo apt install ubuntu-archive-keyring
DIST=bionic git-pbuilder create --mirror=http://us.archive.ubuntu.com/ubuntu 
--components="main universe"
DIST=focal git-pbuilder create --mirror=http://us.archive.ubuntu.com/ubuntu 
--components="main universe"

then e.g.:

gbp buildpackage --git-dist=bionic

Cheers,
 -- Guido

> 
> Thanks!
> 
> --Quanah
> 
> --
> 
> Quanah Gibson-Mount
> Product Architect
> Symas Corporation
> Packaged, certified, and supported LDAP solutions powered by OpenLDAP:
> 
> ___
> git-buildpackage mailing list
> git-buildpackage@lists.sigxcpu.org
> http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage
___
git-buildpackage mailing list
git-buildpackage@lists.sigxcpu.org
http://lists.sigxcpu.org/mailman/listinfo/git-buildpackage