Re: Using microdnf in boostrap mock?
* Pavel Raiskup: > Can you elaborate a bit more please? What is the practical difference > between mock's bootstrap chroot and the manual technique you propose? Koji has access to a compatible RPM for the unpacking, something that may not be available on a developer workstation. Thanks, Florian -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
Dne 29. 11. 20 v 1:44 Miro Hrončok napsal(a): > > I guess I should have searched a bit better before asking the questiong. > > I found the microdnf option in mock: > > https://github.com/rpm-software-management/mock/issues/76 > https://github.com/rpm-software-management/microdnf/pull/28 But is is documented only in site-defaults.cfg. Point taken. I started https://github.com/rpm-software-management/mock/wiki/Feature-package-managers and I will try to add more content to that. Though, you already get the information from these PRs. :) -- Miroslav Suchy, RHCA Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
Additionally, the config in attachemnt will use fedora-33 for bootstrap and
fedora-34 for building.
Here is the important part:
config_opts['use_bootstrap'] = True
config_opts['use_bootstrap_image'] = True
config_opts['bootstrap_image'] = 'fedora:33'
config_opts['target_arch'] = 'x86_64'
config_opts['root'] = 'fedora-rawhide-{{ target_arch }}'
config_opts['bootstrap_root'] = 'fedora-33-{{ target_arch }}'
config_opts['chroot_name'] = 'fedora-rawhide-{{ target_arch }}'
config_opts['bootstrap_chroot_name'] = 'fedora-33-{{ target_arch }}'
config_opts['releasever'] = 34
config_opts['bootstrap_releasever'] = 33
Disclaimer - I believe that this can be even simplier.
Functional one is in the attachement.
If you will fiddle with:
config_opts['bootstra_dnf.conf']
than you can skip that bootstrap image if that would be a problem for Koji.
--
Miroslav Suchy, RHCA
Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys
#config_opts['update_before_build'] = True
config_opts['dnf_install_command'] = '--help'
config_opts['use_bootstrap'] = True
config_opts['use_bootstrap_image'] = True
config_opts['bootstrap_image'] = 'fedora:33'
config_opts['target_arch'] = 'x86_64'
config_opts['root'] = 'fedora-rawhide-{{ target_arch }}'
config_opts['bootstrap_root'] = 'fedora-33-{{ target_arch }}'
config_opts['chroot_name'] = 'fedora-rawhide-{{ target_arch }}'
config_opts['bootstrap_chroot_name'] = 'fedora-33-{{ target_arch }}'
config_opts['releasever'] = 34
config_opts['bootstrap_releasever'] = 33
config_opts['chroot_setup_cmd'] = 'install @{% if mirrored %}buildsys-{% endif
%}build'
config_opts['dnf.conf'] = ('\n'
'[main]\n'
'keepcache=1\n'
'debuglevel=2\n'
'reposdir=/dev/null\n'
'logfile=/var/log/yum.log\n'
'retries=20\n'
'obsoletes=1\n'
'gpgcheck=0\n'
'assumeyes=1\n'
'syslog_ident=mock\n'
'syslog_device=\n'
'install_weak_deps=0\n'
'metadata_expire=0\n'
'best=1\n'
'module_platform_id=platform:f{{ releasever }}\n'
'protected_packages=\n'
'user_agent={{ user_agent }}\n'
'\n'
'{%- macro rawhide_gpg_keys() -%}\n'
'file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-$releasever-primary\n'
'{%- for version in [releasever|int, releasever|int - 1]\n'
'%} file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-{{ '
'version }}-primary\n'
'{%- endfor %}\n'
'{%- endmacro %}\n'
'\n'
'# repos\n'
'\n'
'[local]\n'
'name=local\n'
'baseurl=https://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/\n'
'cost=2000\n'
'enabled={{ not mirrored }}\n'
'skip_if_unavailable=False\n'
'\n'
'[local-source]\n'
'name=local-source\n'
'baseurl=https://kojipkgs.fedoraproject.org/repos/rawhide/latest/src/\n'
'cost=2000\n'
'enabled=0\n'
'skip_if_unavailable=False\n'
'\n'
'{% if mirrored %}\n'
'[fedora]\n'
'name=fedora\n'
'metalink=https://mirrors.fedoraproject.org/metalink?repo=rawhide&arch=$basearch\n'
'gpgkey={{ rawhide_gpg_keys() }}\n'
'gpgcheck=1\n'
'skip_if_unavailable=False\n'
'\n'
'[fedora-debuginfo]\n'
'name=Fedora Rawhide - Debug\n'
'metalink=https://mirrors.fedoraproject.org/metalink?repo=rawhide-debug&arch=$basearch\n'
'enabled=0\n'
'gpgkey={{ rawhide_gpg_keys() }}\n'
'gpgcheck=1\n'
'skip_if_unavailable=False\n'
'\n'
'[fedora-source]\n'
'name=fedora-source\n'
'metalink=https://mirrors.fedoraproject.org/metalink?repo=rawhide-source&arch=$basearch\n'
'gpgkey={{ rawhide_gpg_keys() }}\n'
'gpgcheck=1\n'
'enabled=0\n'
'skip_if_unavailable=False\n'
'\n'
'# modular\n'
'\n'
'[rawhide-modular]\n'
'name=Fedora - Modular Rawhide - Developmental packages for the next Fedora '
'release\n'
'metalink=https://mirrors.fedoraproject.org/metalink?repo=rawhide-modular&arch=$basearch\n'
'# if you want to enable it, you should set best=0\n'
'# see https://bugzilla.redhat.com/show_bug.cgi?id=1673851\n'
'enabled=0\n'
'gpgcheck=1\n'
'gpgkey={{ rawhide_gpg_keys() }}\n'
'skip_if_unavailable=False\n'
'\n'
'[rawhide-modular-debuginfo]\n'
'name=Fedora - Modular Rawhide - Debug\n'
'metalink=https://mirrors.fedoraproject.org/metalink?repo=rawhide-modular-debug&arch=$basearch\n'
'enabled=0\n'
'gpgcheck=1\n'
'gpgkey={{ rawhide_gpg_keys() }}\n'
'skip_if_unavailable=False\n'
'\n'
'[rawhide-modular-source]\n'
'name=Fedora - Modular Rawhide - Source\n'
'metalink=https://mirrors.fedoraproject.org/metalink?repo=rawhide-modular-source&arch=$basearch\n'
'enabled=0\n'
'gpgcheck=1\n'
'gpgkey={{ rawhide_gpg_keys() }}\n'
'skip_if_unavailable=False\n'
'{% endif %}\n')
___
buildsys mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
Dne 30. 11. 20 v 13:44 Neal Gompa napsal(a): > Then that's easy: we define a list of packages that are needed to get > the package manager working, and Koji would just download and unpack > them with no script execution, then it would manually run some > predefined scripts and invoke the package manager after that. That is not needed. This: config_opts['update_before_build'] = True config_opts['dnf_install_command'] = '--help' config_opts['use_bootstrap'] = True config_opts['use_bootstrap_image'] = True config_opts['bootstrap_image'] = 'fedora:rawhide' The last 3 lines enables this feature: https://github.com/rpm-software-management/mock/wiki/Feature-container-for-bootstrap Then config_opts['dnf_install_command'] = '--help' just shortcut dnf install dnf dnf-plugins-core after downloading the image. It is there just to make sure that we have DNF in that container and not just microdnf. Rawhide image has it there. So this step is redundant. Fedora-infra will need whitelist fedora registry on Koji builders so they are network reachable. -- Miroslav Suchy, RHCA Red Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On Mon, 30 Nov 2020 at 08:01, Pavel Raiskup wrote: > On Monday, November 30, 2020 1:44:14 PM CET Neal Gompa wrote: > > On Mon, Nov 30, 2020 at 7:41 AM Stephen John Smoogen > wrote: > > > > > > > > > > > > On Mon, 30 Nov 2020 at 06:57, Florian Weimer > wrote: > > >> > > >> * Miro Hrončok: > > >> > > >> > I was wondering whether it might be possible to use microdnf instead > > >> > of dnf in the boostrap mock chroots. Since dnf depends on Python, > > >> > using the boostrap mode now complicates upgrading Pythons to a newer > > >> > version. > > >> > > >> Would it be possible to use rpm-ostree with a vendored RPM > > >> implementation? That should provide even better isolation. > > >> > > > > > > koji has a very strong idea of what it wants inside a chroot and what > it > > > does not (and may never) want. Getting rpm-ostree or other items to > work > > > inside of the koji mock interface is probably looking at a large > project > > > (going from how many other 'simple' things have ended up being much > more > > > complicated than expected by the original proposers). My usual way of > > > figuring out how to make anything work in koji is to see what the > oldest > > > environment it needs to work on by a customer and see how I would do > it in > > > RHEL-5 (now maybe RHEL-6). > > > > Then that's easy: we define a list of packages that are needed to get > > the package manager working, and Koji would just download and unpack > > them with no script execution, then it would manually run some > > predefined scripts and invoke the package manager after that. > > Can you elaborate a bit more please? What is the practical difference > between > mock's bootstrap chroot and the manual technique you propose? > > Pavel > > Koji has to be programmed to know about the mock's bootstrap chroot and be able to use it. Koji 'handwrites' a mock config for every build and calls mock with specific instructions. I think the manual technique is already built into koji for certain cases but takes configuration on releng to make work. [So the difference would be one is very old, awkward, but there.. the other is a project plan with the koji team.] > > Aside from me doing this before to do RHEL 5 -> RHEL 7 upgrades by > > hand, this is also the strategy used by the openSUSE Build Service to > > work around this problem. > > > > > > > > -- > > 真実はいつも一つ！/ Always, there's only one truth! > > ___ > > buildsys mailing list -- [email protected] > > To unsubscribe send an email to [email protected] > > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > > List Archives: > https://lists.fedoraproject.org/archives/list/[email protected] > > > > > > ___ > buildsys mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/[email protected] > -- Stephen J Smoogen. ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On Mon, Nov 30, 2020 at 8:01 AM Pavel Raiskup wrote: > > On Monday, November 30, 2020 1:44:14 PM CET Neal Gompa wrote: > > On Mon, Nov 30, 2020 at 7:41 AM Stephen John Smoogen > > wrote: > > > > > > > > > > > > On Mon, 30 Nov 2020 at 06:57, Florian Weimer wrote: > > >> > > >> * Miro Hrončok: > > >> > > >> > I was wondering whether it might be possible to use microdnf instead > > >> > of dnf in the boostrap mock chroots. Since dnf depends on Python, > > >> > using the boostrap mode now complicates upgrading Pythons to a newer > > >> > version. > > >> > > >> Would it be possible to use rpm-ostree with a vendored RPM > > >> implementation? That should provide even better isolation. > > >> > > > > > > koji has a very strong idea of what it wants inside a chroot and what it > > > does not (and may never) want. Getting rpm-ostree or other items to work > > > inside of the koji mock interface is probably looking at a large project > > > (going from how many other 'simple' things have ended up being much more > > > complicated than expected by the original proposers). My usual way of > > > figuring out how to make anything work in koji is to see what the oldest > > > environment it needs to work on by a customer and see how I would do it in > > > RHEL-5 (now maybe RHEL-6). > > > > Then that's easy: we define a list of packages that are needed to get > > the package manager working, and Koji would just download and unpack > > them with no script execution, then it would manually run some > > predefined scripts and invoke the package manager after that. > > Can you elaborate a bit more please? What is the practical difference between > mock's bootstrap chroot and the manual technique you propose? > This does not require dependency resolution to be verified, unless you use the dependency resolver in the COPR/Koji code to create the list (that's what OBS does). -- 真実はいつも一つ！/ Always, there's only one truth! ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On Monday, November 30, 2020 1:44:14 PM CET Neal Gompa wrote: > On Mon, Nov 30, 2020 at 7:41 AM Stephen John Smoogen wrote: > > > > > > > > On Mon, 30 Nov 2020 at 06:57, Florian Weimer wrote: > >> > >> * Miro Hrončok: > >> > >> > I was wondering whether it might be possible to use microdnf instead > >> > of dnf in the boostrap mock chroots. Since dnf depends on Python, > >> > using the boostrap mode now complicates upgrading Pythons to a newer > >> > version. > >> > >> Would it be possible to use rpm-ostree with a vendored RPM > >> implementation? That should provide even better isolation. > >> > > > > koji has a very strong idea of what it wants inside a chroot and what it > > does not (and may never) want. Getting rpm-ostree or other items to work > > inside of the koji mock interface is probably looking at a large project > > (going from how many other 'simple' things have ended up being much more > > complicated than expected by the original proposers). My usual way of > > figuring out how to make anything work in koji is to see what the oldest > > environment it needs to work on by a customer and see how I would do it in > > RHEL-5 (now maybe RHEL-6). > > Then that's easy: we define a list of packages that are needed to get > the package manager working, and Koji would just download and unpack > them with no script execution, then it would manually run some > predefined scripts and invoke the package manager after that. Can you elaborate a bit more please? What is the practical difference between mock's bootstrap chroot and the manual technique you propose? Pavel > Aside from me doing this before to do RHEL 5 -> RHEL 7 upgrades by > hand, this is also the strategy used by the openSUSE Build Service to > work around this problem. > > > > -- > 真実はいつも一つ！/ Always, there's only one truth! > ___ > buildsys mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/[email protected] > ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On Mon, Nov 30, 2020 at 7:41 AM Stephen John Smoogen wrote: > > > > On Mon, 30 Nov 2020 at 06:57, Florian Weimer wrote: >> >> * Miro Hrončok: >> >> > I was wondering whether it might be possible to use microdnf instead >> > of dnf in the boostrap mock chroots. Since dnf depends on Python, >> > using the boostrap mode now complicates upgrading Pythons to a newer >> > version. >> >> Would it be possible to use rpm-ostree with a vendored RPM >> implementation? That should provide even better isolation. >> > > koji has a very strong idea of what it wants inside a chroot and what it does > not (and may never) want. Getting rpm-ostree or other items to work inside of > the koji mock interface is probably looking at a large project (going from > how many other 'simple' things have ended up being much more complicated than > expected by the original proposers). My usual way of figuring out how to make > anything work in koji is to see what the oldest environment it needs to work > on by a customer and see how I would do it in RHEL-5 (now maybe RHEL-6). > Then that's easy: we define a list of packages that are needed to get the package manager working, and Koji would just download and unpack them with no script execution, then it would manually run some predefined scripts and invoke the package manager after that. Aside from me doing this before to do RHEL 5 -> RHEL 7 upgrades by hand, this is also the strategy used by the openSUSE Build Service to work around this problem. -- 真実はいつも一つ！/ Always, there's only one truth! ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On Mon, Nov 30, 2020 at 7:39 AM Florian Weimer wrote: > > * Neal Gompa: > > > On Mon, Nov 30, 2020 at 6:57 AM Florian Weimer wrote: > >> > >> * Miro Hrončok: > >> > >> > I was wondering whether it might be possible to use microdnf instead > >> > of dnf in the boostrap mock chroots. Since dnf depends on Python, > >> > using the boostrap mode now complicates upgrading Pythons to a newer > >> > version. > >> > >> Would it be possible to use rpm-ostree with a vendored RPM > >> implementation? That should provide even better isolation. > > > > We're making this too complicated. The only real thing we have to do > > is use OCI images to populate the bootstrap environment rather than > > having DNF create them on the fly. > > But these images have to come from somewhere, too. Their build process > still needs to solve the bootstrap issue. For our purposes, it solves the problem. The main issue for us is that since Koji does not support generating the bootstrap itself (this is being passed to Mock), you cannot control the content going into bootstrap creation. Copr has this issue too. But we don't need to solve that problem to solve Miro's issue. -- 真実はいつも一つ！/ Always, there's only one truth! ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On Mon, 30 Nov 2020 at 06:57, Florian Weimer wrote: > * Miro Hrončok: > > > I was wondering whether it might be possible to use microdnf instead > > of dnf in the boostrap mock chroots. Since dnf depends on Python, > > using the boostrap mode now complicates upgrading Pythons to a newer > > version. > > Would it be possible to use rpm-ostree with a vendored RPM > implementation? That should provide even better isolation. > > koji has a very strong idea of what it wants inside a chroot and what it does not (and may never) want. Getting rpm-ostree or other items to work inside of the koji mock interface is probably looking at a large project (going from how many other 'simple' things have ended up being much more complicated than expected by the original proposers). My usual way of figuring out how to make anything work in koji is to see what the oldest environment it needs to work on by a customer and see how I would do it in RHEL-5 (now maybe RHEL-6). > Thanks, > Florian > -- > Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, > Commercial register: Amtsgericht Muenchen, HRB 153243, > Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael > O'Neill > ___ > buildsys mailing list -- [email protected] > To unsubscribe send an email to [email protected] > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/[email protected] > -- Stephen J Smoogen. ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
* Neal Gompa: > On Mon, Nov 30, 2020 at 6:57 AM Florian Weimer wrote: >> >> * Miro Hrončok: >> >> > I was wondering whether it might be possible to use microdnf instead >> > of dnf in the boostrap mock chroots. Since dnf depends on Python, >> > using the boostrap mode now complicates upgrading Pythons to a newer >> > version. >> >> Would it be possible to use rpm-ostree with a vendored RPM >> implementation? That should provide even better isolation. > > We're making this too complicated. The only real thing we have to do > is use OCI images to populate the bootstrap environment rather than > having DNF create them on the fly. But these images have to come from somewhere, too. Their build process still needs to solve the bootstrap issue. Thanks, Florian -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On Mon, Nov 30, 2020 at 6:57 AM Florian Weimer wrote: > > * Miro Hrončok: > > > I was wondering whether it might be possible to use microdnf instead > > of dnf in the boostrap mock chroots. Since dnf depends on Python, > > using the boostrap mode now complicates upgrading Pythons to a newer > > version. > > Would it be possible to use rpm-ostree with a vendored RPM > implementation? That should provide even better isolation. We're making this too complicated. The only real thing we have to do is use OCI images to populate the bootstrap environment rather than having DNF create them on the fly. -- 真実はいつも一つ！/ Always, there's only one truth! ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
* Miro Hrončok: > I was wondering whether it might be possible to use microdnf instead > of dnf in the boostrap mock chroots. Since dnf depends on Python, > using the boostrap mode now complicates upgrading Pythons to a newer > version. Would it be possible to use rpm-ostree with a vendored RPM implementation? That should provide even better isolation. Thanks, Florian -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On Monday, November 30, 2020 10:41:20 AM CET Miro Hrončok wrote: > > So perhaps we we shouldn't install packages from the side-tag into the > > bootstrap? > > Yes, that would be ideal. So we'd have to have two repositories in Koji's mock config, e.g.: [build] enabled=0 baseurl=... [side-tag] enabled=1 baseurl=... ... And the config would have to have something like: config_opts['dnf_install_command'] += " --disablerepo=side-tag --enablerepo=build" It should make sure that DNF from [build] repo is installed. We have similar hack [1] in mock-core-configs for the el6/7 chroots with scl repositories, because some historically shipped SCLs had broken Provides: that were breaking the bootstrap chroot install transaction. [1] https://github.com/rpm-software-management/mock/blob/70ab0b923c9d65e29c113379e66ef05251e9a201/mock-core-configs/etc/mock/templates/centos-7.tpl#L10 > > Also note there's --use-bootstrap-image option that could help here. > > Do you think we could use that in Koji? Technically it should be possible. But AFAIK it's not yet widely tested feature (can be newly tested in Copr). The question is though what podman image to use in Koji. If we use the default one (e.g. fedora:33 for fedora-33-x86_64), it may not be updated often/fast enough. Also I'm not sure if we have podman support for all our architectures. Pavel ___ buildsys mailing list -- [email protected] To unsubscribe send an email to [email protected] Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On 11/30/20 10:14 AM, Pavel Raiskup wrote:
On Sunday, November 29, 2020 1:44:37 AM CET Miro Hrončok wrote:
On 11/29/20 1:28 AM, Miro Hrončok wrote:
Hello,
I was wondering whether it might be possible to use microdnf instead of dnf in
the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode
now complicates upgrading Pythons to a newer version.
https://pagure.io/releng/issue/9308#comment-703106
Basically, if/when Koji uses the bootstrap mock mode, as soon as we start
bootstrapping (different meaning) Python in a side tag, dnf becomes temporarily
uninstallable in that side tag and we can no longer do any builds.
This can be worked around by not using the boostrap mock option in Koji (ideally
only for Python upgrade side tags, but I am not yet sure if that's possible), or
by eliminating Python libraries out of the bootstrap chroot, hence the idea
about using microdnf.
I've started with this config:
include('/etc/mock/fedora-rawhide-x86_64.cfg')
config_opts['root'] = 'fedora-rawhide-microdnf'
config_opts['package_manager'] = 'dnf'
config_opts['dnf_command'] = '/usr/bin/microdnf'
config_opts['dnf_install_command'] = 'install microdnf'
config_opts['system_dnf_command'] = '/usr/bin/dnf'
config_opts['dnf_common_opts'] = ['--allowerasing']
But it fails pretty soon with:
error: (--setopt) Unknown tsflag: nocontexts
And when I patch that option out, I still get:
error: The "--installroot" argument must be used together with "--config",
"--noplugins", "--setopt=cachedir=", "--setopt=reposdir=",
"--setopt=varsdir=" arguments.
But I guess that if I figure the right options for dnf_common_opts, this might
work...? Is there some crucial functionality that microdnf might be missing that
would prevent it from creating mock chroots like this?
I guess I should have searched a bit better before asking the questiong.
I found the microdnf option in mock:
https://github.com/rpm-software-management/mock/issues/76
https://github.com/rpm-software-management/microdnf/pull/28
But using config_opts['package_manager'] = 'microdnf' still installs dnf into
the bootstrap chroot, which is confusing.
It is likely we'll have to fix Mock too, dunno. Because of the missing
functionality in microdnf nobody had enough motivation to spend more time
on the microdnf support in mock so far.
I'm just not sure it is a valid use-case to fall-back to microdnf in this
case. The point of bootstrap was to install the final chroot by the
**default** tooling from that chroot. Not only to absorb the "RPM"
differences (like sqlite vs BD) but also to stop playing with
the small nuances between 'yum' in RHEL vs. 'yum' in Fedora, or even
'dnf' in F33 vs. 'dnf' in F31.
Going with microdnf is a bit against that principle. At least as long as
microdnf isn't guaranteed to e.g. calculate 100% compatible transactions
with DNF, or as long as 'microdnf' is not the default Mock installer for
Fedora chroots.
Thanks, Pavel.
That makes perfect sense.
While it is less likely, I'm afraid that similar problems may happen with
microdnf too (microdnf still depends on some libraries).
While technically yes, it's easier to manage around a single C library bootstrap
than around dozen of Python packages with many more build dependencies.
So perhaps we we
shouldn't install packages from the side-tag into the bootstrap?
Yes, that would be ideal.
Also note there's --use-bootstrap-image option that could help here.
Do you think we could use that in Koji?
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
buildsys mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On Sunday, November 29, 2020 1:44:37 AM CET Miro Hrončok wrote:
> On 11/29/20 1:28 AM, Miro Hrončok wrote:
> > Hello,
> >
> > I was wondering whether it might be possible to use microdnf instead of dnf
> > in
> > the boostrap mock chroots. Since dnf depends on Python, using the boostrap
> > mode
> > now complicates upgrading Pythons to a newer version.
> >
> > https://pagure.io/releng/issue/9308#comment-703106
> >
> > Basically, if/when Koji uses the bootstrap mock mode, as soon as we start
> > bootstrapping (different meaning) Python in a side tag, dnf becomes
> > temporarily
> > uninstallable in that side tag and we can no longer do any builds.
> >
> > This can be worked around by not using the boostrap mock option in Koji
> > (ideally
> > only for Python upgrade side tags, but I am not yet sure if that's
> > possible), or
> > by eliminating Python libraries out of the bootstrap chroot, hence the idea
> > about using microdnf.
> >
> > I've started with this config:
> >
> > include('/etc/mock/fedora-rawhide-x86_64.cfg')
> > config_opts['root'] = 'fedora-rawhide-microdnf'
> > config_opts['package_manager'] = 'dnf'
> > config_opts['dnf_command'] = '/usr/bin/microdnf'
> > config_opts['dnf_install_command'] = 'install microdnf'
> > config_opts['system_dnf_command'] = '/usr/bin/dnf'
> > config_opts['dnf_common_opts'] = ['--allowerasing']
> >
> > But it fails pretty soon with:
> >
> > error: (--setopt) Unknown tsflag: nocontexts
> >
> > And when I patch that option out, I still get:
> >
> > error: The "--installroot" argument must be used together with
> > "--config",
> > "--noplugins", "--setopt=cachedir=", "--setopt=reposdir=",
> > "--setopt=varsdir=" arguments.
> >
> > But I guess that if I figure the right options for dnf_common_opts, this
> > might
> > work...? Is there some crucial functionality that microdnf might be missing
> > that
> > would prevent it from creating mock chroots like this?
>
> I guess I should have searched a bit better before asking the questiong.
>
> I found the microdnf option in mock:
>
> https://github.com/rpm-software-management/mock/issues/76
> https://github.com/rpm-software-management/microdnf/pull/28
>
> But using config_opts['package_manager'] = 'microdnf' still installs dnf into
> the bootstrap chroot, which is confusing.
It is likely we'll have to fix Mock too, dunno. Because of the missing
functionality in microdnf nobody had enough motivation to spend more time
on the microdnf support in mock so far.
I'm just not sure it is a valid use-case to fall-back to microdnf in this
case. The point of bootstrap was to install the final chroot by the
**default** tooling from that chroot. Not only to absorb the "RPM"
differences (like sqlite vs BD) but also to stop playing with
the small nuances between 'yum' in RHEL vs. 'yum' in Fedora, or even
'dnf' in F33 vs. 'dnf' in F31.
Going with microdnf is a bit against that principle. At least as long as
microdnf isn't guaranteed to e.g. calculate 100% compatible transactions
with DNF, or as long as 'microdnf' is not the default Mock installer for
Fedora chroots.
While it is less likely, I'm afraid that similar problems may happen with
microdnf too (microdnf still depends on some libraries). So perhaps we we
shouldn't install packages from the side-tag into the bootstrap?
Also note there's --use-bootstrap-image option that could help here.
Pavel
> But I got around that with:
>
> include('/etc/mock/fedora-rawhide-x86_64.cfg')
> config_opts['root'] = 'fedora-rawhide-microdnf'
> config_opts['package_manager'] = 'microdnf'
> config_opts['microdnf_command'] = '/usr/bin/microdnf'
> config_opts['microdnf_install_command'] = 'dnf-install microdnf
> distribution-gpg-keys'
> config_opts['system_dnf_command'] = '/usr/bin/dnf'
> config_opts['microdnf_common_opts'] = ['--allowerasing']
>
>
> However, it still ends with:
>
> Curl error (37): Couldn't read a file:// file for
> file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary
> [Couldn't open file
> /usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary]
>
> Running `mock -r fedora-rawhide-microdnf --init` twice works around that
> problem, and it successfully crates the boostraop chroot, but then, trying to
> create the actual one, it fails with:
>
> Start: microdnf install
> Directory /var/lib/mock/fedora-rawhide-microdnf/root doesn't look like it has
> an
> OS tree. Refusing.
>
>
> --
> Miro Hrončok
> --
> Phone: +420777974800
> IRC: mhroncok
> ___
> buildsys mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https:
Re: Using microdnf in boostrap mock?
On Sat, Nov 28, 2020 at 7:44 PM Miro Hrončok wrote:
>
> On 11/29/20 1:28 AM, Miro Hrončok wrote:
> > Hello,
> >
> > I was wondering whether it might be possible to use microdnf instead of dnf
> > in
> > the boostrap mock chroots. Since dnf depends on Python, using the boostrap
> > mode
> > now complicates upgrading Pythons to a newer version.
> >
> > https://pagure.io/releng/issue/9308#comment-703106
> >
> > Basically, if/when Koji uses the bootstrap mock mode, as soon as we start
> > bootstrapping (different meaning) Python in a side tag, dnf becomes
> > temporarily
> > uninstallable in that side tag and we can no longer do any builds.
> >
> > This can be worked around by not using the boostrap mock option in Koji
> > (ideally
> > only for Python upgrade side tags, but I am not yet sure if that's
> > possible), or
> > by eliminating Python libraries out of the bootstrap chroot, hence the idea
> > about using microdnf.
> >
> > I've started with this config:
> >
> > include('/etc/mock/fedora-rawhide-x86_64.cfg')
> > config_opts['root'] = 'fedora-rawhide-microdnf'
> > config_opts['package_manager'] = 'dnf'
> > config_opts['dnf_command'] = '/usr/bin/microdnf'
> > config_opts['dnf_install_command'] = 'install microdnf'
> > config_opts['system_dnf_command'] = '/usr/bin/dnf'
> > config_opts['dnf_common_opts'] = ['--allowerasing']
> >
> > But it fails pretty soon with:
> >
> > error: (--setopt) Unknown tsflag: nocontexts
> >
> > And when I patch that option out, I still get:
> >
> > error: The "--installroot" argument must be used together with
> > "--config",
> > "--noplugins", "--setopt=cachedir=", "--setopt=reposdir=",
> > "--setopt=varsdir=" arguments.
> >
> > But I guess that if I figure the right options for dnf_common_opts, this
> > might
> > work...? Is there some crucial functionality that microdnf might be missing
> > that
> > would prevent it from creating mock chroots like this?
>
> I guess I should have searched a bit better before asking the questiong.
>
> I found the microdnf option in mock:
>
> https://github.com/rpm-software-management/mock/issues/76
> https://github.com/rpm-software-management/microdnf/pull/28
>
> But using config_opts['package_manager'] = 'microdnf' still installs dnf into
> the bootstrap chroot, which is confusing. But I got around that with:
>
> include('/etc/mock/fedora-rawhide-x86_64.cfg')
> config_opts['root'] = 'fedora-rawhide-microdnf'
> config_opts['package_manager'] = 'microdnf'
> config_opts['microdnf_command'] = '/usr/bin/microdnf'
> config_opts['microdnf_install_command'] = 'dnf-install microdnf
> distribution-gpg-keys'
> config_opts['system_dnf_command'] = '/usr/bin/dnf'
> config_opts['microdnf_common_opts'] = ['--allowerasing']
>
>
> However, it still ends with:
>
> Curl error (37): Couldn't read a file:// file for
> file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary
> [Couldn't open file
> /usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary]
>
> Running `mock -r fedora-rawhide-microdnf --init` twice works around that
> problem, and it successfully crates the boostraop chroot, but then, trying to
> create the actual one, it fails with:
>
> Start: microdnf install
> Directory /var/lib/mock/fedora-rawhide-microdnf/root doesn't look like it has
> an
> OS tree. Refusing.
>
This is a legitimate bug with microdnf and deserves a bug report in
Red Hat Bugzilla or GitHub.
The installroot support in microdnf should generally work, as I
verified it for image builds with kiwi[1] this past week.
I think the next release of microdnf should have enough for the
bootstrap phase (especially with the modularity stuff now sorted out),
but using it in mock in all stages requires getting a builddep
implementation[2].
[1]: https://bodhi.fedoraproject.org/updates/FEDORA-2020-9ce3983748
[2]: https://github.com/rpm-software-management/microdnf/issues/82
--
真実はいつも一つ！/ Always, there's only one truth!
___
buildsys mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[email protected]
Re: Using microdnf in boostrap mock?
On 11/29/20 1:28 AM, Miro Hrončok wrote:
Hello,
I was wondering whether it might be possible to use microdnf instead of dnf in
the boostrap mock chroots. Since dnf depends on Python, using the boostrap mode
now complicates upgrading Pythons to a newer version.
https://pagure.io/releng/issue/9308#comment-703106
Basically, if/when Koji uses the bootstrap mock mode, as soon as we start
bootstrapping (different meaning) Python in a side tag, dnf becomes temporarily
uninstallable in that side tag and we can no longer do any builds.
This can be worked around by not using the boostrap mock option in Koji (ideally
only for Python upgrade side tags, but I am not yet sure if that's possible), or
by eliminating Python libraries out of the bootstrap chroot, hence the idea
about using microdnf.
I've started with this config:
include('/etc/mock/fedora-rawhide-x86_64.cfg')
config_opts['root'] = 'fedora-rawhide-microdnf'
config_opts['package_manager'] = 'dnf'
config_opts['dnf_command'] = '/usr/bin/microdnf'
config_opts['dnf_install_command'] = 'install microdnf'
config_opts['system_dnf_command'] = '/usr/bin/dnf'
config_opts['dnf_common_opts'] = ['--allowerasing']
But it fails pretty soon with:
error: (--setopt) Unknown tsflag: nocontexts
And when I patch that option out, I still get:
error: The "--installroot" argument must be used together with "--config",
"--noplugins", "--setopt=cachedir=", "--setopt=reposdir=",
"--setopt=varsdir=" arguments.
But I guess that if I figure the right options for dnf_common_opts, this might
work...? Is there some crucial functionality that microdnf might be missing that
would prevent it from creating mock chroots like this?
I guess I should have searched a bit better before asking the questiong.
I found the microdnf option in mock:
https://github.com/rpm-software-management/mock/issues/76
https://github.com/rpm-software-management/microdnf/pull/28
But using config_opts['package_manager'] = 'microdnf' still installs dnf into
the bootstrap chroot, which is confusing. But I got around that with:
include('/etc/mock/fedora-rawhide-x86_64.cfg')
config_opts['root'] = 'fedora-rawhide-microdnf'
config_opts['package_manager'] = 'microdnf'
config_opts['microdnf_command'] = '/usr/bin/microdnf'
config_opts['microdnf_install_command'] = 'dnf-install microdnf
distribution-gpg-keys'
config_opts['system_dnf_command'] = '/usr/bin/dnf'
config_opts['microdnf_common_opts'] = ['--allowerasing']
However, it still ends with:
Curl error (37): Couldn't read a file:// file for
file:///usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary
[Couldn't open file
/usr/share/distribution-gpg-keys/fedora/RPM-GPG-KEY-fedora-0-primary]
Running `mock -r fedora-rawhide-microdnf --init` twice works around that
problem, and it successfully crates the boostraop chroot, but then, trying to
create the actual one, it fails with:
Start: microdnf install
Directory /var/lib/mock/fedora-rawhide-microdnf/root doesn't look like it has an
OS tree. Refusing.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
buildsys mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/[email protected]
