Re: bootstrapping without .spec modification

2021-01-31 Thread Kalev Lember 

On 1/31/21 11:55, Zbigniew Jędrzejewski-Szmek wrote:

I think those worries are overblown. When a packager has the rights to
build an rpm (in a side-tag or not), they also have the right to
commit whatever they want to the spec file.

Just taking a sentence out of context here:

I believe you are incorrect here and koji doesn't apply any restrictions 
on who can build packages (secure boot packages have a restriction but 
that's a different story). As far as I am aware, anyone who has access 
to koji can build anything that's already commited to git.


Or to put it another way:

There's fine grained restrictions on git access. The same restrictions 
do not apply to koji.


--
Kalev
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-31 Thread Zbigniew Jędrzejewski-Szmek 
On Sat, Jan 30, 2021 at 12:03:21PM -0800, Kevin Fenzi wrote:
> On Fri, Jan 29, 2021 at 03:49:28PM +0100, Miro Hrončok wrote:
> > 
> > Currently a Koji task says:
> > 
> > Source: 
> > git+https://src.fedoraproject.org/rpms/python3.9.git#563b4f4b59a9f81095acfe30899fdc4b040c1b9b
> > Build Target: f34-rebuild
> > Options:
> >   fail_fast = True
> >   wait_builds =
> > 
> > Example taken from 
> > https://koji.fedoraproject.org/koji/taskinfo?taskID=60654655
> > 
> > With this feature, it would say:
> > 
> > Source: 
> > git+https://src.fedoraproject.org/rpms/python3.9.git#563b4f4b59a9f81095acfe30899fdc4b040c1b9b
> > Build Target: f34-rebuild
> > Options:
> >   fail_fast = True
> >   wait_builds =
> > Macro overrides:
> >   _with_bootstrap: 1
> > 
> > And when reproducing, you'd just run the same command, e.g.
> > 
> > $ koji build f34-rebuild --fail-fast --macro '_with_bootstrap 1' 
> > 'git+https://src.fedoraproject.org/rpms/python3.9.git#563b4f4b59a9f81095acfe30899fdc4b040c1b9b'
> > 
> > The Koji web interface could even display the command.
> > 
> > That's very much as reproduce as now.
> 
> So, koji already can set macros per tag.
> For example, eln sig uses this: 
> 
> ➜  ~ koji taginfo eln-build
> Tag: eln-build [22493]
> Arches: i686 x86_64 aarch64 ppc64le s390x
> Groups: appliance-build, build, livecd-build, livemedia-build, srpm-build
> Tag options:
>   mock.new_chroot : 0  [f34]
>   mock.package_manager : 'dnf' [f34]
>   rpm.macro.dist : ('%{!?distprefix0:%{?distprefix}}%{expand:%{lua:for 
> i=0, do '
>  'print("%{?distprefix" .. i .."}") '
>  'end}}.eln%{eln}%{?with_bootstrap:~bootstrap}')
>   rpm.macro.eln : 108
>   rpm.macro.rhel : 9
> This tag is a buildroot for one or more targets
> Current repo: repo#2902927: 2021-01-30 19:46:56.747509+00:00
> Targets that build from this tag:
>   eln-rebuild
>   eln
>   eln-candidate
> Inheritance:
>   0 eln [22492]
>   5 f34-build [27045]
> 
> We had a request to enable this for side-tags: 
> 
> https://pagure.io/fedora-infrastructure/issue/9048

That'd be great.

> what was passed in for the build so if you just had the src.rpm you
> could figure out what was going on?

Yeah, I think we should consider the creation of record so this can be
reviewed after the fact as a requirement for enablement.

> but didn't want to due to how this could be used. 
> 
> My thought was to try and ask koji developers to add some kind of
> 'non-mergable' bit on side-tags. So, if you want to use a side tag to
> play around with/test something you could, but if you adjusted things
> beyond some whitelist, the side-tag would not be mergable back into
> rawhide, and we could allow _with_bootstrap. 

I think those worries are overblown. When a packager has the rights to
build an rpm (in a side-tag or not), they also have the right to
commit whatever they want to the spec file. That packager also already
has the ability to influence the build by tagging things into the
side-tag. This type of modification does not require modification of
the spec file. There isn't any fundamental difference in influencing
the build through macros compared to the other ways. We just need to
record what was set, similarly to the list of rpms that is pulled into
the buildroot.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-30 Thread Kevin Fenzi 
On Fri, Jan 29, 2021 at 03:49:28PM +0100, Miro Hrončok wrote:
> 
> Currently a Koji task says:
> 
> Source: 
> git+https://src.fedoraproject.org/rpms/python3.9.git#563b4f4b59a9f81095acfe30899fdc4b040c1b9b
> Build Target: f34-rebuild
> Options:
>   fail_fast = True
>   wait_builds =
> 
> Example taken from 
> https://koji.fedoraproject.org/koji/taskinfo?taskID=60654655
> 
> With this feature, it would say:
> 
> Source: 
> git+https://src.fedoraproject.org/rpms/python3.9.git#563b4f4b59a9f81095acfe30899fdc4b040c1b9b
> Build Target: f34-rebuild
> Options:
>   fail_fast = True
>   wait_builds =
> Macro overrides:
>   _with_bootstrap: 1
> 
> And when reproducing, you'd just run the same command, e.g.
> 
> $ koji build f34-rebuild --fail-fast --macro '_with_bootstrap 1' 
> 'git+https://src.fedoraproject.org/rpms/python3.9.git#563b4f4b59a9f81095acfe30899fdc4b040c1b9b'
> 
> The Koji web interface could even display the command.
> 
> That's very much as reproduce as now.

So, koji already can set macros per tag.
For example, eln sig uses this: 

➜  ~ koji taginfo eln-build
Tag: eln-build [22493]
Arches: i686 x86_64 aarch64 ppc64le s390x
Groups: appliance-build, build, livecd-build, livemedia-build, srpm-build
Tag options:
  mock.new_chroot : 0  [f34]
  mock.package_manager : 'dnf' [f34]
  rpm.macro.dist : ('%{!?distprefix0:%{?distprefix}}%{expand:%{lua:for i=0, 
do '
 'print("%{?distprefix" .. i .."}") '
 'end}}.eln%{eln}%{?with_bootstrap:~bootstrap}')
  rpm.macro.eln : 108
  rpm.macro.rhel : 9
This tag is a buildroot for one or more targets
Current repo: repo#2902927: 2021-01-30 19:46:56.747509+00:00
Targets that build from this tag:
  eln-rebuild
  eln
  eln-candidate
Inheritance:
  0 eln [22492]
  5 f34-build [27045]

We had a request to enable this for side-tags: 

https://pagure.io/fedora-infrastructure/issue/9048

but didn't want to due to how this could be used. 

My thought was to try and ask koji developers to add some kind of
'non-mergable' bit on side-tags. So, if you want to use a side tag to
play around with/test something you could, but if you adjusted things
beyond some whitelist, the side-tag would not be mergable back into
rawhide, and we could allow _with_bootstrap. 

Ideally though this could be something in rpm... so it keeps track of
what was passed in for the build so if you just had the src.rpm you
could figure out what was going on?

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Miro Hrončok 

On 29. 01. 21 15:02, Richard W.M. Jones wrote:

On Fri, Jan 29, 2021 at 12:44:58PM +0100, Miro Hrončok wrote:

For Koji, you cannot install arbitrary packages.

What if instead, Koji allowed to set arbitrary macros on builds (and
it keeps their definition for further reference). That way, you will
be able to do:

  $ fedpkg build --with bootstrap
  $ koji wait-repo ...
  $ fedpkg build

 From the same commit. No package installation required.


How would it "keep their definition"?  It sounds like it could be a
trap making it hard to understand how to reproduce a build issue /
reproduce a build at all.

Having said that, it's something I've wanted in the past and it'd be a
nice feature.


Currently a Koji task says:

Source: 
git+https://src.fedoraproject.org/rpms/python3.9.git#563b4f4b59a9f81095acfe30899fdc4b040c1b9b

Build Target: f34-rebuild
Options:
  fail_fast = True
  wait_builds =

Example taken from https://koji.fedoraproject.org/koji/taskinfo?taskID=60654655



With this feature, it would say:

Source: 
git+https://src.fedoraproject.org/rpms/python3.9.git#563b4f4b59a9f81095acfe30899fdc4b040c1b9b

Build Target: f34-rebuild
Options:
  fail_fast = True
  wait_builds =
Macro overrides:
  _with_bootstrap: 1


And when reproducing, you'd just run the same command, e.g.

$ koji build f34-rebuild --fail-fast --macro '_with_bootstrap 1' 
'git+https://src.fedoraproject.org/rpms/python3.9.git#563b4f4b59a9f81095acfe30899fdc4b040c1b9b'


The Koji web interface could even display the command.


That's very much as reproduce as now.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Robin Lee 
On Fri, Jan 29, 2021 at 7:33 PM Zbigniew Jędrzejewski-Szmek
 wrote:
>
> Hello fellow packagers!
>
> The subject of bootstrapping came up on fedora-devel recently.
> I had the following idea, about which I would love to hear some feedback:
>
> == Problem:
> building packages with bootstrap currently involves doing *two*
> patches to the spec file: first to add '%global _without_bootstrap 1',
> then comes a rebuild, second to remove the macro, and then comes
> another rebuild.
>
> == Partial solution
> Let's have an rpm that provides a single file that sets the macro for us:
> $ rpm -qpl noarch/rpm-with-bootstrap-0-1.fc34.noarch.rpm
> /usr/lib/rpm/macros.d/macros.rpm-with-bootstrap
> $ cat rpm-macros-bootstrap/macros.rpm-with-bootstrap
> # Enable %%with_bootstrap for all builds
> %_with_bootstrap 1

For my experience, a boolean bootstrap macro is not enough
for bootstrapping the whole Feodra distribution. Especially for
cross-bootstrapping Fedora to a new architecture.

We may need a few bootstrap stages.
For each stage we can define different value for an rpm macro.
For example:
%define _bootstrap_stage 1
%define _bootstrap_stage 2


And the %_bootstrap or %bootstrap macro may have been used
by different packages for different semantics.
It is better to use a brand new macro for distribution bootstrap.

-robin

>
> Then we can do the following:
> $ rpmdev-bumpspec 'Do rebuild w/ bootstrap'
> $ mock -i rpm-with-bootstrap
> $ fedpkg mockbuild
> $ rpmdev-bumpspec 'Do rebuild w/o bootstrap'
> $ mock -i rpm-without-bootstrap [3]
> $ fedpkg mockbuild
> Voilà!
>
> The same pattern should work with side-tags in koji.
>
> I prepared a poc implementation in [1] which builds
> rpm-{with,without}-{bootstrap,tests,lto}, and a test package [2] which
> prints the values of %with_bootstrap, %with_tests, %with_lto.
>
> == Full solution
> If we have automatic version bumps, this would become even simpler:
> $ mock -i rpm-with-bootstrap
> $ fedpkg mockbuild
> $ mock --dnf-cmd remove rpm-with-bootstrap
> $ fedpkg mockbuild
>
> My idea would be to submit [1] for package-review so that it's
> generally available.
>
> Note that this works if the package we're building uses
>   %bcond_with bootstrap
> or
>   %bcond_without bootstrap
> I picked %{with bootstrap}, %{with lto}, %{with tests} as
> generally-useful settings. I think it is worth standarizing the names
> like this, and at least %{with bootstrap} and %{with tests} could be
> added to packaging guidelines.
>
> [1] https://pagure.io/rpm-macros-bootstrap
> [2] https://pagure.io/rpm-macros-test
> [3] rpm-without-bootstrap has Conflicts:rpm-with-bootstrap, so installing
> one forces the other out. Alternatively, rpm-with-bootstrap may just
> be uninstalled.
>
> Zbyszek
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: 
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Miroslav Suchý 

Dne 29. 01. 21 v 13:44 Miro Hrončok napsal(a):

I'm confused. How is this different than:

  $ mock --with bootstrap

?


   mock --with bootstrap

simply define bootstrap macro and build
With the new option we can try to build **without* that macro, if the build fails then again **with* macro, and if it 
succeed then again **without** that macro. I guess it may be more useful together with --chain.


--
Miroslav Suchy, RHCA
Red Hat, Associate Manager, Community Packaging Tools, #brno, #fedora-buildsys
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Richard W.M. Jones 
On Fri, Jan 29, 2021 at 12:44:58PM +0100, Miro Hrončok wrote:
> For Koji, you cannot install arbitrary packages.
> 
> What if instead, Koji allowed to set arbitrary macros on builds (and
> it keeps their definition for further reference). That way, you will
> be able to do:
> 
>  $ fedpkg build --with bootstrap
>  $ koji wait-repo ...
>  $ fedpkg build
> 
> From the same commit. No package installation required.

How would it "keep their definition"?  It sounds like it could be a
trap making it hard to understand how to reproduce a build issue /
reproduce a build at all.

Having said that, it's something I've wanted in the past and it'd be a
nice feature.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Zbigniew Jędrzejewski-Szmek 
On Fri, Jan 29, 2021 at 01:42:43PM +0100, Miro Hrončok wrote:
> On 29. 01. 21 13:32, Zbigniew Jędrzejewski-Szmek wrote:
> >On Fri, Jan 29, 2021 at 12:44:58PM +0100, Miro Hrončok wrote:
> >>For Koji, you cannot install arbitrary packages.
> >I thought we can tag packages into a side-tag? If the
> >rpm-with-bootstrap was available as a normal package, it should
> >be possible to tag the built rpms into the side-tag.
> 
> Yes, but tagging makes them available, not installed.

That kills my idea ;(

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Miro Hrončok 

On 29. 01. 21 13:32, Miroslav Suchý wrote:

Dne 29. 01. 21 v 12:25 Zbigniew Jędrzejewski-Szmek napsal(a):

$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ rpmdev-bumpspec 'Do rebuild w/o bootstrap'
$ mock -i rpm-without-bootstrap [3]


Or we can implement

mock --try-bootstrap-macro


I'm confused. How is this different than:

 $ mock --with bootstrap

?

Which can set the bootstrap macro - that is much easier than having package 
which provides this macro.

In fact it is already doable with -D option.

When it will toggled using config_opts[] koji then can decide whether to use it 
or not.


The only question is: how to call this config/cmdline option? Because we already 
use the term "bootstrap chroot" which can be confusing.




--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Miro Hrončok 

On 29. 01. 21 13:33, Vít Ondruch wrote:


And please also note, that you can use this for modules, where you can 
specify macros for specific module in modulemd file


But that is in fact a "trick" because MSB creates a package and installs it into 
the buildroot. Koji has no knowledge about the macros.


--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Miro Hrončok 

On 29. 01. 21 13:32, Zbigniew Jędrzejewski-Szmek wrote:

On Fri, Jan 29, 2021 at 12:44:58PM +0100, Miro Hrončok wrote:

For Koji, you cannot install arbitrary packages.

I thought we can tag packages into a side-tag? If the
rpm-with-bootstrap was available as a normal package, it should
be possible to tag the built rpms into the side-tag.


Yes, but tagging makes them available, not installed.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Miro Hrončok 

On 29. 01. 21 13:38, Daniel Mach wrote:
* don't forget that NVR has to be unique in koji so you can't build the same 
build twice. Having an ability to set %dist via koji might be nice 
for bootstrapping.


A bootstrap %bcond already amends dist to be .fc34~bootstrap when enabled.

(The %bcond needs to be defined before Release: to have a different NEVR).

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Daniel Mach 



On 1/29/21 12:44 PM, Miro Hrončok wrote:

On 29. 01. 21 12:25, Zbigniew Jędrzejewski-Szmek wrote:

Hello fellow packagers!

The subject of bootstrapping came up on fedora-devel recently.
I had the following idea, about which I would love to hear some feedback:

== Problem:
building packages with bootstrap currently involves doing *two*
patches to the spec file: first to add '%global _without_bootstrap 1',
then comes a rebuild, second to remove the macro, and then comes
another rebuild.

== Partial solution
Let's have an rpm that provides a single file that sets the macro for us:
$ rpm -qpl noarch/rpm-with-bootstrap-0-1.fc34.noarch.rpm
/usr/lib/rpm/macros.d/macros.rpm-with-bootstrap
$ cat rpm-macros-bootstrap/macros.rpm-with-bootstrap
# Enable %%with_bootstrap for all builds
%_with_bootstrap 1

Then we can do the following:
$ rpmdev-bumpspec 'Do rebuild w/ bootstrap'
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ rpmdev-bumpspec 'Do rebuild w/o bootstrap'
$ mock -i rpm-without-bootstrap [3]
$ fedpkg mockbuild
Voilà!

The same pattern should work with side-tags in koji.

I prepared a poc implementation in [1] which builds
rpm-{with,without}-{bootstrap,tests,lto}, and a test package [2] which
prints the values of %with_bootstrap, %with_tests, %with_lto.

== Full solution
If we have automatic version bumps, this would become even simpler:
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ mock --dnf-cmd remove rpm-with-bootstrap
$ fedpkg mockbuild


A more general note: Your examples involve mock, not Koji.

For mock, you can already do:

  $ fedpkg mockbuild --with bootstrap
  $ mock --install 
  $ fedpkg mockbuild

For Koji, you cannot install arbitrary packages.

What if instead, Koji allowed to set arbitrary macros on builds (and it 
keeps their definition for further reference). That way, you will be 
able to do:


  $ fedpkg build --with bootstrap
  $ koji wait-repo ...
  $ fedpkg build

 From the same commit. No package installation required.



Yes, defining rpm macros in koji would solve that for the build system.
I'd love to have the following features:
* general macros that are not package or arch specific (probably most of 
them)

* per-(package,arch) macro overrides
* macro inheritance across koji tags
* don't forget that NVR has to be unique in koji so you can't build the 
same build twice. Having an ability to set %dist via koji might be nice 
for bootstrapping.


The downside is that anyone that is not using koji would have to 
retrieve the macros from somewhere (export macros into a RPM file and 
ship it as part of the release?). But that's only if the macros are used 
for building production RPMs that land in Fedora repos. RPMs used for 
bootstrapping don't have to be rebuildable outside the build system.



It seems to be related to this:
RFC: Feature macros (aka USE flags)
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/4DWDPKYBOXTCGKXJTOIVPO34A5BTOE3T/


And it's also not far from defining build macros in modules:
https://docs.fedoraproject.org/en-US/modularity/building-modules/fedora/defining-modules/#_build_macros_optional
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Vít Ondruch 


Dne 29. 01. 21 v 12:44 Miro Hrončok napsal(a):

On 29. 01. 21 12:25, Zbigniew Jędrzejewski-Szmek wrote:

Hello fellow packagers!

The subject of bootstrapping came up on fedora-devel recently.
I had the following idea, about which I would love to hear some 
feedback:


== Problem:
building packages with bootstrap currently involves doing *two*
patches to the spec file: first to add '%global _without_bootstrap 1',
then comes a rebuild, second to remove the macro, and then comes
another rebuild.

== Partial solution
Let's have an rpm that provides a single file that sets the macro for 
us:

$ rpm -qpl noarch/rpm-with-bootstrap-0-1.fc34.noarch.rpm
/usr/lib/rpm/macros.d/macros.rpm-with-bootstrap
$ cat rpm-macros-bootstrap/macros.rpm-with-bootstrap
# Enable %%with_bootstrap for all builds
%_with_bootstrap 1

Then we can do the following:
$ rpmdev-bumpspec 'Do rebuild w/ bootstrap'
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ rpmdev-bumpspec 'Do rebuild w/o bootstrap'
$ mock -i rpm-without-bootstrap [3]
$ fedpkg mockbuild
Voilà!

The same pattern should work with side-tags in koji.

I prepared a poc implementation in [1] which builds
rpm-{with,without}-{bootstrap,tests,lto}, and a test package [2] which
prints the values of %with_bootstrap, %with_tests, %with_lto.

== Full solution
If we have automatic version bumps, this would become even simpler:
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ mock --dnf-cmd remove rpm-with-bootstrap
$ fedpkg mockbuild


A more general note: Your examples involve mock, not Koji.

For mock, you can already do:

 $ fedpkg mockbuild --with bootstrap
 $ mock --install 
 $ fedpkg mockbuild

For Koji, you cannot install arbitrary packages.

What if instead, Koji allowed to set arbitrary macros on builds (and 
it keeps their definition for further reference).



https://pagure.io/koji/issue/416

https://pagure.io/koji/pull-request/898


And please also note, that you can use this for modules, where you can 
specify macros for specific module in modulemd file.



Vít



That way, you will be able to do:

 $ fedpkg build --with bootstrap
 $ koji wait-repo ...
 $ fedpkg build

From the same commit. No package installation required.





OpenPGP_signature
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Zbigniew Jędrzejewski-Szmek 
On Fri, Jan 29, 2021 at 12:44:58PM +0100, Miro Hrončok wrote:
> For Koji, you cannot install arbitrary packages.

I thought we can tag packages into a side-tag? If the
rpm-with-bootstrap was available as a normal package, it should
be possible to tag the built rpms into the side-tag.

> What if instead, Koji allowed to set arbitrary macros on builds (and
> it keeps their definition for further reference). That way, you will
> be able to do:
> 
>  $ fedpkg build --with bootstrap
>  $ koji wait-repo ...
>  $ fedpkg build
> 
> From the same commit. No package installation required.

OK, that's be nicer then my idea. But it needs support from koji.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Miroslav Suchý 

Dne 29. 01. 21 v 12:25 Zbigniew Jędrzejewski-Szmek napsal(a):

$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ rpmdev-bumpspec 'Do rebuild w/o bootstrap'
$ mock -i rpm-without-bootstrap [3]


Or we can implement

mock --try-bootstrap-macro

Which can set the bootstrap macro - that is much easier than having package 
which provides this macro.
In fact it is already doable with -D option.

When it will toggled using config_opts[] koji then can decide whether to use it 
or not.

The only question is: how to call this config/cmdline option? Because we already use the term "bootstrap chroot" which 
can be confusing.


--
Miroslav Suchy, RHCA
Red Hat, Associate Manager, Community Packaging Tools, #brno, #fedora-buildsys
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Miro Hrončok 

On 29. 01. 21 12:25, Zbigniew Jędrzejewski-Szmek wrote:

Hello fellow packagers!

The subject of bootstrapping came up on fedora-devel recently.
I had the following idea, about which I would love to hear some feedback:

== Problem:
building packages with bootstrap currently involves doing *two*
patches to the spec file: first to add '%global _without_bootstrap 1',
then comes a rebuild, second to remove the macro, and then comes
another rebuild.

== Partial solution
Let's have an rpm that provides a single file that sets the macro for us:
$ rpm -qpl noarch/rpm-with-bootstrap-0-1.fc34.noarch.rpm
/usr/lib/rpm/macros.d/macros.rpm-with-bootstrap
$ cat rpm-macros-bootstrap/macros.rpm-with-bootstrap
# Enable %%with_bootstrap for all builds
%_with_bootstrap 1

Then we can do the following:
$ rpmdev-bumpspec 'Do rebuild w/ bootstrap'
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ rpmdev-bumpspec 'Do rebuild w/o bootstrap'
$ mock -i rpm-without-bootstrap [3]
$ fedpkg mockbuild
Voilà!

The same pattern should work with side-tags in koji.

I prepared a poc implementation in [1] which builds
rpm-{with,without}-{bootstrap,tests,lto}, and a test package [2] which
prints the values of %with_bootstrap, %with_tests, %with_lto.

== Full solution
If we have automatic version bumps, this would become even simpler:
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ mock --dnf-cmd remove rpm-with-bootstrap
$ fedpkg mockbuild


A more general note: Your examples involve mock, not Koji.

For mock, you can already do:

 $ fedpkg mockbuild --with bootstrap
 $ mock --install 
 $ fedpkg mockbuild

For Koji, you cannot install arbitrary packages.

What if instead, Koji allowed to set arbitrary macros on builds (and it keeps 
their definition for further reference). That way, you will be able to do:


 $ fedpkg build --with bootstrap
 $ koji wait-repo ...
 $ fedpkg build

From the same commit. No package installation required.

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

Re: bootstrapping without .spec modification

2021-01-29 Thread Miro Hrončok 

On 29. 01. 21 12:25, Zbigniew Jędrzejewski-Szmek wrote:

Hello fellow packagers!

The subject of bootstrapping came up on fedora-devel recently.
I had the following idea, about which I would love to hear some feedback:

== Problem:
building packages with bootstrap currently involves doing *two*
patches to the spec file: first to add '%global _without_bootstrap 1',
then comes a rebuild, second to remove the macro, and then comes
another rebuild.

== Partial solution
Let's have an rpm that provides a single file that sets the macro for us:
$ rpm -qpl noarch/rpm-with-bootstrap-0-1.fc34.noarch.rpm
/usr/lib/rpm/macros.d/macros.rpm-with-bootstrap
$ cat rpm-macros-bootstrap/macros.rpm-with-bootstrap
# Enable %%with_bootstrap for all builds
%_with_bootstrap 1

Then we can do the following:
$ rpmdev-bumpspec 'Do rebuild w/ bootstrap'
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ rpmdev-bumpspec 'Do rebuild w/o bootstrap'
$ mock -i rpm-without-bootstrap [3]
$ fedpkg mockbuild
Voilà!

The same pattern should work with side-tags in koji.

I prepared a poc implementation in [1] which builds
rpm-{with,without}-{bootstrap,tests,lto}, and a test package [2] which
prints the values of %with_bootstrap, %with_tests, %with_lto.


Honestly, this feels very magical to me. Useful maybe, but I prefer to be 
explicit in the spec.


OTOH if you disable tests like this, koschei will still build with tests, so 
this is excellent for temporary tests disablement :)



== Full solution
If we have automatic version bumps, this would become even simpler:
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ mock --dnf-cmd remove rpm-with-bootstrap
$ fedpkg mockbuild


Not that %{with bootstrap} also mangles the dist tag (if defined before the 
Release tag), so no bump needed already \o/



My idea would be to submit [1] for package-review so that it's
generally available.

Note that this works if the package we're building uses
   %bcond_with bootstrap
or
   %bcond_without bootstrap
I picked %{with bootstrap}, %{with lto}, %{with tests} as
generally-useful settings. I think it is worth standarizing the names
like this, and at least %{with bootstrap} and %{with tests} could be
added to packaging guidelines.


We have recently tried to standardize %{with tests} but there was some pushback.

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/443LZIX4XLZL6NMF3M7HAGHKPNA4TRYN/

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org

bootstrapping without .spec modification

2021-01-29 Thread Zbigniew Jędrzejewski-Szmek 
Hello fellow packagers!

The subject of bootstrapping came up on fedora-devel recently.
I had the following idea, about which I would love to hear some feedback:

== Problem:
building packages with bootstrap currently involves doing *two*
patches to the spec file: first to add '%global _without_bootstrap 1',
then comes a rebuild, second to remove the macro, and then comes
another rebuild.

== Partial solution
Let's have an rpm that provides a single file that sets the macro for us:
$ rpm -qpl noarch/rpm-with-bootstrap-0-1.fc34.noarch.rpm 
/usr/lib/rpm/macros.d/macros.rpm-with-bootstrap
$ cat rpm-macros-bootstrap/macros.rpm-with-bootstrap 
# Enable %%with_bootstrap for all builds
%_with_bootstrap 1

Then we can do the following:
$ rpmdev-bumpspec 'Do rebuild w/ bootstrap'
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ rpmdev-bumpspec 'Do rebuild w/o bootstrap'
$ mock -i rpm-without-bootstrap [3]
$ fedpkg mockbuild
Voilà!

The same pattern should work with side-tags in koji.

I prepared a poc implementation in [1] which builds
rpm-{with,without}-{bootstrap,tests,lto}, and a test package [2] which
prints the values of %with_bootstrap, %with_tests, %with_lto.

== Full solution
If we have automatic version bumps, this would become even simpler:
$ mock -i rpm-with-bootstrap
$ fedpkg mockbuild
$ mock --dnf-cmd remove rpm-with-bootstrap
$ fedpkg mockbuild

My idea would be to submit [1] for package-review so that it's
generally available.

Note that this works if the package we're building uses
  %bcond_with bootstrap
or
  %bcond_without bootstrap
I picked %{with bootstrap}, %{with lto}, %{with tests} as
generally-useful settings. I think it is worth standarizing the names
like this, and at least %{with bootstrap} and %{with tests} could be
added to packaging guidelines.

[1] https://pagure.io/rpm-macros-bootstrap
[2] https://pagure.io/rpm-macros-test
[3] rpm-without-bootstrap has Conflicts:rpm-with-bootstrap, so installing
one forces the other out. Alternatively, rpm-with-bootstrap may just
be uninstalled.

Zbyszek
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org