Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Ondřej Budai]

On Wed, Jun 28, 2023 at 2:53 PM Neal Gompa  wrote:

> On Wed, Jun 28, 2023 at 8:25 AM Ondřej Budai  wrote:
> >
> > Yep, that's a shortcoming of how we currently upload the builds to koji.
> It's something we would like to tackle in the upcoming quarter, see the
> tracking ticket: https://issues.redhat.com/browse/COMPOSER-1801 (public
> link).
> >
>
> I can't comment on that ticket. :(
>
>
FTR, this is now fixed! :) A permission issue on our side.

Ondřej
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Thu, 2023-06-29 at 22:00 +0200, Simon de Vlieger wrote:
> 
> Also, for understanding I've been diving through the kickstart 
> repositories to define a common base set and interdependencies. I've 
> graphed out the kickstart files and their relationships here: 
> https://supakeen.fedorapeople.org/kickstart-graph.png it seems to have 
> found some includes to kickstarts which no longer exist in the repositories!

That is entirely possible. There may be some kickstarts in there that
aren't actually *used* for anything any more. Don't get me wrong, this
stuff could definitely get improved, but it is important to understand
exactly how the current mess works before you set about fixing it. :D
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Simon de Vlieger]

On 6/28/23 17:06, Adam Williamson wrote:

On Wed, 2023-06-28 at 13:01 +0200, Ondřej Budai wrote:

I already answered you here:
https://pagure.io/fedora-workstation/issue/384#comment-862709

TL;DR: Let's figure this out when we are migrating other artifacts. For
now, the blueprint will be empty/minimal.


Thanks a lot for engaging with the feedback. However, I'm not convinced
by that answer, and I would prefer to follow up here; I suspect
feedback to an issue in the workstation tracker might well not be
captured in the Change process.

I think the points discussed in the ticket (layering/inheritance, and
package removals) are critical and they need an answer to be figured
out *before* we start switching Fedora live images to be built with a
different tool. I don't see how it can be good for Fedora if we have
*some* live images built with livemedia-creator and *some* live images
built with Image Builder - but if we start converting images without
figuring out what to do about layering and package exclusions, that's
exactly what we risk.

We do really kinda need inheritance to maintain the live image
definitions sensibly. (Also, a related point Neal didn't mention: we
share some elements of configuration between the live images and the
aarch64 disk images, since both are defined by kickstarts in the
fedora-kickstarts repo.) And we really need package exclusions to keep
some images to a sensible size.

We don't necessarily need these things to be completely implemented in
order to switch Workstation over, I don't think, but I think we should
at least have a *plan* for them. And ideally a timeframe.


Hey Adam,

I've made a beginning with starting to track work ongoing on building 
Fedora within the projects that make up image builder. It's far from 
complete (I'm still backfilling with previous relevant issues and 
creating new issues) but it will give a bit better insight into options 
being considered and work being done to improve support for Fedora: 
https://github.com/orgs/osbuild/projects/3


I'm still working on making the plan more concrete but I think it's 
definitely feasible to, as a followup (likely for Fedora 40) allow 
blueprints to define excludes.


Still working on things related to inheritance, there's two ways of 
going about it. One is a direct support for including blueprint 
snippet(s) in other blueprints (this would be non-standard for the 
current TOML format that the languages are in). The other is more a 
mitigation and it would be to make it so that all current kickstarts can 
be defined without inheritance and then to later work that into a form 
where inheritance is brought back in.


Also, for understanding I've been diving through the kickstart 
repositories to define a common base set and interdependencies. I've 
graphed out the kickstart files and their relationships here: 
https://supakeen.fedorapeople.org/kickstart-graph.png it seems to have 
found some includes to kickstarts which no longer exist in the repositories!


Regards,

Simon

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Davide Cavalca]

On 2023-06-29 09:42, Adam Williamson wrote:

On Thu, 2023-06-29 at 02:22 -0400, Neal Gompa wrote:

And since Lorax (which is what we use
for live and ARM images) requires Anaconda to understand the target
system to install, it couldn't be used for creating these images
either because that requires teaching Anaconda about Apple Silicon
Macs.


Hmm, well, why don't we do that? It knows/knew about PPC Macs and Intel
Macs, after all...


This is planned, see https://pagure.io/fedora-asahi/project/issue/9 and 
https://pagure.io/fedora-asahi/project/issue/10. It's worth noting that 
these systems boot _very_ differently from standard aarch64 machines 
(see 
https://github.com/AsahiLinux/docs/wiki/Open-OS-Ecosystem-on-Apple-Silicon-Macs 
for details).


Cheers
Davide
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Thu, 2023-06-29 at 02:22 -0400, Neal Gompa wrote:
> And since Lorax (which is what we use
> for live and ARM images) requires Anaconda to understand the target
> system to install, it couldn't be used for creating these images
> either because that requires teaching Anaconda about Apple Silicon
> Macs.

Hmm, well, why don't we do that? It knows/knew about PPC Macs and Intel
Macs, after all...
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Neal Gompa]

On Thu, Jun 29, 2023 at 1:06 AM Simon de Vlieger  wrote:
>
> On 6/28/23 22:32, Neal Gompa wrote:
> > On Wed, Jun 28, 2023 at 4:26 PM Simon de Vlieger  wrote:
> >>
> >> On 6/28/23 21:12, Adam Williamson wrote:
> >>
> >>> The current Workstation live has the package exclusions from fedora-
> >>> workstation-common.ks in it:
> >>>
> >>> https://pagure.io/fedora-kickstarts/blob/main/f/fedora-workstation-common.ks
> >>>
> >>> so it excludes three groups that are part of the 'standard' for live
> >>> images but which Workstation doesn't want to include, and it excludes
> >>> two packages that usually get pulled in as part of the installer
> >>> environment. The reiserfs-utils exclusion could actually be dropped as
> >>> we dropped reiserfs-utils from comps long ago, but the gfs2-utils
> >>> exclusion is still 'active'.
> >>
> >> Thanks, yes I saw this when I ran ksflatten on it to come up with the
> >> package set that was needed for the live installer that can be built by
> >> osbuild-composer at this moment.
> >>
> >>> There is a bit of "defining base Fedora" - that's what fedora-live-
> >>> base.ks does - but there is other stuff too. One very prevalent pattern
> >>> is that we share a lot of stuff between live image and disk image
> >>> definitions. So for e.g. for Workstation, we have these chains:
> >>>
> >>> fedora-live-workstation.ks inherits from fedora-live-base.ks and
> >>> fedora-workstation-common.ks
> >>> fedora-disk-workstation.ks inherits from fedora-disk-base.ks, fedora-
> >>> disk-xbase.ks and fedora-workstation-common.ks
> >>>
> >>> so things that are 'basic' to each particular type of image are shared,
> >>> but also things that are 'basic' to each edition or spin are shared, so
> >>> we're not doing them twice between the lives and the disk images.
> >>>
> >>> Then we have things like labs/spins that are based on desktop
> >>> spins/editions, but extend them. For e.g., the Design Suite lab is
> >>> based on Workstation, so it inherits from fedora-live-workstation.ks .
> >>> Astronomy_KDE and Scientific_KDE, obviously, inherit from the KDE
> >>> kickstarts.
> >>>
> >>> There's also a 'minimization' pattern where several kickstarts inherit
> >>> from fedora-live-minimization.ks , which does/did some shared
> >>> minimization stuff. I kinda disliked that pattern and I've managed to
> >>> pare that file down to just `-hplip` now, but that's still there as I
> >>> haven't managed to shift it.
> >>>
> >>> So, there's a lot going on with the inheritance stuff. :D It definitely
> >>> needs to be evaluated at least. You can just do `grep include *.ks` in
> >>> the fedora-kickstarts repo to get a feel for everything.
> >>
> >> Thanks for that write up, I'm going to go spelunking in this repository
> >> and come up with a plan on how to support an approach like this to
> >> define editions/spins in blueprints so there's no wall in front of us
> >> for future building of those images as well.
> >>
> >> I can see it hopefully becoming *easier* to define spins, editions, and
> >> remixes of Fedora in the future.
> >>
> >
> > Comparatively, the Fedora Asahi Remix is defined using KIWI
> > descriptions using composition and inheritance of profiles and config
> > snippets.
> >
> > Top level file:
> > https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/config.xml
> >
> > Desktop environment fragment:
> > https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/components/desktop-environments.xml
> > Common packages:
> > https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/components/base.xml
> > Boot packages: 
> > https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/components/boot.xml
> > Workstation platform:
> > https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/platforms/workstation.xml
> >
> > I would say personally that not being able to do this is a blocker for
> > adopting osbuild-composer blueprints.
>
> Thank you Neal. Is there a reason why Fedora Asahi does not use the
> kickstart system? If so it'd be nice if we can support that use case as
> well in osbuild-composer :)
>
> I'll take the kiwi example files (and the kickstarts) in mind to work on
> a direction for blueprints to take to support these usecases.
>

I wouldn't subject my worst enemies to Oz and ImageFactory (which is
what we use for Cloud images). And since Lorax (which is what we use
for live and ARM images) requires Anaconda to understand the target
system to install, it couldn't be used for creating these images
either because that requires teaching Anaconda about Apple Silicon
Macs. And reintroducing appliance-tools for Fedora images might make
some people upset. :)

As for why kiwi? I understand it quite well, and the kiwi team has
been extremely receptive to community needs above and beyond virtually
any other team I've worked with ever. I became a member of that team a
few years ago too. And there is Koji integration for kiwi, which means
that it can be (eventually) 

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Simon de Vlieger]

On 6/28/23 22:32, Neal Gompa wrote:

On Wed, Jun 28, 2023 at 4:26 PM Simon de Vlieger  wrote:


On 6/28/23 21:12, Adam Williamson wrote:


The current Workstation live has the package exclusions from fedora-
workstation-common.ks in it:

https://pagure.io/fedora-kickstarts/blob/main/f/fedora-workstation-common.ks

so it excludes three groups that are part of the 'standard' for live
images but which Workstation doesn't want to include, and it excludes
two packages that usually get pulled in as part of the installer
environment. The reiserfs-utils exclusion could actually be dropped as
we dropped reiserfs-utils from comps long ago, but the gfs2-utils
exclusion is still 'active'.


Thanks, yes I saw this when I ran ksflatten on it to come up with the
package set that was needed for the live installer that can be built by
osbuild-composer at this moment.


There is a bit of "defining base Fedora" - that's what fedora-live-
base.ks does - but there is other stuff too. One very prevalent pattern
is that we share a lot of stuff between live image and disk image
definitions. So for e.g. for Workstation, we have these chains:

fedora-live-workstation.ks inherits from fedora-live-base.ks and
fedora-workstation-common.ks
fedora-disk-workstation.ks inherits from fedora-disk-base.ks, fedora-
disk-xbase.ks and fedora-workstation-common.ks

so things that are 'basic' to each particular type of image are shared,
but also things that are 'basic' to each edition or spin are shared, so
we're not doing them twice between the lives and the disk images.

Then we have things like labs/spins that are based on desktop
spins/editions, but extend them. For e.g., the Design Suite lab is
based on Workstation, so it inherits from fedora-live-workstation.ks .
Astronomy_KDE and Scientific_KDE, obviously, inherit from the KDE
kickstarts.

There's also a 'minimization' pattern where several kickstarts inherit
from fedora-live-minimization.ks , which does/did some shared
minimization stuff. I kinda disliked that pattern and I've managed to
pare that file down to just `-hplip` now, but that's still there as I
haven't managed to shift it.

So, there's a lot going on with the inheritance stuff. :D It definitely
needs to be evaluated at least. You can just do `grep include *.ks` in
the fedora-kickstarts repo to get a feel for everything.


Thanks for that write up, I'm going to go spelunking in this repository
and come up with a plan on how to support an approach like this to
define editions/spins in blueprints so there's no wall in front of us
for future building of those images as well.

I can see it hopefully becoming *easier* to define spins, editions, and
remixes of Fedora in the future.



Comparatively, the Fedora Asahi Remix is defined using KIWI
descriptions using composition and inheritance of profiles and config
snippets.

Top level file:
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/config.xml

Desktop environment fragment:
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/components/desktop-environments.xml
Common packages:
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/components/base.xml
Boot packages: 
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/components/boot.xml
Workstation platform:
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/platforms/workstation.xml

I would say personally that not being able to do this is a blocker for
adopting osbuild-composer blueprints.


Thank you Neal. Is there a reason why Fedora Asahi does not use the 
kickstart system? If so it'd be nice if we can support that use case as 
well in osbuild-composer :)


I'll take the kiwi example files (and the kickstarts) in mind to work on 
a direction for blueprints to take to support these usecases.


Regards,

Simon
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Neal Gompa]

On Wed, Jun 28, 2023 at 4:26 PM Simon de Vlieger  wrote:
>
> On 6/28/23 21:12, Adam Williamson wrote:
>
> > The current Workstation live has the package exclusions from fedora-
> > workstation-common.ks in it:
> >
> > https://pagure.io/fedora-kickstarts/blob/main/f/fedora-workstation-common.ks
> >
> > so it excludes three groups that are part of the 'standard' for live
> > images but which Workstation doesn't want to include, and it excludes
> > two packages that usually get pulled in as part of the installer
> > environment. The reiserfs-utils exclusion could actually be dropped as
> > we dropped reiserfs-utils from comps long ago, but the gfs2-utils
> > exclusion is still 'active'.
>
> Thanks, yes I saw this when I ran ksflatten on it to come up with the
> package set that was needed for the live installer that can be built by
> osbuild-composer at this moment.
>
> > There is a bit of "defining base Fedora" - that's what fedora-live-
> > base.ks does - but there is other stuff too. One very prevalent pattern
> > is that we share a lot of stuff between live image and disk image
> > definitions. So for e.g. for Workstation, we have these chains:
> >
> > fedora-live-workstation.ks inherits from fedora-live-base.ks and
> > fedora-workstation-common.ks
> > fedora-disk-workstation.ks inherits from fedora-disk-base.ks, fedora-
> > disk-xbase.ks and fedora-workstation-common.ks
> >
> > so things that are 'basic' to each particular type of image are shared,
> > but also things that are 'basic' to each edition or spin are shared, so
> > we're not doing them twice between the lives and the disk images.
> >
> > Then we have things like labs/spins that are based on desktop
> > spins/editions, but extend them. For e.g., the Design Suite lab is
> > based on Workstation, so it inherits from fedora-live-workstation.ks .
> > Astronomy_KDE and Scientific_KDE, obviously, inherit from the KDE
> > kickstarts.
> >
> > There's also a 'minimization' pattern where several kickstarts inherit
> > from fedora-live-minimization.ks , which does/did some shared
> > minimization stuff. I kinda disliked that pattern and I've managed to
> > pare that file down to just `-hplip` now, but that's still there as I
> > haven't managed to shift it.
> >
> > So, there's a lot going on with the inheritance stuff. :D It definitely
> > needs to be evaluated at least. You can just do `grep include *.ks` in
> > the fedora-kickstarts repo to get a feel for everything.
>
> Thanks for that write up, I'm going to go spelunking in this repository
> and come up with a plan on how to support an approach like this to
> define editions/spins in blueprints so there's no wall in front of us
> for future building of those images as well.
>
> I can see it hopefully becoming *easier* to define spins, editions, and
> remixes of Fedora in the future.
>

Comparatively, the Fedora Asahi Remix is defined using KIWI
descriptions using composition and inheritance of profiles and config
snippets.

Top level file:
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/config.xml

Desktop environment fragment:
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/components/desktop-environments.xml
Common packages:
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/components/base.xml
Boot packages: 
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/components/boot.xml
Workstation platform:
https://pagure.io/fedora-asahi/kiwi-descriptions/blob/rawhide/f/platforms/workstation.xml

I would say personally that not being able to do this is a blocker for
adopting osbuild-composer blueprints.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Simon de Vlieger]

On 6/28/23 21:12, Adam Williamson wrote:


The current Workstation live has the package exclusions from fedora-
workstation-common.ks in it:

https://pagure.io/fedora-kickstarts/blob/main/f/fedora-workstation-common.ks

so it excludes three groups that are part of the 'standard' for live
images but which Workstation doesn't want to include, and it excludes
two packages that usually get pulled in as part of the installer
environment. The reiserfs-utils exclusion could actually be dropped as
we dropped reiserfs-utils from comps long ago, but the gfs2-utils
exclusion is still 'active'.


Thanks, yes I saw this when I ran ksflatten on it to come up with the 
package set that was needed for the live installer that can be built by 
osbuild-composer at this moment.



There is a bit of "defining base Fedora" - that's what fedora-live-
base.ks does - but there is other stuff too. One very prevalent pattern
is that we share a lot of stuff between live image and disk image
definitions. So for e.g. for Workstation, we have these chains:

fedora-live-workstation.ks inherits from fedora-live-base.ks and
fedora-workstation-common.ks
fedora-disk-workstation.ks inherits from fedora-disk-base.ks, fedora-
disk-xbase.ks and fedora-workstation-common.ks

so things that are 'basic' to each particular type of image are shared,
but also things that are 'basic' to each edition or spin are shared, so
we're not doing them twice between the lives and the disk images.

Then we have things like labs/spins that are based on desktop
spins/editions, but extend them. For e.g., the Design Suite lab is
based on Workstation, so it inherits from fedora-live-workstation.ks .
Astronomy_KDE and Scientific_KDE, obviously, inherit from the KDE
kickstarts.

There's also a 'minimization' pattern where several kickstarts inherit
from fedora-live-minimization.ks , which does/did some shared
minimization stuff. I kinda disliked that pattern and I've managed to
pare that file down to just `-hplip` now, but that's still there as I
haven't managed to shift it.

So, there's a lot going on with the inheritance stuff. :D It definitely
needs to be evaluated at least. You can just do `grep include *.ks` in
the fedora-kickstarts repo to get a feel for everything.


Thanks for that write up, I'm going to go spelunking in this repository 
and come up with a plan on how to support an approach like this to 
define editions/spins in blueprints so there's no wall in front of us 
for future building of those images as well.


I can see it hopefully becoming *easier* to define spins, editions, and 
remixes of Fedora in the future.


Regards,

Simon

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Wed, 2023-06-28 at 18:44 +0200, Simon de Vlieger wrote:
> 
> What I would like to know is what is the minimum customizations needed 
> in blueprints for this change proposal and what is necessary for 
> supporting editions and spins down the line.

Thanks, Simon. So, we obviously don't need layering/inheritance
*implemented* to build a single live image. I'm not saying we do, just
that we need a plan to address the need that is currently addressed by
kickstart inheritance.

The current Workstation live has the package exclusions from fedora-
workstation-common.ks in it:

https://pagure.io/fedora-kickstarts/blob/main/f/fedora-workstation-common.ks

so it excludes three groups that are part of the 'standard' for live
images but which Workstation doesn't want to include, and it excludes
two packages that usually get pulled in as part of the installer
environment. The reiserfs-utils exclusion could actually be dropped as
we dropped reiserfs-utils from comps long ago, but the gfs2-utils
exclusion is still 'active'.
> 
> The second one is likely harder but seems to be based very much on 
> 'thats how we do it now'. Personally I'd prefer having base fedora 
> defined in code and each edition/spin being a small blueprint that only 
> adds onto that base Fedora. That way spins would be less affected by 
> changes in inherited kickstarts/blueprints.
> 
> I'll dive into the kickstart repository to see what exactly is being 
> done but it's likely that a lot of what happens in kickstarts currently 
> already happens in code in osbuild-composer (e.g. adjustments to 
> bootloaders/partitions/extra packages to install based on the output 
> image type and architecture).

It's not really, or not entirely, about either of those. Ensuring all
bootloader packages is present is really handled by comps - they are in
the anaconda-tools group, and we always pull that into live images
(it's in fedora-live-base.ks ).

There is a bit of "defining base Fedora" - that's what fedora-live-
base.ks does - but there is other stuff too. One very prevalent pattern
is that we share a lot of stuff between live image and disk image
definitions. So for e.g. for Workstation, we have these chains:

fedora-live-workstation.ks inherits from fedora-live-base.ks and
fedora-workstation-common.ks
fedora-disk-workstation.ks inherits from fedora-disk-base.ks, fedora-
disk-xbase.ks and fedora-workstation-common.ks

so things that are 'basic' to each particular type of image are shared,
but also things that are 'basic' to each edition or spin are shared, so
we're not doing them twice between the lives and the disk images. 

Then we have things like labs/spins that are based on desktop
spins/editions, but extend them. For e.g., the Design Suite lab is
based on Workstation, so it inherits from fedora-live-workstation.ks .
Astronomy_KDE and Scientific_KDE, obviously, inherit from the KDE
kickstarts.

There's also a 'minimization' pattern where several kickstarts inherit
from fedora-live-minimization.ks , which does/did some shared
minimization stuff. I kinda disliked that pattern and I've managed to
pare that file down to just `-hplip` now, but that's still there as I
haven't managed to shift it.

So, there's a lot going on with the inheritance stuff. :D It definitely
needs to be evaluated at least. You can just do `grep include *.ks` in
the fedora-kickstarts repo to get a feel for everything.
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Jiří Konečný]

Adam Williamson  napsal:
>On Wed, 2023-06-28 at 18:21 +0200, Jiří Konečný wrote:
>> > 
>> > I think the points discussed in the ticket (layering/inheritance, and
>> > package removals) are critical and they need an answer to be figured
>> > out *before* we start switching Fedora live images to be built with a
>> > different tool. I don't see how it can be good for Fedora if we have
>> > *some* live images built with livemedia-creator and *some* live images
>> > built with Image Builder - but if we start converting images without
>> > figuring out what to do about layering and package exclusions, that's
>> > exactly what we risk.
>> 
>> Honestly, I would rather like having exactly that. Image Builder slowly 
>> taking over the image builds.
>
>Slowly taking over is fine. But there needs to be a clear path to
>slowly taking over. If we have already spotted a wall built across that
>path, our response cannot be "eh, we'll keep driving along and deal
>with the wall when it's a bit closer". We need, at minimum, a Wall
>Removal Plan. :D
>
>>  If you do that at once you can break more and have less time for 
>> improvements.
>> Even now, livemedia-creator used to build Live is not the same workflow as 
>> boot.iso builds by Lorax (even when it is the same codebase).
>
>Indeed. This is part of the bitter past experience which is informing
>my posts in this thread!
>
>(Although, of course, building a live image is just a different thing
>from building an installer image.)
>> 
>> I would be surprised (maybe I'll be :D) if there is no solution for merging 
>> TOML files. In case of kickstart we had to reinvent the wheel because 
>> Kickstart is a new format. With TOML you have benefit of existing ecosystem.
>
>"Kickstart is a new format"? Kickstart is at least 11 years older than
>TOML, though probably much more (with a quick search, the earliest
>definite reference to kickstart that I can find dates to Red Hat Linux
>8.0, released in 2002; TOML's first release was in 2013, per
>Wikipedia).

Sorry, I wanted to write 'was' instead of 'is'. I meant that Kickstart was 
invented for Anaconda and used only there. Noone adopted this format outside of 
Anaconda. My reaction wasn't about how old but more about it is a single 
purpose format. That means, the tooling was created by just a small group of 
people who care about Anaconda. As I'm aware of, all the tooling for Kickstart 
was created by single team.
This is not the case for TOML format.

Jirka
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Simon de Vlieger]

On 6/28/23 17:06, Adam Williamson wrote:

On Wed, 2023-06-28 at 13:01 +0200, Ondřej Budai wrote:

I already answered you here:
https://pagure.io/fedora-workstation/issue/384#comment-862709

TL;DR: Let's figure this out when we are migrating other artifacts. For
now, the blueprint will be empty/minimal.


Thanks a lot for engaging with the feedback. However, I'm not convinced
by that answer, and I would prefer to follow up here; I suspect
feedback to an issue in the workstation tracker might well not be
captured in the Change process.

I think the points discussed in the ticket (layering/inheritance, and
package removals) are critical and they need an answer to be figured
out *before* we start switching Fedora live images to be built with a
different tool. I don't see how it can be good for Fedora if we have
*some* live images built with livemedia-creator and *some* live images
built with Image Builder - but if we start converting images without
figuring out what to do about layering and package exclusions, that's
exactly what we risk.

We do really kinda need inheritance to maintain the live image
definitions sensibly. (Also, a related point Neal didn't mention: we
share some elements of configuration between the live images and the
aarch64 disk images, since both are defined by kickstarts in the
fedora-kickstarts repo.) And we really need package exclusions to keep
some images to a sensible size.

We don't necessarily need these things to be completely implemented in
order to switch Workstation over, I don't think, but I think we should
at least have a *plan* for them. And ideally a timeframe.


Hey Adam (and Neal),

The good news is that the above *can* be done with `osbuild-composer`. 
The bad news is that you can't do it in a blueprint.


Generally we have defined distributions in code, and all of the above 
including sharing, exclusions, inheritance, can be done in those 
definitions in code.


We are in the process of splitting those definitions out of our main 
repository so pull requests and adjustments to those definitions are 
easier and involve less knowledge of the project.


With that out of the way, I would much prefer if all of the above would 
be achievable in blueprints.


What I would like to know is what is the minimum customizations needed 
in blueprints for this change proposal and what is necessary for 
supporting editions and spins down the line.


Taking from a bit further in the thread as well I currently have the 
following list of attention items for *this* change proposal:


- sensible debug logs in koji

And for followup work to support spins, editions, other image formats:

- package exclusions in blueprints
- blueprint inheritance

Of those the first one clashes a bit with the principles of making it 
hard(er) to break your image that osbuild-composer wants to follow but I 
personally see no issue with handing people that footgun.


The second one is likely harder but seems to be based very much on 
'thats how we do it now'. Personally I'd prefer having base fedora 
defined in code and each edition/spin being a small blueprint that only 
adds onto that base Fedora. That way spins would be less affected by 
changes in inherited kickstarts/blueprints.


I'll dive into the kickstart repository to see what exactly is being 
done but it's likely that a lot of what happens in kickstarts currently 
already happens in code in osbuild-composer (e.g. adjustments to 
bootloaders/partitions/extra packages to install based on the output 
image type and architecture).


Regards,

Simon
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Wed, 2023-06-28 at 18:21 +0200, Jiří Konečný wrote:
> > 
> > I think the points discussed in the ticket (layering/inheritance, and
> > package removals) are critical and they need an answer to be figured
> > out *before* we start switching Fedora live images to be built with a
> > different tool. I don't see how it can be good for Fedora if we have
> > *some* live images built with livemedia-creator and *some* live images
> > built with Image Builder - but if we start converting images without
> > figuring out what to do about layering and package exclusions, that's
> > exactly what we risk.
> 
> Honestly, I would rather like having exactly that. Image Builder slowly 
> taking over the image builds.

Slowly taking over is fine. But there needs to be a clear path to
slowly taking over. If we have already spotted a wall built across that
path, our response cannot be "eh, we'll keep driving along and deal
with the wall when it's a bit closer". We need, at minimum, a Wall
Removal Plan. :D

>  If you do that at once you can break more and have less time for 
> improvements.
> Even now, livemedia-creator used to build Live is not the same workflow as 
> boot.iso builds by Lorax (even when it is the same codebase).

Indeed. This is part of the bitter past experience which is informing
my posts in this thread!

(Although, of course, building a live image is just a different thing
from building an installer image.)
> 
> I would be surprised (maybe I'll be :D) if there is no solution for merging 
> TOML files. In case of kickstart we had to reinvent the wheel because 
> Kickstart is a new format. With TOML you have benefit of existing ecosystem.

"Kickstart is a new format"? Kickstart is at least 11 years older than
TOML, though probably much more (with a quick search, the earliest
definite reference to kickstart that I can find dates to Red Hat Linux
8.0, released in 2002; TOML's first release was in 2013, per
Wikipedia).
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Jiří Konečný]

Hi all,

Adam Williamson  napsal:
>On Wed, 2023-06-28 at 13:01 +0200, Ondřej Budai wrote:
>> I already answered you here:
>> https://pagure.io/fedora-workstation/issue/384#comment-862709
>> 
>> TL;DR: Let's figure this out when we are migrating other artifacts. For
>> now, the blueprint will be empty/minimal.
>
>Thanks a lot for engaging with the feedback. However, I'm not convinced
>by that answer, and I would prefer to follow up here; I suspect
>feedback to an issue in the workstation tracker might well not be
>captured in the Change process.
>
>I think the points discussed in the ticket (layering/inheritance, and
>package removals) are critical and they need an answer to be figured
>out *before* we start switching Fedora live images to be built with a
>different tool. I don't see how it can be good for Fedora if we have
>*some* live images built with livemedia-creator and *some* live images
>built with Image Builder - but if we start converting images without
>figuring out what to do about layering and package exclusions, that's
>exactly what we risk.

Honestly, I would rather like having exactly that. Image Builder slowly taking 
over the image builds. If you do that at once you can break more and have less 
time for improvements.
Even now, livemedia-creator used to build Live is not the same workflow as 
boot.iso builds by Lorax (even when it is the same codebase).

It's basically the same approach with Anaconda, replace just one for now to 
improve things based on feedback.

>
>We do really kinda need inheritance to maintain the live image
>definitions sensibly. (Also, a related point Neal didn't mention: we
>share some elements of configuration between the live images and the
>aarch64 disk images, since both are defined by kickstarts in the
>fedora-kickstarts repo.) And we really need package exclusions to keep
>some images to a sensible size.

I would be surprised (maybe I'll be :D) if there is no solution for merging 
TOML files. In case of kickstart we had to reinvent the wheel because Kickstart 
is a new format. With TOML you have benefit of existing ecosystem.

Best Regards,
Jirka
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Wed, 2023-06-28 at 13:01 +0200, Ondřej Budai wrote:
> I already answered you here:
> https://pagure.io/fedora-workstation/issue/384#comment-862709
> 
> TL;DR: Let's figure this out when we are migrating other artifacts. For
> now, the blueprint will be empty/minimal.

Thanks a lot for engaging with the feedback. However, I'm not convinced
by that answer, and I would prefer to follow up here; I suspect
feedback to an issue in the workstation tracker might well not be
captured in the Change process.

I think the points discussed in the ticket (layering/inheritance, and
package removals) are critical and they need an answer to be figured
out *before* we start switching Fedora live images to be built with a
different tool. I don't see how it can be good for Fedora if we have
*some* live images built with livemedia-creator and *some* live images
built with Image Builder - but if we start converting images without
figuring out what to do about layering and package exclusions, that's
exactly what we risk.

We do really kinda need inheritance to maintain the live image
definitions sensibly. (Also, a related point Neal didn't mention: we
share some elements of configuration between the live images and the
aarch64 disk images, since both are defined by kickstarts in the
fedora-kickstarts repo.) And we really need package exclusions to keep
some images to a sensible size.

We don't necessarily need these things to be completely implemented in
order to switch Workstation over, I don't think, but I think we should
at least have a *plan* for them. And ideally a timeframe.
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Wed, 2023-06-28 at 13:23 +0200, Ondřej Budai wrote:
> We can definitely flatten the JSON output into something that resembles a
> log file. I will let you know when this is done.
> 
> Note that every task also produces a manifest file - this is extremely
> useful, because you can just feed it to osbuild locally. Since the manifest
> basically fully specifies an image build (with locked package versions
> including their hashes), there's a high chance that you will be just able
> to reproduce the issue locally and use any tools you want for deeper
> debugging. This is one of the goals of the project: Make image builds more
> reproducible.

Unless the tool can pull older builds directly from Koji, I don't see
how this will work for more than a day or so. It's a cool capability,
but it seems to hinge on older builds being available, which they kinda
aren't very easily.

Reproducibility is awesome, but it doesn't replace good debug output.
For a start, you still need the useful output to debug the problem when
you reproduce it. And second, I assume building a live image is still
gonna take a significant amount of time, like 30 minutes? Typically, we
can debug a simple live image failure in about thirty seconds ("oh,
look at anaconda.log, it failed because of a broken dependency" is the
most common scenario) and probably fix it in less time than it would
take to reproduce the build.
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Wed, 2023-06-28 at 12:58 +0200, Ondřej Budai wrote:
> Hi Adam and Ben!
> 
> On Mon, Jun 26, 2023 at 10:14 PM Adam Williamson 
> wrote:
> 
> > 
> > 
> > In terms of testing: we already test the existing toolchain pretty
> > effectively in practice. openQA builds Workstation and KDE live images
> > with every relevant critical path update. If an update breaks live
> > image creation, we find out about it right away, and the update is
> > gated.
> > 
> 
> Can you elaborate the testing setup more? Does openQA somehow call
> livemedia-creator locally, or does it just offload the task to koji? If the
> latter is true, that this shouldn't be an issue. The former option is more
> tricky, we would have to somehow adjust openQA to use a different tool.

It mimics what Koji does:

https://pagure.io/fedora-qa/os-autoinst-distri-fedora/blob/main/f/tests/_live_build.pm

If this Change goes through, I will change the openQA test
appropriately.

> > 
> > In terms of file formats: is toml really that much better known or
> > better than kickstart, for an audience of Fedora devs, for the purpose
> > of building a live image? Especially now we (finally) got the livesys
> > stuff out of the kickstarts and they're mostly pretty simple? As a
> > specific nitpick, the documentation on the Image Builder format makes
> > it clear that it supports comps groups, but it's less clear if it
> > supports comps *environment* groups. It really needs to do so to
> > maintain parity with livemedia-creator; we don't want to have to define
> > all the environment groups twice (once in comps, once in imagebuilder
> > TOML files). If it does support this, it should be made clear in the
> > docs.
> > 
> 
> Yes, it does support environment groups.
> 
> Regarding kickstarts vs. blueprint (TOML): From my point of view,
> kickstarts are a specific thing to the "Fedora ecosystem", whereas TOML is
> widely used also in other areas (Go/Rust), thus there is a bigger ecosystem
> of tooling for it. I would also like to introduce JSONSchema (conversion
> between JSON and TOML is mostly lossless) for blueprints, which would allow
> e.g. linting of blueprints. Yet another step in making the tooling more
> user-friendly. Anyway, I don't want to spend much time on this, it feels
> like this is more a matter of taste.

I agree it's a matter of taste, but that was kinda my point: switching
from ks to toml is advertised as a "benefit" of the Change, but to me
it feels like a very small one if it is one at all. (we do have a
checker for kickstart files, ksvalidator, btw). It's part of the
overall point that this Change doesn't really seem to constitute a huge
improvement, therefore it should be set a high bar for not introducing
any *disadvantages* compared to the previous approach.
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Wed, 2023-06-28 at 04:23 -0600, Jonathan Steffan wrote:
> On Wed, Jun 28, 2023 at 12:52 AM Adam Williamson 
> wrote:
> 
> > On Tue, 2023-06-27 at 17:59 -0600, Jonathan Steffan wrote:
> > > The Koji integration leaves many things to be desired. I've had to dig
> > far
> > > more than needed for osbuildimage stuff.
> > 
> [...]
> 
> > > Maybe I'm just using it wrong, but I've not found a different way.
> > 
> > For compose debugging I don't think this part would be terrible in
> > practice, most of the time.
> > 
> > https://pagure.io/releng/failed-composes/issues
> 
> 
> I'm glad there is something better than what I'd found. I'm not
> necessarily trying to debug composes, just find them and use/test them as
> needed. If something like this continued after changing to osbuildimage
> tasks, that would work.

composetracker works at a level where the nature of the task doesn't
exactly matter - it works off pungi metadata, I believe. So it should
still collect failed osbuildimage tasks, as long as they're part of a
compose. The source for composetracker is at
https://pagure.io/releng/compose-tracker , if you want to see how it
works.

failed-compose tickets are a good way to find *failed* attempts to
build deliverables. If you want to find *successful* ones, my
fedora_nightlies is quite popular -
https://openqa.fedoraproject.org/nightlies.html . That, again, works
off Pungi metadata, so it will find anything that looks like an 'image'
in the mainline Fedora composes (for Rawhide and for Branched, when
Branched exists).

> >  Personally I don't usually have
> > much call for downloading the actual artifact from Koji - I'd usually
> > get it from the compose - but maybe you have a workflow where it's
> > important?
> > 
> 
> I've regularly used the images produced at
> https://koji.fedoraproject.org/koji/builds?type=image=-build_id for
> doing live system testing for package updates and other changes where I
> don't have a local install already available. It's really nice to have
> these artifacts so easily understandable and accessible. Maybe my workflow
> is off, but having such artifacts available has been super useful.

Unless I'm missing something, the nightlies page above should help you.
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Neal Gompa]

On Wed, Jun 28, 2023 at 8:25 AM Ondřej Budai  wrote:
>
>
> On Wed, Jun 28, 2023 at 2:20 PM Jonathan Steffan  
> wrote:
>>
>> On Wed, Jun 28, 2023 at 5:23 AM Ondřej Budai  wrote:
>
> > Maybe I'm just using it wrong, but I've not found a different way.
>>>
>>>
>>> I will take a look. The difference between livemedia and osbuildImage is 
>>> that the osbuildImage task is defined by a plugin and it interacts with 
>>> koji via the content generator API. Koji sometimes have issues presenting 
>>> information nicely for CGs. If anyone has more experience with Koji 
>>> internals, help would be definitely appreciated.
>>
>>
>> It would be great to address this before adopting the proposed change. I do 
>> wonder if there is a way to change the "osbuildimage" to be much more 
>> similar to an "image" build. Or something like that. The information 
>> conveyed by an "image" vs "osbuildimage" is so much more useful. I don't 
>> know the Koji internals much anymore, but there has to be a way to make this 
>> better.
>>
>>> We can definitely flatten the JSON output into something that resembles a 
>>> log file. I will let you know when this is done.
>>>
>>> Note that every task also produces a manifest file - this is extremely 
>>> useful, because you can just feed it to osbuild locally. Since the manifest 
>>> basically fully specifies an image build (with locked package versions 
>>> including their hashes), there's a high chance that you will be just able 
>>> to reproduce the issue locally and use any tools you want for deeper 
>>> debugging. This is one of the goals of the project: Make image builds more 
>>> reproducible.
>>
>>
>> Both the JSON and raw logs are useful. Very glad to know that the manifest 
>> will allow recreation of the build artifact. I'd like to see all of that 
>> presented similar to a build that packagers are used to (e.g. 
>> https://koji.fedoraproject.org/koji/buildinfo?buildID=698). Maybe this 
>> is more a Koji enhancement, but I still question moving this proposal 
>> forward without addressing the buildsys UX/UI. Let's be sure to not bury 
>> very useful things when making changes.
>
>
>
> Yep, that's a shortcoming of how we currently upload the builds to koji. It's 
> something we would like to tackle in the upcoming quarter, see the tracking 
> ticket: https://issues.redhat.com/browse/COMPOSER-1801 (public link).
>

I can't comment on that ticket. :(



-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Ondřej Budai]

On Wed, Jun 28, 2023 at 2:20 PM Jonathan Steffan 
wrote:

> On Wed, Jun 28, 2023 at 5:23 AM Ondřej Budai  wrote:
>
>> > Maybe I'm just using it wrong, but I've not found a different way.

>>>
>> I will take a look. The difference between livemedia and osbuildImage is
>> that the osbuildImage task is defined by a plugin
>>  and it interacts with koji
>> via the content generator API
>> . Koji sometimes have
>> issues presenting information nicely for CGs. If anyone has more experience
>> with Koji internals, help would be definitely appreciated.
>>
>
> It would be great to address this before adopting the proposed change. I
> do wonder if there is a way to change the "osbuildimage" to be much more
> similar to an "image" build. Or something like that. The information
> conveyed by an "image" vs "osbuildimage" is so much more useful. I don't
> know the Koji internals much anymore, but there has to be a way to make
> this better.
>
> We can definitely flatten the JSON output into something that resembles a
>> log file. I will let you know when this is done.
>>
>> Note that every task also produces a manifest file - this is extremely
>> useful, because you can just feed it to osbuild locally. Since the manifest
>> basically fully specifies an image build (with locked package versions
>> including their hashes), there's a high chance that you will be just able
>> to reproduce the issue locally and use any tools you want for deeper
>> debugging. This is one of the goals of the project: Make image builds more
>> reproducible.
>>
>
> Both the JSON and raw logs are useful. Very glad to know that the manifest
> will allow recreation of the build artifact. I'd like to see all of that
> presented similar to a build that packagers are used to (e.g.
> https://koji.fedoraproject.org/koji/buildinfo?buildID=698). Maybe
> this is more a Koji enhancement, but I still question moving this proposal
> forward without addressing the buildsys UX/UI. Let's be sure to not bury
> very useful things when making changes.
>


Yep, that's a shortcoming of how we currently upload the builds to koji.
It's something we would like to tackle in the upcoming quarter, see the
tracking ticket: https://issues.redhat.com/browse/COMPOSER-1801 (public
link).

Cheers,
Ondřej
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Jonathan Steffan]

On Wed, Jun 28, 2023 at 5:23 AM Ondřej Budai  wrote:

> > Maybe I'm just using it wrong, but I've not found a different way.
>>>
>>
> I will take a look. The difference between livemedia and osbuildImage is
> that the osbuildImage task is defined by a plugin
>  and it interacts with koji via
> the content generator API
> . Koji sometimes have
> issues presenting information nicely for CGs. If anyone has more experience
> with Koji internals, help would be definitely appreciated.
>

It would be great to address this before adopting the proposed change. I do
wonder if there is a way to change the "osbuildimage" to be much more
similar to an "image" build. Or something like that. The information
conveyed by an "image" vs "osbuildimage" is so much more useful. I don't
know the Koji internals much anymore, but there has to be a way to make
this better.

We can definitely flatten the JSON output into something that resembles a
> log file. I will let you know when this is done.
>
> Note that every task also produces a manifest file - this is extremely
> useful, because you can just feed it to osbuild locally. Since the manifest
> basically fully specifies an image build (with locked package versions
> including their hashes), there's a high chance that you will be just able
> to reproduce the issue locally and use any tools you want for deeper
> debugging. This is one of the goals of the project: Make image builds more
> reproducible.
>

Both the JSON and raw logs are useful. Very glad to know that the manifest
will allow recreation of the build artifact. I'd like to see all of that
presented similar to a build that packagers are used to (e.g.
https://koji.fedoraproject.org/koji/buildinfo?buildID=698). Maybe this
is more a Koji enhancement, but I still question moving this proposal
forward without addressing the buildsys UX/UI. Let's be sure to not bury
very useful things when making changes.

-- 
Jonathan Steffan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Ondřej Budai]

On Wed, Jun 28, 2023 at 12:24 PM Jonathan Steffan 
wrote:

> On Wed, Jun 28, 2023 at 12:52 AM Adam Williamson <
> adamw...@fedoraproject.org> wrote:
>
>> On Tue, 2023-06-27 at 17:59 -0600, Jonathan Steffan wrote:
>> > The Koji integration leaves many things to be desired. I've had to dig
>> far
>> > more than needed for osbuildimage stuff.
>>
> [...]
>
>> > Maybe I'm just using it wrong, but I've not found a different way.
>>
>
I will take a look. The difference between livemedia and osbuildImage is
that the osbuildImage task is defined by a plugin
 and it interacts with koji via
the content generator API .
Koji sometimes have issues presenting information nicely for CGs. If anyone
has more experience with Koji internals, help would be definitely
appreciated.


>
>> For compose debugging I don't think this part would be terrible in
>> practice, most of the time.
>>
>> https://pagure.io/releng/failed-composes/issues
>
>
> I'm glad there is something better than what I'd found. I'm not
> necessarily trying to debug composes, just find them and use/test them as
> needed. If something like this continued after changing to osbuildimage
> tasks, that would work.
>
>
>>  Personally I don't usually have
>> much call for downloading the actual artifact from Koji - I'd usually
>> get it from the compose - but maybe you have a workflow where it's
>> important?
>>
>
> I've regularly used the images produced at
> https://koji.fedoraproject.org/koji/builds?type=image=-build_id for
> doing live system testing for package updates and other changes where I
> don't have a local install already available. It's really nice to have
> these artifacts so easily understandable and accessible. Maybe my workflow
> is off, but having such artifacts available has been super useful. The
> osbuildimage tasks, not so much. If we start to migrate the live image
> creation into an osbuildimage task, as currently implemented, it will
> diminish the value of automatically generated install/live images. It has
> been very helpful to be able to grab an ISO from Koji and test whatever is
> needed in a live/virt environment without having to do a full install and
> keep it updated. Keeping that very easily accessible would be awesome.
>
> Personally, I'm more worried about the logs being JSON. Machine-
>> parseable is nice, I guess, but in practice it's usually humans who
>> parse logs, at least in this case, and I see no really nice human way
>> to consume that log.
>
>
> I agree.
>
> Maybe raw text in addition to the JSON logs would be better for this
> workload. For me, the logs on an "image" build are much more familiar to
> regular packagers doing builds and they make sense. I have to dig for
> logs/artifacts in two different places when trying to understand
> "osbuildimage" tasks. The tools for reading JSON are available, but it'd be
> great to add in some easily accessible raw logs.
>

We can definitely flatten the JSON output into something that resembles a
log file. I will let you know when this is done.

Note that every task also produces a manifest file - this is extremely
useful, because you can just feed it to osbuild locally. Since the manifest
basically fully specifies an image build (with locked package versions
including their hashes), there's a high chance that you will be just able
to reproduce the issue locally and use any tools you want for deeper
debugging. This is one of the goals of the project: Make image builds more
reproducible.

Cheers,
Ondřej
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Ondřej Budai]

Hi Kevin!

On Tue, Jun 27, 2023 at 12:41 AM Kevin Fenzi  wrote:

> > * Other developers:
> > Our focus for this change is specifically on Fedora Workstation.
> > Nevertheless, we are open to collaborating with all spins/SIG to
> > transition their build pipelines to Image Builder. However, for the
> > initial switch, we aim to minimize the impact by focusing on a single
> > artifact. We anticipate that more artifacts will be transitioned in
> > subsequent releases of Fedora Linux.
>
> I realize you have to start somewhere, but I'm not sure this is
> something that should be driven by the SIGs/spin owners. I mean if the
> images produced are the same, why do they care? :)
>

We are happy to drive this. We just don't want to do any changes without
them. :)


>
> Would it be better to start with a non blocking deliverable and get
> things working, then just switch all the rest?
>

It's important to understand how we got here: We are collaborating with the
installer team on the new Web UI - e.g. we provided the tooling for
building the preview images
 last
cycle, helping with setting up image builder upstream for their CI and we
did some debugging. Thus, it makes sense for us to focus on the same
artifact as the installer team is concentrating on - the Workstation Live
ISO.

Also, another advantage from my point of view is that people really care
about this artifact, so the bar is set high for us. I'm afraid that if we
switched some less-used artifacts, some issues caused by changing the
underlying tool might go live unnoticed. I fully acknowledge that our
approach has also a somewhat high risk, however as the change is trivially
revertible, the risk is manageable.

Anyway, I'm happy for any suggestions how to do this differently.


>
> If we are moving from kickstarts to blueprints, I would guess we need a
> new repo for that and help converting things.
>
> Do we have a sense that livemedia-creator is going to keep being
> used/work? There is likely a bunch of docs pointing to it that I would
> suspect should be updated to point people to them.
>
> > * Release engineering: [https://pagure.io/releng/issue/11500 #11500]
> > Provide ppc64le machines to Image Builder. Switch the pungi config to
> > use Image Builder.
>
> Yeah. ;)
>
> I take it you have x86_64 and aarch64 in hand?
> When we move all images over, will it be able to handle the capacity?
>

Do you know how many builds for Workstation live ISO we have on one day?
I'm 99% sure we can handle it, internally, we are building about 100 images
a day for RHEL with basically the same infra as is currently used for
Fedora.

Cheers,
Ondřej
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Ondřej Budai]

I already answered you here:
https://pagure.io/fedora-workstation/issue/384#comment-862709

TL;DR: Let's figure this out when we are migrating other artifacts. For
now, the blueprint will be empty/minimal.

Cheers,
Ondřej

On Mon, Jun 26, 2023 at 10:34 PM Neal Gompa  wrote:

> On Mon, Jun 26, 2023 at 4:14 PM Adam Williamson
>  wrote:
> >
> > On Mon, 2023-06-26 at 15:33 +0100, Aoife Moloney wrote:
> > > https://fedoraproject.org/wiki/Changes/FedoraWorkstationImageBuilder
> > >
> > > This document represents a proposed Change. As part of the Changes
> > > process, proposals are publicly announced in order to receive
> > > community feedback. This proposal will only be implemented if approved
> > > by the Fedora Engineering Steering Committee.
> > >
> > >
> > >
> > > == Summary ==
> > > Image Builder is a set of modern tools for building operating system
> > > images. Its goal is to make the builds reliable and reproducible.
> > > Moreover, it's designed to give the end users a simple workflow to
> > > build their own custom images. The aim of this change is to switch the
> > > build tool for Fedora Workstation live ISO from livemedia-creator to
> > > Image Builder.
> > >
> > > == Owner ==
> > > * Name: [[User:obudai|Ondřej Budai]]
> > > * Email: obu...@redhat.com
> > > * Name: [[User:Supakeen|Simon de Vlieger]]
> > > * Email: supak...@redhat.com
> > > * Name: [[User:jkonecny|Jiří Konečný]]
> > > * Email: jkone...@redhat.com
> > >
> > >
> > >
> > > == Detailed Description ==
> > > Image builder is currently getting support for building
> > > [https://github.com/osbuild/osbuild-composer/pull/3440 live ISOs].
> > > Once the PR implementing the new image type is merged,
> > > [
> https://pagure.io/pungi-fedora/blob/8b2c85799cce5c096484cdebe1dbe521eaf7fe92/f/fedora.conf#_530
> > > the pungi-fedora configuration] needs to be updated. This change will
> > > ensure that pungi creates an osbuildImage task in koji instead of the
> > > currently used livemedia one.
> > >
> > > Pungi and Koji already support Image Builder, so no additional work is
> > > required there (refer to the
> > > [
> https://docs.pagure.org/pungi/configuration.html#osbuild-composer-for-building-images
> > > pungi] and [https://github.com/osbuild/koji-osbuild/ koji]
> > > documentation). The only missing part in terms of infrastructure is
> > > provisioning ppc64le worker machines for Image Builder, see the
> > > relevant [https://pagure.io/fedora-infrastructure/issue/11243
> > > fedora-infra ticket].
> > >
> > > Note that Image Builder is already used for
> > > [https://fedoraproject.org/wiki/Changes/IoTArtifactsWithOSBuild
> > > building ISOs and raw disks of Fedora IoT]. Therefore, this proposal
> > > does not introduce a new tool to the Fedora build pipeline.
> > >
> > > == Feedback ==
> > >
> > > Currently, Image Builder does not populate the DNF database correctly,
> > > leading to all RPMs installed on the target system being marked as
> > > user-installed. This is [https://github.com/osbuild/osbuild/issues/455
> > > a known issue] that the team is planning to address as soon as the
> > > initial support for live ISOs is merged.
> > >
> > >
> > > == Benefit to Fedora ==
> > > The maintainer team of Image Builder believes that the project
> > > undergoes more comprehensive testing compared to
> > > lorax/livemedia-creator. Thus, by switching to Image Builder, Fedora
> > > should experience fewer issues with the image building pipeline.
> > >
> > > Another advantage is the project's emphasis on making Image Builder
> > > more user-friendly. End users can easily build their own customized
> > > version of the live ISO using a simple
> > > [
> https://www.osbuild.org/guides/image-builder-on-premises/blueprint-reference.html
> > > TOML blueprint file] and a
> > > [
> https://www.osbuild.org/guides/image-builder-on-premises/building-an-image-from-cli.html
> > > CLI interface]. This approach, utilizing well-known file formats, is a
> > > positive step compared to livemedia-creator's kickstart files. More
> > > information about building customized images can found on
> > > [https://major.io/p/build-custom-centos-stream-cloud-image/ Major
> > > Hayden's blog] or in
> > > [https://www.youtube.com/watch?v=PsQySAEOeFs=17001s a conference
> > > talk] given by Ondřej Budai, one of the proposal owners. Moreover,
> > > Image Builder provides [https://github.com/osbuild/cockpit-composer/ a
> > > graphical interface] for visually defining blueprints, further
> > > simplifying the workflow.
> > >
> > > We believe that Image Builder can also be beneficial to the
> > > [https://fedoraproject.org/wiki/Respins-SIG Respins SIG] as it nicely
> > > aligns with their objective of providing a simple method for building
> > > up-to-date, customizable images.
> > >
> > > == Scope ==
> > > * Proposal owners:
> > > Finishing implementing support for the live ISO upstream and
> > > collaborate with release engineering to switch the pungi config to use
> > > Image 

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Ondřej Budai]

Hi Adam and Ben!

On Mon, Jun 26, 2023 at 10:14 PM Adam Williamson 
wrote:

>
>
> In terms of testing: we already test the existing toolchain pretty
> effectively in practice. openQA builds Workstation and KDE live images
> with every relevant critical path update. If an update breaks live
> image creation, we find out about it right away, and the update is
> gated.
>

Can you elaborate the testing setup more? Does openQA somehow call
livemedia-creator locally, or does it just offload the task to koji? If the
latter is true, that this shouldn't be an issue. The former option is more
tricky, we would have to somehow adjust openQA to use a different tool.


>
> In terms of file formats: is toml really that much better known or
> better than kickstart, for an audience of Fedora devs, for the purpose
> of building a live image? Especially now we (finally) got the livesys
> stuff out of the kickstarts and they're mostly pretty simple? As a
> specific nitpick, the documentation on the Image Builder format makes
> it clear that it supports comps groups, but it's less clear if it
> supports comps *environment* groups. It really needs to do so to
> maintain parity with livemedia-creator; we don't want to have to define
> all the environment groups twice (once in comps, once in imagebuilder
> TOML files). If it does support this, it should be made clear in the
> docs.
>

Yes, it does support environment groups.

Regarding kickstarts vs. blueprint (TOML): From my point of view,
kickstarts are a specific thing to the "Fedora ecosystem", whereas TOML is
widely used also in other areas (Go/Rust), thus there is a bigger ecosystem
of tooling for it. I would also like to introduce JSONSchema (conversion
between JSON and TOML is mostly lossless) for blueprints, which would allow
e.g. linting of blueprints. Yet another step in making the tooling more
user-friendly. Anyway, I don't want to spend much time on this, it feels
like this is more a matter of taste.


>
> I notice the Change doesn't say much about debugging failed builds.
> This is very important. Failed livemedia-creator tasks are quite easy
> to debug. livemedia-creator as run in Koji by Pungi is very good at
> providing sufficient logs to determine what the heck went wrong if an
> image build fails. Other tools - notably ImageFactory - are notoriously
> much worse at this. Where does Image Builder fall?
>

Let's talk about this in the branch about logs in koji.


>
> As I said, I'm not saying we shouldn't do this. But I do think that if
> it doesn't provide any really compelling benefits over livemedia-
> creator, we should set a high bar for it also not having any
> significant *drawbacks* compared with livemedia-creator.

-- 
> Adam Williamson (he/him/his)
> Fedora QA
> Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
> https://www.happyassassin.net
>
>
> Ben Cotton:
> What's the impact to QA and release processes? Will the Workstation
> images still be in the regular large compose, or will they be a
> separate compose that will need to be managed independently?

Images will still be in the regular compose, image builder can be just
called via Pungi, so one compose can use multiple image building tools.
Regarding the impact: I'm mostly concerned about the openQA workflow, see
the paragraph above.

Cheers,
Ondřej
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Jonathan Steffan]

On Wed, Jun 28, 2023 at 12:52 AM Adam Williamson 
wrote:

> On Tue, 2023-06-27 at 17:59 -0600, Jonathan Steffan wrote:
> > The Koji integration leaves many things to be desired. I've had to dig
> far
> > more than needed for osbuildimage stuff.
>
[...]

> > Maybe I'm just using it wrong, but I've not found a different way.
>
> For compose debugging I don't think this part would be terrible in
> practice, most of the time.
>
> https://pagure.io/releng/failed-composes/issues


I'm glad there is something better than what I'd found. I'm not
necessarily trying to debug composes, just find them and use/test them as
needed. If something like this continued after changing to osbuildimage
tasks, that would work.


>  Personally I don't usually have
> much call for downloading the actual artifact from Koji - I'd usually
> get it from the compose - but maybe you have a workflow where it's
> important?
>

I've regularly used the images produced at
https://koji.fedoraproject.org/koji/builds?type=image=-build_id for
doing live system testing for package updates and other changes where I
don't have a local install already available. It's really nice to have
these artifacts so easily understandable and accessible. Maybe my workflow
is off, but having such artifacts available has been super useful. The
osbuildimage tasks, not so much. If we start to migrate the live image
creation into an osbuildimage task, as currently implemented, it will
diminish the value of automatically generated install/live images. It has
been very helpful to be able to grab an ISO from Koji and test whatever is
needed in a live/virt environment without having to do a full install and
keep it updated. Keeping that very easily accessible would be awesome.

Personally, I'm more worried about the logs being JSON. Machine-
> parseable is nice, I guess, but in practice it's usually humans who
> parse logs, at least in this case, and I see no really nice human way
> to consume that log.


I agree.

Maybe raw text in addition to the JSON logs would be better for this
workload. For me, the logs on an "image" build are much more familiar to
regular packagers doing builds and they make sense. I have to dig for
logs/artifacts in two different places when trying to understand
"osbuildimage" tasks. The tools for reading JSON are available, but it'd be
great to add in some easily accessible raw logs.

-- 
Jonathan Steffan
jonathanstef...@gmail.com
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Tue, 2023-06-27 at 17:59 -0600, Jonathan Steffan wrote:
> 
> The Koji integration leaves many things to be desired. I've had to dig far
> more than needed for osbuildimage stuff.
> 
> For example,
> https://koji.fedoraproject.org/koji/tasks?state=all=tree=osbuildImage=-id
> is nearly useless to know what is going on. You have to go into each task
> to find what you are looking for and you end up with some json logs
> https://koji.fedoraproject.org/koji/taskinfo?taskID=102669680 and
> ultimately finding a link to the build
> https://koji.fedoraproject.org/koji/buildinfo?buildID=653 to get to the
> image. Maybe I'm just using it wrong, but I've not found a different way.

For compose debugging I don't think this part would be terrible in
practice, most of the time. I agree that there not being any useful
information about what each osbuild image each task is for, in the
summary, is a pain. But *normally* how we wind up debugging failed
composes is through the failed compose tracker:

https://pagure.io/releng/failed-composes/issues

which knows what each task is and tells you in the issue:

https://pagure.io/releng/failed-composes/issue/5143

so you already know when you get to the task what it is, and the task
logs usually are what we want to see. Personally I don't usually have
much call for downloading the actual artifact from Koji - I'd usually
get it from the compose - but maybe you have a workflow where it's
important?

Personally, I'm more worried about the logs being JSON. Machine-
parseable is nice, I guess, but in practice it's usually humans who
parse logs, at least in this case, and I see no really nice human way
to consume that log. Opening it as raw JSON in my browser gives me a
mess (it's not formatted for human consumption, and is stuffed with
linebreak characters and stuff, much like ImageFactory logs). Opening
it with a JSON-reading extension I have gives me a mess with more
nested menus to click on. Opening it in a text editor gives me the same
mess as opening it in the browser.

Can we have this thing generate easily-human-readable logs as well
as/instead of the JSON-formatted ones?
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Jonathan Steffan]

On Mon, Jun 26, 2023 at 8:34 AM Aoife Moloney  wrote:

> [...]
> Pungi and Koji already support Image Builder, so no additional work is
> required there (refer to the
> [
> https://docs.pagure.org/pungi/configuration.html#osbuild-composer-for-building-images
> pungi] and [https://github.com/osbuild/koji-osbuild/ koji]
> documentation). The only missing part in terms of infrastructure is
> provisioning ppc64le worker machines for Image Builder, see the
> relevant [https://pagure.io/fedora-infrastructure/issue/11243
> fedora-infra ticket].
>
> Note that Image Builder is already used for
> [https://fedoraproject.org/wiki/Changes/IoTArtifactsWithOSBuild
> building ISOs and raw disks of Fedora IoT]. Therefore, this proposal
> does not introduce a new tool to the Fedora build pipeline.
>

The Koji integration leaves many things to be desired. I've had to dig far
more than needed for osbuildimage stuff.

For example,
https://koji.fedoraproject.org/koji/tasks?state=all=tree=osbuildImage=-id
is nearly useless to know what is going on. You have to go into each task
to find what you are looking for and you end up with some json logs
https://koji.fedoraproject.org/koji/taskinfo?taskID=102669680 and
ultimately finding a link to the build
https://koji.fedoraproject.org/koji/buildinfo?buildID=653 to get to the
image. Maybe I'm just using it wrong, but I've not found a different way.

Compare that confusion to something as useful as the image builds at
https://koji.fedoraproject.org/koji/builds?type=image=-build_id where
you know exactly what everythign is and a single click on a build
https://koji.fedoraproject.org/koji/buildinfo?buildID=698 gives me
logs, isos, and all artifacts I'm interested in looking at.

Please include improving the Koji integration before making this change.
Those of use that spelunk around the automatic image creation would be
greatly appreciative.

-- 
Jonathan Steffan
jonathanstef...@gmail.com
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Mon, 2023-06-26 at 23:57 +0200, Kevin Kofler via devel wrote:
> > Image Builder is a set of modern tools for building operating system
> > images. Its goal is to make the builds reliable and reproducible.
> > Moreover, it's designed to give the end users a simple workflow to
> > build their own custom images. The aim of this change is to switch the
> > build tool for Fedora Workstation live ISO from livemedia-creator to
> > Image Builder.
> 
> Yet another image building tool? I remember not too long ago when livemedia-
> creator was the shiny new thing instead of livecd-creator. (Yet, I found 
> livecd-creator to work much better for me to create my Kannolo images than 
> livemedia-creator.)

I agree it doesn't feel like long ago, but, per
https://pagure.io/fedora-kickstarts/pull-request/24 , it seems like it
happened *seven years ago*. In Fedora 24.

Don't mind me, I'll just be over here, gradually crumbling into a pile
of dust...
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Kevin Fenzi]

On Mon, Jun 26, 2023 at 03:33:26PM +0100, Aoife Moloney wrote:
> https://fedoraproject.org/wiki/Changes/FedoraWorkstationImageBuilder
> 
> This document represents a proposed Change. As part of the Changes
> process, proposals are publicly announced in order to receive
> community feedback. This proposal will only be implemented if approved
> by the Fedora Engineering Steering Committee.
> 
> 
> 
> == Summary ==
> Image Builder is a set of modern tools for building operating system
> images. Its goal is to make the builds reliable and reproducible.
> Moreover, it's designed to give the end users a simple workflow to
> build their own custom images. The aim of this change is to switch the
> build tool for Fedora Workstation live ISO from livemedia-creator to
> Image Builder.

...snip...

> * Other developers:
> Our focus for this change is specifically on Fedora Workstation.
> Nevertheless, we are open to collaborating with all spins/SIG to
> transition their build pipelines to Image Builder. However, for the
> initial switch, we aim to minimize the impact by focusing on a single
> artifact. We anticipate that more artifacts will be transitioned in
> subsequent releases of Fedora Linux.

I realize you have to start somewhere, but I'm not sure this is
something that should be driven by the SIGs/spin owners. I mean if the
images produced are the same, why do they care? :) 

Would it be better to start with a non blocking deliverable and get
things working, then just switch all the rest?

If we are moving from kickstarts to blueprints, I would guess we need a
new repo for that and help converting things. 

Do we have a sense that livemedia-creator is going to keep being
used/work? There is likely a bunch of docs pointing to it that I would
suspect should be updated to point people to them.

> * Release engineering: [https://pagure.io/releng/issue/11500 #11500]
> Provide ppc64le machines to Image Builder. Switch the pungi config to
> use Image Builder.

Yeah. ;) 

I take it you have x86_64 and aarch64 in hand? 
When we move all images over, will it be able to handle the capacity?

I'm hoping it's easy to debug failures. We can debug livemedia-creator
pretty well at this point. :)

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Kevin Kofler via devel]

> Image Builder is a set of modern tools for building operating system
> images. Its goal is to make the builds reliable and reproducible.
> Moreover, it's designed to give the end users a simple workflow to
> build their own custom images. The aim of this change is to switch the
> build tool for Fedora Workstation live ISO from livemedia-creator to
> Image Builder.

Yet another image building tool? I remember not too long ago when livemedia-
creator was the shiny new thing instead of livecd-creator. (Yet, I found 
livecd-creator to work much better for me to create my Kannolo images than 
livemedia-creator.)

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Neal Gompa]

On Mon, Jun 26, 2023 at 4:14 PM Adam Williamson
 wrote:
>
> On Mon, 2023-06-26 at 15:33 +0100, Aoife Moloney wrote:
> > https://fedoraproject.org/wiki/Changes/FedoraWorkstationImageBuilder
> >
> > This document represents a proposed Change. As part of the Changes
> > process, proposals are publicly announced in order to receive
> > community feedback. This proposal will only be implemented if approved
> > by the Fedora Engineering Steering Committee.
> >
> >
> >
> > == Summary ==
> > Image Builder is a set of modern tools for building operating system
> > images. Its goal is to make the builds reliable and reproducible.
> > Moreover, it's designed to give the end users a simple workflow to
> > build their own custom images. The aim of this change is to switch the
> > build tool for Fedora Workstation live ISO from livemedia-creator to
> > Image Builder.
> >
> > == Owner ==
> > * Name: [[User:obudai|Ondřej Budai]]
> > * Email: obu...@redhat.com
> > * Name: [[User:Supakeen|Simon de Vlieger]]
> > * Email: supak...@redhat.com
> > * Name: [[User:jkonecny|Jiří Konečný]]
> > * Email: jkone...@redhat.com
> >
> >
> >
> > == Detailed Description ==
> > Image builder is currently getting support for building
> > [https://github.com/osbuild/osbuild-composer/pull/3440 live ISOs].
> > Once the PR implementing the new image type is merged,
> > [https://pagure.io/pungi-fedora/blob/8b2c85799cce5c096484cdebe1dbe521eaf7fe92/f/fedora.conf#_530
> > the pungi-fedora configuration] needs to be updated. This change will
> > ensure that pungi creates an osbuildImage task in koji instead of the
> > currently used livemedia one.
> >
> > Pungi and Koji already support Image Builder, so no additional work is
> > required there (refer to the
> > [https://docs.pagure.org/pungi/configuration.html#osbuild-composer-for-building-images
> > pungi] and [https://github.com/osbuild/koji-osbuild/ koji]
> > documentation). The only missing part in terms of infrastructure is
> > provisioning ppc64le worker machines for Image Builder, see the
> > relevant [https://pagure.io/fedora-infrastructure/issue/11243
> > fedora-infra ticket].
> >
> > Note that Image Builder is already used for
> > [https://fedoraproject.org/wiki/Changes/IoTArtifactsWithOSBuild
> > building ISOs and raw disks of Fedora IoT]. Therefore, this proposal
> > does not introduce a new tool to the Fedora build pipeline.
> >
> > == Feedback ==
> >
> > Currently, Image Builder does not populate the DNF database correctly,
> > leading to all RPMs installed on the target system being marked as
> > user-installed. This is [https://github.com/osbuild/osbuild/issues/455
> > a known issue] that the team is planning to address as soon as the
> > initial support for live ISOs is merged.
> >
> >
> > == Benefit to Fedora ==
> > The maintainer team of Image Builder believes that the project
> > undergoes more comprehensive testing compared to
> > lorax/livemedia-creator. Thus, by switching to Image Builder, Fedora
> > should experience fewer issues with the image building pipeline.
> >
> > Another advantage is the project's emphasis on making Image Builder
> > more user-friendly. End users can easily build their own customized
> > version of the live ISO using a simple
> > [https://www.osbuild.org/guides/image-builder-on-premises/blueprint-reference.html
> > TOML blueprint file] and a
> > [https://www.osbuild.org/guides/image-builder-on-premises/building-an-image-from-cli.html
> > CLI interface]. This approach, utilizing well-known file formats, is a
> > positive step compared to livemedia-creator's kickstart files. More
> > information about building customized images can found on
> > [https://major.io/p/build-custom-centos-stream-cloud-image/ Major
> > Hayden's blog] or in
> > [https://www.youtube.com/watch?v=PsQySAEOeFs=17001s a conference
> > talk] given by Ondřej Budai, one of the proposal owners. Moreover,
> > Image Builder provides [https://github.com/osbuild/cockpit-composer/ a
> > graphical interface] for visually defining blueprints, further
> > simplifying the workflow.
> >
> > We believe that Image Builder can also be beneficial to the
> > [https://fedoraproject.org/wiki/Respins-SIG Respins SIG] as it nicely
> > aligns with their objective of providing a simple method for building
> > up-to-date, customizable images.
> >
> > == Scope ==
> > * Proposal owners:
> > Finishing implementing support for the live ISO upstream and
> > collaborate with release engineering to switch the pungi config to use
> > Image Builder.
> >
> > * Other developers:
> > Our focus for this change is specifically on Fedora Workstation.
> > Nevertheless, we are open to collaborating with all spins/SIG to
> > transition their build pipelines to Image Builder. However, for the
> > initial switch, we aim to minimize the impact by focusing on a single
> > artifact. We anticipate that more artifacts will be transitioned in
> > subsequent releases of Fedora Linux.
> >
> > * Release engineering: 

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Adam Williamson]

On Mon, 2023-06-26 at 15:33 +0100, Aoife Moloney wrote:
> https://fedoraproject.org/wiki/Changes/FedoraWorkstationImageBuilder
> 
> This document represents a proposed Change. As part of the Changes
> process, proposals are publicly announced in order to receive
> community feedback. This proposal will only be implemented if approved
> by the Fedora Engineering Steering Committee.
> 
> 
> 
> == Summary ==
> Image Builder is a set of modern tools for building operating system
> images. Its goal is to make the builds reliable and reproducible.
> Moreover, it's designed to give the end users a simple workflow to
> build their own custom images. The aim of this change is to switch the
> build tool for Fedora Workstation live ISO from livemedia-creator to
> Image Builder.
> 
> == Owner ==
> * Name: [[User:obudai|Ondřej Budai]]
> * Email: obu...@redhat.com
> * Name: [[User:Supakeen|Simon de Vlieger]]
> * Email: supak...@redhat.com
> * Name: [[User:jkonecny|Jiří Konečný]]
> * Email: jkone...@redhat.com
> 
> 
> 
> == Detailed Description ==
> Image builder is currently getting support for building
> [https://github.com/osbuild/osbuild-composer/pull/3440 live ISOs].
> Once the PR implementing the new image type is merged,
> [https://pagure.io/pungi-fedora/blob/8b2c85799cce5c096484cdebe1dbe521eaf7fe92/f/fedora.conf#_530
> the pungi-fedora configuration] needs to be updated. This change will
> ensure that pungi creates an osbuildImage task in koji instead of the
> currently used livemedia one.
> 
> Pungi and Koji already support Image Builder, so no additional work is
> required there (refer to the
> [https://docs.pagure.org/pungi/configuration.html#osbuild-composer-for-building-images
> pungi] and [https://github.com/osbuild/koji-osbuild/ koji]
> documentation). The only missing part in terms of infrastructure is
> provisioning ppc64le worker machines for Image Builder, see the
> relevant [https://pagure.io/fedora-infrastructure/issue/11243
> fedora-infra ticket].
> 
> Note that Image Builder is already used for
> [https://fedoraproject.org/wiki/Changes/IoTArtifactsWithOSBuild
> building ISOs and raw disks of Fedora IoT]. Therefore, this proposal
> does not introduce a new tool to the Fedora build pipeline.
> 
> == Feedback ==
> 
> Currently, Image Builder does not populate the DNF database correctly,
> leading to all RPMs installed on the target system being marked as
> user-installed. This is [https://github.com/osbuild/osbuild/issues/455
> a known issue] that the team is planning to address as soon as the
> initial support for live ISOs is merged.
> 
> 
> == Benefit to Fedora ==
> The maintainer team of Image Builder believes that the project
> undergoes more comprehensive testing compared to
> lorax/livemedia-creator. Thus, by switching to Image Builder, Fedora
> should experience fewer issues with the image building pipeline.
> 
> Another advantage is the project's emphasis on making Image Builder
> more user-friendly. End users can easily build their own customized
> version of the live ISO using a simple
> [https://www.osbuild.org/guides/image-builder-on-premises/blueprint-reference.html
> TOML blueprint file] and a
> [https://www.osbuild.org/guides/image-builder-on-premises/building-an-image-from-cli.html
> CLI interface]. This approach, utilizing well-known file formats, is a
> positive step compared to livemedia-creator's kickstart files. More
> information about building customized images can found on
> [https://major.io/p/build-custom-centos-stream-cloud-image/ Major
> Hayden's blog] or in
> [https://www.youtube.com/watch?v=PsQySAEOeFs=17001s a conference
> talk] given by Ondřej Budai, one of the proposal owners. Moreover,
> Image Builder provides [https://github.com/osbuild/cockpit-composer/ a
> graphical interface] for visually defining blueprints, further
> simplifying the workflow.
> 
> We believe that Image Builder can also be beneficial to the
> [https://fedoraproject.org/wiki/Respins-SIG Respins SIG] as it nicely
> aligns with their objective of providing a simple method for building
> up-to-date, customizable images.
> 
> == Scope ==
> * Proposal owners:
> Finishing implementing support for the live ISO upstream and
> collaborate with release engineering to switch the pungi config to use
> Image Builder.
> 
> * Other developers:
> Our focus for this change is specifically on Fedora Workstation.
> Nevertheless, we are open to collaborating with all spins/SIG to
> transition their build pipelines to Image Builder. However, for the
> initial switch, we aim to minimize the impact by focusing on a single
> artifact. We anticipate that more artifacts will be transitioned in
> subsequent releases of Fedora Linux.
> 
> * Release engineering: [https://pagure.io/releng/issue/11500 #11500]
> Provide ppc64le machines to Image Builder. Switch the pungi config to
> use Image Builder.
> 
> * Policies and guidelines: N/A (not needed for this Change)
> 
> * Trademark approval: N/A (not needed for this Change)
> 

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Richard Shaw]

On Mon, Jun 26, 2023 at 10:02 AM Michal Schorm  wrote:

> > Moreover, it's designed to give the end users a simple workflow to build
> their own custom images.
>
> That's something I'd be interested in.
> I always have an USB drive (or several) with Fedora ISO(s) lying
> around, being handy from time to time.
> Having them customized *easily*, sounds like a great thing to have.
>

Yes, one of the reasons I still keep System Rescue CD around is it already
has gparted installed which is one of the rescue apps I use the most often.

Thanks,
Richard
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Ben Cotton]

On Mon, Jun 26, 2023 at 10:34 AM Aoife Moloney  wrote:
>
> * Other developers:
> Our focus for this change is specifically on Fedora Workstation.
> Nevertheless, we are open to collaborating with all spins/SIG to
> transition their build pipelines to Image Builder. However, for the
> initial switch, we aim to minimize the impact by focusing on a single
> artifact. We anticipate that more artifacts will be transitioned in
> subsequent releases of Fedora Linux.

What's the impact to QA and release processes? Will the Workstation
images still be in the regular large compose, or will they be a
separate compose that will need to be managed independently?

-- 
Ben Cotton (he/him)
TZ=America/Indiana/Indianapolis
https://fedoraproject.org/wiki/User:Bcotton
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: F39 Change Proposal: Build Fedora Workstation live ISO with Image Builder (System-Wide)

[Michal Schorm]

> Moreover, it's designed to give the end users a simple workflow to build 
> their own custom images.

That's something I'd be interested in.
I always have an USB drive (or several) with Fedora ISO(s) lying
around, being handy from time to time.
Having them customized *easily*, sounds like a great thing to have.

--

Michal Schorm
Software Engineer
Core Services - Databases Team
Red Hat

--

On Mon, Jun 26, 2023 at 4:34 PM Aoife Moloney  wrote:
>
> https://fedoraproject.org/wiki/Changes/FedoraWorkstationImageBuilder
>
> This document represents a proposed Change. As part of the Changes
> process, proposals are publicly announced in order to receive
> community feedback. This proposal will only be implemented if approved
> by the Fedora Engineering Steering Committee.
>
>
>
> == Summary ==
> Image Builder is a set of modern tools for building operating system
> images. Its goal is to make the builds reliable and reproducible.
> Moreover, it's designed to give the end users a simple workflow to
> build their own custom images. The aim of this change is to switch the
> build tool for Fedora Workstation live ISO from livemedia-creator to
> Image Builder.
>
> == Owner ==
> * Name: [[User:obudai|Ondřej Budai]]
> * Email: obu...@redhat.com
> * Name: [[User:Supakeen|Simon de Vlieger]]
> * Email: supak...@redhat.com
> * Name: [[User:jkonecny|Jiří Konečný]]
> * Email: jkone...@redhat.com
>
>
>
> == Detailed Description ==
> Image builder is currently getting support for building
> [https://github.com/osbuild/osbuild-composer/pull/3440 live ISOs].
> Once the PR implementing the new image type is merged,
> [https://pagure.io/pungi-fedora/blob/8b2c85799cce5c096484cdebe1dbe521eaf7fe92/f/fedora.conf#_530
> the pungi-fedora configuration] needs to be updated. This change will
> ensure that pungi creates an osbuildImage task in koji instead of the
> currently used livemedia one.
>
> Pungi and Koji already support Image Builder, so no additional work is
> required there (refer to the
> [https://docs.pagure.org/pungi/configuration.html#osbuild-composer-for-building-images
> pungi] and [https://github.com/osbuild/koji-osbuild/ koji]
> documentation). The only missing part in terms of infrastructure is
> provisioning ppc64le worker machines for Image Builder, see the
> relevant [https://pagure.io/fedora-infrastructure/issue/11243
> fedora-infra ticket].
>
> Note that Image Builder is already used for
> [https://fedoraproject.org/wiki/Changes/IoTArtifactsWithOSBuild
> building ISOs and raw disks of Fedora IoT]. Therefore, this proposal
> does not introduce a new tool to the Fedora build pipeline.
>
> == Feedback ==
>
> Currently, Image Builder does not populate the DNF database correctly,
> leading to all RPMs installed on the target system being marked as
> user-installed. This is [https://github.com/osbuild/osbuild/issues/455
> a known issue] that the team is planning to address as soon as the
> initial support for live ISOs is merged.
>
>
> == Benefit to Fedora ==
> The maintainer team of Image Builder believes that the project
> undergoes more comprehensive testing compared to
> lorax/livemedia-creator. Thus, by switching to Image Builder, Fedora
> should experience fewer issues with the image building pipeline.
>
> Another advantage is the project's emphasis on making Image Builder
> more user-friendly. End users can easily build their own customized
> version of the live ISO using a simple
> [https://www.osbuild.org/guides/image-builder-on-premises/blueprint-reference.html
> TOML blueprint file] and a
> [https://www.osbuild.org/guides/image-builder-on-premises/building-an-image-from-cli.html
> CLI interface]. This approach, utilizing well-known file formats, is a
> positive step compared to livemedia-creator's kickstart files. More
> information about building customized images can found on
> [https://major.io/p/build-custom-centos-stream-cloud-image/ Major
> Hayden's blog] or in
> [https://www.youtube.com/watch?v=PsQySAEOeFs=17001s a conference
> talk] given by Ondřej Budai, one of the proposal owners. Moreover,
> Image Builder provides [https://github.com/osbuild/cockpit-composer/ a
> graphical interface] for visually defining blueprints, further
> simplifying the workflow.
>
> We believe that Image Builder can also be beneficial to the
> [https://fedoraproject.org/wiki/Respins-SIG Respins SIG] as it nicely
> aligns with their objective of providing a simple method for building
> up-to-date, customizable images.
>
> == Scope ==
> * Proposal owners:
> Finishing implementing support for the live ISO upstream and
> collaborate with release engineering to switch the pungi config to use
> Image Builder.
>
> * Other developers:
> Our focus for this change is specifically on Fedora Workstation.
> Nevertheless, we are open to collaborating with all spins/SIG to
> transition their build pipelines to Image Builder. However, for the
> initial switch, we aim to minimize the impact by focusing on a single
> artifact. We anticipate