Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2024-05-27 Thread Daan De Meyer
There have indeed been quite a few improvements to mkosi lately so I understand that it might not have been suitable when you were working on this. Note that when it comes to building cross building, we have CI in mkosi that verifies that verifies that mkosi can do cross distribution image build

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2024-05-23 Thread Michal Domonkos
Argh... :facepalm: :smile: re-reading mkosi's man page again, it of course supports multiple `Distribution=` values... meaning that, if that distro's package manager is available on your host, it'll be used. Either way, the point still kinda holds :smile: -- Reply to this email directly or v

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2024-05-23 Thread Michal Domonkos
> ... however it requires a reasonably recent OS version (and/or mkosi) on the > host, as well as a reasonably recent package manager of the OS you're > targeting. Oops, I got this wrong: Mkosi doesn't require the *target* OS's package manager. It uses the *native* one, whichever it is on the

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2024-05-23 Thread Michal Domonkos
Oh, I'd missed your comment @DaanDeMeyer, sorry! Yep, I indeed *did* consider mkosi initially and even had a branch where I played with it for a couple of weeks. However, later I realized the philosophy of mkosi wasn't completely in line with our requirements (and that's OK!), which were: 1. R

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2024-03-15 Thread Daan De Meyer
@dmnks Next time shout at me about what's missing! I Would have been happy to discuss improvements to mkosi to make it work for your use case. (I would have commented but had no clue you were considering it) -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-m

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-10-31 Thread Panu Matilainen
Closed #2643 as completed via #2733. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#event-10820445930 You are receiving this because you are subscribed to this thread. Message ID: ___ R

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-27 Thread Michal Domonkos
Yep, thanks. I noticed this too on Hacker News yesterday and was almost going to post the same here :smile: -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#issuecomment-1736872845 You are receiving this because you are subscribed

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-26 Thread ニール・ゴンパ
Well, apparently this is now a thing: https://macoscontainers.org/ -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#issuecomment-1736575490 You are receiving this because you are subscribed to this thread. Message ID: __

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-14 Thread Michal Domonkos
> > None of this is inherently Linux specific, however to make it reasonably > > fast and efficient, you need something like OverlayFS which is currently > > only available on Linux (and [#2559 > > (comment)](https://github.com/rpm-software-management/rpm/pull/2559#issuecomment-165921) > >

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-14 Thread Michal Domonkos
> I will also point out there are openSUSE containers that use DNF too. 😉 Interesting, thanks for sharing. I don't think it changes anything discussed here so far, though :smile: -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#is

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-14 Thread ニール・ゴンパ
> None of this is inherently Linux specific, however to make it reasonably fast > and efficient, you need something like OverlayFS which is currently only > available on Linux (and > https://github.com/rpm-software-management/rpm/pull/2559#issuecomment-165921 > on some BSDs through fuse-ove

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-14 Thread Michal Domonkos
One more thing to add to the above: *This* ticket is about using OCI images for the test root creation, on Linux distributions. Any kind of non-Linux work would need to be tracked in a separate ticket. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-managem

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-14 Thread Michal Domonkos
> Keep in mind, we need a way to run it directly on the host, because all this > fanciness you're talking about doesn't exist on non-Linux platforms. In > particular, I would like to be able to run the test suite for RPM on macOS > still. 😅 In essence, what the test-suite needs is a filesystem

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-13 Thread ニール・ゴンパ
I will also point out there are openSUSE containers that use DNF too. 😉 -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#issuecomment-1718407495 You are receiving this because you are subscribed to this thread. Message ID:

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-13 Thread ニール・ゴンパ
Keep in mind, we need a way to run it directly on the host, because all this fanciness you're talking about doesn't exist on non-Linux platforms. In particular, I would like to be able to run the test suite for RPM on macOS still. 😅 -- Reply to this email directly or view it on GitHub: https:

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-12 Thread Panu Matilainen
Oh, that's a nice bonus. Assuming of course it actually works :smile: -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#issuecomment-1715127114 You are receiving this because you are subscribed to this thread. Message ID: __

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-11 Thread Michal Domonkos
Oh, Ubuntu 22.04 LTS (our lowest common denominator) ships Podman in the official repos. Nice. We can then drop Docker support altogether and just use the same backend for local and CI purposes. Most likely (still need to verify that the Podman version in Ubuntu works the same way). -- Reply t

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-11 Thread Michal Domonkos
> We'll want to be able to do stuff like use cmake to compile something against > rpm in the test-suite (but that's yet another story). Oh, this is actually a very good point. It's not something I had in mind previously, but it's yet another reason to just reuse the same Dockerfile image for b

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-11 Thread Michal Domonkos
> Oh but it wasn't added to the Dockerfile but mktree.fedora, where on that > third (?) level of inception dbus-devel is _not_ installed because that's > just rpm's own build-dependency whereas mktree.fedora prepares an environment > for just the test-suite. 😄 Yup, I can see how this is confusi

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-11 Thread Panu Matilainen
> Actually, adding `dbus-libs` to the Dockerfile wasn't needed because it > already contains it 😄 It gets pulled in as a dependency of `dbus-devel` which > we install there in order to build RPM (it's not needed in `mktree.fedora` > obviously). Oh but it wasn't added to the Dockerfile but mktre

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-08 Thread Michal Domonkos
Anyway. Having slept on this a couple of times, it's become clear that we just don't want to deal with all this bootstrapping business and basically re-implement `mkosi` (which we can't use for other reasons as outlined above). So, circling back to where we started, all we really need is to outs

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-08 Thread Michal Domonkos
Ah, OK, I think this is the error I saw initially (and now too): ``` error: unpacking of archive failed on file /dev: cpio: chown failed - Device or resource busy error: filesystem-84.87-12.1.i586: install failed ( 4/27) Installing: filesystem-84.87-12.1.i586 .

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-08 Thread Michal Domonkos
> ## Problem > > It turns out that, while this is quite easy to do on Fedora with DNF and > `unshare(1)`, it is not as easy on other distros that I've tried, namely > OpenSUSE where Zypper doesn't seem to like being run through `unshare(1)`, > and would likely require `sudo` instead Hmm, not s

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-07 Thread Michal Domonkos
In terms of usefulness/viability, the worst is option 3, followed by option 2 and then 1. Basically, the biggest drawbacks of 2) and 3) are in the area of local development with `make shell` and `make env`. These are new features that weren't there before the recent test-suite rework, so making

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-07 Thread Michal Domonkos
To summarize again, these are our options to resolve the ticket: ### 1) No change, keep the native & podman backends Image building: * Done by a host specific script, typically with a native package manager (DNF/Zypper/...) Container layering: * Host and per-test isolation is done with Bubblewr

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-07 Thread Michal Domonkos
One way to fix *that* would be to simply build and test with the `mktree.fedora` backend even in the CI, but from a Fedora container, due to the build deps. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#issuecomment-1709969191 Y

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-07 Thread Michal Domonkos
And yes, that's still double-maintenance in a sense, but it's due to the fact that we do *not* need build deps of RPM in the local testing scenario, as those are installed on the host (RPM is built on the host). So, not really much of a problem. -- Reply to this email directly or view it on Gi

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-07 Thread Michal Domonkos
> The other important point here is to avoid tests behaving differently locally > and in CI (AIUI CI didn't need the dbus-libs tweak in > [4712c9c](https://github.com/rpm-software-management/rpm/commit/4712c9c37bbf28f56f1e386df788dac440cc4cb8) > but local usage does, and that's unwanted double-m

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-07 Thread Michal Domonkos
I've come to realize that, by losing host-specific mktree scripts (e.g. `mktree.fedora`), we would also lose a couple of nice features along with it, namely: 1) Ability to simplify test tree management with the host package manager The recently added `make env` target runs a shell on the h

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-05 Thread Michal Domonkos
Ack, it's a confirmed *bug* now and will be dealt with like any other. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#issuecomment-1706551024 You are receiving this because you are subscribed to this thread. Message ID: __

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-05 Thread Panu Matilainen
Yep, to me that'd be among the top issues to address. -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#issuecomment-1706548969 You are receiving this because you are subscribed to this thread. Message ID: ___

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-05 Thread Michal Domonkos
Yup, just the fact that we currently have *two* distinct ways to create a test image is awkward and just attracts all kinds of bugs :laughing: -- Reply to this email directly or view it on GitHub: https://github.com/rpm-software-management/rpm/issues/2643#issuecomment-1706543839 You are receivi

Re: [Rpm-maint] [rpm-software-management/rpm] RFE: Standardize on OCI images for test-suite, even locally (Issue #2643)

2023-09-05 Thread Panu Matilainen
> So, to summarize, the goal of this ticket is to merge mktree.fedora and > mktree.podman such that a host-specific Dockerfile. is used to build > the image, instead of a mktree. script that does that from scratch. > It also has to support reusing the local build artifacts for local > developme