Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Andreas Schneider
On Saturday, 30 March 2024 10:37:44 CEST Richard W.M. Jones wrote: > These are just my thoughts on a Saturday morning. Feedback welcome of > course. I find the use of the ifunc attribute is really uncommon at this place. I would expect it in ffmpeg or some media codecs. In xz it looks like it

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Matthew Miller, Unit tests, even though in theory developer should mock dependencies to isolate their code to the maximum, in reality, it is not that clear cut. Therefore, those unit tests do serve to some extent as a validation that their code works with the system libraries and platforms

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Chris, The specific points of entry were evading the strength of open source: many skilled eyes. Therefore, there is value in programmatically enforcing that everything used as an input in a build must have been exposed to *normal opensource workflows*. It is a very simple principle, yet very

[Bug 2271173] perl-Business-ISBN-Data-20240323.001 is available

2024-04-01 Thread bugzilla
https://bugzilla.redhat.com/show_bug.cgi?id=2271173 Fedora Update System changed: What|Removed |Added Fixed In Version|perl-Business-ISBN-Data-202 |perl-Business-ISBN-Data-202

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Chris Adams
Once upon a time, Gabriel Somlo said: > IMHO, there's no good way to *programmatically* protect ourselves > from a malicious upstream on which we depend. If their goal is to > compromise us, they will work around whatever programmatic/technical > measures we happen to have in place at the time

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Gabriel Somlo
> On Mon, Apr 1, 2024 at 17:11:46 -0400, Matthew Miller via devel wrote: > On Sat, Mar 30, 2024 at 08:11:38PM +0100, Kevin Kofler via devel wrote: > > Unit tests are something for upstream developers. They should NEVER be run > > in a distribution build. > > Even in the few little packages I'm

[Bug 2272395] perl-IO-Compress-2.208 is available

2024-04-01 Thread bugzilla
https://bugzilla.redhat.com/show_bug.cgi?id=2272395 Fedora Update System changed: What|Removed |Added Fixed In Version||perl-IO-Compress-2.208-1.fc

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 23:37 +0200, Kevin Kofler via devel wrote: > Adam Williamson wrote: > > > * Deleting ALL files automatically generated or imported by autotools in > > > %prep, THEN running "autoreconf -i -f". (DO NOT trust autoreconf, it > > > would NOT have done the right thing here. Delete

Re: xz backdoor

2024-04-01 Thread Gary Buhrmaster
On Mon, Apr 1, 2024 at 9:17 PM Matthew Miller wrote: > > On Mon, Apr 01, 2024 at 05:47:10PM +, Gary Buhrmaster wrote: > > It does bring up a potential point that perhaps > > Fedora should have an additional repo (let's > > call it "emergency fixes") that is not community > > mirrored (so any

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Kevin Kofler via devel
Adam Williamson wrote: >> * Deleting ALL files automatically generated or imported by autotools in >> %prep, THEN running "autoreconf -i -f". (DO NOT trust autoreconf, it >> would NOT have done the right thing here. Delete the files, THEN run >> autoreconf.) > > No. This would not have avoided

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Jakub Jelinek
On Mon, Apr 01, 2024 at 01:36:48PM -0400, Peter Jones wrote: > Unrelated to the idea that some packages are special in this way, it's > probably worth writing some static analysis tools we could put into > rpm-inspect to detect when (a) a binary grows new public keys it didn't > have before, and

Re: xz backdoor

2024-04-01 Thread Matthew Miller
On Mon, Apr 01, 2024 at 05:47:10PM +, Gary Buhrmaster wrote: > It does bring up a potential point that perhaps > Fedora should have an additional repo (let's > call it "emergency fixes") that is not community > mirrored (so any mirrors for load sharing > would be fully controlled by the

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Matthew Miller
On Sat, Mar 30, 2024 at 08:11:38PM +0100, Kevin Kofler via devel wrote: > Unit tests are something for upstream developers. They should NEVER be run > in a distribution build. Even in the few little packages I'm still responsible for, I sometimes see unit test failures. The developer ran the

Summary/Minutes from today's FESCo Meeting (2024-04-01)

2024-04-01 Thread Josh Stone
(sorry for the crazy links -- meetbot didn't grab the intended name) Text Log:

Re: xz backdoor

2024-04-01 Thread Michael Catanzaro
On Mon, Apr 1 2024 at 10:25:16 AM -07:00:00, Adam Williamson wrote: Oh, ISWYM. Well, I suppose yes, that does happen to be true. We could communicate that if it's done very carefully and made really clear that it's about the *time frame*, nothing to do with the repositories. It's been

Re: Outage alert: openQA result reporting (affects critical path gating), nightly page updating, candidate compose nominations

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 08:53 -0700, Adam Williamson wrote: > Hi folks! I just discovered this so I'm still investigating it, but > wanted to give a quick heads-up. > > It looks like the message consumers on openqa01 all broke on Saturday > when a fedora-messaging update landed. This affects a lot

Re: xz backdoor

2024-04-01 Thread Sandro
On 01-04-2024 19:12, Adam Williamson wrote: On Mon, 2024-04-01 at 09:32 -0500, Michael Catanzaro wrote: On Sun, Mar 31 2024 at 06:52:53 PM +00:00:00, Christopher Klooz wrote: "Fedora Linux 40 branched users (i.e. pre-Beta) likely received the potentially vulnerable 5.6.0-2.fc40 build if the

Re: xz backdoor

2024-04-01 Thread Gary Buhrmaster
On Mon, Apr 1, 2024 at 5:27 PM Kevin Fenzi wrote: > Yes. The downgrade was pushed out on friday along with the f40 one. Of course, your mirror may vary as to availability (as I recall, in my particular case, my test VM for rawhide did not get the update for a day or so). It does bring up a

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Peter Jones
> (3) We should have a "security path", like "critical path". > > sshd is linked to a lot of libraries: > > /lib64/libaudit.so.1audit-libs > /lib64/libc.so.6glibc > /lib64/libcap-ng.so.0 libcap-ng > /lib64/libcap.so.2 libcap > /lib64/libcom_err.so.2

Re: xz backdoor

2024-04-01 Thread Christopher Klooz
On 01/04/2024 19.27, Kevin Fenzi wrote: On Mon, Apr 01, 2024 at 05:07:13PM +, Christopher Klooz wrote: On 31/03/2024 23.08, Kevin Fenzi wrote: On Sun, Mar 31, 2024 at 10:30:23PM +0200, Leon Fauster via devel wrote: Not sure, if it was already mentioned -> containers. I had here a toolbox

Re: xz backdoor

2024-04-01 Thread Kevin Fenzi
On Mon, Apr 01, 2024 at 05:07:13PM +, Christopher Klooz wrote: > > On 31/03/2024 23.08, Kevin Fenzi wrote: > > On Sun, Mar 31, 2024 at 10:30:23PM +0200, Leon Fauster via devel wrote: > > > Not sure, if it was already mentioned -> containers. I had here a toolbox > > > environment with F40.

Re: xz backdoor

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 12:16 -0500, Michael Catanzaro wrote: > On Mon, Apr 1 2024 at 10:12:55 AM -07:00:00, Adam Williamson > wrote: > > This is not really correct, or at least at all relevant. The bug > > wasn't > > in F40 Beta simply because the update never made it to 'stable'. Only > >

Re: xz backdoor

2024-04-01 Thread Michael Catanzaro
On Mon, Apr 1 2024 at 10:12:55 AM -07:00:00, Adam Williamson wrote: This is not really correct, or at least at all relevant. The bug wasn't in F40 Beta simply because the update never made it to 'stable'. Only 'stable' packages go into *composes*. However, saying that is not really useful

Re: xz backdoor

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 09:32 -0500, Michael Catanzaro wrote: > On Sun, Mar 31 2024 at 06:52:53 PM +00:00:00, Christopher Klooz > wrote: > > "Fedora Linux 40 branched users (i.e. pre-Beta) likely received the > > potentially vulnerable 5.6.0-2.fc40 build if the system updated > > between March

Re: What we mean when we talk about "supply chains" [was Re: Three steps we could take to make supply chain attacks a bit harder]

2024-04-01 Thread Gary Buhrmaster
On Mon, Apr 1, 2024 at 4:42 PM Adam Williamson wrote: > I think we *are* part of a supply chain, regardless of any handwaving > about The Open Source Model. And, more importantly, the industry has agreed to use the term supply chain. Is the term perhaps overloaded, or perhaps too

Re: xz backdoor

2024-04-01 Thread Christopher Klooz
On 31/03/2024 23.08, Kevin Fenzi wrote: On Sun, Mar 31, 2024 at 10:30:23PM +0200, Leon Fauster via devel wrote: Not sure, if it was already mentioned -> containers. I had here a toolbox environment with F40. That I had not in my first actions on the screen. The last state had 5.6.0-3 installed

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Understood. However, at least for those unit tests run in the %check, it is going to be almost unfeasible, because of the variability of the way things are done in the different programming ecosystems. In Java, unit tests are nicely separated in a different folder (i.e., `src/test`), but in

Re: xz backdoor

2024-04-01 Thread Christopher Klooz
On 01/04/2024 16.32, Michael Catanzaro wrote: On Sun, Mar 31 2024 at 06:52:53 PM +00:00:00, Christopher Klooz wrote: "Fedora Linux 40 branched users (i.e. pre-Beta) likely received the potentially vulnerable 5.6.0-2.fc40 build if the system updated between March 2nd and March 6th. Fedora

What we mean when we talk about "supply chains" [was Re: Three steps we could take to make supply chain attacks a bit harder]

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 12:27 -0400, Neal Gompa wrote: > > > > ii) the fact that this attack reinforces the painful truth that > > sophisticated attackers *are* extremely interested in attacking the > > supply chain of which we form a significant component > > Can we please reframe it for what it

Schedule for Monday's FESCo Meeting (2024-04-01)

2024-04-01 Thread Josh Stone
Following is the list of topics that will be discussed in the FESCo meeting Monday at 19:30 UTC in #meeting:fedoraproject.org on Matrix. To convert UTC to your local time, take a look at http://fedoraproject.org/wiki/UTCHowto or run: date -d '2024-04-01 19:30 UTC' Links to all issues to be

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 05:58 -0700, Carlos Rodriguez-Fernandez wrote: > Test isolation is still assuming the attack comes in the test phase. As I initially suggested it, it does not. My suggestion was that we ensure the test code is not available to the prep / build / install phases *at all*, and

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Neal Gompa
On Mon, Apr 1, 2024 at 12:22 PM Adam Williamson wrote: > > On Mon, 2024-04-01 at 10:56 +, Zbigniew Jędrzejewski-Szmek wrote: > > On Sun, Mar 31, 2024 at 07:54:08PM +0200, Kevin Kofler via devel wrote: > > > Adam Williamson wrote: > > > > Maybe this needs to go on the growing pile of reasons

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 10:56 +, Zbigniew Jędrzejewski-Szmek wrote: > On Sun, Mar 31, 2024 at 07:54:08PM +0200, Kevin Kofler via devel wrote: > > Adam Williamson wrote: > > > Maybe this needs to go on the growing pile of reasons why the > > > traditional Linux model *does* need to go away. Maybe

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Scott Schmit
On Mon, Apr 01, 2024 at 02:23:19PM -, François Rigault wrote: > > Those blobs were not in systemd, > > that was not my point, nevertheless putting it this way: nobody knows. > > For the example about compression methods you could generate your binary > using a piece of code, that can be

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Miroslav Suchý
Dne 01. 04. 24 v 3:16 dop. Kilian Hanich via devel napsal(a): Also, I have seen build setups which encode the status of tests in the eventual binary and as such info page or integrated bug report generators. Often because some distros sometimes turned them off or ships software even with failed

Re: Outage alert: openQA result reporting (affects critical path gating), nightly page updating, candidate compose nominations

2024-04-01 Thread Adam Williamson
On Mon, 2024-04-01 at 08:53 -0700, Adam Williamson wrote: > as a stopgap I > will manually trigger submission of all reports from the last couple of > days shortly. correction: I won't do this right away, as there would be a flood of duplicate reports if I did then fix the consumers. If I can't

Outage alert: openQA result reporting (affects critical path gating), nightly page updating, candidate compose nominations

2024-04-01 Thread Adam Williamson
Hi folks! I just discovered this so I'm still investigating it, but wanted to give a quick heads-up. It looks like the message consumers on openqa01 all broke on Saturday when a fedora-messaging update landed. This affects a lot of things, but by far the most important is that openQA test results

Fedora 40 compose report: 20240401.n.0 changes

2024-04-01 Thread Fedora Branched Report
OLD: Fedora-40-20240331.n.0 NEW: Fedora-40-20240401.n.0 = SUMMARY = Added images:3 Dropped images: 0 Added packages: 4 Dropped packages:0 Upgraded packages: 82 Downgraded packages: 0 Size of added packages: 4.82 MiB Size of dropped packages:0 B Size

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Definitely this attack leveraged places where eyes don't look: distributed tar.gz and blobs. I put the PoC to flag those two in github[1] Example output: $ ./rpmseclint tests/rpmseclint-test.spec -Diff- ~ test.txt + additional.txt + blob.txt.gz -Blobs application/gzip

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
I created a discussion issue for this idea: https://github.com/rpm-software-management/rpm/discussions/3009 I think it worth pursuing further. On 4/1/24 04:46, Neal Gompa wrote: On Mon, Apr 1, 2024 at 7:38 AM Zbigniew Jędrzejewski-Szmek wrote: On Sun, Mar 31, 2024 at 11:20:17PM -0700, Adam

Re: xz backdoor

2024-04-01 Thread Michael Catanzaro
On Sun, Mar 31 2024 at 06:52:53 PM +00:00:00, Christopher Klooz wrote: "Fedora Linux 40 branched users (i.e. pre-Beta) likely received the potentially vulnerable 5.6.0-2.fc40 build if the system updated between March 2nd and March 6th. Fedora Linux 40 Beta users only using stable repositories

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread François Rigault
> Those blobs were not in systemd, that was not my point, nevertheless putting it this way: nobody knows. For the example about compression methods you could generate your binary using a piece of code, that can be reviewed (maybe using a fixed seed as inspired by

Fedora rawhide compose report: 20240401.n.0 changes

2024-04-01 Thread Fedora Rawhide Report
OLD: Fedora-Rawhide-20240331.n.0 NEW: Fedora-Rawhide-20240401.n.0 = SUMMARY = Added images:3 Dropped images: 1 Added packages: 1 Dropped packages:0 Upgraded packages: 17 Downgraded packages: 0 Size of added packages: 68.66 KiB Size of dropped packages:0 B

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Carlos Rodriguez-Fernandez
Test isolation is still assuming the attack comes in the test phase. The attack can come in the `make`, or in the `make install` too. That's why the idea of other techniques being discussed are still valid, but perhaps not abstracted out enough for a wider defense. However, the test

Re: xz backdoor

2024-04-01 Thread Christopher Klooz
On 31/03/2024 21.33, Sandro wrote: On 31-03-2024 20:54, Christopher Klooz wrote: On 31/03/2024 20.52, Christopher Klooz wrote: On 31/03/2024 20.21, Michael Catanzaro wrote: On Sun, Mar 31 2024 at 09:56:04 AM -05:00:00, Michael Catanzaro wrote: I'm really frustrated with our communication

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Sam Varshavchik
Christoph Erhardt writes: I strongly oppose this suggestion. While it would have prevented this particular backdoor as a side-effect, the primary effect of going without unit tests would be an outsize hole in Fedora's QA. There have been several suggestions here for ways that this specific

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Stephen Smoogen
On Mon, 1 Apr 2024 at 04:47, François Rigault wrote: > To echo > > > To trust code, it needs to be reviewed. > > If the code is reviewed, and the build system is sane, [..] > > I deduce from your response that the binary tests committed in systemd > were not reviewed neither by co-maintainers

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Neal Gompa
On Mon, Apr 1, 2024 at 7:38 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Sun, Mar 31, 2024 at 11:20:17PM -0700, Adam Williamson wrote: > > On Sun, 2024-03-31 at 22:13 -0700, Carlos Rodriguez-Fernandez wrote: > > > Adam, > > > > > > Is there a way already to achieve test isolation during the rpm

[Bug 2272395] perl-IO-Compress-2.208 is available

2024-04-01 Thread bugzilla
https://bugzilla.redhat.com/show_bug.cgi?id=2272395 Fedora Update System changed: What|Removed |Added Status|ASSIGNED|MODIFIED --- Comment #1 from

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Zbigniew Jędrzejewski-Szmek
On Sun, Mar 31, 2024 at 11:20:17PM -0700, Adam Williamson wrote: > On Sun, 2024-03-31 at 22:13 -0700, Carlos Rodriguez-Fernandez wrote: > > Adam, > > > > Is there a way already to achieve test isolation during the rpm build? > > Nothing systematic that I'm aware of, no. It would be tricky

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Apr 01, 2024 at 09:06:16AM +0900, Dominique Martinet wrote: > Scott Schmit wrote on Sun, Mar 31, 2024 at 05:02:44PM -0400: > > Deleting the tests makes no sense to me either, but it seems like a > > mechanism that ensures the test code can't change the build outputs (or > > a mechanism to

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Apr 01, 2024 at 08:46:39AM -, François Rigault wrote: > To echo > > > To trust code, it needs to be reviewed. > > If the code is reviewed, and the build system is sane, [..] > > I deduce from your response that the binary tests committed in > systemd were not reviewed neither by

[Bug 2272395] perl-IO-Compress-2.208 is available

2024-04-01 Thread bugzilla
https://bugzilla.redhat.com/show_bug.cgi?id=2272395 Paul Howarth changed: What|Removed |Added Doc Type|--- |If docs needed, set a value

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Zbigniew Jędrzejewski-Szmek
On Sun, Mar 31, 2024 at 07:54:08PM +0200, Kevin Kofler via devel wrote: > Adam Williamson wrote: > > Maybe this needs to go on the growing pile of reasons why the > > traditional Linux model *does* need to go away. Maybe Fedora, with its > > foundation of First, should be kind of at the forefront

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread François Rigault
To echo > To trust code, it needs to be reviewed. > If the code is reviewed, and the build system is sane, [..] I deduce from your response that the binary tests committed in systemd were not reviewed neither by co-maintainers nor by downstream package maintainers. I understand that the build

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Adam Williamson
On Sun, 2024-03-31 at 20:27 -0700, Adam Williamson wrote: > > > What WOULD have greatly reduced the impact of this attack: > > * NOT enabling updates-testing by default for Branched releases. > > This would only have helped by coincidence - the coincidence that the > compromise was discovered so

Re: Three steps we could take to make supply chain attacks a bit harder

2024-04-01 Thread Adam Williamson
On Sun, 2024-03-31 at 22:13 -0700, Carlos Rodriguez-Fernandez wrote: > Adam, > > Is there a way already to achieve test isolation during the rpm build? Nothing systematic that I'm aware of, no. It would be tricky because there is no one universal Standard Test System (not even within a single