Re: Proposal: drop delta rpms (for real this time)

[Demi Marie Obenour]

On 7/4/23 11:49, Fabio Valentini wrote:
> On Thu, Jun 22, 2023 at 10:32 AM Matthew Miller
>  wrote:
>>
>> On Mon, Jun 12, 2023 at 09:46:26AM -0700, Kevin Fenzi wrote:
>>> I don't think there's a formal change filed yet.
>>>
>>> Matthew: Did you want to do that? Or would you like me or someone else
>>> to do so?
>>
>> I would love someone else to do so, but if no one else wants to, I can. :)
> 
> Well ... has anybody filed a change proposal yet, or should I do that?
> 
> Fabio

Do it!  Also include deltarpm=False by default for attack surface reduction.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Fabio Valentini]

On Thu, Jun 22, 2023 at 10:32 AM Matthew Miller
 wrote:
>
> On Mon, Jun 12, 2023 at 09:46:26AM -0700, Kevin Fenzi wrote:
> > I don't think there's a formal change filed yet.
> >
> > Matthew: Did you want to do that? Or would you like me or someone else
> > to do so?
>
> I would love someone else to do so, but if no one else wants to, I can. :)

Well ... has anybody filed a change proposal yet, or should I do that?

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Matthew Miller]

On Mon, Jun 12, 2023 at 09:46:26AM -0700, Kevin Fenzi wrote:
> I don't think there's a formal change filed yet.
> 
> Matthew: Did you want to do that? Or would you like me or someone else
> to do so?

I would love someone else to do so, but if no one else wants to, I can. :)


-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Sun, Jun 11, 2023 at 11:06:15PM -, Reon Beon via devel wrote:
> Any update? Is this being done?

I don't think there's a formal change filed yet.

Matthew: Did you want to do that? Or would you like me or someone else
to do so?

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Reon Beon via devel]

Any update? Is this being done?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Nico Kadel-Garcia]

On Fri, Mar 3, 2023 at 7:06 AM Geraldo Simião Kutz
 wrote:
>
> Testing F38 KDE spin today, I see this:
> Delta RPMs reduced 235.2 MB to 23.8 MB (89.9%)
>
> So, it seems we do still have some cornercases when deltaRPM shows its value.

Not really. That's local dis, but the aggregate burden of regularly
downloading additional repodata exceeds that pretty quickly in
environments that do daily update audits.

> But, ok, In agree that in most cases it's irrelevant this days.
> +1 for dropping deltarpm
>
> geraldosimiao

It's a complex feature to support, we're not scrambling for slivers of
disk space these days, and maintaining all the deltas grows very
quickly for environments that, in theory, no longer have point
releases to use as references for deltarpm.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Geraldo Simião Kutz]

Testing F38 KDE spin today, I see this:
Delta RPMs reduced 235.2 MB to 23.8 MB (89.9%)

So, it seems we do still have some cornercases when deltaRPM shows its
value.

But, ok, In agree that in most cases it's irrelevant this days.
+1 for dropping deltarpm

geraldosimiao



Em ter., 21 de fev. de 2023 às 17:48, Matthew Miller <
mat...@fedoraproject.org> escreveu:

> I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
> priority. Last time we talked about this we didn't really get anywhere...
>
>
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
>
> ... and that ticket hasn't moved, because fixing it isn't trivial.
>
>
> What we're doing now — as has been the case for several years, already
> noted
> in the previous discussion — has very little end-user value. Also as noted
> in that thread (as in the ticket)... that's unfortunate, because it did
> bring some real benefits (and could possibly do even more.)
>
> But, I think it's time to move on. We have ostree and various
> container-delta approaches. We should focus on those — and give DeltaRPMs a
> sad, fond farewell.
>
> --
> Matthew Miller
> 
> Fedora Project Leader
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Demi Marie Obenour]

On 2/28/23 05:06, Petr Pisar wrote:
> V Tue, Feb 28, 2023 at 02:47:03AM -, Daniel Alley napsal(a):
>> I am not sure whether by "all historical updates" you are only referring to
>> all updates being listed in updateinfo.xml, or all history generally
>> (including old packages).
> 
> The latter.
> 
>> But in the latter case, note that keeping all
>> updates massively inflates the storage requirements for maintaining a copy
>> of the repo, which many (or even most) corporate users do.  This is not
>> a huge problem, generally, but it's also not ideal, and probably isn't the
>> right tradeoff for Fedora.
>>
> I know. I only replied the question.
> 
>> Here[0] for example is RHEL 8 baseos and appstream, for which the difference
>> between storing "only the latest package" and "all the packages listed" is
>> 20x and 10x, respectively.  Metadata size would likewise be larger, meaning
>> DNF clients have more to download.
>>
> I don't say Fedora needs to do it the same way. Fedora could only accumulate
> updateinfos while only retaining the latest package. Would it inflate
> metadata? Definitely. But if you want to deliver the data to the clients, you
> have to store them somewhere. Would that affect all clients? No.
> updateinfo.xml can only be downloaded by clients requesting that data. People
> doing "dnf upgrade" can safely skip it.
> 
> Or Fedora could reverse it: Fedora would run a network service which clients
> would send a list of installed packages and the service would return a list of
> affected packages. At the end, ostree od debuginfod services work like that.

debuginfod clients should be checking the downloaded data against a hash 
included
in the binary being debugged.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Kofler via devel]

Petr Pisar wrote:
> Would that affect all clients? No. updateinfo.xml can only be downloaded
> by clients requesting that data. People doing "dnf upgrade" can safely
> skip it.

Keep in mind that updateinfo.xml is also used by GUI updaters such as 
dnfdragora or plasma-pk-updates to display details about the updates, not 
just for CLI filtering purposes (dnf --security, dnf --advisory=…).

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Petr Pisar]

V Tue, Feb 28, 2023 at 02:47:03AM -, Daniel Alley napsal(a):
> I am not sure whether by "all historical updates" you are only referring to
> all updates being listed in updateinfo.xml, or all history generally
> (including old packages).

The latter.

> But in the latter case, note that keeping all
> updates massively inflates the storage requirements for maintaining a copy
> of the repo, which many (or even most) corporate users do.  This is not
> a huge problem, generally, but it's also not ideal, and probably isn't the
> right tradeoff for Fedora.
> 
I know. I only replied the question.

> Here[0] for example is RHEL 8 baseos and appstream, for which the difference
> between storing "only the latest package" and "all the packages listed" is
> 20x and 10x, respectively.  Metadata size would likewise be larger, meaning
> DNF clients have more to download.
> 
I don't say Fedora needs to do it the same way. Fedora could only accumulate
updateinfos while only retaining the latest package. Would it inflate
metadata? Definitely. But if you want to deliver the data to the clients, you
have to store them somewhere. Would that affect all clients? No.
updateinfo.xml can only be downloaded by clients requesting that data. People
doing "dnf upgrade" can safely skip it.

Or Fedora could reverse it: Fedora would run a network service which clients
would send a list of installed packages and the service would return a list of
affected packages. At the end, ostree od debuginfod services work like that.

-- Petr


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Daniel Alley]

I am not sure whether by "all historical updates" you are only referring to all 
updates being listed in updateinfo.xml, or all history generally (including old 
packages).  But in the latter case, note that keeping all updates massively 
inflates the storage requirements for maintaining a copy of the repo, which 
many (or even most) corporate users do.  This is not a huge problem, generally, 
but it's also not ideal, and probably isn't the right tradeoff for Fedora.

Here[0] for example is RHEL 8 baseos and appstream, for which the difference 
between storing "only the latest package" and "all the packages listed" is 20x 
and 10x, respectively.  Metadata size would likewise be larger, meaning DNF 
clients have more to download.

[0]

[dalley@thinkpad repos]$ rpmrepo details el8-baseos
...
Number of packages:  12910   
Number of unique packages (latest version):  1798
Number of packages (latest 3 versions):  4459
Packages total size: 23.82 GB
Packages total size (latest version):1.4 GB  
Packages total size (latest 3 versions): 4.03 GB 

[dalley@thinkpad repos]$ rpmrepo details el8-appstream
...
Number of packages:  29103   
Number of unique packages (latest version):  5902
Number of packages (latest 3 versions):  12988   
Packages total size: 92.91 GB
Packages total size (latest version):9.12 GB 
Packages total size (latest 3 versions): 23.91 GB
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Petr Pisar]

V Fri, Feb 24, 2023 at 12:08:28PM -0400, Robert Marcano via devel napsal(a):
> On 2/24/23 12:01 PM, Chris Adams wrote:
> > Once upon a time, Robert Marcano via devel  
> > said:
> > > Does DNF on RHEL for example do something different when --security
> > > is involved? Because the RHEL documentation talks about it as a
> > > feature to use. Is a lack of metadata for previous updates the
> > > problem or the implementation?
> > 
> > Just a guess, but... updates in RHEL are different from updates in
> > Fedora because of policy.  In RHEL, updates outside of a point release
> > are much more targeted - mostly security and significant bug fixes.
> > Since there are fewer updates, the security updates stick around for a
> > while and stand out more.
> > 
> > In Fedora, essentially anything can be updated at any time for any
> > reason, whenever the packager(s) want.  It could be a minor bugfix, a
> > new upstream release, etc.  So the update "churn" tends to be higher.
> > There could be a security update today to a package (maybe just by
> > applying a quick patch), and then maybe upstream incorporates the patch
> > next week (along with other changes) and the Fedora packager updates to
> > that release.  From the Fedora point of view, the second new package is
> > not addressing any security issue, because the first new package did.
> > 
> > Neither are wrong, they're just different polices.
> 
> Right, but does a security update replaced by a non security update will
> hide the first security update on RHEL like happens on Fedora?
> 
> Because if the problem is how DNF process --security and not how Fedora and
> RHEL push security updates metadata, The Red Hat documenting how to use
> dnf-automatic to only install security updates is probably not at all
> secure. Just wondering where is the problem, metadata or implementation.

I think that DNF works same both in Fedora dnf in RHEL. The main difference is
that RHEL repositories contain all historical updates. Therefore DNF can see
security updateinfo data even for packages whose latest update is
a non-security.

-- Petr


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Kofler via devel]

Kevin Kofler via devel wrote:
> I would personally not miss --security, but what is really useful is the
> --advisory=… flag that used to be implemented by the same plugin. (They
> are now both built in in DNF.) That is invaluable to test individual
> updates from updates-testing. I would consider the absence of --advisory=…
> a dealbreaker.

PS: Note that --advisory=… actually works on *any* update, not just security 
updates. But the functionality is often lumped together with --security, 
which is why I am mentioning it here.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Fri, Feb 24, 2023 at 12:12:41PM -0800, Gordon Messmer wrote:
> On 2023-02-21 12:48, Matthew Miller wrote:
> > I was asked to weigh in onhttps://pagure.io/releng/issue/7215  as a
> > priority. Last time we talked about this we didn't really get anywhere...
> > 
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
> > 
> > ... and that ticket hasn't moved, because fixing it isn't trivial.
> 
> 
> In that thread, Kevin noted:
> 
> * only can make drpms at createrepo_c run time, this means you can't do
> them later/on the side/keep previous ones around/etc.
> * pungi doesn't have access to older rpms at the right time to generate
> more and has no way to keep them accross pushes.
> 
> If someone were interested in working on this, to fix either delta rpms, or
> the vanishing security advisory problem, or both: Are there any tools,
> repos, collections of scripts, etc that they would need to update in
> addition to those two?

Well, thats a pretty wide scope... a lot of it depends on _how_ the fix
would work.

I think Kevin's (the other Kevin) suggestion to enhance bodhi to inject
into repodata a 'last security fix for this package' for every package
in the current compose might at least help to fix that part of the missing
security update problem (combined with a dnf change to read and use this data).
So that would need bodhi and dnf work. 

It's not too clear what the best solution to drpms would be to me. 
If we moved drpm creation out of composes, the standlone app would still
need to know what to make drpms against. Against GA content would be
somewhat easy, but against previous updates... it would need to perhaps
query bodhi for that? I suspect it would need to download all the rpms
involved, so thats a ton of disk space, or it would need to have some
other way to access them all (perhaps if we ran it in infra it could
have ro access to the koji data). So that would need a new application
made, dnf may need changes to look for the drpms in another repo, or
perhaps we could just add a drpms repo (I don't know how dnf detects
drpms off hand, but I know currently they are in the main repodata).

I think it's probibly time to just let drpms go though. Bandwith vs
cpu/disk space isn't as much of a good tradeoff these days. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Gordon Messmer]

On 2023-02-21 12:48, Matthew Miller wrote:

I was asked to weigh in onhttps://pagure.io/releng/issue/7215  as a
priority. Last time we talked about this we didn't really get anywhere...

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E

... and that ticket hasn't moved, because fixing it isn't trivial.



In that thread, Kevin noted:

* only can make drpms at createrepo_c run time, this means you can't do
them later/on the side/keep previous ones around/etc.
* pungi doesn't have access to older rpms at the right time to generate
more and has no way to keep them accross pushes.

If someone were interested in working on this, to fix either delta rpms, 
or the vanishing security advisory problem, or both: Are there any 
tools, repos, collections of scripts, etc that they would need to update 
in addition to those two?


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Michael Catanzaro]

On Fri, Feb 24 2023 at 11:42:17 AM -0400, Robert Marcano via devel 
 wrote:

Does DNF on RHEL for example do something different when --security is
involved? Because the RHEL documentation talks about it as a feature 
to

use. Is a lack of metadata for previous updates the problem or the
implementation?


So if you are missing a non-security update, then any security update 
built against it will break your system, exactly the same as in the 
scenarios proposed in the very first comment in this thread. A scenario 
like libsoup depending on an nghttp update could easily happen in your 
RHEL updates just the same as it could in Fedora.


Perhaps --security might work sometimes or even most of the time, but 
it cannot work safely in general.


Michael

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Fri, Feb 24, 2023 at 06:03:39AM +0100, Kevin Kofler via devel wrote:
> Kevin Fenzi wrote:
> > I'm not sure what a solution could be. Keep every update in updateinfo
> > so dnf could tell you that there's 2 updates and 1 is security and the
> > other bugfix? but then we would need to also keep those updates around
> > to update to.
> 
> Add a repodata field "last security update EVR" that would be filled in by 
> Bodhi for any non-security update. Then DNF would just need to check whether 
> the installed EVR is less than the value of that field and treat the update 
> as a security update in that case.

That might be workable. Someone willing to file a bodhi RFE/PR? ;) 

Then we would need buyin from dnf folks... 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Fri, Feb 24, 2023 at 05:56:01AM -, Daniel Alley wrote:
> Are you saying that DNF does an exact version match instead of making the
> assumption that packages with version >= X contain a fix for a security bug
> which the updateinfo declares to be fixed in X? 
> Or that the updateinfo itself gets purged of advisories that don't apply to 
> the latest versions available.

updateinfo is created by bodhi on every push with the current data. 

So consider: 

You have foo-1.0-1.fc37 in the base repo
foo-1.1-1.fc37 comes out as an update and it fixes a security bug.
later foo-1.2-1.fc37 comes out and it's an enhancement. 

Users that updated to 1.1-1.fc37 will just see the enhancement update. 

Users that just installed or haven't updated to 1.1-1.fc37 will see just
'an enhancement update to 1.2-1.fc37' and --security will not update the
package. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[drago01]

On Wednesday, February 22, 2023, Kevin Fenzi  wrote:

> On Wed, Feb 22, 2023 at 06:18:23AM -0500, Neal Gompa wrote:
> > On Tue, Feb 21, 2023 at 3:48 PM Matthew Miller 
> wrote:
> > >
> > > I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
> > > priority. Last time we talked about this we didn't really get
> anywhere...
> > >
> > > https://lists.fedoraproject.org/archives/list/devel@lists.
> fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#
> RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
> > >
> > > ... and that ticket hasn't moved, because fixing it isn't trivial.
> > >
> > >
> > > What we're doing now — as has been the case for several years, already
> noted
> > > in the previous discussion — has very little end-user value. Also as
> noted
> > > in that thread (as in the ticket)... that's unfortunate, because it did
> > > bring some real benefits (and could possibly do even more.)
> > >
> >
> > Our tooling has been broken for a long time and contributions to that
> > tooling is just not going to happen since nobody can run this stuff
> > outside of Fedora infrastructure. It's a sad state of affairs indeed.
>
> It's not "broken" just because it doesn't do what you would like it to
> do. Please can we not disparage other peoples work?


Well everything is someone's work. Being that defensive makes discussion on
improving things unnecessary hard. I doubt people mean this comments as
personal attacks on the people who did this work.

Back on topic: using fedora without having an adequate internet connection
is not really "fun" even with delta roms due to the amount and frequency of
updates. From that pov the can go if the costs of keeping them is not
justified.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Robert Marcano via devel]

On 2/24/23 12:35 PM, Gordon Messmer wrote:

On 2023-02-24 07:42, Robert Marcano via devel wrote:
Does DNF on RHEL for example do something different when --security is 
involved? Because the RHEL documentation talks about it as a feature 
to use. Is a lack of metadata for previous updates the problem or the 
implementation? 



I don't have the log, but I checked this about a month ago:

I can set up an 8.3 system (I used a UBI container, to be specific) and 
subscribe to Red Hat's repositories. Since 8.3, there has been a 
security update to bash, but the most recent bash package is not a 
security fix. If I run |dnf update --security bash|, the system will 
offer the most recent bash package, even though it is not a security 
fix. Naturally, if I run |dnf update bash|, I get the same offer.


So on RHEL, dnf will evidently offer to update a package to the current 
version if any of the available update candidates are marked as a 
security update.  And there are multiple update candidates in RHEL 
repositories, as well as CentOS Stream repositories, unlike Fedora.


Thanks for testing it. So, if there is a desire to "fix" this in Fedora, 
having the repository includes the latest package marked as a security 
update and the latest one. I am not sure that will solve much because it 
still depends, as other posts said, Fedora has more open rules for 
updates, than enterprise distributions and therea are entire version 
jumps without tracking errata metadata.


So, from my point of view the biggest problem with "dnf update 
--security" on Fedora is that rpm doesn't track minor-version 
dependencies of libraries without versioned symbols, which means that 
"dnf update --security" can easily break the system by updating a leaf 
package but not updating dependencies that have added new interfaces 
that it requires.  (But I'm working on fixing that.)

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Gordon Messmer]

On 2023-02-24 07:42, Robert Marcano via devel wrote:
Does DNF on RHEL for example do something different when --security is 
involved? Because the RHEL documentation talks about it as a feature 
to use. Is a lack of metadata for previous updates the problem or the 
implementation? 



I don't have the log, but I checked this about a month ago:

I can set up an 8.3 system (I used a UBI container, to be specific) and 
subscribe to Red Hat's repositories. Since 8.3, there has been a 
security update to bash, but the most recent bash package is not a 
security fix. If I run |dnf update --security bash|, the system will 
offer the most recent bash package, even though it is not a security 
fix. Naturally, if I run |dnf update bash|, I get the same offer.


So on RHEL, dnf will evidently offer to update a package to the current 
version if any of the available update candidates are marked as a 
security update.  And there are multiple update candidates in RHEL 
repositories, as well as CentOS Stream repositories, unlike Fedora.


So, from my point of view the biggest problem with "dnf update 
--security" on Fedora is that rpm doesn't track minor-version 
dependencies of libraries without versioned symbols, which means that 
"dnf update --security" can easily break the system by updating a leaf 
package but not updating dependencies that have added new interfaces 
that it requires.  (But I'm working on fixing that.)

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Robert Marcano via devel]

On 2/24/23 12:01 PM, Chris Adams wrote:

Once upon a time, Robert Marcano via devel  said:

Does DNF on RHEL for example do something different when --security
is involved? Because the RHEL documentation talks about it as a
feature to use. Is a lack of metadata for previous updates the
problem or the implementation?


Just a guess, but... updates in RHEL are different from updates in
Fedora because of policy.  In RHEL, updates outside of a point release
are much more targeted - mostly security and significant bug fixes.
Since there are fewer updates, the security updates stick around for a
while and stand out more.

In Fedora, essentially anything can be updated at any time for any
reason, whenever the packager(s) want.  It could be a minor bugfix, a
new upstream release, etc.  So the update "churn" tends to be higher.
There could be a security update today to a package (maybe just by
applying a quick patch), and then maybe upstream incorporates the patch
next week (along with other changes) and the Fedora packager updates to
that release.  From the Fedora point of view, the second new package is
not addressing any security issue, because the first new package did.

Neither are wrong, they're just different polices.


Right, but does a security update replaced by a non security update will 
hide the first security update on RHEL like happens on Fedora?


Because if the problem is how DNF process --security and not how Fedora 
and RHEL push security updates metadata, The Red Hat documenting how to 
use dnf-automatic to only install security updates is probably not at 
all secure. Just wondering where is the problem, metadata or implementation.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Chris Adams]

Once upon a time, Robert Marcano via devel  said:
> Does DNF on RHEL for example do something different when --security
> is involved? Because the RHEL documentation talks about it as a
> feature to use. Is a lack of metadata for previous updates the
> problem or the implementation?

Just a guess, but... updates in RHEL are different from updates in
Fedora because of policy.  In RHEL, updates outside of a point release
are much more targeted - mostly security and significant bug fixes.
Since there are fewer updates, the security updates stick around for a
while and stand out more.

In Fedora, essentially anything can be updated at any time for any
reason, whenever the packager(s) want.  It could be a minor bugfix, a
new upstream release, etc.  So the update "churn" tends to be higher.
There could be a security update today to a package (maybe just by
applying a quick patch), and then maybe upstream incorporates the patch
next week (along with other changes) and the Fedora packager updates to
that release.  From the Fedora point of view, the second new package is
not addressing any security issue, because the first new package did.

Neither are wrong, they're just different polices.
-- 
Chris Adams 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Robert Marcano via devel]

On 2/23/23 8:24 PM, Dennis Gilmore via devel wrote:



On Tue, Feb 21, 2023 at 2:48 PM Matthew Miller > wrote:


I was asked to weigh in on https://pagure.io/releng/issue/7215
 as a
priority. Last time we talked about this we didn't really get
anywhere...


https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
 


... and that ticket hasn't moved, because fixing it isn't trivial.


What we're doing now — as has been the case for several years,
already noted
in the previous discussion — has very little end-user value. Also as
noted
in that thread (as in the ticket)... that's unfortunate, because it did
bring some real benefits (and could possibly do even more.)

But, I think it's time to move on. We have ostree and various
container-delta approaches. We should focus on those — and give
DeltaRPMs a
sad, fond farewell.


The last updates just now on three different machines gave me
Delta RPMs reduced 284.9 MB of updates to 281.0 MB (1.4% saved)
Delta RPMs reduced 14.3 MB of updates to 3.3 MB (76.9% saved)
  and the third had no Delta RPMs

Outside of specific instances, the first or last results are typical.  I 
think it is time to say goodbye. Times are very different from what they 
were when we added support.


Results may be better for applications that require a lot of data 
outside the projects binaries, like games related packages as 
wesnoth-data, built from the wesnoth main spec file, but not 
xonotic-data (1.1G) that is a split from the binaries spec files.


Disabling deltarpms may not change much for a lot of people, but it may 
affect a few ones using these games. maybe packagers will be forces to 
split their data to different specs, how easy it is depends on the build 
process of those games.




Dennis

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Robert Marcano via devel]

On 2/23/23 8:04 PM, Michael Catanzaro wrote:
On Fri, Feb 24 2023 at 12:00:40 AM +0100, Björn Persson 
 wrote:

There are also other dangers with installing only security fixes. If a
bugfix is released and packaged, and later it's discovered that the bug
had security implications, then no security update will be made because
the fix is already packaged. It might be possible to set a security
flag on the update after the fact, but nobody will bother with that.

I would therefore advise against using --security. If one can't install
all the updates continuously, then one should use a more stable
distribution than Fedora.


tbh I'd go so far as to propose that this functionality should not be 
reimplemented in dnf5 at all.




Does DNF on RHEL for example do something different when --security is 
involved? Because the RHEL documentation talks about it as a feature to 
use. Is a lack of metadata for previous updates the problem or the 
implementation?



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/

List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Daniel Alley]

Are you saying that DNF does an exact version match instead of making the 
assumption that packages with version >= X contain a fix for a security bug 
which the updateinfo declares to be fixed in X?  Or that the updateinfo itself 
gets purged of advisories that don't apply to the latest versions available.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Kofler via devel]

Kevin Fenzi wrote:
> I'm not sure what a solution could be. Keep every update in updateinfo
> so dnf could tell you that there's 2 updates and 1 is security and the
> other bugfix? but then we would need to also keep those updates around
> to update to.

Add a repodata field "last security update EVR" that would be filled in by 
Bodhi for any non-security update. Then DNF would just need to check whether 
the installed EVR is less than the value of that field and treat the update 
as a security update in that case.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Kofler via devel]

Michael Catanzaro wrote:
> On Fri, Feb 24 2023 at 12:00:40 AM +0100, Björn Persson
>  wrote:
>> There are also other dangers with installing only security fixes. If a
>> bugfix is released and packaged, and later it's discovered that the
>> bug
>> had security implications, then no security update will be made
>> because
>> the fix is already packaged. It might be possible to set a security
>> flag on the update after the fact, but nobody will bother with that.
>> 
>> I would therefore advise against using --security. If one can't
>> install
>> all the updates continuously, then one should use a more stable
>> distribution than Fedora.
> 
> tbh I'd go so far as to propose that this functionality should not be
> reimplemented in dnf5 at all.

I would personally not miss --security, but what is really useful is the 
--advisory=… flag that used to be implemented by the same plugin. (They are 
now both built in in DNF.) That is invaluable to test individual updates 
from updates-testing. I would consider the absence of --advisory=… a 
dealbreaker.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Reon Beon via devel]

Should it be replaced with rsync?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Dennis Gilmore via devel]

On Tue, Feb 21, 2023 at 2:48 PM Matthew Miller 
wrote:

> I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
> priority. Last time we talked about this we didn't really get anywhere...
>
>
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
>
> ... and that ticket hasn't moved, because fixing it isn't trivial.
>
>
> What we're doing now — as has been the case for several years, already
> noted
> in the previous discussion — has very little end-user value. Also as noted
> in that thread (as in the ticket)... that's unfortunate, because it did
> bring some real benefits (and could possibly do even more.)
>
> But, I think it's time to move on. We have ostree and various
> container-delta approaches. We should focus on those — and give DeltaRPMs a
> sad, fond farewell.
>

The last updates just now on three different machines gave me
Delta RPMs reduced 284.9 MB of updates to 281.0 MB (1.4% saved)
Delta RPMs reduced 14.3 MB of updates to 3.3 MB (76.9% saved)
 and the third had no Delta RPMs

Outside of specific instances, the first or last results are typical.  I
think it is time to say goodbye. Times are very different from what they
were when we added support.

Dennis
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Demi Marie Obenour]

On 2/23/23 18:00, Björn Persson wrote:
> Gordon Messmer wrote:
>> Contrary-wise: Because Fedora updates only contains the latest built, 
>> once a build marked as a security fix is obsoleted by another build, 
>> there is no longer any indication that a security issue existed in any 
>> version, at which point "dnf update --security" no longer works.
> 
> There are also other dangers with installing only security fixes. If a
> bugfix is released and packaged, and later it's discovered that the bug
> had security implications, then no security update will be made because
> the fix is already packaged. It might be possible to set a security
> flag on the update after the fact, but nobody will bother with that.
> 
> I would therefore advise against using --security. If one can't install
> all the updates continuously, then one should use a more stable
> distribution than Fedora.
> 
> Björn Persson

I actually use --security for the *opposite* purpose: to get security
updates from updates-testing.  Only problem I can remember having is broken
syntax highlighting from a somewhat recent vim update.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Demi Marie Obenour]

On 2/23/23 12:52, Matthew Miller wrote:
> On Wed, Feb 22, 2023 at 10:33:27AM -0800, Kevin Fenzi wrote:
>> Just FYI, we do not produce drpms for rawhide/branched at all (since
>> 2017 ish).
> [...]
>> Its one line in bodhi pungi config:
>> createrepo_deltas = False
>>> If shut off, can it be turned back on again (in case of Regrets)?
>> Just change the one line back to True (well, it's more complex because
>> we only actually do drpms for the 'Everything' repo, not others, but
>> it's one line).
> 
> 
> So, it sounds like "remove the step in the release SOP to turn them _on_ for
> the branch at release time" would be the easiest way to go. And the default
> config could be changed in DNF at any time for F38+.

I would like to see the DNF config changed in F38, for attack surface
reduction reasons.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Michael Catanzaro]

On Fri, Feb 24 2023 at 12:00:40 AM +0100, Björn Persson 
 wrote:

There are also other dangers with installing only security fixes. If a
bugfix is released and packaged, and later it's discovered that the 
bug
had security implications, then no security update will be made 
because

the fix is already packaged. It might be possible to set a security
flag on the update after the fact, but nobody will bother with that.

I would therefore advise against using --security. If one can't 
install

all the updates continuously, then one should use a more stable
distribution than Fedora.


tbh I'd go so far as to propose that this functionality should not be 
reimplemented in dnf5 at all.


___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Björn Persson]

Gordon Messmer wrote:
> Contrary-wise: Because Fedora updates only contains the latest built, 
> once a build marked as a security fix is obsoleted by another build, 
> there is no longer any indication that a security issue existed in any 
> version, at which point "dnf update --security" no longer works.

There are also other dangers with installing only security fixes. If a
bugfix is released and packaged, and later it's discovered that the bug
had security implications, then no security update will be made because
the fix is already packaged. It might be possible to set a security
flag on the update after the fact, but nobody will bother with that.

I would therefore advise against using --security. If one can't install
all the updates continuously, then one should use a more stable
distribution than Fedora.

Björn Persson


pgp_GPr1Z148d.pgp
Description: OpenPGP digital signatur
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Thu, Feb 23, 2023 at 10:15:42AM -0800, Gordon Messmer wrote:
> On 2023-02-23 10:05, Gordon Messmer wrote:
> > Contrary-wise: Because Fedora updates only contains the latest built,
> > once a build marked as a security fix is obsoleted by another build,
> > there is no longer any indication that a security issue existed in any
> > version, at which point "dnf update --security" no longer works.
> 
> 
> For example, https://bodhi.fedoraproject.org/updates/FEDORA-2022-839fd408a5
> is no longer an indication of a problem in a default package:
> 
> $ podman run --rm -it fedora:37
> [root@d1c2aa7da870 /]# rpm -qa vim\*
> vim-data-9.0.475-1.fc37.noarch
> vim-minimal-9.0.475-1.fc37.x86_64
> [root@d1c2aa7da870 /]# dnf update --security vim\*
> No security updates needed for "vim*", but 2 updates available
> Dependencies resolved.
> Nothing to do.
> Complete!
> 
> > That might be a problem only for systems that are updated less
> > frequently than the window between a security update and a later build,
> > I still think it's a flaw that should be fixed.
> 
> (And I probably shouldn't have phrased this as if it's very limited. 
> Anything installed from the installation media or "fedora" repo without full
> updates would definitely have security issues that weren't reflected in the
> package set selected by "dnf update --security")

For this reason, bodhi used to mark such packages for the rest of the
release. Ie, if you mark foo-1.0-1.fc37 a security update, forever after
that foo package gets 'security' in the updateinfo. I think this was
dropped because it confused too many people and it also didn't really
express the actual problem here. 

I'm not sure what a solution could be. Keep every update in updateinfo
so dnf could tell you that there's 2 updates and 1 is security and the
other bugfix? but then we would need to also keep those updates around
to update to. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Thu, Feb 23, 2023 at 10:46:03AM +0100, Vít Ondruch wrote:
> 
> Dne 22. 02. 23 v 5:38 Kevin Kofler via devel napsal(a):
> > Kevin Fenzi wrote:
> > > I fear the only way to fix it is to get pungi to merge entire repos, and
> > > I don't think thats anything that pungi wants to be on the hook for
> > > doing.
> > But there are more things than just DeltaRPMs it could be useful for. E.g.,
> > I remember that in ancient times (pre Core-Extras Merge), some Fedora
> > repository (IIRC, the old Fedora Extras) used to ship not only the latest
> > build for each package, but the TWO latest builds, so that if the latest
> > build was broken, you could easily downgrade to the previous one. That
> > should really be done in all the rolling repositories (updates, updates-
> > testing, Rawhide).
> 
> 
> Once there was this DNF RFE:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1397174

yeah. We actually do have a repo with all updates for stable releases,
which we had to setup for a corner fedora coreos corner case. I suppose
we could enable that more globally if the desire is for easy downgrades
in stable releases. I think they would be much more common in
rawhide/branched though. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Gordon Messmer]

On 2023-02-23 10:05, Gordon Messmer wrote:
Contrary-wise: Because Fedora updates only contains the latest built, 
once a build marked as a security fix is obsoleted by another build, 
there is no longer any indication that a security issue existed in any 
version, at which point "dnf update --security" no longer works. 



For example, 
https://bodhi.fedoraproject.org/updates/FEDORA-2022-839fd408a5 is no 
longer an indication of a problem in a default package:


$ podman run --rm -it fedora:37
[root@d1c2aa7da870 /]# rpm -qa vim\*
vim-data-9.0.475-1.fc37.noarch
vim-minimal-9.0.475-1.fc37.x86_64
[root@d1c2aa7da870 /]# dnf update --security vim\*
No security updates needed for "vim*", but 2 updates available
Dependencies resolved.
Nothing to do.
Complete!

That might be a problem only for systems that are updated less 
frequently than the window between a security update and a later 
build, I still think it's a flaw that should be fixed. 


(And I probably shouldn't have phrased this as if it's very limited.  
Anything installed from the installation media or "fedora" repo without 
full updates would definitely have security issues that weren't 
reflected in the package set selected by "dnf update --security")

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Gordon Messmer]

On 2023-02-22 10:48, Kevin Fenzi wrote:

On Wed, Feb 22, 2023 at 05:38:23AM +0100, Kevin Kofler via devel wrote:


I remember that in ancient times (pre Core-Extras Merge), some Fedora
repository (IIRC, the old Fedora Extras) used to ship not only the latest
build for each package, but the TWO latest builds,

Cons:
* It will allow for more easily tricking people into downgrading to a
version that has serious security problems so they could be exploited.



Contrary-wise: Because Fedora updates only contains the latest built, 
once a build marked as a security fix is obsoleted by another build, 
there is no longer any indication that a security issue existed in any 
version, at which point "dnf update --security" no longer works.


That might be a problem only for systems that are updated less 
frequently than the window between a security update and a later build, 
I still think it's a flaw that should be fixed.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Matthew Miller]

On Wed, Feb 22, 2023 at 10:33:27AM -0800, Kevin Fenzi wrote:
> Just FYI, we do not produce drpms for rawhide/branched at all (since
> 2017 ish).
[...]
> Its one line in bodhi pungi config:
> createrepo_deltas = False
> > If shut off, can it be turned back on again (in case of Regrets)?
> Just change the one line back to True (well, it's more complex because
> we only actually do drpms for the 'Everything' repo, not others, but
> it's one line).


So, it sounds like "remove the step in the release SOP to turn them _on_ for
the branch at release time" would be the easiest way to go. And the default
config could be changed in DNF at any time for F38+.

It doesn't sound like it's really worthwhile to bother changing F36 or F37.



-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Matthew Miller]

On Wed, Feb 22, 2023 at 10:56:53AM -0500, Demi Marie Obenour wrote:
> > Could we do this as a two-step approach? First change the default to
> > not use deltas but still allow people to opt-in to it. Then (assuming
> > we can track this, which maybe we can't) see how much they're used
> > before we decide to pull the plug on producing them.
> That would be absolutely awesome.

I don't think we can actually tell easily. Additionally, we can't actually
tell the important thing, which was "how useful were they really?" — if we
have a million people using them but getting an average 0.01% size
benefit... that probably doesn't outweigh the costs.

But, we _can_ tell (again, see previous discussion) that what we're
currently providing is really unlikely to be very useful. So, I'm not
actually sure this approach buys us anything.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Matthew Miller]

On Wed, Feb 22, 2023 at 01:41:43PM +0100, Luna Jernberg wrote:
> Dislike -1

This kind of "vote" isn't really very helpful. Of course not everyone is
going to like everything. As I said initially, it's unfortunate that we
aren't in a better place here. But we're in the place we're in. If you have
a constructive, alternate proposal, let's hear that, please.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Vít Ondruch]

Dne 22. 02. 23 v 5:38 Kevin Kofler via devel napsal(a):

Kevin Fenzi wrote:

I fear the only way to fix it is to get pungi to merge entire repos, and
I don't think thats anything that pungi wants to be on the hook for
doing.

But there are more things than just DeltaRPMs it could be useful for. E.g.,
I remember that in ancient times (pre Core-Extras Merge), some Fedora
repository (IIRC, the old Fedora Extras) used to ship not only the latest
build for each package, but the TWO latest builds, so that if the latest
build was broken, you could easily downgrade to the previous one. That
should really be done in all the rolling repositories (updates, updates-
testing, Rawhide).



Once there was this DNF RFE:

https://bugzilla.redhat.com/show_bug.cgi?id=1397174


Vít




 Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue


OpenPGP_signature
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Wed, Feb 22, 2023 at 04:44:29PM -0500, Neal Gompa wrote:
> > We could merge current repos into the newly created one from pungi, but
> > it would be a vast amount of work. It would mean all repodata would need
> > to be regenerated. That said, it could be done, but no one has had the
> > cycles to work on it.
> >
> 
> Maybe we should just publish into Pulp and have them do that bit for
> us automatically. I wouldn't personally be comfortable with that until
> Pulp is available in Fedora for people to easily deploy, but I know
> the COPR folks are investigating it for their tooling. And the
> AlmaLinux folks use Pulp for their system for similar reasons.
> 
> Pulp is designed to handle super-huge repositories like that.

I haven't looked at pulp in years... but back when I did it seemed
pretty complex and heavy, but sure we could look at it again. I suspect some
of the things we do wouldn't be to compatible, like bodhi injecting
security/bugfix/enhancement data into repodata after it's made. 

In any case we don't have it now, so I still think dropping drpms for
now is the way to go. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Gregory Bartholomew]

On Wed, Feb 22, 2023 at 12:49 PM Kevin Fenzi  wrote:

> It does have advantages for sure.
>
> Pros:
> * can dnf downgrade easily.
> * can choose not to upgrade something thats a big change you aren't
> ready for.
>
>
I don't know enough about RPM packaging or DNF CoW to say either way, but I
wonder if these enhancements might also contribute to getting DNF CoW's
"reusable extents" feature working? If I understood the FOSDEM presentation
correctly, DNF CoW would not only save the installer from having to rewrite
the file to the user's disk on update when the file has not changed, it
would also not download the file in the first place if it hasn't changed.
Presumably this depends on the same functionality that deltarpm would need
to identify unchanged files in RPM packages? So if these enhancements to
deltarpm would also contribute to DNF CoW, I would consider that another
plus.

Here is the DNF CoW FOSDEM presentation if anyone is interested in it.
https://www.youtube.com/watch?v=-qYgkcs6Uxk

Just my two cents.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Neal Gompa]

On Wed, Feb 22, 2023 at 1:40 PM Kevin Fenzi  wrote:
>
> The reason it behaves this way is because koji has no concept of 'newer
> version' or 'older version'. It has only when a valid build was tagged
> into a tag, and pungi operates on that with 'give me the latest tagged
> packages in these tags'.
>

We have compose metadata for every single compose that has succeeded
and failed. In that metadata, we know what NVRs from Koji were pulled
in, and we can reconstitute any historical update push with that data.

> We could merge current repos into the newly created one from pungi, but
> it would be a vast amount of work. It would mean all repodata would need
> to be regenerated. That said, it could be done, but no one has had the
> cycles to work on it.
>

Maybe we should just publish into Pulp and have them do that bit for
us automatically. I wouldn't personally be comfortable with that until
Pulp is available in Fedora for people to easily deploy, but I know
the COPR folks are investigating it for their tooling. And the
AlmaLinux folks use Pulp for their system for similar reasons.

Pulp is designed to handle super-huge repositories like that.




--
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Stephen Smoogen]

On Wed, 22 Feb 2023 at 15:56, Daniel Alley  wrote:

> Well, regarding the "based on something", you can hand off a list of
> packages to createrepo_c with --pkglist, and avoid the need to download
> files with --update + --skip-stat.  Unfortunately that doesn't help you
> with the package file management.  In a vacuum --baseurl would help here
> because you could have one root directory, however in reality it breaks
> repository mirroring because any mirror be telling clients to fetch the
> packages from the source-of-truth.
>
> I'm not 100% sure how --basedir works, the description is a bit vague.
>
> Another option is to use something like Pulp which stores all the
> information required for metadata generation inside Postgresql and thus can
> do so without ever touching the packages / headers again.  That approach
> isn't necessarily free of downsides either, but it does abstract the whole
> file management problem.
>

I think we need to begin by figuring out what happens in at least the
'pungi' part of the daily 'let's make updates and rawhide'. There are a LOT
of things which are going on which interrelate with each other and are
prone to cascading breakage when something is 'added', 'removed', 'fixed',
or 'changed'. There are also some hard resource limitations on how much
CPU/disk/memory is available, how close things must be to 'work', and
places where things break a lot but trying to remove/fix/change would
require longer downtimes than the project has allowed in the past.

I say this from having done all of the above at one point or another and
caused all kinds of chaos in doing so. I have probably used up more of
Kevin's patience on those than was right.



-- 
Stephen Smoogen, Red Hat Automotive
Let us be kind to one another, for most of us are fighting a hard battle.
-- Ian MacClaren
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Wed, Feb 22, 2023 at 08:12:01PM +, Mattia Verga via devel wrote:
> So, does that mean that every time a compose is performed pungi
> downloads all RPMs tagged with a specific tag in a directory and then it
> operates on those to create repository metadata? Or it just downloads
> the rpms headers, or whatever?

Yes. It downloads them all, because it needs the ones signed with the
specific keys in it's config. It can optionally however look at a
previous config and decide that nothing changed in koji so it can reuse
them, but in practice I think this doesn't happen much.

> Can we have a persistent compose root directory as base, then download
> the latest builds (if newer), do the pruning "based on something",
> create the repo and save the state as the base for the next run? Or will
> that eat way too much disk space? On the other hand we could save
> bandwith because we would not download all the rpms which were not
> updated from the previous run... what's more valuable for our
> infrastructure, disk space or network bandwith usage?

Definitely disk space is more important right now. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Wed, Feb 22, 2023 at 12:43:20PM -0800, Adam Williamson wrote:
> > It's not "broken" just because it doesn't do what you would like it to
> > do. Please can we not disparage other peoples work?
> > 
> > The reason it behaves this way is because koji has no concept of 'newer
> > version' or 'older version'. It has only when a valid build was tagged
> > into a tag, and pungi operates on that with 'give me the latest tagged
> > packages in these tags'. 
> > 
> > We could merge current repos into the newly created one from pungi, but
> > it would be a vast amount of work. It would mean all repodata would need
> > to be regenerated. That said, it could be done, but no one has had the
> > cycles to work on it.
> 
> I'm guessing this would also make composes slower and less reliable?

yep.

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Daniel Alley]

Well, regarding the "based on something", you can hand off a list of packages 
to createrepo_c with --pkglist, and avoid the need to download files with 
--update + --skip-stat.  Unfortunately that doesn't help you with the package 
file management.  In a vacuum --baseurl would help here because you could have 
one root directory, however in reality it breaks repository mirroring because 
any mirror be telling clients to fetch the packages from the source-of-truth.

I'm not 100% sure how --basedir works, the description is a bit vague.

Another option is to use something like Pulp which stores all the information 
required for metadata generation inside Postgresql and thus can do so without 
ever touching the packages / headers again.  That approach isn't necessarily 
free of downsides either, but it does abstract the whole file management 
problem.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Adam Williamson]

On Wed, 2023-02-22 at 10:40 -0800, Kevin Fenzi wrote:
> On Wed, Feb 22, 2023 at 06:18:23AM -0500, Neal Gompa wrote:
> > On Tue, Feb 21, 2023 at 3:48 PM Matthew Miller  
> > wrote:
> > > 
> > > I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
> > > priority. Last time we talked about this we didn't really get anywhere...
> > > 
> > > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
> > > 
> > > ... and that ticket hasn't moved, because fixing it isn't trivial.
> > > 
> > > 
> > > What we're doing now — as has been the case for several years, already 
> > > noted
> > > in the previous discussion — has very little end-user value. Also as noted
> > > in that thread (as in the ticket)... that's unfortunate, because it did
> > > bring some real benefits (and could possibly do even more.)
> > > 
> > 
> > Our tooling has been broken for a long time and contributions to that
> > tooling is just not going to happen since nobody can run this stuff
> > outside of Fedora infrastructure. It's a sad state of affairs indeed.
> 
> It's not "broken" just because it doesn't do what you would like it to
> do. Please can we not disparage other peoples work?
> 
> The reason it behaves this way is because koji has no concept of 'newer
> version' or 'older version'. It has only when a valid build was tagged
> into a tag, and pungi operates on that with 'give me the latest tagged
> packages in these tags'. 
> 
> We could merge current repos into the newly created one from pungi, but
> it would be a vast amount of work. It would mean all repodata would need
> to be regenerated. That said, it could be done, but no one has had the
> cycles to work on it.

I'm guessing this would also make composes slower and less reliable?
-- 
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @ad...@fosstodon.org
https://www.happyassassin.net



___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Mattia Verga via devel]

Il 22/02/23 19:46, Kevin Fenzi ha scritto:
> On Wed, Feb 22, 2023 at 08:37:00AM +, Richard W.M. Jones wrote:
>> It's also been a long time since I've seen any benefit.
>>
>> Can anyone summarise quickly why delta RPMs don't work?  It seems like
>> they "obviously" should be possible, because there must be a lot of
>> commonality in the content of adjacent RPM versions ...
> pungi creates composes. It does this by asking koji for 'whats the
> latest rpm tagged into this tag'. It then operates on those packages.
> It has little concept of other repos/things other than koji.
>
> So, in order to do what might be expected it would have to:
>
> * query koji/download things as it does now.
> * copy in the current repos
> * possibly prune based on something (only 2 copies, only X days old)
> * then go on with it's repo creation, etc.
>
So, does that mean that every time a compose is performed pungi
downloads all RPMs tagged with a specific tag in a directory and then it
operates on those to create repository metadata? Or it just downloads
the rpms headers, or whatever?

Can we have a persistent compose root directory as base, then download
the latest builds (if newer), do the pruning "based on something",
create the repo and save the state as the base for the next run? Or will
that eat way too much disk space? On the other hand we could save
bandwith because we would not download all the rpms which were not
updated from the previous run... what's more valuable for our
infrastructure, disk space or network bandwith usage?

Mattia

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Wed, Feb 22, 2023 at 12:51:54PM -0600, Chris Adams wrote:
> Once upon a time, Kevin Fenzi  said:
> > Pros:
> > * can dnf downgrade easily.
> 
> This assumes the N-1 release is working though, which is not always
> valid when there's an issue (sometimes release N doesn't work, so
> there's release N+1 that still has a problem, but then release N-1 would
> be gone).

Right. Also, for things that incompatibly upgrade your data, rolling
back will not be as easy as a dnf downgrade. ;( 
Luckily those things are rare.
 
> > Cons:
> 
> Wouldn't it also increase the repodata size significantly, which is
> already a problem for dnf resource usage, especially on small devices
> like Raspberry Pi?

Yep. Thats another con.

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Chris Adams]

Once upon a time, Kevin Fenzi  said:
> Pros:
> * can dnf downgrade easily.

This assumes the N-1 release is working though, which is not always
valid when there's an issue (sometimes release N doesn't work, so
there's release N+1 that still has a problem, but then release N-1 would
be gone).

> Cons:

Wouldn't it also increase the repodata size significantly, which is
already a problem for dnf resource usage, especially on small devices
like Raspberry Pi?

-- 
Chris Adams 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Wed, Feb 22, 2023 at 05:38:23AM +0100, Kevin Kofler via devel wrote:
> Kevin Fenzi wrote:
> > I fear the only way to fix it is to get pungi to merge entire repos, and
> > I don't think thats anything that pungi wants to be on the hook for
> > doing.
> 
> But there are more things than just DeltaRPMs it could be useful for. E.g., 
> I remember that in ancient times (pre Core-Extras Merge), some Fedora 
> repository (IIRC, the old Fedora Extras) used to ship not only the latest 
> build for each package, but the TWO latest builds, so that if the latest 
> build was broken, you could easily downgrade to the previous one. That 
> should really be done in all the rolling repositories (updates, updates-
> testing, Rawhide).

It does have advantages for sure. 

Pros:
* can dnf downgrade easily.
* can choose not to upgrade something thats a big change you aren't
ready for.

Cons:
* It takes a bunch more mirror space.
* It would make composes take longer.
* It will allow for more easily tricking people into downgrading to a
version that has serious security problems so they could be exploited.

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Wed, Feb 22, 2023 at 08:37:00AM +, Richard W.M. Jones wrote:
> It's also been a long time since I've seen any benefit.
> 
> Can anyone summarise quickly why delta RPMs don't work?  It seems like
> they "obviously" should be possible, because there must be a lot of
> commonality in the content of adjacent RPM versions ...

pungi creates composes. It does this by asking koji for 'whats the
latest rpm tagged into this tag'. It then operates on those packages. 
It has little concept of other repos/things other than koji.

So, in order to do what might be expected it would have to: 

* query koji/download things as it does now.
* copy in the current repos
* possibly prune based on something (only 2 copies, only X days old)
* then go on with it's repo creation, etc. 

Last I checked pungi developers weren't too excited by pungi growing
that ability/code. I can try asking again. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Wed, Feb 22, 2023 at 06:18:23AM -0500, Neal Gompa wrote:
> On Tue, Feb 21, 2023 at 3:48 PM Matthew Miller  
> wrote:
> >
> > I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
> > priority. Last time we talked about this we didn't really get anywhere...
> >
> > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
> >
> > ... and that ticket hasn't moved, because fixing it isn't trivial.
> >
> >
> > What we're doing now — as has been the case for several years, already noted
> > in the previous discussion — has very little end-user value. Also as noted
> > in that thread (as in the ticket)... that's unfortunate, because it did
> > bring some real benefits (and could possibly do even more.)
> >
> 
> Our tooling has been broken for a long time and contributions to that
> tooling is just not going to happen since nobody can run this stuff
> outside of Fedora infrastructure. It's a sad state of affairs indeed.

It's not "broken" just because it doesn't do what you would like it to
do. Please can we not disparage other peoples work?

The reason it behaves this way is because koji has no concept of 'newer
version' or 'older version'. It has only when a valid build was tagged
into a tag, and pungi operates on that with 'give me the latest tagged
packages in these tags'. 

We could merge current repos into the newly created one from pungi, but
it would be a vast amount of work. It would mean all repodata would need
to be regenerated. That said, it could be done, but no one has had the
cycles to work on it.

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Tue, Feb 21, 2023 at 05:13:26PM -0500, Matthew Miller wrote:
> On Tue, Feb 21, 2023 at 04:44:31PM -0500, Stephen Smoogen wrote:
> > So do we kill it for:
> > 
> > F39/F40 and beyond?
> > F38 and beyond?
> > X-date and all releases?
> 
> My normal response would be... well, I missed the F38 change deadline by a
> wide margin, so F39+.

Just FYI, we do not produce drpms for rawhide/branched at all (since
2017 ish).

> But, I think we could stop producing deltarpms for current releases without
> really affecting those releases (there would just not be more created, which
> as previously explored, would not really have a strong effect). So... I
> wouldn't leave the other options out of the question. 
> 
> What do infra / releng folks think?
> 
> How easy is it to shut things off?

Its one line in bodhi pungi config:
createrepo_deltas = False

> If shut off, can it be turned back on again (in case of Regrets)?

Just change the one line back to True (well, it's more complex because
we only actually do drpms for the 'Everything' repo, not others, but
it's one line).

> Once shut off, is decomissioning infrastructure around it a Project, or just
> more shutting off?

As far as I can think, nothing else needs to be done. They will just
disappear. 

> What risks are there?

None that leap to mind.

> And... how much would be saved in:
> 
>  * compute resources

Some, but not a lot, as we only delta against the previous update
composes and thus don't do too many.

>  * storage

100's of MB... not much at all.

>  * delays
>  * ongoing maintenance work 
>  * other?

Nothing comes to mind. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Matthew Miller]

On Wed, Feb 22, 2023 at 06:18:23AM -0500, Neal Gompa wrote:
> Please don't try to equate those things to DeltaRPMs, unless you're
> trying to equate their general uselessness. OSTree and

Neal, hyperbole like "general uselessness" is not appropriate in talking
about other people's work. In any case, CoreOS is our fourth-most-common
variant at this point, so clearly some people are finding it useful.

> "container-delta" stuff is not generally useful or applicable for
> Fedora users and they won't be for a very long time.

Maybe, depending on your definition of "very long time". There is work in
the right direction, and supporting that can only help.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Demi Marie Obenour]

On 2/22/23 10:39, Ben Cotton wrote:
> On Tue, Feb 21, 2023 at 3:48 PM Matthew Miller  
> wrote:
>>
>> What we're doing now — as has been the case for several years, already noted
>> in the previous discussion — has very little end-user value. Also as noted
>> in that thread (as in the ticket)... that's unfortunate, because it did
>> bring some real benefits (and could possibly do even more.)
> 
> The fact that the value of deltas requires frequent updates means that
> most people don't get the benefit. And since delta RPMs trade
> bandwidth for CPU, it probably makes things worse for folks in
> developing countries. So I agree, it's probably not worth keeping
> deltas as the default.
> 
>> But, I think it's time to move on. We have ostree and various
>> container-delta approaches. We should focus on those — and give DeltaRPMs a
>> sad, fond farewell.
> 
> Could we do this as a two-step approach? First change the default to
> not use deltas but still allow people to opt-in to it. Then (assuming
> we can track this, which maybe we can't) see how much they're used
> before we decide to pull the plug on producing them.

That would be absolutely awesome.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Ahmed Almeleh]

I agree with the idea that Ben suggested of not enabling deltas by default
and giving users the option until a certain time where it can be phased out
fully.

I remember in my personal experience that delta slowed down update
durations.

On Wed, 22 Feb 2023, 15:40 Ben Cotton,  wrote:

> On Tue, Feb 21, 2023 at 3:48 PM Matthew Miller 
> wrote:
> >
> > What we're doing now — as has been the case for several years, already
> noted
> > in the previous discussion — has very little end-user value. Also as
> noted
> > in that thread (as in the ticket)... that's unfortunate, because it did
> > bring some real benefits (and could possibly do even more.)
>
> The fact that the value of deltas requires frequent updates means that
> most people don't get the benefit. And since delta RPMs trade
> bandwidth for CPU, it probably makes things worse for folks in
> developing countries. So I agree, it's probably not worth keeping
> deltas as the default.
>
> > But, I think it's time to move on. We have ostree and various
> > container-delta approaches. We should focus on those — and give
> DeltaRPMs a
> > sad, fond farewell.
>
> Could we do this as a two-step approach? First change the default to
> not use deltas but still allow people to opt-in to it. Then (assuming
> we can track this, which maybe we can't) see how much they're used
> before we decide to pull the plug on producing them.
>
> --
> Ben Cotton
> He / Him / His
> Fedora Program Manager
> Red Hat
> TZ=America/Indiana/Indianapolis
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Ben Cotton]

On Tue, Feb 21, 2023 at 3:48 PM Matthew Miller  wrote:
>
> What we're doing now — as has been the case for several years, already noted
> in the previous discussion — has very little end-user value. Also as noted
> in that thread (as in the ticket)... that's unfortunate, because it did
> bring some real benefits (and could possibly do even more.)

The fact that the value of deltas requires frequent updates means that
most people don't get the benefit. And since delta RPMs trade
bandwidth for CPU, it probably makes things worse for folks in
developing countries. So I agree, it's probably not worth keeping
deltas as the default.

> But, I think it's time to move on. We have ostree and various
> container-delta approaches. We should focus on those — and give DeltaRPMs a
> sad, fond farewell.

Could we do this as a two-step approach? First change the default to
not use deltas but still allow people to opt-in to it. Then (assuming
we can track this, which maybe we can't) see how much they're used
before we decide to pull the plug on producing them.

--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Luna Jernberg]

Dislike -1

On 2/22/23, Steve Cossette  wrote:
> Yeah I'd say +1 from here too. Was just wondering about this yesterday.
>
> Le mer. 22 févr. 2023, 05 h 52, Kamil Paral  a écrit :
>
>> On Tue, Feb 21, 2023 at 9:48 PM Matthew Miller 
>> wrote:
>>
>>> But, I think it's time to move on. We have ostree and various
>>> container-delta approaches. We should focus on those — and give
>>> DeltaRPMs
>>> a
>>> sad, fond farewell.
>>>
>>
>> +1 from me. It will speed up the compose, and I haven't seen a positive
>> impact from deltarpms in a long time. They are rarely available and the
>> saved bandwidth is usually negligible (and reassembling of the rpm is
>> usually quite slow).
>>
>> ___
>> devel mailing list -- devel@lists.fedoraproject.org
>> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
>> Fedora Code of Conduct:
>> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives:
>> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
>> Do not reply to spam, report it:
>> https://pagure.io/fedora-infrastructure/new_issue
>>
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Steve Cossette]

Yeah I'd say +1 from here too. Was just wondering about this yesterday.

Le mer. 22 févr. 2023, 05 h 52, Kamil Paral  a écrit :

> On Tue, Feb 21, 2023 at 9:48 PM Matthew Miller 
> wrote:
>
>> But, I think it's time to move on. We have ostree and various
>> container-delta approaches. We should focus on those — and give DeltaRPMs
>> a
>> sad, fond farewell.
>>
>
> +1 from me. It will speed up the compose, and I haven't seen a positive
> impact from deltarpms in a long time. They are rarely available and the
> saved bandwidth is usually negligible (and reassembling of the rpm is
> usually quite slow).
>
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
>
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Neal Gompa]

On Tue, Feb 21, 2023 at 3:48 PM Matthew Miller  wrote:
>
> I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
> priority. Last time we talked about this we didn't really get anywhere...
>
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
>
> ... and that ticket hasn't moved, because fixing it isn't trivial.
>
>
> What we're doing now — as has been the case for several years, already noted
> in the previous discussion — has very little end-user value. Also as noted
> in that thread (as in the ticket)... that's unfortunate, because it did
> bring some real benefits (and could possibly do even more.)
>

Our tooling has been broken for a long time and contributions to that
tooling is just not going to happen since nobody can run this stuff
outside of Fedora infrastructure. It's a sad state of affairs indeed.

> But, I think it's time to move on. We have ostree and various
> container-delta approaches. We should focus on those — and give DeltaRPMs a
> sad, fond farewell.
>

Please don't try to equate those things to DeltaRPMs, unless you're
trying to equate their general uselessness. OSTree and
"container-delta" stuff is not generally useful or applicable for
Fedora users and they won't be for a very long time.

And they might never be useful because OSTree's approach requires us
to use OSTree remotes (which we're killing for OCI remotes), and
there's no standard for "container-delta" since the baseline OCI
format isn't amenable to delta fetching.







--
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Neal Gompa]

On Wed, Feb 22, 2023 at 3:37 AM Richard W.M. Jones  wrote:
>
> It's also been a long time since I've seen any benefit.
>
> Can anyone summarise quickly why delta RPMs don't work?  It seems like
> they "obviously" should be possible, because there must be a lot of
> commonality in the content of adjacent RPM versions ...

They don't work because we compose updates wrong. Instead of building
on top of previous updates, we throw them away and rebuild from the
latest. Since we don't merge previous composes into a new compose, we
are missing too much stuff for DeltaRPMs to be continuously useful.


-- 
真実はいつも一つ！/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kamil Paral]

On Tue, Feb 21, 2023 at 9:48 PM Matthew Miller 
wrote:

> But, I think it's time to move on. We have ostree and various
> container-delta approaches. We should focus on those — and give DeltaRPMs a
> sad, fond farewell.
>

+1 from me. It will speed up the compose, and I haven't seen a positive
impact from deltarpms in a long time. They are rarely available and the
saved bandwidth is usually negligible (and reassembling of the rpm is
usually quite slow).
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Gary Buhrmaster]

On Tue, Feb 21, 2023 at 8:48 PM Matthew Miller  wrote:

> What we're doing now — as has been the case for several years, already noted
> in the previous discussion — has very little end-user value.

While occasionally I have seen a small decrease in
the size of the files transferred (which certainly can
benefit some people some of the time), the total
elapsed time of the transaction has always ended
up being higher as the recreation of the original
rpm exceeds the time that it would have taken
me to just download the full new rpm (with an
admittedly reasonably high speed network
provider in my environment).

However, that is an end user experience.  What
about the mirror providers?  Even a small
percentage of bandwidth savings might be
useful for them (depending on their cost model)
at the scale(s) they may be operating.  Has
anyone asked those providers, or do all (or
most) now have a network cost structure
such that it does not matter?
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Richard W.M. Jones]

It's also been a long time since I've seen any benefit.

Can anyone summarise quickly why delta RPMs don't work?  It seems like
they "obviously" should be possible, because there must be a lot of
commonality in the content of adjacent RPM versions ...

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
nbdkit - Flexible, fast NBD server with plugins
https://gitlab.com/nbdkit/nbdkit
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Kofler via devel]

Kevin Fenzi wrote:
> I fear the only way to fix it is to get pungi to merge entire repos, and
> I don't think thats anything that pungi wants to be on the hook for
> doing.

But there are more things than just DeltaRPMs it could be useful for. E.g., 
I remember that in ancient times (pre Core-Extras Merge), some Fedora 
repository (IIRC, the old Fedora Extras) used to ship not only the latest 
build for each package, but the TWO latest builds, so that if the latest 
build was broken, you could easily downgrade to the previous one. That 
should really be done in all the rolling repositories (updates, updates-
testing, Rawhide).

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Demi Marie Obenour]

On 2/21/23 17:13, Matthew Miller wrote:
> On Tue, Feb 21, 2023 at 04:44:31PM -0500, Stephen Smoogen wrote:
>> So do we kill it for:
>>
>> F39/F40 and beyond?
>> F38 and beyond?
>> X-date and all releases?
> 
> My normal response would be... well, I missed the F38 change deadline by a
> wide margin, so F39+.
> 
> But, I think we could stop producing deltarpms for current releases without
> really affecting those releases (there would just not be more created, which
> as previously explored, would not really have a strong effect). So... I
> wouldn't leave the other options out of the question.

Can we also disable deltarpms in the F38 repo files?  It’s a 1-line change,
trivially revertable, and it measurably reduces the attack surface of DNF.
If no deltarpms are being generated, there is no point in DNF looking for
them.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Demi Marie Obenour]

On 2/21/23 16:44, Stephen Smoogen wrote:
> On Tue, 21 Feb 2023 at 15:48, Matthew Miller 
> wrote:
> 
>> I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
>> priority. Last time we talked about this we didn't really get anywhere...
>>
>>
>> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
>>
>> ... and that ticket hasn't moved, because fixing it isn't trivial.
>>
>>
> So do we kill it for:
> 
> F39/F40 and beyond?
> F38 and beyond?
> X-date and all releases?

F38+?  Also maybe disable deltarpms in dnf.conf, to reduce attack surface.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Matthew Miller]

On Tue, Feb 21, 2023 at 04:44:31PM -0500, Stephen Smoogen wrote:
> So do we kill it for:
> 
> F39/F40 and beyond?
> F38 and beyond?
> X-date and all releases?

My normal response would be... well, I missed the F38 change deadline by a
wide margin, so F39+.

But, I think we could stop producing deltarpms for current releases without
really affecting those releases (there would just not be more created, which
as previously explored, would not really have a strong effect). So... I
wouldn't leave the other options out of the question. 

What do infra / releng folks think?

How easy is it to shut things off?

If shut off, can it be turned back on again (in case of Regrets)?

Once shut off, is decomissioning infrastructure around it a Project, or just
more shutting off?

What risks are there?

And... how much would be saved in:

 * compute resources
 * storage
 * delays
 * ongoing maintenance work 
 * other?




-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin Fenzi]

On Tue, Feb 21, 2023 at 03:48:06PM -0500, Matthew Miller wrote:
> I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
> priority. Last time we talked about this we didn't really get anywhere...
> 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
> 
> ... and that ticket hasn't moved, because fixing it isn't trivial.

Yeah. ;( 

I fear the only way to fix it is to get pungi to merge entire repos, and
I don't think thats anything that pungi wants to be on the hook for
doing. 

> What we're doing now — as has been the case for several years, already noted
> in the previous discussion — has very little end-user value. Also as noted
> in that thread (as in the ticket)... that's unfortunate, because it did
> bring some real benefits (and could possibly do even more.)
> 
> But, I think it's time to move on. We have ostree and various
> container-delta approaches. We should focus on those — and give DeltaRPMs a
> sad, fond farewell.

yeah, I agree. 

kevin


signature.asc
Description: PGP signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Stephen Smoogen]

On Tue, 21 Feb 2023 at 15:48, Matthew Miller 
wrote:

> I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
> priority. Last time we talked about this we didn't really get anywhere...
>
>
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
>
> ... and that ticket hasn't moved, because fixing it isn't trivial.
>
>
So do we kill it for:

F39/F40 and beyond?
F38 and beyond?
X-date and all releases?


>
> What we're doing now — as has been the case for several years, already
> noted
> in the previous discussion — has very little end-user value. Also as noted
> in that thread (as in the ticket)... that's unfortunate, because it did
> bring some real benefits (and could possibly do even more.)
>
> But, I think it's time to move on. We have ostree and various
> container-delta approaches. We should focus on those — and give DeltaRPMs a
> sad, fond farewell.
>
> --
> Matthew Miller
> 
> Fedora Project Leader
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue
>


-- 
Stephen Smoogen, Red Hat Automotive
Let us be kind to one another, for most of us are fighting a hard battle.
-- Ian MacClaren
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Leslie Satenstein via devel]

Got it. 


Leslie Satenstein
 

On Tuesday, February 21, 2023 at 03:48:23 p.m. EST, Matthew Miller 
 wrote:  
 
 I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
priority. Last time we talked about this we didn't really get anywhere...

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E

... and that ticket hasn't moved, because fixing it isn't trivial.


What we're doing now — as has been the case for several years, already noted
in the previous discussion — has very little end-user value. Also as noted
in that thread (as in the ticket)... that's unfortunate, because it did
bring some real benefits (and could possibly do even more.)

But, I think it's time to move on. We have ostree and various
container-delta approaches. We should focus on those — and give DeltaRPMs a
sad, fond farewell.

-- 
Matthew Miller

Fedora Project Leader
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue
  ___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Kevin P. Fleming]

On 2/21/23 15:53, Fabio Valentini wrote:

It's been a long, long, time since any dnf transaction I ran printed
positive news about deltarpms ... most of the time there either aren't
any, or using them actually increased data download.
A recent transaction went so poorly that it prompted me to disable 
deltarpms completely on my system.


--
Kevin P. Fleming
He/Him/His
Principal Program Manager, RHEL
Red Hat US/Eastern Time Zone
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue

Re: Proposal: drop delta rpms (for real this time)

[Fabio Valentini]

On Tue, Feb 21, 2023 at 9:48 PM Matthew Miller  wrote:
>
> I was asked to weigh in on https://pagure.io/releng/issue/7215 as a
> priority. Last time we talked about this we didn't really get anywhere...
>
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/JYKVELSBJQMEK6KEFXG354ZDZDDX4C5G/#RLEUYSWOUVUS53YAP7WQQNN7HNEBIC4E
>
> ... and that ticket hasn't moved, because fixing it isn't trivial.
>
>
> What we're doing now — as has been the case for several years, already noted
> in the previous discussion — has very little end-user value. Also as noted
> in that thread (as in the ticket)... that's unfortunate, because it did
> bring some real benefits (and could possibly do even more.)
>
> But, I think it's time to move on. We have ostree and various
> container-delta approaches. We should focus on those — and give DeltaRPMs a
> sad, fond farewell.

It's been a long, long, time since any dnf transaction I ran printed
positive news about deltarpms ... most of the time there either aren't
any, or using them actually increased data download.

So, I agree. It's time to pull the plug.

Fabio
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: 
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Do not reply to spam, report it: 
https://pagure.io/fedora-infrastructure/new_issue