Fedora Rawhide-20181117.n.0 compose check report

2018-11-17 Thread Fedora compose checker
No missing expected images.

Failed openQA tests: 14/142 (x86_64), 2/24 (i386), 1/2 (arm)

New failures (same test did not fail in Rawhide-20181114.n.0):

ID: 309623  Test: x86_64 Server-dvd-iso realmd_join_sssd
URL: https://openqa.fedoraproject.org/tests/309623
ID: 309630  Test: x86_64 Server-dvd-iso server_freeipa_replication_client
URL: https://openqa.fedoraproject.org/tests/309630
ID: 309702  Test: x86_64 universal install_blivet_btrfs@uefi
URL: https://openqa.fedoraproject.org/tests/309702
ID: 309736  Test: x86_64 universal install_rescue_encrypted@uefi
URL: https://openqa.fedoraproject.org/tests/309736
ID: 309768  Test: i386 universal install_blivet_lvmthin
URL: https://openqa.fedoraproject.org/tests/309768

Old failures (same test failed in Rawhide-20181114.n.0):

ID: 309608  Test: x86_64 Server-dvd-iso modularity_tests
URL: https://openqa.fedoraproject.org/tests/309608
ID: 309646  Test: x86_64 Workstation-live-iso 
desktop_notifications_postinstall
URL: https://openqa.fedoraproject.org/tests/309646
ID: 309649  Test: x86_64 Workstation-boot-iso memory_check
URL: https://openqa.fedoraproject.org/tests/309649
ID: 309650  Test: x86_64 Workstation-boot-iso memory_check@uefi
URL: https://openqa.fedoraproject.org/tests/309650
ID: 309653  Test: i386 Workstation-boot-iso memory_check
URL: https://openqa.fedoraproject.org/tests/309653
ID: 309669  Test: arm Minimal-raw_xz-raw.xz 
install_arm_image_deployment_upload
URL: https://openqa.fedoraproject.org/tests/309669
ID: 309674  Test: x86_64 Silverblue-dvd_ostree-iso install_default@uefi
URL: https://openqa.fedoraproject.org/tests/309674
ID: 309690  Test: x86_64 universal install_cyrillic_language
URL: https://openqa.fedoraproject.org/tests/309690
ID: 309691  Test: x86_64 universal install_arabic_language
URL: https://openqa.fedoraproject.org/tests/309691
ID: 309741  Test: x86_64 universal upgrade_server_64bit
URL: https://openqa.fedoraproject.org/tests/309741
ID: 309742  Test: x86_64 universal upgrade_server_domain_controller
URL: https://openqa.fedoraproject.org/tests/309742
ID: 309778  Test: x86_64 Silverblue-dvd_ostree-iso install_default_upload
URL: https://openqa.fedoraproject.org/tests/309778

Soft failed openQA tests: 5/142 (x86_64), 4/24 (i386)
(Tests completed, but using a workaround for a known bug)

New soft failures (same test did not soft fail in Rawhide-20181114.n.0):

ID: 309631  Test: i386 Server-boot-iso install_default
URL: https://openqa.fedoraproject.org/tests/309631
ID: 309637  Test: x86_64 Workstation-live-iso install_default_upload
URL: https://openqa.fedoraproject.org/tests/309637
ID: 309721  Test: x86_64 universal upgrade_desktop_64bit
URL: https://openqa.fedoraproject.org/tests/309721
ID: 309729  Test: x86_64 universal upgrade_desktop_encrypted_64bit
URL: https://openqa.fedoraproject.org/tests/309729
ID: 309769  Test: i386 universal upgrade_desktop_32bit
URL: https://openqa.fedoraproject.org/tests/309769

Old soft failures (same test soft failed in Rawhide-20181114.n.0):

ID: 309632  Test: i386 Server-dvd-iso install_default
URL: https://openqa.fedoraproject.org/tests/309632
ID: 309704  Test: x86_64 universal upgrade_2_desktop_64bit
URL: https://openqa.fedoraproject.org/tests/309704
ID: 309707  Test: x86_64 universal upgrade_2_desktop_encrypted_64bit
URL: https://openqa.fedoraproject.org/tests/309707
ID: 309770  Test: i386 universal upgrade_2_desktop_32bit
URL: https://openqa.fedoraproject.org/tests/309770

Passed openQA tests: 116/142 (x86_64), 18/24 (i386)

New passes (same test did not pass in Rawhide-20181114.n.0):

ID: 309604  Test: x86_64 Server-boot-iso install_default
URL: https://openqa.fedoraproject.org/tests/309604
ID: 309605  Test: x86_64 Server-boot-iso install_default@uefi
URL: https://openqa.fedoraproject.org/tests/309605
ID: 309613  Test: x86_64 Server-dvd-iso base_services_start
URL: https://openqa.fedoraproject.org/tests/309613
ID: 309633  Test: x86_64 Everything-boot-iso install_default
URL: https://openqa.fedoraproject.org/tests/309633
ID: 309634  Test: x86_64 Everything-boot-iso install_default@uefi
URL: https://openqa.fedoraproject.org/tests/309634
ID: 309635  Test: i386 Everything-boot-iso install_default
URL: https://openqa.fedoraproject.org/tests/309635
ID: 309638  Test: x86_64 Workstation-live-iso install_default@uefi
URL: https://openqa.fedoraproject.org/tests/309638
ID: 309651  Test: x86_64 Workstation-boot-iso install_default@uefi
URL: https://openqa.fedoraproject.org/tests/309651
ID: 309663  Test: x86_64 KDE-live-iso desktop_notifications_postinstall
URL: https://openqa.fedoraproject.org/tests/309663
ID: 309693  Test: x86_64 universal install_kickstart_firewall_disabled
URL: https://openqa.fedoraproject.org/tests/309693
ID: 309695  Test: x86_64 universal install_kickstart_user_creation
URL: https://openqa.fedoraproject.org/tests/309695
ID: 309697  Test: x8

Re: Proposal: Faster composes by eliminating deltarpms and using zchunked rpms instead

2018-11-17 Thread Leigh Scott
+1 to anything to rid me of deltarpms, I currently have to disable this lame 
default.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Proposal: Faster composes by eliminating deltarpms and using zchunked rpms instead

2018-11-17 Thread Jonathan Dieter
On Sat, 2018-11-17 at 14:43 -0600, Chris Adams wrote:
> Once upon a time, Jonathan Dieter  said:
> > The benefit of zchunked rpms is that, when downloading an updated rpm,
> > you would only need to download the chunks that have changed from
> > what's on your system.
> 
> How well do web servers and caches handle range requests?  I haven't
> really paid attention to range requests in a long time; at one point
> IIRC mirrors would often disable them because of "download accelerators"
> that would open multiple connections to download parts of the same ISO
> in parallel (hogging server resources).

When I did the original POC testing, out of Fedora's 150 mirrors, 3
didn't support range requests at all and 3 supported a limited number
of ranges in a single http request.

Zchunk doesn't open extra connections to the server, but instead
combines as many ranges as the server supports into a single request. 
Currently, if a server doesn't support ranges, zchunk will just
download the full file, but this could be changed to try a different
server.

Jonathan
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Automating package maintainers responsivity check

2018-11-17 Thread Philip Kovacs
 There already is a fedora_active_user script of sorts 
https://github.com/pypingou/fedora-active-user.
I would not be in favor of any respond or die automation.   We volunteer our 
time and effort to be packagersand the job is often thankless enough as it is.  
Having some additional automation orphan your packagesbecause you happen to be 
away for a bit is overkill.   You want to attract packagers, not irritate them.
On Saturday, November 17, 2018, 11:58:27 AM EST, Mattia Verga 
 wrote:  
 
 Il 11/17/18 4:50 PM, Ron Olson ha scritto:
> What about packages that see infrequent updates; I maintain Nethack 
> and the Dev Team can and does take years between releases. If it's 
> just a blanket email to ask the packagers if they're still interested 
> that's one thing, but going off package updates may be problematic for 
> some folks.
>
>
I'm thinking about a script which would run every month and:

- checks maintainer activities in git / bugzilla / mailing lists 
(something like fedora-active-user already does) in the last six months
- if the user hasn't done any activity, send an email with a link
   - if the user visit the link, do not bother them for the next 6 months
   - if the user doesn't visit the link, send a second email after one 
week and a third after another week
- after three emails without response, orphan their packages and inform 
devel list

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
  ___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Proposal: Faster composes by eliminating deltarpms and using zchunked rpms instead

2018-11-17 Thread Jonathan Dieter
On Sat, 2018-11-17 at 14:36 -0500, Neal Gompa wrote:
> On Sat, Nov 17, 2018 at 1:15 PM Jonathan Dieter  wrote:
> > Neal, thanks so much for your thoughts on this.  Responses inline:
> > 
> > On Sat, 2018-11-17 at 09:53 -0500, Neal Gompa wrote:
> > 
> > > If we're really considering changing the RPM file format, then we need
> > > a proper discussion on rpm-maint@ and rpm-ecosystem@ mailing lists on
> > > rpm.org. Can you please start a targeted discussion there?
> > 
> > Sure.
> > 
> > > But addressing the specific concrete suggestion here, there's a few
> > > concerns I have:
> > > 
> > > 1. This is a huge format break, which means that for the first time in
> > > a _very_ long time, it would not be possible to reuse RHEL for Fedora
> > > infrastructure _at all_. That's going to be a difficult problem.
> > > There's a large legacy of systems that won't be able to handle that
> > > new format, and unfortunately, rpm is not parallel installable in the
> > > same manner as something like GCC or Python currently. Making it
> > > parallel installable *is* possible (I've done it, and there have been
> > > other attempts before), but it's not a supported thing. This is
> > > probably the thing that would trigger a major version bump for RPM,
> > > since it's a new archive format.
> > 
> > Agreed, that this would be a massive format change and should therefore
> > be a major version bump for RPM.  New versions of RPM should still be
> > able to read and install old-format rpms, but, as you point out, old
> > versions of RPM won't be able to read or install new-format rpms.
> > Unfortunately, I don't see any way around this.
> > 
> 
> I don't think there's a way around it either. I just hope we do better
> than the last time someone tried to do this...

+1

> > > 2. This also means the _entire_ ecosystem of RPM archive parsers will
> > > break. This is not particularly insurmountable, actually, as the RPM
> > > file format was not particularly well documented, and a new format is
> > > an opportunity to revisit some of those old issues and try to do a
> > > better job this go around. But it's still a challenge to deal with.
> > 
> > Yes, this is going to be quite a bit of work.
> > 
> > > 3. When you refer to the rpm cpio, I assume you're referring to only
> > > the archive payload, right? Typically the payload is what is
> > > compressed, and the headers are not. It sounds like you're proposing
> > > both aspects to be compressed, and compressed differently. If we made
> > > the RPM header an uncompressed zchunk stream and the RPM payload a
> > > zstd-compressed zchunk stream, would we be able to support fetching
> > > header deltas for retrieving extra information on the fly? Say, for
> > > example, attributes like arch color, filecap properties, and so on,
> > > that aren't in the rpm-md data for things like transaction tests
> > > without the whole RPM?
> > 
> > Yes, I'm referring the the archive payload as the cpio.  The zchunk
> > format supports the idea of separate data streams, and I was planning
> > to use that to put the headers in one stream and the archive payload in
> > another.  If the header chunks are first in the zchunk file, then they
> > could be read without needing to read any of the rest of the file.
> > And, yes, we could make the header stream uncompressed if that made it
> > easier to parse.
> > 
> 
> Whether it's compressed or not isn't terribly important, but what is
> important is being able to validate the correctness before beginning
> any processing, including decompression.

Absolutely!  This includes both the rpm header and the rpm archive
data, and that's why we store both the compressed and uncompressed
checksums of the chunks.

> > > 4. I'd actually rather make it easier for the header streams to be
> > > fetched instead of trying to make specific attributes easier in the
> > > header payload. History has shown that any attempt at foresight here
> > > tends to fail miserably, and common attributes are already specified
> > > in the rpm-md primary.xml anyway, so if you're fetching the header to
> > > retrieve an attribute, you *need* to do something weird anyway.
> > 
> > The main purpose of putting separate attributes in the zchunk header is
> > so programs like 'file' can determine some basic information about an
> > rpm without needing to parse the full rpm header.  This data would also
> > be in the rpm header, so programs that read the rpm header wouldn't
> > care about the attributes in the zchunk header.
> > 
> 
> I see, so some simple hints for stuff like that? But that would still
> require awareness of the format to some degree. I guess we'd have a
> specific lead magic to let tools know to look for them...

Yeah, the code would be maybe a hundred lines, max, that could be
copylib'd into file, etc.

> > > 5. I'm not exactly sure what you mean by zchunk signing...
> > 
> > The zchunk format supports signing, but just for the zchunk header.
> > Because the header contain

Re: Proposal: Faster composes by eliminating deltarpms and using zchunked rpms instead

2018-11-17 Thread Kevin Kofler
Jonathan Dieter wrote:
> My proposal would be to make zchunk the rpm compression format for
> Fedora.

Given that:
1. zchunk is based on zstd, which is typically less efficient in terms of
   compression ratio than xz, depending on settings
   (see, e.g., https://github.com/inikep/lzbench), and
2. zchunk can by design only compress chunks individually and not benefit
   from the space savings of a solid archive with a global dictionary,
I fear that this is going to significantly increase the size of the RPMs, 
which matters:
* for the initial downloads,
* for storage (e.g., keepcache=1, local mirrors, etc.), and
* for the people not using deltas for whatever reason.

I think zchunk makes a lot of sense for the metadata, but I am not convinced 
that it is the right choice for the RPMs themselves.

Kevin Kofler
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Proposal: Faster composes by eliminating deltarpms and using zchunked rpms instead

2018-11-17 Thread Chris Adams
Once upon a time, Jonathan Dieter  said:
> The benefit of zchunked rpms is that, when downloading an updated rpm,
> you would only need to download the chunks that have changed from
> what's on your system.

How well do web servers and caches handle range requests?  I haven't
really paid attention to range requests in a long time; at one point
IIRC mirrors would often disable them because of "download accelerators"
that would open multiple connections to download parts of the same ISO
in parallel (hogging server resources).
-- 
Chris Adams 
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Proposal: Faster composes by eliminating deltarpms and using zchunked rpms instead

2018-11-17 Thread Neal Gompa
On Sat, Nov 17, 2018 at 1:15 PM Jonathan Dieter  wrote:
>
> Neal, thanks so much for your thoughts on this.  Responses inline:
>
> On Sat, 2018-11-17 at 09:53 -0500, Neal Gompa wrote:
> 
> > If we're really considering changing the RPM file format, then we need
> > a proper discussion on rpm-maint@ and rpm-ecosystem@ mailing lists on
> > rpm.org. Can you please start a targeted discussion there?
>
> Sure.
>
> > But addressing the specific concrete suggestion here, there's a few
> > concerns I have:
> >
> > 1. This is a huge format break, which means that for the first time in
> > a _very_ long time, it would not be possible to reuse RHEL for Fedora
> > infrastructure _at all_. That's going to be a difficult problem.
> > There's a large legacy of systems that won't be able to handle that
> > new format, and unfortunately, rpm is not parallel installable in the
> > same manner as something like GCC or Python currently. Making it
> > parallel installable *is* possible (I've done it, and there have been
> > other attempts before), but it's not a supported thing. This is
> > probably the thing that would trigger a major version bump for RPM,
> > since it's a new archive format.
>
> Agreed, that this would be a massive format change and should therefore
> be a major version bump for RPM.  New versions of RPM should still be
> able to read and install old-format rpms, but, as you point out, old
> versions of RPM won't be able to read or install new-format rpms.
> Unfortunately, I don't see any way around this.
>

I don't think there's a way around it either. I just hope we do better
than the last time someone tried to do this...

> > 2. This also means the _entire_ ecosystem of RPM archive parsers will
> > break. This is not particularly insurmountable, actually, as the RPM
> > file format was not particularly well documented, and a new format is
> > an opportunity to revisit some of those old issues and try to do a
> > better job this go around. But it's still a challenge to deal with.
>
> Yes, this is going to be quite a bit of work.
>
> > 3. When you refer to the rpm cpio, I assume you're referring to only
> > the archive payload, right? Typically the payload is what is
> > compressed, and the headers are not. It sounds like you're proposing
> > both aspects to be compressed, and compressed differently. If we made
> > the RPM header an uncompressed zchunk stream and the RPM payload a
> > zstd-compressed zchunk stream, would we be able to support fetching
> > header deltas for retrieving extra information on the fly? Say, for
> > example, attributes like arch color, filecap properties, and so on,
> > that aren't in the rpm-md data for things like transaction tests
> > without the whole RPM?
>
> Yes, I'm referring the the archive payload as the cpio.  The zchunk
> format supports the idea of separate data streams, and I was planning
> to use that to put the headers in one stream and the archive payload in
> another.  If the header chunks are first in the zchunk file, then they
> could be read without needing to read any of the rest of the file.
> And, yes, we could make the header stream uncompressed if that made it
> easier to parse.
>

Whether it's compressed or not isn't terribly important, but what is
important is being able to validate the correctness before beginning
any processing, including decompression.

> > 4. I'd actually rather make it easier for the header streams to be
> > fetched instead of trying to make specific attributes easier in the
> > header payload. History has shown that any attempt at foresight here
> > tends to fail miserably, and common attributes are already specified
> > in the rpm-md primary.xml anyway, so if you're fetching the header to
> > retrieve an attribute, you *need* to do something weird anyway.
>
> The main purpose of putting separate attributes in the zchunk header is
> so programs like 'file' can determine some basic information about an
> rpm without needing to parse the full rpm header.  This data would also
> be in the rpm header, so programs that read the rpm header wouldn't
> care about the attributes in the zchunk header.
>

I see, so some simple hints for stuff like that? But that would still
require awareness of the format to some degree. I guess we'd have a
specific lead magic to let tools know to look for them...

> > 5. I'm not exactly sure what you mean by zchunk signing...
>
> The zchunk format supports signing, but just for the zchunk header.
> Because the header contains the checksums for each chunk, this
> establishes a chain of trust for verifying the whole file.  Which
> brings me to...
>
> > 6. I'm wondering why we can't do a perfect reconstruction of the
> > original RPM, given two RPM sources that are both zchunked? We can
> > pull it off with repodata, so what's different about RPM that makes
> > that not doable?
>
> The problem is that, unlike the repodata, once an rpm is installed, the
> package file is deleted and the data is only available on

Re: Automating package maintainers responsivity check

2018-11-17 Thread Leigh Scott

> Maybe a tool that checks a packager activity in the last 6 months and if 
> What do you think?

Better make it a year as people are entitled to a break!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Proposal: Faster composes by eliminating deltarpms and using zchunked rpms instead

2018-11-17 Thread Jonathan Dieter
Neal, thanks so much for your thoughts on this.  Responses inline:

On Sat, 2018-11-17 at 09:53 -0500, Neal Gompa wrote:

> If we're really considering changing the RPM file format, then we need
> a proper discussion on rpm-maint@ and rpm-ecosystem@ mailing lists on
> rpm.org. Can you please start a targeted discussion there?

Sure.

> But addressing the specific concrete suggestion here, there's a few
> concerns I have:
> 
> 1. This is a huge format break, which means that for the first time in
> a _very_ long time, it would not be possible to reuse RHEL for Fedora
> infrastructure _at all_. That's going to be a difficult problem.
> There's a large legacy of systems that won't be able to handle that
> new format, and unfortunately, rpm is not parallel installable in the
> same manner as something like GCC or Python currently. Making it
> parallel installable *is* possible (I've done it, and there have been
> other attempts before), but it's not a supported thing. This is
> probably the thing that would trigger a major version bump for RPM,
> since it's a new archive format.

Agreed, that this would be a massive format change and should therefore
be a major version bump for RPM.  New versions of RPM should still be
able to read and install old-format rpms, but, as you point out, old
versions of RPM won't be able to read or install new-format rpms. 
Unfortunately, I don't see any way around this.

> 2. This also means the _entire_ ecosystem of RPM archive parsers will
> break. This is not particularly insurmountable, actually, as the RPM
> file format was not particularly well documented, and a new format is
> an opportunity to revisit some of those old issues and try to do a
> better job this go around. But it's still a challenge to deal with.

Yes, this is going to be quite a bit of work.

> 3. When you refer to the rpm cpio, I assume you're referring to only
> the archive payload, right? Typically the payload is what is
> compressed, and the headers are not. It sounds like you're proposing
> both aspects to be compressed, and compressed differently. If we made
> the RPM header an uncompressed zchunk stream and the RPM payload a
> zstd-compressed zchunk stream, would we be able to support fetching
> header deltas for retrieving extra information on the fly? Say, for
> example, attributes like arch color, filecap properties, and so on,
> that aren't in the rpm-md data for things like transaction tests
> without the whole RPM?

Yes, I'm referring the the archive payload as the cpio.  The zchunk
format supports the idea of separate data streams, and I was planning
to use that to put the headers in one stream and the archive payload in
another.  If the header chunks are first in the zchunk file, then they
could be read without needing to read any of the rest of the file. 
And, yes, we could make the header stream uncompressed if that made it
easier to parse.

> 4. I'd actually rather make it easier for the header streams to be
> fetched instead of trying to make specific attributes easier in the
> header payload. History has shown that any attempt at foresight here
> tends to fail miserably, and common attributes are already specified
> in the rpm-md primary.xml anyway, so if you're fetching the header to
> retrieve an attribute, you *need* to do something weird anyway.

The main purpose of putting separate attributes in the zchunk header is
so programs like 'file' can determine some basic information about an
rpm without needing to parse the full rpm header.  This data would also
be in the rpm header, so programs that read the rpm header wouldn't
care about the attributes in the zchunk header.

> 5. I'm not exactly sure what you mean by zchunk signing...

The zchunk format supports signing, but just for the zchunk header. 
Because the header contains the checksums for each chunk, this
establishes a chain of trust for verifying the whole file.  Which
brings me to...

> 6. I'm wondering why we can't do a perfect reconstruction of the
> original RPM, given two RPM sources that are both zchunked? We can
> pull it off with repodata, so what's different about RPM that makes
> that not doable?

The problem is that, unlike the repodata, once an rpm is installed, the
package file is deleted and the data is only available on the system in
its uncompressed installed form.  If we're trying to use that data to
rebuild an rpm, we have two options.

   1. Compress the data using the same method that was used to create the
  original rpm.  This is what applydeltarpm does, and is why it's so
  heavy on the CPU.
   2. Store the data uncompressed in the rebuilt rpm.  This isn't feasible
  with deltarpm, but, if we store both compressed hashes and
  uncompressed hashes in the zchunk header, we can do this in zchunk. 
  When running checking the signature, zchunk verifies the header
  against the signature first, and then checks each chunk to see if it
  passes *either* the compressed or uncompressed 

Re: Automating package maintainers responsivity check

2018-11-17 Thread Mattia Verga
Il 11/17/18 4:50 PM, Ron Olson ha scritto:
> What about packages that see infrequent updates; I maintain Nethack 
> and the Dev Team can and does take years between releases. If it's 
> just a blanket email to ask the packagers if they're still interested 
> that's one thing, but going off package updates may be problematic for 
> some folks.
>
>
I'm thinking about a script which would run every month and:

- checks maintainer activities in git / bugzilla / mailing lists 
(something like fedora-active-user already does) in the last six months
- if the user hasn't done any activity, send an email with a link
   - if the user visit the link, do not bother them for the next 6 months
   - if the user doesn't visit the link, send a second email after one 
week and a third after another week
- after three emails without response, orphan their packages and inform 
devel list

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Fedora Lifecycles: imagine longer-term possibilities

2018-11-17 Thread Gerald Henriksen
On Tue, 13 Nov 2018 18:36:38 -0500, you wrote:

>But there are some good cases for a longer lifecycle. For one thing,
>this has been a really big blocker for getting Fedora shipped on
>hardware. 

In a later message you also bring up the reluctance of Universities to
use Fedora - a bad sign given that is a good source of future
developers/packagers/testers/users.

Which thus results in perhaps another issue to consider, release
dates.

What is the best time for these manufacturers to have a release?

It has been mentioned that they would likely need a release for 6
months before releasing hardware with it, and that would likely be
similar to any organization looking to roll it out onto desktops.

So should Fedora, if it goes down this path, look at having (September
-6) as a goal, to allow for the start of the school year and then the
approaching Christmas shopping season?  Or maybe (August - 6) to allow
for back to school?

A need for an early March / late February release is perhaps just as
important a question given the changes that would require in release
planning as whether to do a LTS style release.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Fedora Lifecycles: imagine longer-term possibilities

2018-11-17 Thread Gerald Henriksen
On Fri, 16 Nov 2018 17:58:36 +0100, you wrote:

>Gerald Henriksen wrote:
>> I think the problem is that for a consumer / desktop oriented product
>> - which we seem to be talking about given that this appears to be
>> driven in part by the desire of hardware vendors - the RHEL/CentOS
>> release cycle leads to problems for several years worth of hardware
>> releases.
>> 
>> If you are a hardware vendor would you want to be releasing a laptop
>> running CentOS 7.x today, with its outdated versions of much of the
>> software (that is a value to the enterprise server market, but less so
>> consumer)?
>
>But the same goes for ANY LTS distribution!
>
>No distro is going to do LTS releases with 37 month support every 6 months. 
>It would mean supporting 7 releases at once!

Of course not, which is one of the reasons why moving to a 12 month
release cycle may be a better idea, with a sort of LTS release maybe
every 2 years.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Upgraded from F28 x96_64 XFCE Spin to F29 => rough audio

2018-11-17 Thread Samuel Sieb

On 11/17/18 1:33 AM, Philip Rhoades wrote:

On 2018-11-17 19:46, Samuel Sieb wrote:

On 11/17/18 12:17 AM, Philip Rhoades wrote:
My problem is not crackling_or_skipping - it is more just noisy.  
That link is a fix for a PA problem and is not specific to the 
F28->F29 upgrade which has caused the current problem.  I don't want 
to have to re-install PA now - to my way of thinking, that just adds 
a layer of complexity to the problem.


Does booting an F28 kernel fix the audio?  If so, then file a kernel bug.


Ah, good thinking - I will boot on the F28 LiveUSB . . and report back.


I was actually meaning one of the previous kernel versions from before 
the upgrade, but a live USB is a good test as well.

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Automating package maintainers responsivity check

2018-11-17 Thread Ron Olson
What about packages that see infrequent updates; I maintain Nethack and the Dev 
Team can and does take years between releases. If it's just a blanket email to 
ask the packagers if they're still interested that's one thing, but going off 
package updates may be problematic for some folks.


> On Nov 17, 2018, at 6:36 AM, Manas Mangaonkar  
> wrote:
> 
> Yes, this could be a good idea. Might be too small as a gsoc project though. 
> 
> I'd happily take this as i am anyways going to try for gsoc 19 with 
> fedora,this could be a good starting point. 
> 
> A simple dockerized/contenerized python based app could be done as a very 
> simple implementation.
> 
> 
> 
> On Nov 17, 2018 4:35 PM, "Mattia Verga"  wrote:
> Too often there are "non responsive maintainer" messages here.
> 
> Sometimes this is because a maintainer has lost their interest in Fedora 
> and left their responsibilities without communication, which means their 
> packages can be left unmaintained for long time in repositories before 
> someone notice that.
> 
> At other times the maintainer is still active, but for some reason they 
> are unreachable to users (email change, etc.).
> 
> I would propose some sort of automatic check of maintainer responsivity. 
> Maybe a tool that checks a packager activity in the last 6 months and if 
> there is no activity then sends an email asking the maintainer to 
> confirm they're still involved in Fedora. In case there's no reply, 
> either automatically orphan their packages or notify someone 
> (devel-list) to try to reach them.
> 
> What do you think?
> 
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
> 
> ___
> devel mailing list -- devel@lists.fedoraproject.org
> To unsubscribe send an email to devel-le...@lists.fedoraproject.org
> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: 
> https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Proposal: Faster composes by eliminating deltarpms and using zchunked rpms instead

2018-11-17 Thread Neal Gompa
On Fri, Nov 16, 2018 at 6:03 PM Jonathan Dieter  wrote:
>
>
> *Changes*
> The zchunk format would need to be extended to allow for a zchunked rpm
> to contain both the uncompressed chunks that were already on the local
> system and the newly downloaded compressed chunks while still passing
> signature verification.  This would also require moving signature
> verification to zchunk.
>
> The rpm file format has to be changed because the zchunk header needs
> to be at the beginning of the file in order for the zchunk library
> figure out which chunks it needs to download.  My suggestions for
> changes to the rpm file format are as follows:
>
>1. Signing should be moved to the zchunk format as described at the
>   beginning of this section
>2. The rpm header should be stored in one stream inside the zchunk
>   file.  This allows it to be easily extracted separately from the
>   data
>3. The rpm cpio should be stored in a second stream inside the zchunk
>   file.
>4. At minimum, an optional zchunk element should be set to identify
>   zchunk rpms as rpms rather than regular zchunk files.  If desired,
>   optional elements could also be set containing %{name}, %[version},
>   %{release}, %{arch} and %{epoch}.  This would allow this information
>   to be read easily without needing to extract the rpm header stream.
>
> *Final notes*
> I realize this is a massive proposal, zchunk is still very young, and
> we're still working on getting the dnf zchunk pull requests reviewed.
> I do think it's feasible and provides an opportunity to eliminate a
> pain point from our compose process while still reducing the download
> size for our users.
>

If we're really considering changing the RPM file format, then we need
a proper discussion on rpm-maint@ and rpm-ecosystem@ mailing lists on
rpm.org. Can you please start a targeted discussion there?

But addressing the specific concrete suggestion here, there's a few
concerns I have:

1. This is a huge format break, which means that for the first time in
a _very_ long time, it would not be possible to reuse RHEL for Fedora
infrastructure _at all_. That's going to be a difficult problem.
There's a large legacy of systems that won't be able to handle that
new format, and unfortunately, rpm is not parallel installable in the
same manner as something like GCC or Python currently. Making it
parallel installable *is* possible (I've done it, and there have been
other attempts before), but it's not a supported thing. This is
probably the thing that would trigger a major version bump for RPM,
since it's a new archive format.

2. This also means the _entire_ ecosystem of RPM archive parsers will
break. This is not particularly insurmountable, actually, as the RPM
file format was not particularly well documented, and a new format is
an opportunity to revisit some of those old issues and try to do a
better job this go around. But it's still a challenge to deal with.

3. When you refer to the rpm cpio, I assume you're referring to only
the archive payload, right? Typically the payload is what is
compressed, and the headers are not. It sounds like you're proposing
both aspects to be compressed, and compressed differently. If we made
the RPM header an uncompressed zchunk stream and the RPM payload a
zstd-compressed zchunk stream, would we be able to support fetching
header deltas for retrieving extra information on the fly? Say, for
example, attributes like arch color, filecap properties, and so on,
that aren't in the rpm-md data for things like transaction tests
without the whole RPM?

4. I'd actually rather make it easier for the header streams to be
fetched instead of trying to make specific attributes easier in the
header payload. History has shown that any attempt at foresight here
tends to fail miserably, and common attributes are already specified
in the rpm-md primary.xml anyway, so if you're fetching the header to
retrieve an attribute, you *need* to do something weird anyway.

5. I'm not exactly sure what you mean by zchunk signing...

6. I'm wondering why we can't do a perfect reconstruction of the
original RPM, given two RPM sources that are both zchunked? We can
pull it off with repodata, so what's different about RPM that makes
that not doable?


-- 
真実はいつも一つ!/ Always, there's only one truth!
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Automating package maintainers responsivity check

2018-11-17 Thread Manas Mangaonkar
Yes, this could be a good idea. Might be too small as a gsoc project
though.

I'd happily take this as i am anyways going to try for gsoc 19 with
fedora,this could be a good starting point.

A simple dockerized/contenerized python based app could be done as a very
simple implementation.



On Nov 17, 2018 4:35 PM, "Mattia Verga"  wrote:

Too often there are "non responsive maintainer" messages here.

Sometimes this is because a maintainer has lost their interest in Fedora
and left their responsibilities without communication, which means their
packages can be left unmaintained for long time in repositories before
someone notice that.

At other times the maintainer is still active, but for some reason they
are unreachable to users (email change, etc.).

I would propose some sort of automatic check of maintainer responsivity.
Maybe a tool that checks a packager activity in the last 6 months and if
there is no activity then sends an email asking the maintainer to
confirm they're still involved in Fedora. In case there's no reply,
either automatically orphan their packages or notify someone
(devel-list) to try to reach them.

What do you think?

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Call for libtimidity owners

2018-11-17 Thread Antonio Trande
Regarding https://src.fedoraproject.org/rpms/libtimidity/pull-requests,
please i need permissions to build libtimidity on epel7 branch.

On 15/11/18 13:22, Antonio Trande wrote:
> Hello!
> 
> Please, take a look to
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1630466
> https://src.fedoraproject.org/rpms/libtimidity/pull-requests
> 

-- 
---
Antonio Trande
Fedora Project
mailto 'sagitter at fedoraproject dot org'
GPG key: 0x6e0331dd1699e4d7
GPG key server: https://keys.fedoraproject.org/



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Automating package maintainers responsivity check

2018-11-17 Thread Sheogorath
On 11/17/18 11:14 AM, Mattia Verga wrote:
> I would propose some sort of automatic check of maintainer responsivity. 
> Maybe a tool that checks a packager activity in the last 6 months and if 
> there is no activity then sends an email asking the maintainer to 
> confirm they're still involved in Fedora. In case there's no reply, 
> either automatically orphan their packages or notify someone 
> (devel-list) to try to reach them.
> 
> What do you think?
> 

So basically a heartbeat for developers? Maybe make it even more
automatic. When people are not active, they get an email about the
branching. when they don't react their package is not branched to the
new release. This will cause broken builds, yes. But that's actually the
point since people will ask the maintainer to branch it **now**. When
people don't react it's time to get a new maintainer. And the people who
are maybe interested in being this new maintainer: The people who need
the package.

Maybe I'm a bit to easy with taking packages away from unresponsive
maintainers, but I think this would be a quite easy way to take care of
those things.

-- 
Signed
Sheogorath



signature.asc
Description: OpenPGP digital signature
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Automating package maintainers responsivity check

2018-11-17 Thread Mattia Verga
Too often there are "non responsive maintainer" messages here.

Sometimes this is because a maintainer has lost their interest in Fedora 
and left their responsibilities without communication, which means their 
packages can be left unmaintained for long time in repositories before 
someone notice that.

At other times the maintainer is still active, but for some reason they 
are unreachable to users (email change, etc.).

I would propose some sort of automatic check of maintainer responsivity. 
Maybe a tool that checks a packager activity in the last 6 months and if 
there is no activity then sends an email asking the maintainer to 
confirm they're still involved in Fedora. In case there's no reply, 
either automatically orphan their packages or notify someone 
(devel-list) to try to reach them.

What do you think?

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Upgraded from F28 x96_64 XFCE Spin to F29 => rough audio

2018-11-17 Thread Philip Rhoades

Samuel,


On 2018-11-17 19:46, Samuel Sieb wrote:

On 11/17/18 12:17 AM, Philip Rhoades wrote:
My problem is not crackling_or_skipping - it is more just noisy.  That 
link is a fix for a PA problem and is not specific to the F28->F29 
upgrade which has caused the current problem.  I don't want to have to 
re-install PA now - to my way of thinking, that just adds a layer of 
complexity to the problem.


Does booting an F28 kernel fix the audio?  If so, then file a kernel 
bug.



Ah, good thinking - I will boot on the F28 LiveUSB . . and report back.

Thanks!

Phil.
--
Philip Rhoades

PO Box 896
Cowra  NSW  2794
Australia
E-mail:  p...@pricom.com.au
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Upgraded from F28 x96_64 XFCE Spin to F29 => rough audio

2018-11-17 Thread Samuel Sieb

On 11/17/18 12:17 AM, Philip Rhoades wrote:
My problem is not crackling_or_skipping - it is more just noisy.  That 
link is a fix for a PA problem and is not specific to the F28->F29 
upgrade which has caused the current problem.  I don't want to have to 
re-install PA now - to my way of thinking, that just adds a layer of 
complexity to the problem.


Does booting an F28 kernel fix the audio?  If so, then file a kernel bug.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Upgraded from F28 x96_64 XFCE Spin to F29 => rough audio

2018-11-17 Thread Philip Rhoades

Leigh,


On 2018-11-17 18:19, Leigh Scott wrote:

Try

https://fedoraproject.org/wiki/How_to_debug_PulseAudio_problems#Playback_problems.2C_crackling_or_skipping



My problem is not crackling_or_skipping - it is more just noisy.  That 
link is a fix for a PA problem and is not specific to the F28->F29 
upgrade which has caused the current problem.  I don't want to have to 
re-install PA now - to my way of thinking, that just adds a layer of 
complexity to the problem.


Thanks anyway,

Phil.
--
Philip Rhoades

PO Box 896
Cowra  NSW  2794
Australia
E-mail:  p...@pricom.com.au
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org


Re: Fedora Lifecycles: imagine longer-term possibilities

2018-11-17 Thread Leigh Scott
> On Tue, Nov 13, 2018 at 5:36 PM, Matthew Miller 
>  
> "Canonical Extends Ubuntu 18.04 LTS Linux Support to 10 Years"
> 
> https://www.serverwatch.com/server-news/canonical-extends-ubuntu-18.04-lt...
> 
> I just don't see how we're going to be able to compete with that, not 
> unless our Fedora LTS is just CentOS with different branding.
> 
> Michael

I don't consider RHEL7 or CentOS7 are true LTS releases, they are more like 
tame fedora releases now.
Each point release has major changes, 7.6 switched to mesa/libglvnd.
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org
Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: 
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org