Re: [RFC] delta-repository-metadata

[Ken Dreyer]

On Thu, Apr 27, 2017 at 11:02 AM, Igor Gnatenko
 wrote:
> We would like you to try it out and give feedback, it is very important
> for us! Just send it as reply to this message..


This is an interesting feature, thanks for announcing it and
presenting at DevConf.

Colin Walters made an interesting point in a different context [1]
when it comes to adding additional files to the package manager.
Mirror content updates are not atomic, so mirrors could have some
files but not others. In this context, Colin pointed out that a mirror
could update repomd.xml independently of repomd.xml.asc. This leads to
client races, where the only solution is to retry.

Previously, Debian used to have apt-get download its "Release" files
and GPG signatures in separate "Release.asc" files. With the number of
mirrors and the number of clients, these updates would race frequently
and Debian finally moved to inline-gpg-signing their files as
"InRelease". [2]

When I think about this solution of introducing more metadata files on
the mirrors, I'm wondering if this could introduce more client races
in a similar way? What is the behavior when mirrors misbehave, or are
just a little slow?

- Ken


[1] https://pagure.io/releng/issue/133
[2] 
http://www.chiark.greenend.org.uk/~cjwatson/blog/no-more-hash-sum-mismatch-errors.html
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [RFC] delta-repository-metadata

[Hedayat Vatankhah]

Hi!

I'm trying it under F25. First, thanks for working on it! It'd be a 
great feature. However:


1. I run it, and it was running for too long (probably 10 minutes!) with 
absolutely no log. I found that it is working on a 200M 
filelists.xml.gz.part file in DNF cache (which is apparently an 
uncompressed filelists file). And I guess it also downloads some data 
(at least the .zsync file, and probably also the new data) again without 
any reports to the user.


2. Finally, it finished, and this is the result:

dnf update -vvv --disablerepo="rpmfusion*" --disablerepo=fedora
cachedir: /var/cache/dnf
Loaded plugins: local, noroot, needs-restarting, debuginfo-install, 
zsync, protected_packages, copr, playground, builddep, repomanage, 
Query, reposync, download, config-manager, generate_completion_cache

DNF version: 1.1.10
repo: using cache for: localfedora
not found deltainfo for: Local Fedora 25 - x86_64
not found updateinfo for: Local Fedora 25 - x86_64
repo: using cache for: group_rpm-software-management-deltamd
not found deltainfo for: Copr repo for deltamd owned by 
@rpm-software-management
not found updateinfo for: Copr repo for deltamd owned by 
@rpm-software-management

repo: using cache for: _dnf_local
not found deltainfo for: _dnf_local
not found updateinfo for: _dnf_local
Error: Cache-only enabled but no cache for 'updates'


3. I have not used -C option with DNF, so I wonder why it is 
complaining! Also, the filelists.xml.gz.part file is still there, so I 
guess zsync has not completed its task completely.



Note: I've interrupted the command once the first time after installing 
the plugin, and run it again. I wonder if the interruption has anything 
to do with the problem.



And I think if it is going to always take so much time for filelists 
file, maybe it is better to make zsync use the compressed file instead 
of uncompressed data for big files.



Regards,

Hedayat



/*Igor Gnatenko*/ wrote on Thu, 27 Apr 2017 19:02:02 +0200:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Hello users and developers,

We had presented deltametadata project at DevConf.CZ 2017[0], goal of
this project is to minimize bandwidth required on clients to update
repository metadata (repomd.xml, primary.xml, etc.) which would improve
user experience with DNF.

So, we have DNF plugin which acts

How to try it?
1. dnf -y copr enable @rpm-software-management/deltamd
2. dnf -y install dnf-plugin-zsync

If something doesn't work, look into /var/log/dnf.log.


We would like you to try it out and give feedback, it is very important
for us! Just send it as reply to this message..

P.S. F25 and above are supported


[0] https://www.youtube.com/watch?v=l6uWxg3yMmw
- -- 
Regards,

lab-q Red Hat
-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEEhLFO09aHZVqO+CM6aVcUvRu8X0wFAlkCJAoACgkQaVcUvRu8
X0wrGg/+Lvb/up8swQR6Nd9BacvrubhgELvOcOIsv/69aMPqe9cDqmAKUr33AWik
zUWbAwpkS79BLLKjoxHTxR379Ubmj5mCWyuufg/HCgeCHOZwIBEQzqrBjwKMsIN+
ts6Qk7mo1bsgS3xozx/tRH+N1xhx1O+UInsDxS1qKQ5KIBntf/e8ZrgFL/fbjHYx
/JVlz2p44tZxMHCsZP/hJv+DKV+AaY9/NcvLh2KBvpj75h8cVA3RkSE85YgNFNyD
GJRjkzv+swmwWXVZMd0XNiCKKXn+qGooMsrrjAmbousG/Gd/DmTQekuO/hIkGZ+g
rjSCPRjLXe+utD2lnqK5aWm1nJvaEUTjTAWKYAhV0mAAT/Y4Y8S2XcsdY6EXxiW6
iSYmXa4Rz4rTYiZ+K2XXYoEIGDVSN8C5lQyYy4Qqg2lnAyKDZ1567s/4fCz/IYpB
rpDv+4jX1o3C8RY0XUuCYkt+/IpVH+1/DfkhWJCG2rNLDpnq5R7N4cryJg0MClW1
PUw24GJFbIwWnl6he0n+ShJ6wzZvd2jHwFBVj5a/718wech3YnIfSHf/xJeIE6Nn
dK716wgmHUPyjCwk/cWQWdj5x99lMiDZJIXXFTu0zGLxXcHzz9MnZjDCv219u6XU
Vj0dCKWa3SoLml+zmRkqb+T4acyyXlwPBahRFXN3qBhFpqoA/MY=
=5ar8
-END PGP SIGNATURE-
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org

Re: [RFC] delta-repository-metadata

[Björn 'besser82' Esser]

Am 27.04.2017 um 19:02 schrieb Igor Gnatenko:

P.S. F25 and above are supported


Well, no…  I tried the plugin on fc26 and dnf refused to do anything 
after that:


AttributeError: 'Repo' object has no attribute 'cachedir'

I had to remove the plugin using low-level rpm…

Looks like the code isn't ported to dnf >= 2.0, yet…  :/


Cheers,
  Björn
___
devel mailing list -- devel@lists.fedoraproject.org
To unsubscribe send an email to devel-le...@lists.fedoraproject.org