Bug#1058579: apt: gives misleading error when not finding Packages.xz in Release (not InRelease)

2023-12-13 Thread David Kalnischkies
On Wed, Dec 13, 2023 at 11:29:29AM +0100, Yann Dirson wrote:
> W: Skipping acquire of configured file 'Packages' as repository 
> 'https://gitlab.com/api/v4/projects/xen-project%2Fxen-guest-agent/packages/generic/deb-amd64
>  ci/ InRelease' does not seem to provide it (sources.list entry misspelt?)

Sadly, this relates to your later observation:

> After more testing, it appears that the behavior is different
> depending on the way the flat repository is used:
[…]
> I'm a bit unsure whether this would be expected.

The "funny" thing about flat repositories is that nearly everything can
be one: http://deb.debian.org/debian/dists/sid/main/binary-amd64/ is
e.g. a flat repository as well… unsigned and without any sort of hashes
so apt will hate your guts by default, but with enough force it will
gobble it up anyhow (it contains a Release file, but that is optional as
well…).

So… if a repository you added doesn't contain a Packages file there is
a good chance that you misspelt something which happened to be still
a valid repository. Perhaps nowadays, that apt even dislikes unsigned
repos we could while not trim the support for these things at least
review and edit messages assuming that users will nowadays only interact
with proper repos…

Messages referring to a repository will usually speak about InRelease
as that is convenient to access from a code point of view. It doesn't
mean the file is used – or even exists. Its just that the code who
handles the file also handles if it doesn't exist, manages the fallbacks
and so on. In an ideal world we would have an easier and better way of
referring to a repository, but in this world we so far don't. There
probably is an "upgrade path", but I don't have the cycles to think
about it at the moment (nor while implementing part of that message
a few years ago). It is somehwhat similar with how messages tend to
talk nearly interchangeable about 'Packages' and 'Packages.xz' even
if neither exists and we end up using 'Packages.gz' (and Packages.lz4).

So, the gist is that yes, it can be misleading, but all individual parts
are correct or have at least a good explanation. I also don't see a lot
of ways to improve on it given that the message is already way too long
and we can not go full analyse mode on untrusted data either…


> root@debian:~# curl 
> https://gitlab.com/api/v4/projects/xen-project%2Fxen-guest-agent/packages/generic/deb-amd64/ci/Release
> Date: Wed, 13 Dec 2023 09:55:54 +
> Description: xen-guest-agent CI packages
> Label: xen-guest-agent-ci
> Suite: ci
> MD5Sum:
>  4bbd32da77623285b8a54ef926a1f028  277 Contents-amd64.gz
>  9c8c7743a78d4c850fbdffac54c5e159  340 Contents-amd64.xz
>  557f3042ec38f51a838d518739ecf4c2  925 Packages.gz
>  a3b82abf0ab81c4d5f829ec631c9deb8 1008 Packages.xz

APT wants checksums for the uncompressed Packages file – even if they
are not shipped in the repository – the info has to be there so that
apt can check that the data it got after uncompressing is really the
intended one. It will also be needed "later" for updates for the data
as files could be stored (re)compressed [different compressor versions
can produce different files having different checksums. Also, we tend to
use compressions for storage which aren't available online to begin
with], not to mention reverify, pdiffs, …

I said wants as there is still support for repos who don't have hashes
and so if you play your cards right, you could manage to make apt accept
your repo, but please don't. Just provide checksums for the uncompressed
files and be done.



So, in summary, I am not sure what to do about this… on some level
I agree, others I consider a user error, but if I look at the cost-
benefit ratio of it all I don't see the parts changing that would need
changing anytime soon so that this report has a very good chance of
staying open until the heat death of the universe (or apt being removed
from Debian, but that is only a theoretical possibility of course).


Best regards

David Kalnischkies


signature.asc
Description: PGP signature


Bug#1058579: apt: gives misleading error when not finding Packages.xz in Release (not InRelease)

2023-12-13 Thread Yann Dirson
Package: apt
Version: 2.6.1

# context

The repo is served as a "generic package repo" on gitlab.  As a first step I'm
putting unsigned Release file there, because setting sigs there is another
adventure.

So I have Release not InRelease, and since it's 2 packages I chose to spare
space using just a Package.gz, hoping for maximum compatibility (apparently
mistakenly so).


# observations

When updating, apt acknowledges it got Release not InRelease, but its error 
message
seems to imply it checked *InRelease* to find a *Packages* file:

root@debian:~# apt update
Hit:1 http://deb.debian.org/debian bookworm InRelease
Hit:2 http://security.debian.org/debian-security bookworm-security InRelease
   
Ign:3 
https://gitlab.com/api/v4/projects/xen-project%2Fxen-guest-agent/packages/generic/deb-amd64
 ci/ InRelease
Hit:4 
https://gitlab.com/api/v4/projects/xen-project%2Fxen-guest-agent/packages/generic/deb-amd64
 ci/ Release
Ign:5 
https://gitlab.com/api/v4/projects/xen-project%2Fxen-guest-agent/packages/generic/deb-amd64
 ci/ Release.gpg
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
77 packages can be upgraded. Run 'apt list --upgradable' to see them.
W: Skipping acquire of configured file 'Packages' as repository 
'https://gitlab.com/api/v4/projects/xen-project%2Fxen-guest-agent/packages/generic/deb-amd64
 ci/ InRelease' does not seem to provide it (sources.list entry misspelt?)


Using "-oDebug::pkgAcquire::Worker=1 -o Debug::Acquire::https=1" indeed shows 
no attempt
at downloading anything after Release.gpg


# my interpretation

There are 3 misleading items in the same statement:
* it likely did not check *InRelease* contents but really *Release*
* OK it did not find *Packages* but only after looking for *Packages.xz*, and 
since
  adding *Packages* back does work, it does not really push users to use the 
default
  compression format.
* the "sources.list entry misspelt?" suggestion feels to throw the user 
completely off-track:
  as it did find a Release file, the entry surely *does* point to a repo