Bug#818731: approx: weekly cronjob breaks with removal of Files field from Sources

[Ian Campbell]

On Tue, 2016-11-15 at 20:48 -0500, Eric Cooper wrote:
> On Sun, Mar 20, 2016 at 08:04:56AM +, Ian Campbell wrote:
> > I've just received the following in a cron mail:
> >
> > /etc/cron.weekly/approx:
> > File
> /var/cache/approx/debian/dists/experimental/main/source/Sources.xz,
> line 1: missing "Files" field
> > run-parts: /etc/cron.weekly/approx exited with return code 1
> >
> > Which I pressume relates to the changes announced in
> > https://lists.debian.org/debian-devel-announce/2016/03/msg6.htm
> l
> >
> > Tagging as blocking the corresponding archive bug.
> >
> > There is a newer version in sid but the changelog doesn't indicate
> that it
> > resolves this issue.
> 
> I can't reproduce this problem.  When I include a deb-src line for
> experimental,  I don't see any Sources.xz files in my approx server's
> cache.

It perhaps depends on the version of apt being used and which
compression it prefers? My system with experimental configured is a
mostly-testing/some-unstable infrequently updated one, which doesn't
tell us much since I don't know what I was running when that file was
created.

> Can you please send me the output of
> ls -l /var/cache/approx/debian/dists/experimental/main/source/

Some weeks ago my /var/cache/approx filled up so I manually removed
that file and ran the cleanup, which succeeded. However since then I am
sure I've been getting cron messages about other paths, but stupidly I
seem to have been deleting it rather than archiving it.

I've just run the following by hand and it matches what I recall the
most recent cron mail to look like:

root@celaeno:~# /usr/sbin/approx-gc
File /var/cache/approx/debian-multimedia/dists/sid/main/source/Sources.xz, line 
1: missing "Files" field

This will correspond to my mythtv box (another testing/unstable hybrid,
not updated as often as I should) which has in sources.list:

# mythtv, liblame, mplayer etc
deb http://mirror/debian-multimedia/ sid main
deb-src http://mirror/debian-multimedia/ sid main

To answer your question, I see:

root@celaeno:~# ls -lrt /var/cache/approx/debian/dists/experimental/main/source/
total 264
-rw-r--r-- 1 approx approx 261196 Jul 12 03:31 Sources.xz
-- 1 approx approx  0 Oct 19 09:12 Sources
drwxr-xr-x 3 approx approx   4096 Oct 23 04:38 by-hash

Which I suppose is no longer relevant to this bug but I included it
because the files are so old and Sources is zero sized which seemed odd
to me (and doesn't match the official mirrors). I just ran "apt update"
on a system with experimental configured and those files have not
changed (the system also runs cronapt nightly). I then ran "apt update
--print-uris and from that ran "curl -o /dev/null http://mirror/debian/
dists/experimental/main/source/Sources.xz" and now:

root@celaeno:~# ls -lrt /var/cache/approx/debian/dists/experimental/main/source/
total 208
-- 1 approx approx  0 Oct 19 09:12 Sources
drwxr-xr-x 3 approx approx   4096 Oct 23 04:38 by-hash
-rw-r--r-- 1 approx approx 201732 Nov 16 02:18 Sources.xz

I've no idea what is up with a plain "apt update", but not a approx
issue I guess.

WRT this bug I suppose you would now be more interested in:

root@celaeno:~# ls -rtl 
/var/cache/approx/debian-multimedia/dists/sid/main/source
total 348
drwxr-xr-x 2 approx approx   4096 May 22  2013 Sources.diff
-rw-r--r-- 1 approx approx 134988 Jul 24  2015 Sources.gz
-rw-r--r-- 1 approx approx 140282 Jul 30 15:09 Sources.bz2
-rw-r--r-- 1 approx approx  57992 Nov 12 18:08 Sources.xz

A representative first stanza from the Sources.xz is:

Package: 2mandvd-dmo
Binary: 2mandvd, 2mandvd-data
Version: 1:1.8.5-dmo4
Maintainer: Christian Marillat 
Build-Depends: debhelper (>= 9), quilt, ccache, libswscale-dev, 
libavformat-dev, libqt4-dev (>= 4:4.7.2~), libqt4-opengl-dev, libqtwebkit-dev, 
libavcodec-dev
Build-Conflicts: qtbase5-dev
Architecture: any all
Standards-Version: 3.9.7
Format: 1.0
Checksums-Sha256:
 9d5882145478bcecaf43cbecc4f70afead88538f669bcbcc08f76c5114f70316 1920 
2mandvd-dmo_1.8.5-dmo4.dsc
 8d34d7fd748f29ab065cec64817628b573dc4781d48fb4ead049a4b7a362f83b 29534393 
2mandvd-dmo_1.8.5.orig.tar.gz
 7e1c97557334517ed9ac974b0e57abccb09cba8b61eefd6c05265c835f894269 11584 
2mandvd-dmo_1.8.5-dmo4.diff.gz
Homepage: http://2mandvd.tuxfamily.org/
Package-List: 
 2mandvd deb video extra arch=any
 2mandvd-data deb video extra arch=all
Directory: pool/main/2/2mandvd-dmo
Priority: extra
Section: misc

AIUI omitting the Files block with their md5 checksums in favour of
Checksums-Sha256 (or other checksum varieties stronger than md5) is
valid and I think something Debian ftp-masters are moving towards as
well (https://lists.debian.org/debian-devel-announce/2016/03/msg6.h
tml, although I see Files entries in
/var/cache/approx/debian/dists/experimental/main/source/Sources.xz and 
ftp://ftp.debian.org/debian/dists/experimental/main

Bug#818731: approx: weekly cronjob breaks with removal of Files field from Sources

[Eric Cooper]

On Sun, Mar 20, 2016 at 08:04:56AM +, Ian Campbell wrote:
> I've just received the following in a cron mail:
>
> /etc/cron.weekly/approx:
> File /var/cache/approx/debian/dists/experimental/main/source/Sources.xz, line 
> 1: missing "Files" field
> run-parts: /etc/cron.weekly/approx exited with return code 1
>
> Which I pressume relates to the changes announced in
> https://lists.debian.org/debian-devel-announce/2016/03/msg6.html
>
> Tagging as blocking the corresponding archive bug.
>
> There is a newer version in sid but the changelog doesn't indicate that it
> resolves this issue.

I can't reproduce this problem.  When I include a deb-src line for
experimental,  I don't see any Sources.xz files in my approx server's
cache.

Can you please send me the output of
ls -l /var/cache/approx/debian/dists/experimental/main/source/

Thanks, and sorry for the inconvenience.

--
Eric Cooper e c c @ c m u . e d u

Bug#818731: approx: weekly cronjob breaks with removal of Files field from Sources

[Niels Thykier]

On Sun, 20 Mar 2016 08:04:56 + Ian Campbell  wrote:
> Package: approx
> Version: 5.5-1
> Severity: important
> Control: block 818463 by -1
> 
> Dear Maintainer,
> 
> I've just received the following in a cron mail:
> 
> /etc/cron.weekly/approx:
> File /var/cache/approx/debian/dists/experimental/main/source/Sources.xz, line 
> 1: missing "Files" field
> run-parts: /etc/cron.weekly/approx exited with return code 1
> 
> Which I pressume relates to the changes announced in
> https://lists.debian.org/debian-devel-announce/2016/03/msg6.html
> 
> Tagging as blocking the corresponding archive bug.
> 
> There is a newer version in sid but the changelog doesn't indicate that it
> resolves this issue.
> 
> Ian.
> 
> [...]


Hi,

Is there any news on this bug?

It is my understanding that the FTP masters want to retire Packages.gz
shortly before the Stretch release.

Thanks,
~Niels

Bug#818731: approx: weekly cronjob breaks with removal of Files field from Sources

[Ian Campbell]

Package: approx
Version: 5.5-1
Severity: important
Control: block 818463 by -1

Dear Maintainer,

I've just received the following in a cron mail:

/etc/cron.weekly/approx:
File /var/cache/approx/debian/dists/experimental/main/source/Sources.xz, line 
1: missing "Files" field
run-parts: /etc/cron.weekly/approx exited with return code 1

Which I pressume relates to the changes announced in
https://lists.debian.org/debian-devel-announce/2016/03/msg6.html

Tagging as blocking the corresponding archive bug.

There is a newer version in sid but the changelog doesn't indicate that it
resolves this issue.

Ian.

-- System Information:
Debian Release: 8.3
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: armel (armv5tel)

Kernel: Linux 3.16.0-4-kirkwood
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: sysvinit (via /sbin/init)

Versions of packages approx depends on:
ii  adduser   3.113+nmu3
ii  bzip2 1.0.6-7+b3
ii  curl  7.38.0-4+deb8u3
ii  debconf [debconf-2.0] 1.5.56
ii  libc6 2.19-18+deb8u3
ii  libpcre3  2:8.35-3.3+deb8u2
ii  openbsd-inetd [inet-superserver]  0.20140418-2
ii  rsyslog [system-log-daemon]   8.4.2-1+deb8u2
ii  update-inetd  4.43
ii  xz-utils  5.1.1alpha+20120614-2+b3

approx recommends no packages.

Versions of packages approx suggests:
pn  libconfig-model-approx-perl  

-- Configuration Files:
/etc/approx/approx.conf changed:
debian  http://ftp.uk.debian.org/debian
debian-security http://security.debian.org/debian-security
debian-multimedia   http://www.deb-multimedia.org
debian-portshttp://ftp.debian-ports.org/debian
$verbosetrue
$debug  true


-- debconf information:
  approx/port: