Bug#988514: does not work with apt-cacher-ng

[Mike Depot]

I just ran into this bug as well.

>  First: I don't see the URL that fails

I wondered the same thing.  The debug line only shows the name of the file to 
save, not the url it's downloading from.  So in 
simple_cdd/tools/mirror_download.py I changed the log.debug line (54) as 
follows:

log.debug("downloading: %s as %s", url, output)

which confirmed it was getting stuck on the extrafiles file for me as well.

As a workaround, I added the following line to acng.conf :
VfilePatternEx: /extrafiles$

and then restarted the service:
systemctl restart apt-cacher-ng

Bug#988514: does not work with apt-cacher-ng

[Marc Haber]

Hi,

On Fri, May 14, 2021 at 10:36:00AM -0700, Vagrant Cascadian wrote:
> It is probably the "extrafiles"; on my local apt-cacher-ng instance I
> get a 403 error when hitting:
> 
>   http://localhost:3142/deb.debian.org/debian/extrafiles

Confirmed:
1621516920|E|342|127.0.0.1|/extrafiles [HTTP error, code: 403]

> The extrafiles file is used to figure out what files are available and
> the checksums for files not present in debian/dists/*/*Release* (e.g.
> doc/* and tools/*). Maybe you can configure apt-cacher-ng to allow
> "extrafiles" and the relevent files... or omit using them entirely by
> adding any included profiles/*.conf:
> 
>   export OMIT_DOC_TOOLS=1
>   export OMIT_MANUAL=1
>   export OMIT_RELEASE_NOTES=1

All three of those are set, the error still happens.

> Maybe ignore_missing_checksums would work instead

Nope, same error.

Greetings
Marc

-- 
-
Marc Haber | "I don't trust Computers. They | Mailadresse im Header
Leimen, Germany|  lose things."Winona Ryder | Fon: *49 6224 1600402
Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421

Bug#988514: does not work with apt-cacher-ng

[Vagrant Cascadian]

Control: retitle 988514 does not work with apt-cacher-ng

On 2021-05-14, Marc Haber wrote:
> calling build-simple-cdd with --debian-mirror pointing to an
> apt-cacher-ng instalation fails:
> [14/5009]mh@fan:~ $ time build-simple-cdd  --verbose --logfile 
> ./simple-cdd.log --dist buster --locale en_US.UTF-8 --debian-mirror 
> http://debian.debian.zugschlus.de/debian/
> Traceback (most recent call last):
>   File "/usr/bin/build-simple-cdd", line 686, in 
> scdd.build_mirror()
>   File "/usr/bin/build-simple-cdd", line 290, in build_mirror
> self.run_tool("mirror", tool)
>   File "/usr/bin/build-simple-cdd", line 387, in run_tool
> tool.run()
>   File "/usr/lib/python3/dist-packages/simple_cdd/tools/mirror_download.py", 
> line 64, in run
> _download(download_extrafiles_file, extrafiles_file_inlinesig)
>   File "/usr/lib/python3/dist-packages/simple_cdd/tools/mirror_download.py", 
> line 55, in _download
> request.urlretrieve(url, filename=output)
>   File "/usr/lib/python3.9/urllib/request.py", line 239, in urlretrieve
> with contextlib.closing(urlopen(url, data)) as fp:
>   File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen
> return opener.open(url, data, timeout)
>   File "/usr/lib/python3.9/urllib/request.py", line 523, in open
> response = meth(req, response)
>   File "/usr/lib/python3.9/urllib/request.py", line 632, in http_response
> response = self.parent.error(
>   File "/usr/lib/python3.9/urllib/request.py", line 561, in error
> return self._call_chain(*args)
>   File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
> result = func(*args)
>   File "/usr/lib/python3.9/urllib/request.py", line 641, in http_error_default
> raise HTTPError(req.full_url, code, msg, hdrs, fp)
> urllib.error.HTTPError: HTTP Error 403: Forbidden file type or location
>
> First: I don't see the URL that fails

It is probably the "extrafiles"; on my local apt-cacher-ng instance I
get a 403 error when hitting:

  http://localhost:3142/deb.debian.org/debian/extrafiles

The extrafiles file is used to figure out what files are available and
the checksums for files not present in debian/dists/*/*Release* (e.g.
doc/* and tools/*). Maybe you can configure apt-cacher-ng to allow
"extrafiles" and the relevent files... or omit using them entirely by
adding any included profiles/*.conf:

  export OMIT_DOC_TOOLS=1
  export OMIT_MANUAL=1
  export OMIT_RELEASE_NOTES=1


Maybe ignore_missing_checksums would work instead; someone else
implemented that and I haven't tested it.


live well,
  vagrant


signature.asc
Description: PGP signature